[AMDGPU] Move call clobbered return address registers s[30:31] to callee saved range

[RamNalamothu]

Summary

Currently the return address ABI registers s[30:31], which fall in the call
clobbered register range, are added as a live-in on the function entry to
preserve its value when we have calls so that it gets saved and restored
around the calls.

But the DWARF unwind information (CFI) needs to track where the return address
resides in a frame and the above approach makes it difficult to track the
return address when the CFI information is emitted during the frame lowering,
due to the involvment of understanding the control flow.

This patch moves the return address ABI registers s[30:31] into callee saved
registers range and stops adding live-in for return address registers, so that
the CFI machinery will know where the return address resides when CSR
save/restore happen during the frame lowering.

And doing the above poses an issue that now the return instruction uses undefined
register sgpr30_sgpr31. This is resolved by hiding the return address register
use by the return instruction through the SI_RETURN pseudo instruction, which
doesn't take any input operands, until the SI_RETURN pseudo gets lowered to the
S_SETPC_B64_return during the expandPostRAPseudo().

As an added benefit, this patch simplifies overall return instruction handling.

Note: The AMDGPU CFI changes are there only in the downstream code and another
version of this patch will be posted for review for the downstream code.

[sebastian-ne]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

94–96

Can the MVT be deduced from getRegSizeInBits(*getPhysRegClass(Reg))? I think that would look cleaner.

232

Why is the return address only marked as callee-save if this function has calls? I assume it can’t it be overwritten if there are no calls because it is marked as reserved?

Coming back here after reading the tests, I think s[30:31] should also be marked as callee-save if there are no calls. It shouldn’t cause harm if there are no calls (the register is reserved anyway) and it should fix the tests.

llvm/test/CodeGen/AMDGPU/branch-relax-spill.ll

897

This test is not working with this patch. It has no calls, so s[30:31] are not marked as callee-save. But they are overwritten in inline assembly.
Before this patch, they were saved and restored. With this patch this is not the case and the return will jump to an invalid address.

llvm/test/CodeGen/AMDGPU/call-preserved-registers.ll

75–88

I think this test is supposed to clobber the return address in s[30:31].

llvm/test/CodeGen/AMDGPU/callee-frame-setup.ll

424

I think these tests should continue to work without the need to introduce a call.

llvm/test/CodeGen/AMDGPU/gfx-callable-preserved-registers.ll

127

I think this test purposefully clobbered the return address in s[30:31].

[sebastian-ne]

llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll

26–49

This codes gets quite a bit worse. But I’m not sure if we can do anything about it.

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

94–96

Can the MVT be deduced from getRegSizeInBits(*getPhysRegClass(Reg))? I think that would look cleaner.

Will check, thank you.

232

Why is the return address only marked as callee-save if this function has calls? I assume it can’t it be overwritten if there are no calls because it is marked as reserved?

Yes.

Coming back here after reading the tests, I think s[30:31] should also be marked as callee-save if there are no calls. It shouldn’t cause harm if there are no calls (the register is reserved anyway) and it should fix the tests.

AFAIK that's the eventual goal and there is a discussion to pick the ABI registers for return address from the callee saved range. And these changes should work with no/minimal changes.

However, I don't think I understand how the tests will get fixed automatically if the return address registers are marked as callee saved because even the SGPR CSR save/restore happens during SGPRSpillLowering which is where I am manually adding s[30:31] to the CSR list.

llvm/test/CodeGen/AMDGPU/callee-frame-setup.ll

424

I think these tests should continue to work without the need to introduce a call.

Thanks. I will take a re-look at it. I don't recall now why I added the call.

llvm/test/CodeGen/AMDGPU/gfx-callable-preserved-registers.ll

127

I think this test purposefully clobbered the return address in s[30:31].

I believe the intent here is to test the behavior when non callee saved registers are clobbered inside inline asm but not particularly s[30:31] and I guess s[30:31] happened to be used since they are the boundary of non callee save range. The same goes for the other tests which were clobbering s[30:31].

As of now there is no policy on how the inline asm could use the ABI registers. I spoke to @arsenm and he is aware of this test changes.

llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll

26–49

This codes gets quite a bit worse. But I’m not sure if we can do anything about it.

Yeah. This is a known scenario where previously the s[30:31] were being spilled to another SGPR pair, which was used to return, but now since the return address should always be in s[30:31] it has to be spilled into either VGPR or memory.

For now, we will have to live with it.

May be we could do something during SGPRSpillLowering to use the available SGPRs, not only for this case but in general?
I didn't check if that capability is already in place and s[30:31] is a corner case which is not handled.

[scott.linder]

llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll

26–49

I'm not sure I understand how the old code is correct to begin with; does it rely on knowledge about the called function?

I think in a "fixed function ABI" world we can't just clobber (in this example) s[36:37], right?

[sebastian-ne]

llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll

26–49

As you said, it relies on knowledge about the called function.
The IPRA (Inter Procedural Register Allocation) pass looks at direct callees and removes registers from the call-clobbered list if the called function does not touch them.

[arsenm]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

I don't think this register should be reserved, which avoids the need to do this

llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp

630–633 ↗

(On Diff #390096)

I don't think it's necessary to reserve this

[RamNalamothu]

Rebase and fix tests in llvm/test/CodeGen/AMDGPU/callee-frame-setup.ll.

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

94–96

getRegSizeInBits(*getPhysRegClass(Reg)) is not useful as getPhysRegClass(Reg) gives base register class i.e. SReg_32.

/// Return the 'base' register class for this register.
/// e.g. SGPR0 => SReg_32, VGPR => VGPR_32 SGPR0_SGPR1 -> SReg_32, etc.
const TargetRegisterClass *getPhysRegClass(MCRegister Reg) const;

llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp

630–633 ↗

(On Diff #390096)

But otherwise the register allocation thinks it is available for allocation as s[30:31] are scratch registers at the moment and its usage with return instruction is hidden with using SI_RETURN through the register allocation and we don't emit Live-ins anymore.

And we will have to reserve it here even after these registers actually get picked from CSR range because we want to emit save/restore exclusively so that the unwind info machinery knows it and can generate CFI as needed for return address. That's the reason why all of this got started right.

llvm/test/CodeGen/AMDGPU/callee-frame-setup.ll

424

My bad. I understood them wrongly.

Yes, you are right. Removed the calls and fixed the tests accordingly.

[arsenm]

llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp

630–633 ↗

(On Diff #390096)

I think something special happens with the return address we are ignoring, but I haven't followed it all the way through.

AArch64 passes LR to the TargetRegisterInfo constructor, where we're passing the dummy PC_REG. If you replace that with the real return address, does that help?

[arsenm]

llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp

630–633 ↗

(On Diff #390096)

Even if not, AArch64 does not reserve the return address

[RamNalamothu]

llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp

630–633 ↗

(On Diff #390096)

I looked at it already and it appears there is nothing magical about it except getting MCRegisterInfo::getRARegister() to return that register from the constructor.

And we get the same by hardcoding s[30:31] in SIRegisterInfo::getReturnAddressReg().

Though I didn't go through everything that Hexagon does, but they also reserve the return address register.

def retflag : SDNode<"HexagonISD::RET_FLAG", SDTNone,
                      [SDNPHasChain, SDNPOptInGlue, SDNPVariadic]>;
def: Pat<(retflag),   (PS_jmpret (i32 R31))>;

[sebastian-ne]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

Why is the return address not callee-save if there are no calls?

Does removing the && MFI.hasCalls() condition have any bad effects?

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

No functional issues but its just that it will be unnecessary to save/restore when the registers are not clobbered i.e. when we have no calls.

[sebastian-ne]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

But if we have no calls and the register is not clobbered, marking the registers as callee-save shouldn’t save and restore them?

I think it doesn’t hurt to remove this condition. On the contrary, I expect that removing the hasCalls condition will fix the case where s30 or s31 are overwritten in inline-assembly.

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

Sorry, I didn't get your point well.

Are you saying we should save/restore s[30:31] even when we don't have calls i.e. even when s[30:31] is not modified in the current function? Does that help in any way, other than the inline asm scenario?

For the inline asm scenario, as I already mentioned. I heard that we currently don't have any policy and the ABI registers cannot be clobbered like any other scratch register right?
And even if we want to handle that scenario, I guess a blanket removal of hasCalls() condition is not the best/optimal IMO because we will be emitting unnecessary save/restore if the code inside inline asm does not actually modify s[30:31].

[sebastian-ne]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

Ups, ignore my previous comment. I should have read the code again before making that comment.

What I thought this code would do, is add s[30:31] to the callee-save register list if this condition is satisfied. I did not see that it is already spilling it, based on the condition.

Let me try what happens if s[30:31] is (unconditionally) added to the CSR list and the code here is removed.

[sebastian-ne]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

I tested it and I think marking s[30:31] as callee-save register instead of explicitly saving them here is better.

1. We don’t need hasCall to check if s[30:31] may be overwritten.
2. We shouldn’t need to mark the register as reserved (didn’t test this).
3. It “just works”, even if there is inline assembly overwriting s30 or s31 (and there are no unnecessary saves/restores if it isn’t overwritten).

The last point also means you don’t need to add work-arounds to tests by introducing artificial calls or changing the registers in inline assembly, because it will keep working as is.

See this sketched commit: https://github.com/Flakebi/amdvlk-llvm-project/commit/240040601e0f8849acb85141142408b3e6a32ff9
E.g. it fixed the test here: https://github.com/Flakebi/amdvlk-llvm-project/commit/240040601e0f8849acb85141142408b3e6a32ff9#diff-25d99c15c21155a13557b29a1c0edfa792c19040f02a0d7cd676865b46275713L75-R88

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

What you did there is you just took a part of this patch and doing that is incorrect. If you still want to do those experiments, please take the full patch and try making your changes on top of that.

This patch makes changes to return instruction and its use of return address during the instruction selection. I think you missed to fully understand that aspect.

Please read again my earlier response to @arsenm which I copied here.

I don't think it's necessary to reserve this

But otherwise the register allocation thinks it is available for allocation as s[30:31] are scratch registers at the moment and its usage with return instruction is hidden with using SI_RETURN through the register allocation and we don't emit Live-ins anymore.

And we will have to reserve it here even after these registers actually get picked from CSR range because we want to emit save/restore exclusively so that the unwind info machinery knows it and can generate CFI as needed for return address. That's the reason why all of this got started right.

[sebastian-ne]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

I used arc patch to get the diff (https://github.com/Flakebi/amdvlk-llvm-project/commit/240040601e0f8849acb85141142408b3e6a32ff9) and based my commit on top of that, so I hope it got all of your changes?

Unfortunately I’m not familiar with CFI (I guess it refers to call-frame-information and not control-flow-integrity?). Are there more upcoming patches that need the explicit spill here and can’t rely on the llvm infrastructure for CSRs?

I like that this patch simplifies the call-related code in amdgpu, but as you probably noticed, I’m not that happy that inline assembly that touches s[30:31] now ends up with non-working code.

If this is an intended change (and I don’t feel authoritative to say if this is fine or not), I think it should be mentioned in the commit message that s[30:31] is not saved anymore if used in inline assembly. And the tests that intend to test that (like @void_func_void_clobber_s30_s31) should probably just be removed.

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

Ah, my bad. I didn't see the parent commit.

It is call frame information and there will be additional CFI changes but only in the downstream. They are not/can't be up-streamed yet.

This patch is written with the assumption that s[30:31] can't be moved to CSR range right now. Like I mentioned in the beginning, there is already a discussion to pick a different register pair (i.e. s[28:29]) for return address which will be part of the (new) CSR range.

Yes, the condition check in SILowerSGPRSpills can be removed if we can move s[30:31] to CSR range now. @t-tye @arsenm what do you say?

However s[30:31] spill should not come from the register allocation and must be generated only during SGPR/frame lowering for the unwind info (CFI) generation. For this, I still believe s[30:31] has to be reserved so that the register allocation won't interfere with that. Or am I missing something here @scott.linder ?

[RamNalamothu]

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

232

@t-tye @arsenm @scott.linder ping.

[RamNalamothu]

Rebase onto ToT and also made the following changes to address review feedback.

• Move s[30:31] into CSR range
• Do not reserve s[30:31] during register allocation

[sebastian-ne]

Thanks for making these changes!
The patch looks good to me, I’ll leave it for others to accept.

[arsenm]

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

2146

If we're going to just hardcode this to s[30:31], probably should do away with getReturnAddressReg

2149–2154

There's a copyImplicitOperands helper for this

[RamNalamothu]

Rebase onto ToT and use copyImplicitOps.

[RamNalamothu]

I think I have addressed all the outstanding feedback. Is this good to go now?

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

2146

I believe using getReturnAddressReg() doesn't hurt and any future updates can be made easily without changing all the places.

[arsenm]

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

2150

Should break or return true, I don't like switch fallthrough here

[RamNalamothu]

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

2150

Ah, thanks for catching it. Fixed in the committed changes.

[ronlieb]

hi Ram, landed patch breaks https://lab.llvm.org/buildbot/#/builders/193/builds/3514

[RamNalamothu]

Fixed an issue exposed by OpenMP LIT test failures.

determineCalleeSavesSGPR() now explicitly adds S30 and S31 to CSR list
when we have calls and this is needed for cases like D117243.

[RamNalamothu]

Fixed the blender application failure seen in the downstream testing.

Added the test llvm/test/CodeGen/AMDGPU/ipra-return-address-save-restore.ll to catch that scenario.

[arsenm]

llvm/lib/Target/AMDGPU/SIFrameLowering.cpp

1351

Referring to phabricator reviews in a comment is weird

1359

Can just directly check isPhysRegModified(RetAddrReg)

[RamNalamothu]

Address review feedback.

[RamNalamothu]

llvm/lib/Target/AMDGPU/SIFrameLowering.cpp

1351

I have seen this in many places of clang/llvm and in fact in AMDGPU's source/tests as well. But if you say this is not a recommended practice, I will remove those references while landing the patch.

1359

Fixed. Thanks.

[arsenm]

I'm not sure I understand why a separate isPhysRegUsed check is needed since ultimately that's what IPRA is doing, but I don't see how it could hurt

[RamNalamothu]

In D114652#3367310, @arsenm wrote:

I'm not sure I understand why a separate isPhysRegUsed check is needed since ultimately that's what IPRA is doing, but I don't see how it could hurt

The separate isPhysRegModified check is needed because we will have the SI_RETURN pseudo, which doesn't have return address operand/use, when IPRA's register usage collection runs and that hides the return address use.
Since the caller function save/restores return address registers, as the call instruction clobbers them, and return address registers use in the callee function is hidden, the IPRA/register allocator concludes that return address registers can be clobbered without save/restoring them in the callee function if the callee function doesn't have any function calls.

Hope that helps.

llvm/lib/Target/AMDGPU/SIFrameLowering.cpp

void SIFrameLowering::determineCalleeSavesSGPR(MachineFunction &MF,
   // there are any SGPR spills. Whether they are CSR spills or otherwise.
   MachineFrameInfo &FrameInfo = MF.getFrameInfo();
   const bool WillHaveFP =
       FrameInfo.hasCalls() && (AllSavedRegs.any() || MFI->hasSpilledSGPRs());
 
   // FP will be specially managed like SP.
   if (WillHaveFP || hasFP(MF))
     SavedRegs.reset(MFI->getFrameOffsetReg());
+
+  // Return address use with return instruction is hidden through the SI_RETURN
+  // pseudo. Given that and since the IPRA computes actual register usage and
+  // does not use CSR list, the clobbering of return address by function calls
+  // (D117243) or otherwise (D120922) is ignored/not seen by the IPRA's register
+  // usage collection. This will ensure save/restore of return address happens
+  // in those scenarios.
+  const MachineRegisterInfo &MRI = MF.getRegInfo();
+  Register RetAddrReg = TRI->getReturnAddressReg(MF);
+  if (!MFI->isEntryFunction() &&
+      (FrameInfo.hasCalls() || MRI.isPhysRegModified(RetAddrReg))) {
+    SavedRegs.set(TRI->getSubReg(RetAddrReg, AMDGPU::sub0));
+    SavedRegs.set(TRI->getSubReg(RetAddrReg, AMDGPU::sub1));
+  }
 }
 
 bool SIFrameLowering::assignCalleeSavedSpillSlots(
     MachineFunction &MF, const TargetRegisterInfo *TRI,
     std::vector<CalleeSavedInfo> &CSI) const {
   if (CSI.empty())
     return true; // Early exit if no callee saved registers are modified!
 

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

bool SIInstrInfo::expandPostRAPseudo(MachineInstr &MI) const {
   }
   case AMDGPU::EXIT_STRICT_WWM:
   case AMDGPU::EXIT_STRICT_WQM: {
     // This only gets its own opcode so that SIPreAllocateWWMRegs can tell when
     // WWM/STICT_WQM is exited.
     MI.setDesc(get(ST.isWave32() ? AMDGPU::S_MOV_B32 : AMDGPU::S_MOV_B64));
     break;
   }
+  case AMDGPU::SI_RETURN: {
+    const MachineFunction *MF = MBB.getParent();
+    const GCNSubtarget &ST = MF->getSubtarget<GCNSubtarget>();
+    const SIRegisterInfo *TRI = ST.getRegisterInfo();
+    // Hiding the return address use with SI_RETURN may lead to extra kills in
+    // the function and missing live-ins. We are fine in practice because callee
+    // saved register handling ensures the register value is restored before
+    // RET, but we need the undef flag here to appease the MachineVerifier
+    // liveness checks.
+    MachineInstrBuilder MIB =
+        BuildMI(MBB, MI, DL, get(AMDGPU::S_SETPC_B64_return))
+            .addReg(TRI->getReturnAddressReg(*MF), RegState::Undef);
+
+    MIB.copyImplicitOps(MI);
+    MI.eraseFromParent();
+    break;
+  }
   }
   return true;
 }
 
 std::pair<MachineInstr*, MachineInstr*>
 SIInstrInfo::expandMovDPP64(MachineInstr &MI) const {
   assert (MI.getOpcode() == AMDGPU::V_MOV_B64_DPP_PSEUDO);
 
   MachineBasicBlock &MBB = *MI.getParent();
   DebugLoc DL = MBB.findDebugLoc(MI);
   MachineFunction *MF = MBB.getParent();

llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp

 /// Insert restore code for the callee-saved registers used in the function.
 static void insertCSRSaves(MachineBasicBlock &SaveBlock,
                            ArrayRef<CalleeSavedInfo> CSI,
                            LiveIntervals *LIS) {
   MachineFunction &MF = *SaveBlock.getParent();
   const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
   const TargetFrameLowering *TFI = MF.getSubtarget().getFrameLowering();
   const TargetRegisterInfo *TRI = MF.getSubtarget().getRegisterInfo();
+  const GCNSubtarget &ST = MF.getSubtarget<GCNSubtarget>();
+  const SIRegisterInfo *RI = ST.getRegisterInfo();
 
   MachineBasicBlock::iterator I = SaveBlock.begin();
   if (!TFI->spillCalleeSavedRegisters(SaveBlock, I, CSI, TRI)) {
     const MachineRegisterInfo &MRI = MF.getRegInfo();
 
     for (const CalleeSavedInfo &CS : CSI) {
       // Insert the spill to the stack frame.
       MCRegister Reg = CS.getReg();
 
       MachineInstrSpan MIS(I, &SaveBlock);
-      const TargetRegisterClass *RC =
-        TRI->getMinimalPhysRegClass(Reg, MVT::i32);
+      const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(
+          Reg, Reg == RI->getReturnAddressReg(MF) ? MVT::i64 : MVT::i32);
 
       // If this value was already livein, we probably have a direct use of the
       // incoming register value, so don't kill at the spill point. This happens
       // since we pass some special inputs (workgroup IDs) in the callee saved
       // range.
       const bool IsLiveIn = MRI.isLiveIn(Reg);
       TII.storeRegToStackSlot(SaveBlock, I, Reg, !IsLiveIn, CS.getFrameIdx(),
                               RC, TRI);
 

 /// Insert restore code for the callee-saved registers used in the function.
 static void insertCSRRestores(MachineBasicBlock &RestoreBlock,
                               MutableArrayRef<CalleeSavedInfo> CSI,
                               LiveIntervals *LIS) {
   MachineFunction &MF = *RestoreBlock.getParent();
   const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
   const TargetFrameLowering *TFI = MF.getSubtarget().getFrameLowering();
   const TargetRegisterInfo *TRI = MF.getSubtarget().getRegisterInfo();
-
+  const GCNSubtarget &ST = MF.getSubtarget<GCNSubtarget>();
+  const SIRegisterInfo *RI = ST.getRegisterInfo();
   // Restore all registers immediately before the return and any
   // terminators that precede it.
   MachineBasicBlock::iterator I = RestoreBlock.getFirstTerminator();
 
   // FIXME: Just emit the readlane/writelane directly
   if (!TFI->restoreCalleeSavedRegisters(RestoreBlock, I, CSI, TRI)) {
     for (const CalleeSavedInfo &CI : reverse(CSI)) {
       Register Reg = CI.getReg();
-      const TargetRegisterClass *RC =
-        TRI->getMinimalPhysRegClass(Reg, MVT::i32);
+      const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(
+          Reg, Reg == RI->getReturnAddressReg(MF) ? MVT::i64 : MVT::i32);
 
       TII.loadRegFromStackSlot(RestoreBlock, I, Reg, CI.getFrameIdx(), RC, TRI);
       assert(I != RestoreBlock.begin() &&
              "loadRegFromStackSlot didn't insert any code!");
       // Insert in reverse order.  loadRegFromStackSlot can insert
       // multiple instructions.
 
       if (LIS) {
for (unsigned I = 0; CSRegs[I]; ++I) {
         CSI.push_back(CalleeSavedInfo(Reg, JunkFI));
       }
     }
 
     if (!CSI.empty()) {
       for (MachineBasicBlock *SaveBlock : SaveBlocks)
         insertCSRSaves(*SaveBlock, CSI, LIS);
 
       // Add live ins to save blocks.
       assert(SaveBlocks.size() == 1 && "shrink wrapping not fully implemented");
       updateLiveness(MF, CSI);
 
       for (MachineBasicBlock *RestoreBlock : RestoreBlocks)
         insertCSRRestores(*RestoreBlock, CSI, LIS);
       return true;
     }
   }

llvm/test/CodeGen/AMDGPU/branch-relax-spill.ll

bb3:
   tail call void asm sideeffect "; reg use $0", "{s99}"(i32 %sgpr99) #0
   tail call void asm sideeffect "; reg use $0", "{s100}"(i32 %sgpr100) #0
   tail call void asm sideeffect "; reg use $0", "{s101}"(i32 %sgpr101) #0
   tail call void asm sideeffect "; reg use $0", "{vcc_lo}"(i32 %vcc_lo) #0
   tail call void asm sideeffect "; reg use $0", "{vcc_hi}"(i32 %vcc_hi) #0
   ret void
 }
 
 define void @spill_func(i32 addrspace(1)* %arg) #0 {
 ; CHECK-LABEL: spill_func:
 ; CHECK:       ; %bb.0: ; %entry
 ; CHECK-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; CHECK-NEXT:    s_or_saveexec_b64 s[4:5], -1
 ; CHECK-NEXT:    buffer_store_dword v0, off, s[0:3], s32 ; 4-byte Folded Spill
 ; CHECK-NEXT:    buffer_store_dword v1, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
 ; CHECK-NEXT:    s_mov_b64 exec, s[4:5]
 ; CHECK-NEXT:    s_waitcnt expcnt(1)
-; CHECK-NEXT:    v_writelane_b32 v0, s33, 0
-; CHECK-NEXT:    v_writelane_b32 v0, s34, 1
-; CHECK-NEXT:    v_writelane_b32 v0, s35, 2
-; CHECK-NEXT:    v_writelane_b32 v0, s36, 3
-; CHECK-NEXT:    v_writelane_b32 v0, s37, 4
-; CHECK-NEXT:    v_writelane_b32 v0, s38, 5
-; CHECK-NEXT:    v_writelane_b32 v0, s39, 6
-; CHECK-NEXT:    v_writelane_b32 v0, s40, 7
-; CHECK-NEXT:    v_writelane_b32 v0, s41, 8
-; CHECK-NEXT:    v_writelane_b32 v0, s42, 9
-; CHECK-NEXT:    v_writelane_b32 v0, s43, 10
-; CHECK-NEXT:    v_writelane_b32 v0, s44, 11
-; CHECK-NEXT:    v_writelane_b32 v0, s45, 12
-; CHECK-NEXT:    v_writelane_b32 v0, s46, 13
-; CHECK-NEXT:    v_writelane_b32 v0, s47, 14
-; CHECK-NEXT:    v_writelane_b32 v0, s48, 15
-; CHECK-NEXT:    v_writelane_b32 v0, s49, 16
-; CHECK-NEXT:    v_writelane_b32 v0, s50, 17
-; CHECK-NEXT:    v_writelane_b32 v0, s51, 18
-; CHECK-NEXT:    v_writelane_b32 v0, s52, 19
-; CHECK-NEXT:    v_writelane_b32 v0, s53, 20
-; CHECK-NEXT:    v_writelane_b32 v0, s54, 21
-; CHECK-NEXT:    v_writelane_b32 v0, s55, 22
-; CHECK-NEXT:    v_writelane_b32 v0, s56, 23
-; CHECK-NEXT:    v_writelane_b32 v0, s57, 24
-; CHECK-NEXT:    v_writelane_b32 v0, s58, 25
-; CHECK-NEXT:    v_writelane_b32 v0, s59, 26
-; CHECK-NEXT:    v_writelane_b32 v0, s60, 27
-; CHECK-NEXT:    v_writelane_b32 v0, s61, 28
-; CHECK-NEXT:    v_writelane_b32 v0, s62, 29
-; CHECK-NEXT:    v_writelane_b32 v0, s63, 30
-; CHECK-NEXT:    v_writelane_b32 v0, s64, 31
-; CHECK-NEXT:    v_writelane_b32 v0, s65, 32
-; CHECK-NEXT:    v_writelane_b32 v0, s66, 33
-; CHECK-NEXT:    v_writelane_b32 v0, s67, 34
-; CHECK-NEXT:    v_writelane_b32 v0, s68, 35
-; CHECK-NEXT:    v_writelane_b32 v0, s69, 36
-; CHECK-NEXT:    v_writelane_b32 v0, s70, 37
-; CHECK-NEXT:    v_writelane_b32 v0, s71, 38
-; CHECK-NEXT:    v_writelane_b32 v0, s72, 39
-; CHECK-NEXT:    v_writelane_b32 v0, s73, 40
-; CHECK-NEXT:    v_writelane_b32 v0, s74, 41
-; CHECK-NEXT:    v_writelane_b32 v0, s75, 42
-; CHECK-NEXT:    v_writelane_b32 v0, s76, 43
-; CHECK-NEXT:    v_writelane_b32 v0, s77, 44
-; CHECK-NEXT:    v_writelane_b32 v0, s78, 45
-; CHECK-NEXT:    v_writelane_b32 v0, s79, 46
-; CHECK-NEXT:    v_writelane_b32 v0, s80, 47
-; CHECK-NEXT:    v_writelane_b32 v0, s81, 48
-; CHECK-NEXT:    v_writelane_b32 v0, s82, 49
-; CHECK-NEXT:    v_writelane_b32 v0, s83, 50
-; CHECK-NEXT:    v_writelane_b32 v0, s84, 51
-; CHECK-NEXT:    v_writelane_b32 v0, s85, 52
-; CHECK-NEXT:    v_writelane_b32 v0, s86, 53
-; CHECK-NEXT:    v_writelane_b32 v0, s87, 54
-; CHECK-NEXT:    v_writelane_b32 v0, s88, 55
-; CHECK-NEXT:    v_writelane_b32 v0, s89, 56
-; CHECK-NEXT:    v_writelane_b32 v0, s90, 57
+; CHECK-NEXT:    v_writelane_b32 v0, s30, 0
+; CHECK-NEXT:    v_writelane_b32 v0, s31, 1
+; CHECK-NEXT:    v_writelane_b32 v0, s33, 2
+; CHECK-NEXT:    v_writelane_b32 v0, s34, 3
+; CHECK-NEXT:    v_writelane_b32 v0, s35, 4
+; CHECK-NEXT:    v_writelane_b32 v0, s36, 5
+; CHECK-NEXT:    v_writelane_b32 v0, s37, 6
+; CHECK-NEXT:    v_writelane_b32 v0, s38, 7
+; CHECK-NEXT:    v_writelane_b32 v0, s39, 8
+; CHECK-NEXT:    v_writelane_b32 v0, s40, 9
+; CHECK-NEXT:    v_writelane_b32 v0, s41, 10
+; CHECK-NEXT:    v_writelane_b32 v0, s42, 11
+; CHECK-NEXT:    v_writelane_b32 v0, s43, 12
+; CHECK-NEXT:    v_writelane_b32 v0, s44, 13
+; CHECK-NEXT:    v_writelane_b32 v0, s45, 14
+; CHECK-NEXT:    v_writelane_b32 v0, s46, 15
+; CHECK-NEXT:    v_writelane_b32 v0, s47, 16
+; CHECK-NEXT:    v_writelane_b32 v0, s48, 17
+; CHECK-NEXT:    v_writelane_b32 v0, s49, 18
+; CHECK-NEXT:    v_writelane_b32 v0, s50, 19
+; CHECK-NEXT:    v_writelane_b32 v0, s51, 20
+; CHECK-NEXT:    v_writelane_b32 v0, s52, 21
+; CHECK-NEXT:    v_writelane_b32 v0, s53, 22
+; CHECK-NEXT:    v_writelane_b32 v0, s54, 23
+; CHECK-NEXT:    v_writelane_b32 v0, s55, 24
+; CHECK-NEXT:    v_writelane_b32 v0, s56, 25
+; CHECK-NEXT:    v_writelane_b32 v0, s57, 26
+; CHECK-NEXT:    v_writelane_b32 v0, s58, 27
+; CHECK-NEXT:    v_writelane_b32 v0, s59, 28
+; CHECK-NEXT:    v_writelane_b32 v0, s60, 29
+; CHECK-NEXT:    v_writelane_b32 v0, s61, 30
+; CHECK-NEXT:    v_writelane_b32 v0, s62, 31
+; CHECK-NEXT:    v_writelane_b32 v0, s63, 32
+; CHECK-NEXT:    v_writelane_b32 v0, s64, 33
+; CHECK-NEXT:    v_writelane_b32 v0, s65, 34
+; CHECK-NEXT:    v_writelane_b32 v0, s66, 35
+; CHECK-NEXT:    v_writelane_b32 v0, s67, 36
+; CHECK-NEXT:    v_writelane_b32 v0, s68, 37
+; CHECK-NEXT:    v_writelane_b32 v0, s69, 38
+; CHECK-NEXT:    v_writelane_b32 v0, s70, 39
+; CHECK-NEXT:    v_writelane_b32 v0, s71, 40
+; CHECK-NEXT:    v_writelane_b32 v0, s72, 41
+; CHECK-NEXT:    v_writelane_b32 v0, s73, 42
+; CHECK-NEXT:    v_writelane_b32 v0, s74, 43
+; CHECK-NEXT:    v_writelane_b32 v0, s75, 44
+; CHECK-NEXT:    v_writelane_b32 v0, s76, 45
+; CHECK-NEXT:    v_writelane_b32 v0, s77, 46
+; CHECK-NEXT:    v_writelane_b32 v0, s78, 47
+; CHECK-NEXT:    v_writelane_b32 v0, s79, 48
+; CHECK-NEXT:    v_writelane_b32 v0, s80, 49
+; CHECK-NEXT:    v_writelane_b32 v0, s81, 50
+; CHECK-NEXT:    v_writelane_b32 v0, s82, 51
+; CHECK-NEXT:    v_writelane_b32 v0, s83, 52
+; CHECK-NEXT:    v_writelane_b32 v0, s84, 53
+; CHECK-NEXT:    v_writelane_b32 v0, s85, 54
+; CHECK-NEXT:    v_writelane_b32 v0, s86, 55
+; CHECK-NEXT:    v_writelane_b32 v0, s87, 56
+; CHECK-NEXT:    v_writelane_b32 v0, s88, 57
 ; CHECK-NEXT:    s_waitcnt expcnt(0)
-; CHECK-NEXT:    v_writelane_b32 v1, s97, 0
-; CHECK-NEXT:    v_writelane_b32 v0, s91, 58
-; CHECK-NEXT:    v_writelane_b32 v1, s98, 1
-; CHECK-NEXT:    v_writelane_b32 v0, s92, 59
-; CHECK-NEXT:    v_writelane_b32 v1, s99, 2
-; CHECK-NEXT:    v_writelane_b32 v0, s93, 60
-; CHECK-NEXT:    v_writelane_b32 v1, s100, 3
-; CHECK-NEXT:    v_writelane_b32 v0, s94, 61
-; CHECK-NEXT:    v_writelane_b32 v1, s101, 4
-; CHECK-NEXT:    v_writelane_b32 v0, s95, 62
-; CHECK-NEXT:    v_writelane_b32 v1, s30, 5
-; CHECK-NEXT:    s_mov_b32 s29, s12
-; CHECK-NEXT:    v_writelane_b32 v0, s96, 63
-; CHECK-NEXT:    v_writelane_b32 v1, s31, 6
-; CHECK-NEXT:    s_cmp_eq_u32 s29, 0
+; CHECK-NEXT:    v_writelane_b32 v1, s95, 0
+; CHECK-NEXT:    v_writelane_b32 v0, s89, 58
+; CHECK-NEXT:    v_writelane_b32 v1, s96, 1
+; CHECK-NEXT:    v_writelane_b32 v0, s90, 59
+; CHECK-NEXT:    v_writelane_b32 v1, s97, 2
+; CHECK-NEXT:    v_writelane_b32 v0, s91, 60
+; CHECK-NEXT:    v_writelane_b32 v1, s98, 3
+; CHECK-NEXT:    v_writelane_b32 v0, s92, 61
+; CHECK-NEXT:    v_writelane_b32 v1, s99, 4
+; CHECK-NEXT:    v_writelane_b32 v0, s93, 62
+; CHECK-NEXT:    v_writelane_b32 v1, s100, 5
+; CHECK-NEXT:    s_mov_b32 s31, s12
+; CHECK-NEXT:    v_writelane_b32 v0, s94, 63
+; CHECK-NEXT:    v_writelane_b32 v1, s101, 6
+; CHECK-NEXT:    s_cmp_eq_u32 s31, 0
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    s_mov_b32 s0, 0
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    s_mov_b32 s1, 0
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    s_mov_b32 s2, 0

 ; CHECK-NEXT:    ; reg use s3
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use s4
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use s5
 ; CHECK-NEXT:    ;;#ASMEND
-; CHECK-NEXT:    v_readlane_b32 s4, v1, 5
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use s6
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use s7
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use s8

 ; CHECK-NEXT:    ; reg use s101
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use vcc_lo
 ; CHECK-NEXT:    ;;#ASMEND
 ; CHECK-NEXT:    ;;#ASMSTART
 ; CHECK-NEXT:    ; reg use vcc_hi
 ; CHECK-NEXT:    ;;#ASMEND
-; CHECK-NEXT:    v_readlane_b32 s5, v1, 6
-; CHECK-NEXT:    v_readlane_b32 s101, v1, 4
-; CHECK-NEXT:    v_readlane_b32 s100, v1, 3
-; CHECK-NEXT:    v_readlane_b32 s99, v1, 2
-; CHECK-NEXT:    v_readlane_b32 s98, v1, 1
-; CHECK-NEXT:    v_readlane_b32 s97, v1, 0
-; CHECK-NEXT:    v_readlane_b32 s96, v0, 63
-; CHECK-NEXT:    v_readlane_b32 s95, v0, 62
-; CHECK-NEXT:    v_readlane_b32 s94, v0, 61
-; CHECK-NEXT:    v_readlane_b32 s93, v0, 60
-; CHECK-NEXT:    v_readlane_b32 s92, v0, 59
-; CHECK-NEXT:    v_readlane_b32 s91, v0, 58
-; CHECK-NEXT:    v_readlane_b32 s90, v0, 57
-; CHECK-NEXT:    v_readlane_b32 s89, v0, 56
-; CHECK-NEXT:    v_readlane_b32 s88, v0, 55
-; CHECK-NEXT:    v_readlane_b32 s87, v0, 54
-; CHECK-NEXT:    v_readlane_b32 s86, v0, 53
-; CHECK-NEXT:    v_readlane_b32 s85, v0, 52
-; CHECK-NEXT:    v_readlane_b32 s84, v0, 51
-; CHECK-NEXT:    v_readlane_b32 s83, v0, 50
-; CHECK-NEXT:    v_readlane_b32 s82, v0, 49
-; CHECK-NEXT:    v_readlane_b32 s81, v0, 48
-; CHECK-NEXT:    v_readlane_b32 s80, v0, 47
-; CHECK-NEXT:    v_readlane_b32 s79, v0, 46
-; CHECK-NEXT:    v_readlane_b32 s78, v0, 45
-; CHECK-NEXT:    v_readlane_b32 s77, v0, 44
-; CHECK-NEXT:    v_readlane_b32 s76, v0, 43
-; CHECK-NEXT:    v_readlane_b32 s75, v0, 42
-; CHECK-NEXT:    v_readlane_b32 s74, v0, 41
-; CHECK-NEXT:    v_readlane_b32 s73, v0, 40
-; CHECK-NEXT:    v_readlane_b32 s72, v0, 39
-; CHECK-NEXT:    v_readlane_b32 s71, v0, 38
-; CHECK-NEXT:    v_readlane_b32 s70, v0, 37
-; CHECK-NEXT:    v_readlane_b32 s69, v0, 36
-; CHECK-NEXT:    v_readlane_b32 s68, v0, 35
-; CHECK-NEXT:    v_readlane_b32 s67, v0, 34
-; CHECK-NEXT:    v_readlane_b32 s66, v0, 33
-; CHECK-NEXT:    v_readlane_b32 s65, v0, 32
-; CHECK-NEXT:    v_readlane_b32 s64, v0, 31
-; CHECK-NEXT:    v_readlane_b32 s63, v0, 30
-; CHECK-NEXT:    v_readlane_b32 s62, v0, 29
-; CHECK-NEXT:    v_readlane_b32 s61, v0, 28
-; CHECK-NEXT:    v_readlane_b32 s60, v0, 27
-; CHECK-NEXT:    v_readlane_b32 s59, v0, 26
-; CHECK-NEXT:    v_readlane_b32 s58, v0, 25
-; CHECK-NEXT:    v_readlane_b32 s57, v0, 24
-; CHECK-NEXT:    v_readlane_b32 s56, v0, 23
-; CHECK-NEXT:    v_readlane_b32 s55, v0, 22
-; CHECK-NEXT:    v_readlane_b32 s54, v0, 21
-; CHECK-NEXT:    v_readlane_b32 s53, v0, 20
-; CHECK-NEXT:    v_readlane_b32 s52, v0, 19
-; CHECK-NEXT:    v_readlane_b32 s51, v0, 18
-; CHECK-NEXT:    v_readlane_b32 s50, v0, 17
-; CHECK-NEXT:    v_readlane_b32 s49, v0, 16
-; CHECK-NEXT:    v_readlane_b32 s48, v0, 15
-; CHECK-NEXT:    v_readlane_b32 s47, v0, 14
-; CHECK-NEXT:    v_readlane_b32 s46, v0, 13
-; CHECK-NEXT:    v_readlane_b32 s45, v0, 12
-; CHECK-NEXT:    v_readlane_b32 s44, v0, 11
-; CHECK-NEXT:    v_readlane_b32 s43, v0, 10
-; CHECK-NEXT:    v_readlane_b32 s42, v0, 9
-; CHECK-NEXT:    v_readlane_b32 s41, v0, 8
-; CHECK-NEXT:    v_readlane_b32 s40, v0, 7
-; CHECK-NEXT:    v_readlane_b32 s39, v0, 6
-; CHECK-NEXT:    v_readlane_b32 s38, v0, 5
-; CHECK-NEXT:    v_readlane_b32 s37, v0, 4
-; CHECK-NEXT:    v_readlane_b32 s36, v0, 3
-; CHECK-NEXT:    v_readlane_b32 s35, v0, 2
-; CHECK-NEXT:    v_readlane_b32 s34, v0, 1
-; CHECK-NEXT:    v_readlane_b32 s33, v0, 0
-; CHECK-NEXT:    s_or_saveexec_b64 s[6:7], -1
+; CHECK-NEXT:    v_readlane_b32 s101, v1, 6
+; CHECK-NEXT:    v_readlane_b32 s100, v1, 5
+; CHECK-NEXT:    v_readlane_b32 s99, v1, 4
+; CHECK-NEXT:    v_readlane_b32 s98, v1, 3
+; CHECK-NEXT:    v_readlane_b32 s97, v1, 2
+; CHECK-NEXT:    v_readlane_b32 s96, v1, 1
+; CHECK-NEXT:    v_readlane_b32 s95, v1, 0
+; CHECK-NEXT:    v_readlane_b32 s94, v0, 63
+; CHECK-NEXT:    v_readlane_b32 s93, v0, 62
+; CHECK-NEXT:    v_readlane_b32 s92, v0, 61
+; CHECK-NEXT:    v_readlane_b32 s91, v0, 60
+; CHECK-NEXT:    v_readlane_b32 s90, v0, 59
+; CHECK-NEXT:    v_readlane_b32 s89, v0, 58
+; CHECK-NEXT:    v_readlane_b32 s88, v0, 57
+; CHECK-NEXT:    v_readlane_b32 s87, v0, 56
+; CHECK-NEXT:    v_readlane_b32 s86, v0, 55
+; CHECK-NEXT:    v_readlane_b32 s85, v0, 54
+; CHECK-NEXT:    v_readlane_b32 s84, v0, 53
+; CHECK-NEXT:    v_readlane_b32 s83, v0, 52
+; CHECK-NEXT:    v_readlane_b32 s82, v0, 51
+; CHECK-NEXT:    v_readlane_b32 s81, v0, 50
+; CHECK-NEXT:    v_readlane_b32 s80, v0, 49
+; CHECK-NEXT:    v_readlane_b32 s79, v0, 48
+; CHECK-NEXT:    v_readlane_b32 s78, v0, 47
+; CHECK-NEXT:    v_readlane_b32 s77, v0, 46
+; CHECK-NEXT:    v_readlane_b32 s76, v0, 45
+; CHECK-NEXT:    v_readlane_b32 s75, v0, 44
+; CHECK-NEXT:    v_readlane_b32 s74, v0, 43
+; CHECK-NEXT:    v_readlane_b32 s73, v0, 42
+; CHECK-NEXT:    v_readlane_b32 s72, v0, 41
+; CHECK-NEXT:    v_readlane_b32 s71, v0, 40
+; CHECK-NEXT:    v_readlane_b32 s70, v0, 39
+; CHECK-NEXT:    v_readlane_b32 s69, v0, 38
+; CHECK-NEXT:    v_readlane_b32 s68, v0, 37
+; CHECK-NEXT:    v_readlane_b32 s67, v0, 36
+; CHECK-NEXT:    v_readlane_b32 s66, v0, 35
+; CHECK-NEXT:    v_readlane_b32 s65, v0, 34
+; CHECK-NEXT:    v_readlane_b32 s64, v0, 33
+; CHECK-NEXT:    v_readlane_b32 s63, v0, 32
+; CHECK-NEXT:    v_readlane_b32 s62, v0, 31
+; CHECK-NEXT:    v_readlane_b32 s61, v0, 30
+; CHECK-NEXT:    v_readlane_b32 s60, v0, 29
+; CHECK-NEXT:    v_readlane_b32 s59, v0, 28
+; CHECK-NEXT:    v_readlane_b32 s58, v0, 27
+; CHECK-NEXT:    v_readlane_b32 s57, v0, 26
+; CHECK-NEXT:    v_readlane_b32 s56, v0, 25
+; CHECK-NEXT:    v_readlane_b32 s55, v0, 24
+; CHECK-NEXT:    v_readlane_b32 s54, v0, 23
+; CHECK-NEXT:    v_readlane_b32 s53, v0, 22
+; CHECK-NEXT:    v_readlane_b32 s52, v0, 21
+; CHECK-NEXT:    v_readlane_b32 s51, v0, 20
+; CHECK-NEXT:    v_readlane_b32 s50, v0, 19
+; CHECK-NEXT:    v_readlane_b32 s49, v0, 18
+; CHECK-NEXT:    v_readlane_b32 s48, v0, 17
+; CHECK-NEXT:    v_readlane_b32 s47, v0, 16
+; CHECK-NEXT:    v_readlane_b32 s46, v0, 15
+; CHECK-NEXT:    v_readlane_b32 s45, v0, 14
+; CHECK-NEXT:    v_readlane_b32 s44, v0, 13
+; CHECK-NEXT:    v_readlane_b32 s43, v0, 12
+; CHECK-NEXT:    v_readlane_b32 s42, v0, 11
+; CHECK-NEXT:    v_readlane_b32 s41, v0, 10
+; CHECK-NEXT:    v_readlane_b32 s40, v0, 9
+; CHECK-NEXT:    v_readlane_b32 s39, v0, 8
+; CHECK-NEXT:    v_readlane_b32 s38, v0, 7
+; CHECK-NEXT:    v_readlane_b32 s37, v0, 6
+; CHECK-NEXT:    v_readlane_b32 s36, v0, 5
+; CHECK-NEXT:    v_readlane_b32 s35, v0, 4
+; CHECK-NEXT:    v_readlane_b32 s34, v0, 3
+; CHECK-NEXT:    v_readlane_b32 s33, v0, 2
+; CHECK-NEXT:    v_readlane_b32 s31, v0, 1
+; CHECK-NEXT:    v_readlane_b32 s30, v0, 0
+; CHECK-NEXT:    s_or_saveexec_b64 s[4:5], -1
 ; CHECK-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
 ; CHECK-NEXT:    buffer_load_dword v1, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; CHECK-NEXT:    s_mov_b64 exec, s[6:7]
+; CHECK-NEXT:    s_mov_b64 exec, s[4:5]
 ; CHECK-NEXT:    s_waitcnt vmcnt(0)
-; CHECK-NEXT:    s_setpc_b64 s[4:5]
+; CHECK-NEXT:    s_setpc_b64 s[30:31]
 entry:
   %cnd = tail call i32 @llvm.amdgcn.workgroup.id.x() #0
   %sgpr0 = tail call i32 asm sideeffect "s_mov_b32 s0, 0", "={s0}"() #0
   %sgpr1 = tail call i32 asm sideeffect "s_mov_b32 s1, 0", "={s1}"() #0
   %sgpr2 = tail call i32 asm sideeffect "s_mov_b32 s2, 0", "={s2}"() #0
   %sgpr3 = tail call i32 asm sideeffect "s_mov_b32 s3, 0", "={s3}"() #0
   %sgpr4 = tail call i32 asm sideeffect "s_mov_b32 s4, 0", "={s4}"() #0
   %sgpr5 = tail call i32 asm sideeffect "s_mov_b32 s5, 0", "={s5}"() #0

llvm/test/CodeGen/AMDGPU/call-preserved-registers.ll

define amdgpu_kernel void @test_kernel_call_external_void_func_void_clobber_s30_s31_call_external_void_func_void() #0 {
   call void @external_void_func_void()
   ret void
 }
 
 ; GCN-LABEL: {{^}}test_func_call_external_void_func_void_clobber_s30_s31_call_external_void_func_void:
 ; MUBUF:   buffer_store_dword
 ; FLATSCR: scratch_store_dword
 ; GCN: v_writelane_b32 v40, s33, 4
-; GCN: v_writelane_b32 v40, s34, 0
-; GCN: v_writelane_b32 v40, s35, 1
-; GCN: v_writelane_b32 v40, s30, 2
-; GCN: v_writelane_b32 v40, s31, 3
+; GCN: v_writelane_b32 v40, s30, 0
+; GCN: v_writelane_b32 v40, s31, 1
+; GCN: v_writelane_b32 v40, s34, 2
+; GCN: v_writelane_b32 v40, s35, 3
 
 ; GCN: s_swappc_b64
 ; GCN-NEXT: ;;#ASMSTART
 ; GCN-NEXT: ;;#ASMEND
 ; GCN-NEXT: s_swappc_b64
-; MUBUF-DAG:   v_readlane_b32 s4, v40, 2
-; MUBUF-DAG:   v_readlane_b32 s5, v40, 3
-; FLATSCR-DAG: v_readlane_b32 s0, v40, 2
-; FLATSCR-DAG: v_readlane_b32 s1, v40, 3
-; GCN: v_readlane_b32 s35, v40, 1
-; GCN: v_readlane_b32 s34, v40, 0
+; GCN: v_readlane_b32 s35, v40, 3
+; GCN: v_readlane_b32 s34, v40, 2
+; MUBUF-DAG:   v_readlane_b32 s31, v40, 1
+; MUBUF-DAG:   v_readlane_b32 s30, v40, 0
+; FLATSCR-DAG: v_readlane_b32 s31, v40, 1
+; FLATSCR-DAG: v_readlane_b32 s30, v40, 0
 
 ; GCN: v_readlane_b32 s33, v40, 4
 ; MUBUF:   buffer_load_dword
 ; FLATSCR: scratch_load_dword
-; GCN: s_setpc_b64
+; GCN: s_setpc_b64 s[30:31]
 define void @test_func_call_external_void_func_void_clobber_s30_s31_call_external_void_func_void() #0 {
   call void @external_void_func_void()
   call void asm sideeffect "", ""() #0
   call void @external_void_func_void()
   ret void
 }
 
 ; GCN-LABEL: {{^}}test_func_call_external_void_funcx2:

 ; MUBUF:   buffer_load_dword v40
 ; FLATSCR: scratch_load_dword v40
 define void @test_func_call_external_void_funcx2() #0 {
   call void @external_void_func_void()
   call void @external_void_func_void()
   ret void
 }
 
 ; GCN-LABEL: {{^}}void_func_void_clobber_s30_s31:
 ; GCN: s_waitcnt
-; GCN-NEXT: s_mov_b64 [[SAVEPC:s\[[0-9]+:[0-9]+\]]], s[30:31]
+; GCN: v_writelane_b32 v0, s30, 0
+; GCN: v_writelane_b32 v0, s31, 1
 ; GCN-NEXT: #ASMSTART
 ; GCN: ; clobber
 ; GCN-NEXT: #ASMEND
-; GCN-NEXT: s_setpc_b64 [[SAVEPC]]
+; GCN: v_readlane_b32 s31, v0, 1
+; GCN: v_readlane_b32 s30, v0, 0
+; GCN: s_setpc_b64 s[30:31]
 define void @void_func_void_clobber_s30_s31() #2 {
   call void asm sideeffect "; clobber", "~{s[30:31]}"() #0
   ret void
 }
 
 ; GCN-LABEL: {{^}}void_func_void_clobber_vcc:
 ; GCN: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GCN-NEXT: ;;#ASMSTART
 ; GCN-NEXT: ;;#ASMEND
 ; GCN-NEXT: s_setpc_b64 s[30:31]
 define hidden void @void_func_void_clobber_vcc() #2 {
   call void asm sideeffect "", "~{vcc}"() #0

llvm/test/CodeGen/AMDGPU/callee-frame-setup.ll

 ; GCN-DAG: v_writelane_b32 [[CSR_VGPR]], s30,
 ; GCN-DAG: v_writelane_b32 [[CSR_VGPR]], s31,
 
 ; MUBUF-DAG:   buffer_store_dword [[ZERO]], off, s[0:3], s33{{$}}
 ; FLATSCR-DAG: scratch_store_dword off, [[ZERO]], s33{{$}}
 
 ; GCN: s_swappc_b64
 
-; MUBUF-DAG: v_readlane_b32 s5, [[CSR_VGPR]]
-; MUBUF-DAG: v_readlane_b32 s4, [[CSR_VGPR]]
-; FLATSCR-DAG: v_readlane_b32 s0, [[CSR_VGPR]]
-; FLATSCR-DAG: v_readlane_b32 s1, [[CSR_VGPR]]
+; GCN-DAG: v_readlane_b32 s30, [[CSR_VGPR]]
+; GCN-DAG: v_readlane_b32 s31, [[CSR_VGPR]]
 
 ; MUBUF:    s_addk_i32 s32, 0xfc00{{$}}
 ; FLATSCR:  s_add_i32 s32, s32, -16{{$}}
 ; GCN-NEXT: v_readlane_b32 s33, [[CSR_VGPR]], 2
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC1:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT:   buffer_load_dword [[CSR_VGPR]], off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; FLATSCR-NEXT: scratch_load_dword [[CSR_VGPR]], off, s32 offset:4 ; 4-byte Folded Reload
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC1]]
 ; GCN-NEXT: s_waitcnt vmcnt(0)
 
-; GCN-NEXT: s_setpc_b64
+; GCN-NEXT: s_setpc_b64 s[30:31]
 define void @callee_with_stack_and_call() #0 {
   %alloca = alloca i32, addrspace(5)
   store volatile i32 0, i32 addrspace(5)* %alloca
   call void @external_void_func_void()
   ret void
 }
 
 ; Should be able to copy incoming stack pointer directly to inner

 ; MUBUF-DAG:   s_addk_i32 s32, 0x400
 ; FLATSCR-DAG: s_add_i32 s32, s32, 16
 ; GCN-DAG: v_writelane_b32 [[CSR_VGPR]], s33, [[FP_SPILL_LANE:[0-9]+]]
 
 ; GCN-DAG: v_writelane_b32 [[CSR_VGPR]], s30, 0
 ; GCN-DAG: v_writelane_b32 [[CSR_VGPR]], s31, 1
 ; GCN: s_swappc_b64
 
-; MUBUF-DAG: v_readlane_b32 s4, v40, 0
-; MUBUF-DAG: v_readlane_b32 s5, v40, 1
-; FLATSCR-DAG: v_readlane_b32 s0, v40, 0
-; FLATSCR-DAG: v_readlane_b32 s1, v40, 1
+; GCN-DAG: v_readlane_b32 s30, [[CSR_VGPR]], 0
+; GCN-DAG: v_readlane_b32 s31, [[CSR_VGPR]], 1
 
 ; MUBUF:   s_addk_i32 s32, 0xfc00
 ; FLATSCR: s_add_i32 s32, s32, -16
 ; GCN-NEXT: v_readlane_b32 s33, [[CSR_VGPR]], [[FP_SPILL_LANE]]
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC1:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT:   buffer_load_dword [[CSR_VGPR]], off, s[0:3], s32 ; 4-byte Folded Reload
 ; FLATSCR-NEXT: scratch_load_dword [[CSR_VGPR]], off, s32 ; 4-byte Folded Reload
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC1]]
 ; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: s_setpc_b64
+; GCN-NEXT: s_setpc_b64 s[30:31]
 define void @callee_no_stack_with_call() #0 {
   call void @external_void_func_void()
   ret void
 }
 
 declare hidden void @external_void_func_void() #0
 
 ; Make sure if a CSR vgpr is used for SGPR spilling, it is saved and

 }
 
 ; GCN-LABEL: {{^}}no_unused_non_csr_sgpr_for_fp:
 ; GCN: s_waitcnt
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC0:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT:   buffer_store_dword [[CSR_VGPR:v[0-9]+]], off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
 ; FLATSCR-NEXT: scratch_store_dword off, [[CSR_VGPR:v[0-9]+]], s32 offset:4 ; 4-byte Folded Spill
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC0]]
-; GCN-NEXT: v_writelane_b32 v0, s33, 2
+; GCN-NEXT: v_writelane_b32 [[CSR_VGPR]], s33, 2
 ; GCN-NEXT: s_mov_b32 s33, s32
-; GCN-NEXT: v_writelane_b32 v0, s30, 0
+; GCN: v_writelane_b32 [[CSR_VGPR]], s30, 0
 ; GCN: v_mov_b32_e32 [[ZERO:v[0-9]+]], 0
-; GCN: v_writelane_b32 v0, s31, 1
+; MUBUF:        s_addk_i32 s32, 0x300
+; FLATSCR:      s_add_i32 s32, s32, 12
+; GCN: v_writelane_b32 [[CSR_VGPR]], s31, 1
 ; MUBUF:   buffer_store_dword [[ZERO]], off, s[0:3], s33{{$}}
 ; FLATSCR: scratch_store_dword off, [[ZERO]], s33{{$}}
 ; GCN-NEXT:     s_waitcnt vmcnt(0)
 ; GCN: ;;#ASMSTART
-; MUBUF:        s_addk_i32 s32, 0x300
-; MUBUF-NEXT:   v_readlane_b32 s4, v0, 0
-; MUBUF-NEXT:   v_readlane_b32 s5, v0, 1
-; FLATSCR:      s_add_i32 s32, s32, 12
-; FLATSCR-NEXT: v_readlane_b32 s0, v0, 0
-; FLATSCR-NEXT: v_readlane_b32 s1, v0, 1
-; MUBUF-NEXT:   s_addk_i32 s32, 0xfd00
-; FLATSCR-NEXT: s_add_i32 s32, s32, -12
-; GCN-NEXT:     v_readlane_b32 s33, v0, 2
+; GCN:     v_readlane_b32 s31, [[CSR_VGPR]], 1
+; GCN:     v_readlane_b32 s30, [[CSR_VGPR]], 0
+; MUBUF:   s_addk_i32 s32, 0xfd00
+; FLATSCR: s_add_i32 s32, s32, -12
+; GCN-NEXT:     v_readlane_b32 s33, [[CSR_VGPR]], 2
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC1:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT:   buffer_load_dword [[CSR_VGPR]], off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; FLATSCR-NEXT: scratch_load_dword [[CSR_VGPR]], off, s32 offset:4 ; 4-byte Folded Reload
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC1]]
 ; GCN-NEXT: s_waitcnt vmcnt(0)
-; MUBUF-NEXT:   s_setpc_b64 s[4:5]
-; FLATSCR-NEXT: s_setpc_b64 s[0:1]
+; GCN-NEXT: s_setpc_b64 s[30:31]
 define void @no_unused_non_csr_sgpr_for_fp() #1 {
   %alloca = alloca i32, addrspace(5)
   store volatile i32 0, i32 addrspace(5)* %alloca
 
   ; Use all clobberable registers, so FP has to spill to a VGPR.
   call void asm sideeffect "",
     "~{s0},~{s1},~{s2},~{s3},~{s4},~{s5},~{s6},~{s7},~{s8},~{s9}
     ,~{s10},~{s11},~{s12},~{s13},~{s14},~{s15},~{s16},~{s17},~{s18},~{s19}
     ,~{s20},~{s21},~{s22},~{s23},~{s24},~{s25},~{s26},~{s27},~{s28},~{s29}
     ,~{s30},~{s31}"() #0
 
   ret void
 }
 
 ; Need a new CSR VGPR to satisfy the FP spill.
 ; GCN-LABEL: {{^}}no_unused_non_csr_sgpr_for_fp_no_scratch_vgpr:
 ; GCN: s_waitcnt
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC0:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT:   buffer_store_dword [[CSR_VGPR:v[0-9]+]], off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
 ; FLATSCR-NEXT: scratch_store_dword off, [[CSR_VGPR:v[0-9]+]], s32 offset:4 ; 4-byte Folded Spill
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC0]]
 ; GCN-NEXT: v_writelane_b32 [[CSR_VGPR]], s33, 2
 ; GCN-NEXT: s_mov_b32 s33, s32
-; GCN-NEXT: v_writelane_b32 [[CSR_VGPR]], s30, 0
+; MUBUF:       s_addk_i32 s32, 0x300{{$}}
+; FLATSCR:     s_add_i32 s32, s32, 12{{$}}
 
-; GCN-DAG: v_writelane_b32 [[CSR_VGPR]], s31, 1
 ; MUBUF-DAG:   buffer_store_dword
 ; FLATSCR-DAG: scratch_store_dword
-; MUBUF:       s_addk_i32 s32, 0x300{{$}}
-; FLATSCR:     s_add_i32 s32, s32, 12{{$}}
 
-; MUBUF:        v_readlane_b32 s4, [[CSR_VGPR]], 0
-; FLATSCR:      v_readlane_b32 s0, [[CSR_VGPR]], 0
 ; GCN: ;;#ASMSTART
-; MUBUF:        v_readlane_b32 s5, [[CSR_VGPR]], 1
-; FLATSCR:      v_readlane_b32 s1, [[CSR_VGPR]], 1
-; MUBUF-NEXT:   s_addk_i32 s32, 0xfd00{{$}}
-; FLATSCR-NEXT: s_add_i32 s32, s32, -12{{$}}
+; MUBUF:   s_addk_i32 s32, 0xfd00{{$}}
+; FLATSCR: s_add_i32 s32, s32, -12{{$}}
 ; GCN-NEXT: v_readlane_b32 s33, [[CSR_VGPR]], 2
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC1:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT:   buffer_load_dword [[CSR_VGPR]], off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; FLATSCR-NEXT: scratch_load_dword [[CSR_VGPR]], off, s32 offset:4 ; 4-byte Folded Reload
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC1]]
 ; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: s_setpc_b64
+; GCN-NEXT: s_setpc_b64 s[30:31]
 define void @no_unused_non_csr_sgpr_for_fp_no_scratch_vgpr() #1 {
   %alloca = alloca i32, addrspace(5)
   store volatile i32 0, i32 addrspace(5)* %alloca
 
   ; Use all clobberable registers, so FP has to spill to a VGPR.
   call void asm sideeffect "",
     "~{s0},~{s1},~{s2},~{s3},~{s4},~{s5},~{s6},~{s7},~{s8},~{s9}
     ,~{s10},~{s11},~{s12},~{s13},~{s14},~{s15},~{s16},~{s17},~{s18},~{s19}

 ; GCN: s_waitcnt
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC0:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT: s_add_i32 [[SCRATCH_SGPR:s[0-9]+]], s32, 0x40100
 ; MUBUF-NEXT: buffer_store_dword [[CSR_VGPR:v[0-9]+]], off, s[0:3], [[SCRATCH_SGPR]] ; 4-byte Folded Spill
 ; FLATSCR-NEXT: s_add_i32 [[SCRATCH_SGPR:s[0-9]+]], s32, 0x1004
 ; FLATSCR-NEXT: scratch_store_dword off, [[CSR_VGPR:v[0-9]+]], [[SCRATCH_SGPR]] ; 4-byte Folded Spill
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC0]]
 ; GCN-NEXT: v_writelane_b32 [[CSR_VGPR]], s33, 2
-; GCN-DAG:  v_writelane_b32 [[CSR_VGPR]], s30, 0
 ; GCN-DAG:  s_mov_b32 s33, s32
-; GCN-DAG:  v_writelane_b32 [[CSR_VGPR]], s31, 1
 ; MUBUF-DAG:   s_add_i32 s32, s32, 0x40300{{$}}
 ; FLATSCR-DAG: s_addk_i32 s32, 0x100c{{$}}
 ; MUBUF-DAG:   buffer_store_dword
 ; FLATSCR-DAG: scratch_store_dword
 
-; MUBUF:   v_readlane_b32 s4, [[CSR_VGPR]], 0
-; FLATSCR: v_readlane_b32 s0, [[CSR_VGPR]], 0
 ; GCN: ;;#ASMSTART
-; MUBUF:   v_readlane_b32 s5, [[CSR_VGPR]], 1
-; FLATSCR: v_readlane_b32 s1, [[CSR_VGPR]], 1
-; MUBUF-NEXT:   s_add_i32 s32, s32, 0xfffbfd00{{$}}
-; FLATSCR-NEXT: s_addk_i32 s32, 0xeff4{{$}}
+; MUBUF:   s_add_i32 s32, s32, 0xfffbfd00{{$}}
+; FLATSCR: s_addk_i32 s32, 0xeff4{{$}}
 ; GCN-NEXT: v_readlane_b32 s33, [[CSR_VGPR]], 2
 ; GCN-NEXT: s_or_saveexec_b64 [[COPY_EXEC1:s\[[0-9]+:[0-9]+\]]], -1{{$}}
 ; MUBUF-NEXT: s_add_i32 [[SCRATCH_SGPR:s[0-9]+]], s32, 0x40100
 ; MUBUF-NEXT: buffer_load_dword [[CSR_VGPR]], off, s[0:3], [[SCRATCH_SGPR]] ; 4-byte Folded Reload
 ; FLATSCR-NEXT: s_add_i32 [[SCRATCH_SGPR:s[0-9]+]], s32, 0x1004
 ; FLATSCR-NEXT: scratch_load_dword [[CSR_VGPR]], off, [[SCRATCH_SGPR]] ; 4-byte Folded Reload
 ; GCN-NEXT: s_mov_b64 exec, [[COPY_EXEC1]]
 ; GCN-NEXT: s_waitcnt vmcnt(0)
-; GCN-NEXT: s_setpc_b64
+; GCN-NEXT: s_setpc_b64 s[30:31]
 define void @scratch_reg_needed_mubuf_offset([4096 x i8] addrspace(5)* byval([4096 x i8]) align 4 %arg) #1 {
   %alloca = alloca i32, addrspace(5)
   store volatile i32 0, i32 addrspace(5)* %alloca
 
   ; Use all clobberable registers, so FP has to spill to a VGPR.
   call void asm sideeffect "; clobber nonpreserved SGPRs",
     "~{s0},~{s1},~{s2},~{s3},~{s4},~{s5},~{s6},~{s7},~{s8},~{s9}
     ,~{s10},~{s11},~{s12},~{s13},~{s14},~{s15},~{s16},~{s17},~{s18},~{s19}

 ; GCN-NEXT: s_setpc_b64
 define internal void @local_empty_func() #0 {
   ret void
 }
 
 ; An FP is needed, despite not needing any spills
 ; TODO: Ccould see callee does not use stack and omit FP.
 ; GCN-LABEL: {{^}}ipra_call_with_stack:
-; GCN: s_mov_b32 [[FP_COPY:s[0-9]+]], s33
+; GCN: v_writelane_b32 v0, s33, 2
 ; GCN: s_mov_b32 s33, s32
 ; MUBUF:   s_addk_i32 s32, 0x400
 ; FLATSCR: s_add_i32 s32, s32, 16
 ; MUBUF:   buffer_store_dword v{{[0-9]+}}, off, s[0:3], s33{{$}}
 ; FLATSCR: scratch_store_dword off, v{{[0-9]+}}, s33{{$}}
 ; GCN:     s_swappc_b64
 ; MUBUF:   s_addk_i32 s32, 0xfc00
 ; FLATSCR: s_add_i32 s32, s32, -16
-; GCN: s_mov_b32 s33, [[FP_COPY:s[0-9]+]]
+; GCN: v_readlane_b32 s33, v0, 2
 define void @ipra_call_with_stack() #0 {
   %alloca = alloca i32, addrspace(5)
   store volatile i32 0, i32 addrspace(5)* %alloca
   call void @local_empty_func()
   ret void
 }
 
 ; With no free registers, we must spill the FP to memory.

llvm/test/CodeGen/AMDGPU/gfx-callable-preserved-registers.ll

 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 3
 ; GFX9-NEXT:    s_getpc_b64 s[4:5]
 ; GFX9-NEXT:    s_add_u32 s4, s4, external_void_func_void@rel32@lo+4
 ; GFX9-NEXT:    s_addc_u32 s5, s5, external_void_func_void@rel32@hi+12
 ; GFX9-NEXT:    s_swappc_b64 s[30:31], s[4:5]
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    s_swappc_b64 s[30:31], s[4:5]
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 2
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 3
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 2
 ; GFX9-NEXT:    v_readlane_b32 s5, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 4
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)

 ; GFX10-NEXT:    s_add_u32 s4, s4, external_void_func_void@rel32@lo+4
 ; GFX10-NEXT:    s_addc_u32 s5, s5, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 2
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 3
 ; GFX10-NEXT:    s_swappc_b64 s[30:31], s[4:5]
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    s_swappc_b64 s[30:31], s[4:5]
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 2
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 3
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 2
 ; GFX10-NEXT:    v_readlane_b32 s5, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 4
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   call amdgpu_gfx void @external_void_func_void()
   call void asm sideeffect "", ""() #0
   call amdgpu_gfx void @external_void_func_void()
   ret void
 }
 
-define amdgpu_gfx void @void_func_void_clobber_s30_s31() #1 {
-; GFX9-LABEL: void_func_void_clobber_s30_s31:
+define amdgpu_gfx void @void_func_void_clobber_s28_s29() #1 {
+; GFX9-LABEL: void_func_void_clobber_s28_s29:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    s_mov_b64 s[36:37], s[30:31]
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_store_dword v0, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    v_writelane_b32 v0, s28, 0
+; GFX9-NEXT:    v_writelane_b32 v0, s29, 1
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; clobber
 ; GFX9-NEXT:    ;;#ASMEND
-; GFX9-NEXT:    s_setpc_b64 s[36:37]
+; GFX9-NEXT:    v_readlane_b32 s29, v0, 1
+; GFX9-NEXT:    v_readlane_b32 s28, v0, 0
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
-; GFX10-LABEL: void_func_void_clobber_s30_s31:
+; GFX10-LABEL: void_func_void_clobber_s28_s29:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX10-NEXT:    s_mov_b64 s[36:37], s[30:31]
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_store_dword v0, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    v_writelane_b32 v0, s28, 0
+; GFX10-NEXT:    v_writelane_b32 v0, s29, 1
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; clobber
 ; GFX10-NEXT:    ;;#ASMEND
-; GFX10-NEXT:    s_setpc_b64 s[36:37]
-  call void asm sideeffect "; clobber", "~{s[30:31]}"() #0
+; GFX10-NEXT:    v_readlane_b32 s29, v0, 1
+; GFX10-NEXT:    v_readlane_b32 s28, v0, 0
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    s_waitcnt vmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+; GCN:          v_writelane_b32 v0, s28, 0
+; GCN:          v_writelane_b32 v0, s29, 1
+
+; GCN:          v_readlane_b32 s28, v0, 0
+; GCN:          v_readlane_b32 s29, v0, 1
+  call void asm sideeffect "; clobber", "~{s[28:29]}"() #0
   ret void
 }
 
 define amdgpu_gfx void @test_call_void_func_void_mayclobber_s31(i32 addrspace(1)* %out) #0 {
 ; GFX9-LABEL: test_call_void_func_void_mayclobber_s31:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    v_writelane_b32 v40, s33, 3
 ; GFX9-NEXT:    v_writelane_b32 v40, s4, 0
-; GFX9-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
+; GFX9-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 2
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def s31
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    s_mov_b32 s4, s31
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    s_mov_b32 s31, s4
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use s31
 ; GFX9-NEXT:    ;;#ASMEND
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]

 ; GFX10-NEXT:    ; def s31
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    s_mov_b32 s4, s31
 ; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    s_mov_b32 s31, s4
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use s31
 ; GFX10-NEXT:    ;;#ASMEND
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)

 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v40, s30, 0
 ; GFX9-NEXT:    buffer_store_dword v41, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 1
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def v31
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    v_mov_b32_e32 v41, v31
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    v_mov_b32_e32 v31, v41
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use v31
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 2
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;

 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
 ; GFX10-NEXT:    buffer_store_dword v41, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; def v31
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 0
 ; GFX10-NEXT:    v_mov_b32_e32 v41, v31
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 1
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    v_mov_b32_e32 v31, v41
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use v31
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 2
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]

 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 2
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def s33
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    s_mov_b32 s4, s33
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    s_mov_b32 s33, s4
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use s33
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: test_call_void_func_void_preserves_s33:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    v_writelane_b32 v40, s33, 3
 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; def s33
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_writelane_b32 v40, s4, 0
 ; GFX10-NEXT:    s_mov_b32 s4, s33
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 2
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    s_mov_b32 s33, s4
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use s33
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)

 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    v_writelane_b32 v40, s33, 3
 ; GFX9-NEXT:    v_writelane_b32 v40, s4, 0
 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v40, s30, 1
-; GFX9-NEXT:    v_writelane_b32 v40, s31, 2
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def s34
 ; GFX9-NEXT:    ;;#ASMEND
+; GFX9-NEXT:    v_writelane_b32 v40, s31, 2
 ; GFX9-NEXT:    s_mov_b32 s4, s34
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    s_mov_b32 s34, s4
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use s34
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: test_call_void_func_void_preserves_s34:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    v_writelane_b32 v40, s33, 3
 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[36:37]
+; GFX10-NEXT:    s_add_u32 s36, s36, external_void_func_void@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s37, s37, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; def s34
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_writelane_b32 v40, s4, 0
 ; GFX10-NEXT:    s_mov_b32 s4, s34
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 2
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[36:37]
 ; GFX10-NEXT:    s_mov_b32 s34, s4
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use s34
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)

 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v41, s30, 0
 ; GFX9-NEXT:    buffer_store_dword v40, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX9-NEXT:    v_writelane_b32 v41, s31, 1
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def v40
 ; GFX9-NEXT:    ;;#ASMEND
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use v40
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX9-NEXT:    v_readlane_b32 s30, v41, 0
 ; GFX9-NEXT:    v_readlane_b32 s31, v41, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v41, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v41, 2
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v41, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: test_call_void_func_void_preserves_v40:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_store_dword v41, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    v_writelane_b32 v41, s33, 2
 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
 ; GFX10-NEXT:    buffer_store_dword v40, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; def v40
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_writelane_b32 v41, s30, 0
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    v_writelane_b32 v41, s31, 1
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use v40
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX10-NEXT:    v_readlane_b32 s30, v41, 0
 ; GFX10-NEXT:    v_readlane_b32 s31, v41, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v41, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v41, 2
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v41, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]

 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    v_writelane_b32 v40, s33, 2
 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v40, s30, 0
 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 1
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, void_func_void_clobber_s33@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, void_func_void_clobber_s33@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 0
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, void_func_void_clobber_s33@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, void_func_void_clobber_s33@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 2
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: test_call_void_func_void_clobber_s33:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    v_writelane_b32 v40, s33, 2
 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, void_func_void_clobber_s33@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, void_func_void_clobber_s33@rel32@hi+12
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 0
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 1
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, void_func_void_clobber_s33@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, void_func_void_clobber_s33@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 0
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 2
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   call amdgpu_gfx void @void_func_void_clobber_s33()
   ret void
 }
 
 define amdgpu_gfx void @test_call_void_func_void_clobber_s34() #0 {
 ; GFX9-LABEL: test_call_void_func_void_clobber_s34:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    v_writelane_b32 v40, s33, 2
 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v40, s30, 0
 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 1
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, void_func_void_clobber_s34@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, void_func_void_clobber_s34@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 0
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, void_func_void_clobber_s34@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, void_func_void_clobber_s34@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 2
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: test_call_void_func_void_clobber_s34:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    v_writelane_b32 v40, s33, 2
 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, void_func_void_clobber_s34@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, void_func_void_clobber_s34@rel32@hi+12
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 0
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 1
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, void_func_void_clobber_s34@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, void_func_void_clobber_s34@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 0
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 2
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]

 ; GFX9-NEXT:    s_mov_b32 s33, s32
 ; GFX9-NEXT:    s_addk_i32 s32, 0x400
 ; GFX9-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX9-NEXT:    v_writelane_b32 v40, s31, 2
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def s40
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    s_mov_b32 s4, s40
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use s4
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: callee_saved_sgpr_kernel:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_store_dword v40, off, s[0:3], s32 ; 4-byte Folded Spill
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    v_writelane_b32 v40, s33, 3
 ; GFX10-NEXT:    s_mov_b32 s33, s32
 ; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; def s40
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_writelane_b32 v40, s4, 0
 ; GFX10-NEXT:    s_mov_b32 s4, s40
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 2
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use s4
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)

 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def s40
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    s_mov_b32 s4, s40
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; def v32
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    v_mov_b32_e32 v41, v32
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use s4
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    ;;#ASMSTART
 ; GFX9-NEXT:    ; use v41
 ; GFX9-NEXT:    ;;#ASMEND
 ; GFX9-NEXT:    buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX9-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX9-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-NEXT:    buffer_load_dword v40, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]

 ; GFX10-NEXT:    ; def s40
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_writelane_b32 v40, s4, 0
 ; GFX10-NEXT:    s_mov_b32 s4, s40
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; def v32
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    v_mov_b32_e32 v41, v32
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, external_void_func_void@rel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, external_void_func_void@rel32@hi+12
 ; GFX10-NEXT:    v_writelane_b32 v40, s30, 1
 ; GFX10-NEXT:    v_writelane_b32 v40, s31, 2
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, external_void_func_void@rel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, external_void_func_void@rel32@hi+12
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use s4
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    ;;#ASMSTART
 ; GFX10-NEXT:    ; use v41
 ; GFX10-NEXT:    ;;#ASMEND
 ; GFX10-NEXT:    buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s31, v40, 2
+; GFX10-NEXT:    v_readlane_b32 s30, v40, 1
 ; GFX10-NEXT:    v_readlane_b32 s4, v40, 0
 ; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
 ; GFX10-NEXT:    v_readlane_b32 s33, v40, 3
 ; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
 ; GFX10-NEXT:    buffer_load_dword v40, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
 ; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
 ; GFX10-NEXT:    s_mov_b32 exec_lo, s34
 ; GFX10-NEXT:    s_waitcnt vmcnt(0)

llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll

 entry:
   ret i1 1
 }
 
 define amdgpu_gfx void @call_i1() #0 {
 ; GFX9-LABEL: call_i1:
 ; GFX9:       ; %bb.0: ; %entry
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, return_i1@gotpcrel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, return_i1@gotpcrel32@hi+12
-; GFX9-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_store_dword v1, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    v_writelane_b32 v1, s33, 2
+; GFX9-NEXT:    s_mov_b32 s33, s32
+; GFX9-NEXT:    s_addk_i32 s32, 0x400
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, return_i1@gotpcrel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, return_i1@gotpcrel32@hi+12
+; GFX9-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX9-NEXT:    v_writelane_b32 v1, s30, 0
+; GFX9-NEXT:    v_writelane_b32 v1, s31, 1
 ; GFX9-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    s_setpc_b64 s[36:37]
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX9-NEXT:    v_readlane_b32 s31, v1, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v1, 0
+; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
+; GFX9-NEXT:    v_readlane_b32 s33, v1, 2
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_load_dword v1, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: call_i1:
 ; GFX10:       ; %bb.0: ; %entry
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX10-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, return_i1@gotpcrel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, return_i1@gotpcrel32@hi+12
-; GFX10-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_store_dword v1, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    v_writelane_b32 v1, s33, 2
+; GFX10-NEXT:    s_mov_b32 s33, s32
+; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, return_i1@gotpcrel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, return_i1@gotpcrel32@hi+12
+; GFX10-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX10-NEXT:    v_writelane_b32 v1, s30, 0
+; GFX10-NEXT:    v_writelane_b32 v1, s31, 1
 ; GFX10-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    s_setpc_b64 s[36:37]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX10-NEXT:    v_readlane_b32 s31, v1, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v1, 0
+; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
+; GFX10-NEXT:    v_readlane_b32 s33, v1, 2
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_load_dword v1, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    s_waitcnt vmcnt(0)
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
 entry:
   call amdgpu_gfx i1 @return_i1()
   ret void
 }
 
 define amdgpu_gfx i16 @return_i16() #0 {
 ; GFX9-LABEL: return_i16:
 ; GFX9:       ; %bb.0: ; %entry

 entry:
   ret i16 10
 }
 
 define amdgpu_gfx void @call_i16() #0 {
 ; GFX9-LABEL: call_i16:
 ; GFX9:       ; %bb.0: ; %entry
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, return_i16@gotpcrel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, return_i16@gotpcrel32@hi+12
-; GFX9-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_store_dword v1, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    v_writelane_b32 v1, s33, 2
+; GFX9-NEXT:    s_mov_b32 s33, s32
+; GFX9-NEXT:    s_addk_i32 s32, 0x400
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, return_i16@gotpcrel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, return_i16@gotpcrel32@hi+12
+; GFX9-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX9-NEXT:    v_writelane_b32 v1, s30, 0
+; GFX9-NEXT:    v_writelane_b32 v1, s31, 1
 ; GFX9-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    s_setpc_b64 s[36:37]
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX9-NEXT:    v_readlane_b32 s31, v1, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v1, 0
+; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
+; GFX9-NEXT:    v_readlane_b32 s33, v1, 2
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_load_dword v1, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: call_i16:
 ; GFX10:       ; %bb.0: ; %entry
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX10-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, return_i16@gotpcrel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, return_i16@gotpcrel32@hi+12
-; GFX10-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_store_dword v1, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    v_writelane_b32 v1, s33, 2
+; GFX10-NEXT:    s_mov_b32 s33, s32
+; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, return_i16@gotpcrel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, return_i16@gotpcrel32@hi+12
+; GFX10-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX10-NEXT:    v_writelane_b32 v1, s30, 0
+; GFX10-NEXT:    v_writelane_b32 v1, s31, 1
 ; GFX10-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    s_setpc_b64 s[36:37]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX10-NEXT:    v_readlane_b32 s31, v1, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v1, 0
+; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
+; GFX10-NEXT:    v_readlane_b32 s33, v1, 2
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_load_dword v1, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    s_waitcnt vmcnt(0)
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
 entry:
   call amdgpu_gfx i16 @return_i16()
   ret void
 }
 
 define amdgpu_gfx <2 x i16> @return_2xi16() #0 {
 ; GFX9-LABEL: return_2xi16:
 ; GFX9:       ; %bb.0: ; %entry

 entry:
   ret <2 x i16> <i16 1, i16 2>
 }
 
 define amdgpu_gfx void @call_2xi16() #0 {
 ; GFX9-LABEL: call_2xi16:
 ; GFX9:       ; %bb.0: ; %entry
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, return_2xi16@gotpcrel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, return_2xi16@gotpcrel32@hi+12
-; GFX9-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_store_dword v1, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    v_writelane_b32 v1, s33, 2
+; GFX9-NEXT:    s_mov_b32 s33, s32
+; GFX9-NEXT:    s_addk_i32 s32, 0x400
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, return_2xi16@gotpcrel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, return_2xi16@gotpcrel32@hi+12
+; GFX9-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX9-NEXT:    v_writelane_b32 v1, s30, 0
+; GFX9-NEXT:    v_writelane_b32 v1, s31, 1
 ; GFX9-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    s_setpc_b64 s[36:37]
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX9-NEXT:    v_readlane_b32 s31, v1, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v1, 0
+; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
+; GFX9-NEXT:    v_readlane_b32 s33, v1, 2
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_load_dword v1, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: call_2xi16:
 ; GFX10:       ; %bb.0: ; %entry
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX10-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, return_2xi16@gotpcrel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, return_2xi16@gotpcrel32@hi+12
-; GFX10-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_store_dword v1, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    v_writelane_b32 v1, s33, 2
+; GFX10-NEXT:    s_mov_b32 s33, s32
+; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, return_2xi16@gotpcrel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, return_2xi16@gotpcrel32@hi+12
+; GFX10-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX10-NEXT:    v_writelane_b32 v1, s30, 0
+; GFX10-NEXT:    v_writelane_b32 v1, s31, 1
 ; GFX10-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    s_setpc_b64 s[36:37]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX10-NEXT:    v_readlane_b32 s31, v1, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v1, 0
+; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
+; GFX10-NEXT:    v_readlane_b32 s33, v1, 2
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_load_dword v1, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    s_waitcnt vmcnt(0)
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
 entry:
   call amdgpu_gfx <2 x i16> @return_2xi16()
   ret void
 }
 
 define amdgpu_gfx <3 x i16> @return_3xi16() #0 {
 ; GFX9-LABEL: return_3xi16:
 ; GFX9:       ; %bb.0: ; %entry

 entry:
   ret <3 x i16> <i16 1, i16 2, i16 3>
 }
 
 define amdgpu_gfx void @call_3xi16() #0 {
 ; GFX9-LABEL: call_3xi16:
 ; GFX9:       ; %bb.0: ; %entry
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, return_3xi16@gotpcrel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, return_3xi16@gotpcrel32@hi+12
-; GFX9-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_store_dword v2, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    v_writelane_b32 v2, s33, 2
+; GFX9-NEXT:    s_mov_b32 s33, s32
+; GFX9-NEXT:    s_addk_i32 s32, 0x400
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, return_3xi16@gotpcrel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, return_3xi16@gotpcrel32@hi+12
+; GFX9-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX9-NEXT:    v_writelane_b32 v2, s30, 0
+; GFX9-NEXT:    v_writelane_b32 v2, s31, 1
 ; GFX9-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX9-NEXT:    s_setpc_b64 s[36:37]
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX9-NEXT:    v_readlane_b32 s31, v2, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v2, 0
+; GFX9-NEXT:    s_addk_i32 s32, 0xfc00
+; GFX9-NEXT:    v_readlane_b32 s33, v2, 2
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_load_dword v2, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: call_3xi16:
 ; GFX10:       ; %bb.0: ; %entry
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX10-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, return_3xi16@gotpcrel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, return_3xi16@gotpcrel32@hi+12
-; GFX10-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_store_dword v2, off, s[0:3], s32 ; 4-byte Folded Spill
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    v_writelane_b32 v2, s33, 2
+; GFX10-NEXT:    s_mov_b32 s33, s32
+; GFX10-NEXT:    s_addk_i32 s32, 0x200
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, return_3xi16@gotpcrel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, return_3xi16@gotpcrel32@hi+12
+; GFX10-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX10-NEXT:    v_writelane_b32 v2, s30, 0
+; GFX10-NEXT:    v_writelane_b32 v2, s31, 1
 ; GFX10-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
-; GFX10-NEXT:    s_setpc_b64 s[36:37]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX10-NEXT:    v_readlane_b32 s31, v2, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v2, 0
+; GFX10-NEXT:    s_addk_i32 s32, 0xfe00
+; GFX10-NEXT:    v_readlane_b32 s33, v2, 2
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_load_dword v2, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    s_waitcnt vmcnt(0)
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
 entry:
   call amdgpu_gfx <3 x i16> @return_3xi16()
   ret void
 }
 
 ; Check that return values that do not fit in registers do not crash
 
 define amdgpu_gfx <512 x i32> @return_512xi32() #0 {

 entry:
   ret <512 x i32> zeroinitializer
 }
 
 define amdgpu_gfx void @call_512xi32() #0 {
 ; GFX9-LABEL: call_512xi32:
 ; GFX9:       ; %bb.0: ; %entry
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    s_mov_b32 s34, s33
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_store_dword v2, off, s[0:3], s32 offset:2048 ; 4-byte Folded Spill
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    v_writelane_b32 v2, s33, 2
 ; GFX9-NEXT:    s_add_i32 s33, s32, 0x1ffc0
 ; GFX9-NEXT:    s_and_b32 s33, s33, 0xfffe0000
 ; GFX9-NEXT:    s_add_i32 s32, s32, 0x60000
-; GFX9-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX9-NEXT:    s_getpc_b64 s[30:31]
-; GFX9-NEXT:    s_add_u32 s30, s30, return_512xi32@gotpcrel32@lo+4
-; GFX9-NEXT:    s_addc_u32 s31, s31, return_512xi32@gotpcrel32@hi+12
-; GFX9-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX9-NEXT:    s_getpc_b64 s[34:35]
+; GFX9-NEXT:    s_add_u32 s34, s34, return_512xi32@gotpcrel32@lo+4
+; GFX9-NEXT:    s_addc_u32 s35, s35, return_512xi32@gotpcrel32@hi+12
+; GFX9-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
+; GFX9-NEXT:    v_writelane_b32 v2, s30, 0
 ; GFX9-NEXT:    v_lshrrev_b32_e64 v0, 6, s33
+; GFX9-NEXT:    v_writelane_b32 v2, s31, 1
 ; GFX9-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX9-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX9-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX9-NEXT:    v_readlane_b32 s31, v2, 1
+; GFX9-NEXT:    v_readlane_b32 s30, v2, 0
 ; GFX9-NEXT:    s_add_i32 s32, s32, 0xfffa0000
-; GFX9-NEXT:    s_mov_b32 s33, s34
-; GFX9-NEXT:    s_setpc_b64 s[36:37]
+; GFX9-NEXT:    v_readlane_b32 s33, v2, 2
+; GFX9-NEXT:    s_or_saveexec_b64 s[34:35], -1
+; GFX9-NEXT:    buffer_load_dword v2, off, s[0:3], s32 offset:2048 ; 4-byte Folded Reload
+; GFX9-NEXT:    s_mov_b64 exec, s[34:35]
+; GFX9-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: call_512xi32:
 ; GFX10:       ; %bb.0: ; %entry
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX10-NEXT:    s_mov_b32 s34, s33
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_store_dword v2, off, s[0:3], s32 offset:2048 ; 4-byte Folded Spill
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    v_writelane_b32 v2, s33, 2
 ; GFX10-NEXT:    s_add_i32 s33, s32, 0xffe0
 ; GFX10-NEXT:    s_add_i32 s32, s32, 0x30000
 ; GFX10-NEXT:    s_and_b32 s33, s33, 0xffff0000
-; GFX10-NEXT:    s_mov_b64 s[36:37], s[30:31]
-; GFX10-NEXT:    s_getpc_b64 s[30:31]
-; GFX10-NEXT:    s_add_u32 s30, s30, return_512xi32@gotpcrel32@lo+4
-; GFX10-NEXT:    s_addc_u32 s31, s31, return_512xi32@gotpcrel32@hi+12
+; GFX10-NEXT:    s_getpc_b64 s[34:35]
+; GFX10-NEXT:    s_add_u32 s34, s34, return_512xi32@gotpcrel32@lo+4
+; GFX10-NEXT:    s_addc_u32 s35, s35, return_512xi32@gotpcrel32@hi+12
+; GFX10-NEXT:    v_writelane_b32 v2, s30, 0
+; GFX10-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
 ; GFX10-NEXT:    v_lshrrev_b32_e64 v0, 5, s33
-; GFX10-NEXT:    s_load_dwordx2 s[30:31], s[30:31], 0x0
+; GFX10-NEXT:    v_writelane_b32 v2, s31, 1
 ; GFX10-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX10-NEXT:    s_swappc_b64 s[30:31], s[30:31]
+; GFX10-NEXT:    s_swappc_b64 s[30:31], s[34:35]
+; GFX10-NEXT:    v_readlane_b32 s31, v2, 1
+; GFX10-NEXT:    v_readlane_b32 s30, v2, 0
 ; GFX10-NEXT:    s_add_i32 s32, s32, 0xfffd0000
-; GFX10-NEXT:    s_mov_b32 s33, s34
-; GFX10-NEXT:    s_setpc_b64 s[36:37]
+; GFX10-NEXT:    v_readlane_b32 s33, v2, 2
+; GFX10-NEXT:    s_or_saveexec_b32 s34, -1
+; GFX10-NEXT:    buffer_load_dword v2, off, s[0:3], s32 offset:2048 ; 4-byte Folded Reload
+; GFX10-NEXT:    s_waitcnt_depctr 0xffe3
+; GFX10-NEXT:    s_mov_b32 exec_lo, s34
+; GFX10-NEXT:    s_waitcnt vmcnt(0)
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
 entry:
   call amdgpu_gfx <512 x i32> @return_512xi32()
   ret void
 }
 
 attributes #0 = { nounwind }