AMDGPU: Handle llvm.stacksave and llvm.stackrestore

[arsenm]

Summary

Not sure if the only valid use is to have stackrestore directly
consume stacksave outputs or not. Handled exactly like a regular stack
pointer so all the edge cases theoretically should work.

[arsenm]

Add documentation (really need to replace this insufferable table with the less whitespace dependent syntax form)

[Pierre-vh]

llvm/docs/ReleaseNotes.rst

77

Should this mention it's only for p5?

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

2582

So this does:

swizzled SP = unswizzled SP / wavefront size

But the docs say it does the opposite? (converts a swizzled address to unswizzled?)

My understanding of swizzling is pretty shallow, a bit more docs to help clarify this would be nice

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

3537

How does changing the SP address to the swizzled SP help with edge cases?
Also why do we need a new opcode for this?

[arsenm]

llvm/docs/ReleaseNotes.rst

77

Not really, it's all that makes sense. the AMDGPUUsage note is enough

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

2582

I think the terminology is "swizzled" = "vgpr offset" and "unswizzled" = "sgpr offset" to a MUBUF instruction

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

3537

An IR pointer is a vector value. A raw read of SP is the scalar value. The stack increments are really in wavesize chunks. If you do any addressing, or access of the pointer using the MUBUF voffset field you'll end up in the wrong place. The real usage is just write the direct value of a stacksave to a stackrestore call.

We need a new opcode to preserve the semantic intent. Otherwise the fold (whether into an addressing mode, or into the stackrestore call) will need to guess at the interpretation for a copy from a physical register. The call lowering has the same problem for matching the MUBUF addressing modes (GlobalISel uses the pseudo, the DAG gets away with inspecting the PseudoSourceValue for the underlying pointer)

[arsenm]

Fix comment

[arsenm]

9a53f5f5c466b8d2a63827ec33dd97e4333db6ce

llvm/docs/AMDGPUUsage.rst

 
 The AMDGPU backend implements the following LLVM IR intrinsics.
 
 *This section is WIP.*
 
 .. table:: AMDGPU LLVM IR Intrinsics
   :name: amdgpu-llvm-ir-intrinsics-table
 
-  =========================================  ==========================================================
-  LLVM Intrinsic                             Description
-  =========================================  ==========================================================
-  llvm.amdgcn.sqrt                           Provides direct access to v_sqrt_f64, v_sqrt_f32 and v_sqrt_f16
+  ==============================================   ==========================================================
+  LLVM Intrinsic                                   Description
+  ==============================================   ==========================================================
+  llvm.amdgcn.sqrt                                 Provides direct access to v_sqrt_f64, v_sqrt_f32 and v_sqrt_f16
                                                    (on targets with half support). Peforms sqrt function.
 
-  llvm.amdgcn.log                            Provides direct access to v_log_f32 and v_log_f16
+  llvm.amdgcn.log                                  Provides direct access to v_log_f32 and v_log_f16
                                                    (on targets with half support). Peforms log2 function.
 
-  llvm.amdgcn.exp2                           Provides direct access to v_exp_f32 and v_exp_f16
+  llvm.amdgcn.exp2                                 Provides direct access to v_exp_f32 and v_exp_f16
                                                    (on targets with half support). Performs exp2 function.
 
-  :ref:`llvm.frexp <int_frexp>`              Implemented for half, float and double.
+  :ref:`llvm.frexp <int_frexp>`                    Implemented for half, float and double.
 
-  :ref:`llvm.log2 <int_log2>`                Implemented for float and half (and vectors of float or
+  :ref:`llvm.log2 <int_log2>`                      Implemented for float and half (and vectors of float or
                                                    half). Not implemented for double. Hardware provides
                                                    1ULP accuracy for float, and 0.51ULP for half. Float
                                                    instruction does not natively support denormal
                                                    inputs. Backend will optimize out denormal scaling if
                                                    marked with the :ref:`afn <fastmath_afn>` flag.
 
-  :ref:`llvm.sqrt <int_sqrt>`                Implemented for double, float and half (and vectors).
+  :ref:`llvm.sqrt <int_sqrt>`                      Implemented for double, float and half (and vectors).
 
-  :ref:`llvm.log <int_log>`                  Implemented for float and half (and vectors).
+  :ref:`llvm.log <int_log>`                        Implemented for float and half (and vectors).
 
-  :ref:`llvm.exp <int_exp>`                  Implemented for float and half (and vectors).
+  :ref:`llvm.exp <int_exp>`                        Implemented for float and half (and vectors).
 
-  :ref:`llvm.log10 <int_log10>`              Implemented for float and half (and vectors).
+  :ref:`llvm.log10 <int_log10>`                    Implemented for float and half (and vectors).
 
-  :ref:`llvm.exp2 <int_exp2>`                Implemented for float and half (and vectors of float or
+  :ref:`llvm.exp2 <int_exp2>`                      Implemented for float and half (and vectors of float or
                                                    half). Not implemented for double. Hardware provides
                                                    1ULP accuracy for float, and 0.51ULP for half. Float
                                                    instruction does not natively support denormal
                                                    inputs. Backend will optimize out denormal scaling if
                                                    marked with the :ref:`afn <fastmath_afn>` flag.
 
-  llvm.amdgcn.wave.reduce.umin               Performs an arithmetic unsigned min reduction on the unsigned values
+  :ref:`llvm.stacksave.p5 <int_stacksave>`         Implemented, must use the alloca address space.
+  :ref:`llvm.stackrestore.p5 <int_stackrestore>`   Implemented, must use the alloca address space.
+
+  llvm.amdgcn.wave.reduce.umin                     Performs an arithmetic unsigned min reduction on the unsigned values
                                                    provided by each lane in the wavefront.
                                                    Intrinsic takes a hint for reduction strategy using second operand
                                                    0: Target default preference,
                                                    1: `Iterative strategy`, and
                                                    2: `DPP`.
                                                    If target does not support the DPP operations (e.g. gfx6/7),
                                                    reduction will be performed using default iterative strategy.
                                                    Intrinsic is currently only implemented for i32.
 
-  llvm.amdgcn.wave.reduce.umax               Performs an arithmetic unsigned max reduction on the unsigned values
+  llvm.amdgcn.wave.reduce.umax                     Performs an arithmetic unsigned max reduction on the unsigned values
                                                    provided by each lane in the wavefront.
                                                    Intrinsic takes a hint for reduction strategy using second operand
                                                    0: Target default preference,
                                                    1: `Iterative strategy`, and
                                                    2: `DPP`.
                                                    If target does not support the DPP operations (e.g. gfx6/7),
                                                    reduction will be performed using default iterative strategy.
                                                    Intrinsic is currently only implemented for i32.
 
-  =========================================  ==========================================================
+  ==============================================   ==========================================================
 
 .. TODO::
 
    List AMDGPU intrinsics.
 
 LLVM IR Attributes
 ------------------
 

llvm/docs/ReleaseNotes.rst

 ------------------------------
 
 Changes to the AMDGPU Backend
 -----------------------------
 
 * `llvm.sqrt.f64` is now lowered correctly. Use `llvm.amdgcn.sqrt.f64`
   for raw instruction access.
 
+* Implemented `llvm.stacksave` and `llvm.stackrestore` intrinsics.
+
 Changes to the ARM Backend
 --------------------------
 
 Changes to the AVR Backend
 --------------------------
 
 Changes to the DirectX Backend
 ------------------------------

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.h

private:
   void SelectFMAD_FMA(SDNode *N);
   void SelectDSAppendConsume(SDNode *N, unsigned IntrID);
   void SelectDSBvhStackIntrinsic(SDNode *N);
   void SelectDS_GWS(SDNode *N, unsigned IntrID);
   void SelectInterpP1F16(SDNode *N);
   void SelectINTRINSIC_W_CHAIN(SDNode *N);
   void SelectINTRINSIC_WO_CHAIN(SDNode *N);
   void SelectINTRINSIC_VOID(SDNode *N);
+  void SelectWAVE_ADDRESS(SDNode *N);
+  void SelectSTACKRESTORE(SDNode *N);
 
 protected:
   // Include the pieces autogenerated from the target description.
 #include "AMDGPUGenDAGISel.inc"
 };
 
 #endif // LLVM_LIB_TARGET_AMDGPU_AMDGPUISELDAGTODAG_H

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

void AMDGPUDAGToDAGISel::Select(SDNode *N) {
   case ISD::INTRINSIC_WO_CHAIN: {
     SelectINTRINSIC_WO_CHAIN(N);
     return;
   }
   case ISD::INTRINSIC_VOID: {
     SelectINTRINSIC_VOID(N);
     return;
   }
+  case AMDGPUISD::WAVE_ADDRESS: {
+    SelectWAVE_ADDRESS(N);
+    return;
+  }
+  case ISD::STACKRESTORE: {
+    SelectSTACKRESTORE(N);
+    return;
+  }
   }
 
   SelectCode(N);
 }
 
 bool AMDGPUDAGToDAGISel::isUniformBr(const SDNode *N) const {
   const BasicBlock *BB = FuncInfo->MBB->getBasicBlock();
   const Instruction *Term = BB->getTerminator();
case Intrinsic::amdgcn_ds_gws_sema_release_all:
     return;
   default:
     break;
   }
 
   SelectCode(N);
 }
 
+void AMDGPUDAGToDAGISel::SelectWAVE_ADDRESS(SDNode *N) {
+  SDValue Log2WaveSize =
+    CurDAG->getTargetConstant(Subtarget->getWavefrontSizeLog2(), SDLoc(N), MVT::i32);
+  CurDAG->SelectNodeTo(N, AMDGPU::S_LSHR_B32, N->getVTList(),
+                       {N->getOperand(0), Log2WaveSize});
+}
+
+void AMDGPUDAGToDAGISel::SelectSTACKRESTORE(SDNode *N) {
+  SDValue SrcVal = N->getOperand(1);
+  if (SrcVal.getValueType() != MVT::i32) {
+    SelectCode(N); // Emit default error
+    return;
+  }
+
+  SDValue CopyVal;
+  Register SP = TLI->getStackPointerRegisterToSaveRestore();
+  SDLoc SL(N);
+
+  if (SrcVal.getOpcode() == AMDGPUISD::WAVE_ADDRESS) {
+    CopyVal = SrcVal.getOperand(0);
+  } else {
+    SDValue Log2WaveSize = CurDAG->getTargetConstant(
+        Subtarget->getWavefrontSizeLog2(), SL, MVT::i32);
+
+    if (N->isDivergent()) {
+      SrcVal = SDValue(CurDAG->getMachineNode(AMDGPU::V_READFIRSTLANE_B32, SL,
+                                              MVT::i32, SrcVal),
+                       0);
+    }
+
+    CopyVal = SDValue(CurDAG->getMachineNode(AMDGPU::S_LSHL_B32, SL, MVT::i32,
+                                             {SrcVal, Log2WaveSize}),
+                      0);
+  }
+
+  SDValue CopyToSP = CurDAG->getCopyToReg(N->getOperand(0), SL, SP, CopyVal);
+  CurDAG->ReplaceAllUsesOfValueWith(SDValue(N, 0), CopyToSP);
+}
+
 bool AMDGPUDAGToDAGISel::SelectVOP3ModsImpl(SDValue In, SDValue &Src,
                                             unsigned &Mods,
                                             bool IsCanonicalizing,
                                             bool AllowAbs) const {
   Mods = SISrcMods::NONE;
   Src = In;
 
   if (Src.getOpcode() == ISD::FNEG) {

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.h

SDValue addTokenForArgument(SDValue Chain,
                               int ClobberedFI) const;
 
   SDValue lowerUnhandledCall(CallLoweringInfo &CLI,
                              SmallVectorImpl<SDValue> &InVals,
                              StringRef Reason) const;
   SDValue LowerCall(CallLoweringInfo &CLI,
                     SmallVectorImpl<SDValue> &InVals) const override;
 
-  SDValue LowerDYNAMIC_STACKALLOC(SDValue Op,
-                                  SelectionDAG &DAG) const;
-
+  SDValue LowerDYNAMIC_STACKALLOC(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
   SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
   void ReplaceNodeResults(SDNode * N,
                           SmallVectorImpl<SDValue> &Results,
                           SelectionDAG &DAG) const override;
 
   SDValue combineFMinMaxLegacyImpl(const SDLoc &DL, EVT VT, SDValue LHS,
                                    SDValue RHS, SDValue True, SDValue False,
enum NodeType : unsigned {
   ENDPGM_TRAP,
 
   // Return to a shader part's epilog code.
   RETURN_TO_EPILOG,
 
   // Return with values from a non-entry function.
   RET_GLUE,
 
+  // Convert a unswizzled wave uniform stack address to an address compatible
+  // with a vector offset for use in stack access.
+  WAVE_ADDRESS,
+
   DWORDADDR,
   FRACT,
 
   /// CLAMP value between 0.0 and 1.0. NaN clamped to 0, following clamp output
   /// modifier behavior with dx10_enable.
   CLAMP,
 
   // This is SETCC with the full mask result which is used for a compare with a

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp

const char* AMDGPUTargetLowering::getTargetNodeName(unsigned Opcode) const {
   NODE_NAME_CASE(IF)
   NODE_NAME_CASE(ELSE)
   NODE_NAME_CASE(LOOP)
   NODE_NAME_CASE(CALL)
   NODE_NAME_CASE(TC_RETURN)
   NODE_NAME_CASE(TC_RETURN_GFX)
   NODE_NAME_CASE(TRAP)
   NODE_NAME_CASE(RET_GLUE)
+  NODE_NAME_CASE(WAVE_ADDRESS)
   NODE_NAME_CASE(RETURN_TO_EPILOG)
   NODE_NAME_CASE(ENDPGM)
   NODE_NAME_CASE(ENDPGM_TRAP)
   NODE_NAME_CASE(DWORDADDR)
   NODE_NAME_CASE(FRACT)
   NODE_NAME_CASE(SETCC)
   NODE_NAME_CASE(SETREG)
   NODE_NAME_CASE(DENORM_MODE)

llvm/lib/Target/AMDGPU/SIISelLowering.h

bool isEligibleForTailCallOptimization(
     const SmallVectorImpl<SDValue> &OutVals,
     const SmallVectorImpl<ISD::InputArg> &Ins, SelectionDAG &DAG) const;
 
   SDValue LowerCall(CallLoweringInfo &CLI,
                     SmallVectorImpl<SDValue> &InVals) const override;
 
   SDValue lowerDYNAMIC_STACKALLOCImpl(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerDYNAMIC_STACKALLOC(SDValue Op, SelectionDAG &DAG) const;
+  SDValue LowerSTACKSAVE(SDValue Op, SelectionDAG &DAG) const;
 
   Register getRegisterByName(const char* RegName, LLT VT,
                              const MachineFunction &MF) const override;
 
   MachineBasicBlock *splitKillBlock(MachineInstr &MI,
                                     MachineBasicBlock *BB) const;
 
   void bundleInstWithWaitcnt(MachineInstr &MI) const;

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

setOperationAction(ISD::INTRINSIC_W_CHAIN,
                      Custom);
 
   setOperationAction(ISD::INTRINSIC_VOID,
                      {MVT::Other, MVT::v2i16, MVT::v2f16, MVT::v3i16,
                       MVT::v3f16, MVT::v4f16, MVT::v4i16, MVT::f16, MVT::i16,
                       MVT::i8, MVT::i128},
                      Custom);
 
+  setOperationAction(ISD::STACKSAVE, MVT::Other, Custom);
+
   setTargetDAGCombine({ISD::ADD,
                        ISD::UADDO_CARRY,
                        ISD::SUB,
                        ISD::USUBO_CARRY,
                        ISD::FADD,
                        ISD::FSUB,
                        ISD::FMINNUM,
                        ISD::FMAXNUM,
SDValue SITargetLowering::LowerDYNAMIC_STACKALLOC(SDValue Op,
   // much to increment the uniform stack pointer.
   SDValue Size = Op.getOperand(1);
   if (isa<ConstantSDNode>(Size))
       return lowerDYNAMIC_STACKALLOCImpl(Op, DAG); // Use "generic" expansion.
 
   return AMDGPUTargetLowering::LowerDYNAMIC_STACKALLOC(Op, DAG);
 }
 
+SDValue SITargetLowering::LowerSTACKSAVE(SDValue Op, SelectionDAG &DAG) const {
+  if (Op.getValueType() != MVT::i32)
+    return Op; // Defer to cannot select error.
+
+  Register SP = getStackPointerRegisterToSaveRestore();
+  SDLoc SL(Op);
+
+  SDValue CopyFromSP = DAG.getCopyFromReg(Op->getOperand(0), SL, SP, MVT::i32);
+
+  // Convert from wave uniform to swizzled vector address. This should protect
+  // from any edge cases where the stacksave result isn't directly used with
+  // stackrestore.
+  SDValue VectorAddress =
+      DAG.getNode(AMDGPUISD::WAVE_ADDRESS, SL, MVT::i32, CopyFromSP);
+  return DAG.getMergeValues({VectorAddress, CopyFromSP.getValue(1)}, SL);
+}
+
 Register SITargetLowering::getRegisterByName(const char* RegName, LLT VT,
                                              const MachineFunction &MF) const {
   Register Reg = StringSwitch<Register>(RegName)
     .Case("m0", AMDGPU::M0)
     .Case("exec", AMDGPU::EXEC)
     .Case("exec_lo", AMDGPU::EXEC_LO)
     .Case("exec_hi", AMDGPU::EXEC_HI)
     .Case("flat_scratch", AMDGPU::FLAT_SCR)
SDValue SITargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::SMULO:
   case ISD::UMULO:
     return lowerXMULO(Op, DAG);
   case ISD::SMUL_LOHI:
   case ISD::UMUL_LOHI:
     return lowerXMUL_LOHI(Op, DAG);
   case ISD::DYNAMIC_STACKALLOC:
     return LowerDYNAMIC_STACKALLOC(Op, DAG);
+  case ISD::STACKSAVE:
+    return LowerSTACKSAVE(Op, DAG);
   }
   return SDValue();
 }
 
 // Used for D16: Casts the result of an instruction into the right vector,
 // packs values if loads return unpacked values.
 static SDValue adjustLoadValueTypeImpl(SDValue Result, EVT LoadVT,
                                        const SDLoc &DL,

llvm/test/CodeGen/AMDGPU/stacksave_stackrestore.invalid.ll

+; RUN: split-file %s %t
+; RUN: not --crash llc -mtriple=amdgcn-amd-amdpal -mcpu=gfx1030 -filetype=null %t/stacksave-error.ll 2>&1 | FileCheck -check-prefix=ERR-SAVE %s
+; RUN: not --crash llc -mtriple=amdgcn-amd-amdpal -mcpu=gfx1030 -filetype=null %t/stackrestore-error.ll 2>&1 | FileCheck -check-prefix=ERR-RESTORE %s
+
+; Test that an error is produced if stacksave/stackrestore are used
+; with the wrong (default) address space.
+
+;--- stacksave-error.ll
+
+declare ptr @llvm.stacksave.p0()
+
+; ERR-SAVE: LLVM ERROR: Cannot select: {{.+}}: i64,ch = stacksave
+define void @func_store_stacksave() {
+  %stacksave = call ptr @llvm.stacksave.p0()
+  call void asm sideeffect "; use $0", "s"(ptr %stacksave)
+  ret void
+}
+
+;--- stackrestore-error.ll
+
+declare void @llvm.stackrestore.p0(ptr)
+
+; ERR-RESTORE: LLVM ERROR: Cannot select: {{.+}}: ch = stackrestore {{.+}}, {{.+}}
+define amdgpu_gfx void @func_stacksave_sgpr(ptr inreg %stack) {
+  call void @llvm.stackrestore.p0(ptr %stack)
+  ret void
+}

llvm/test/CodeGen/AMDGPU/stacksave_stackrestore.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
+; RUN: llc -mtriple=amdgcn-amd-amdpal -mcpu=gfx1030 < %s | FileCheck -check-prefixes=GCN,WAVE32,WAVE32-OPT %s
+; RUN: llc -mtriple=amdgcn-amd-amdpal -mcpu=gfx1030 -mattr=+wavefrontsize64 < %s | FileCheck -check-prefixes=GCN,WAVE64,WAVE64-OPT %s
+
+; RUN: llc -O0 -mtriple=amdgcn-amd-amdpal -mcpu=gfx1030 < %s | FileCheck -check-prefixes=GCN,WAVE32,WAVE32-O0 %s
+; RUN: llc -O0 -mtriple=amdgcn-amd-amdpal -mcpu=gfx1030 -mattr=+wavefrontsize64 < %s | FileCheck -check-prefixes=GCN,WAVE64,WAVE64-O0 %s
+
+declare ptr addrspace(5) @llvm.stacksave.p5()
+declare void @llvm.stackrestore.p5(ptr addrspace(5))
+
+define hidden void @stack_passed_argument([32 x i32], i32) {
+; GCN-LABEL: stack_passed_argument:
+; GCN:       ; %bb.0:
+; GCN-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GCN-NEXT:    s_setpc_b64 s[30:31]
+  ret void
+}
+
+define void @func_store_stacksave() {
+; WAVE32-OPT-LABEL: func_store_stacksave:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_lshr_b32 s4, s32, 5
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s4
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_store_stacksave:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_lshr_b32 s4, s32, 6
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s4
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_store_stacksave:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s4
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_store_stacksave:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s4
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  ret void
+}
+
+define amdgpu_kernel void @kernel_store_stacksave() {
+; WAVE32-OPT-LABEL: kernel_store_stacksave:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_lshr_b32 s0, s32, 5
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s0
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_endpgm
+;
+; WAVE64-OPT-LABEL: kernel_store_stacksave:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_lshr_b32 s0, s32, 6
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s0
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_endpgm
+;
+; WAVE32-O0-LABEL: kernel_store_stacksave:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s0, s0, 5
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s0
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_endpgm
+;
+; WAVE64-O0-LABEL: kernel_store_stacksave:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s0, s0, 6
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s0
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_endpgm
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  ret void
+}
+
+define amdgpu_kernel void @kernel_store_stacksave_nocall() {
+; WAVE32-OPT-LABEL: kernel_store_stacksave_nocall:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_getpc_b64 s[4:5]
+; WAVE32-OPT-NEXT:    s_mov_b32 s4, s0
+; WAVE32-OPT-NEXT:    v_mov_b32_e32 v0, 0
+; WAVE32-OPT-NEXT:    s_load_dwordx4 s[4:7], s[4:5], 0x0
+; WAVE32-OPT-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_bitset0_b32 s7, 21
+; WAVE32-OPT-NEXT:    s_add_u32 s4, s4, s1
+; WAVE32-OPT-NEXT:    s_addc_u32 s5, s5, 0
+; WAVE32-OPT-NEXT:    s_lshr_b32 s0, s32, 5
+; WAVE32-OPT-NEXT:    v_mov_b32_e32 v1, s0
+; WAVE32-OPT-NEXT:    buffer_store_dword v0, v1, s[4:7], 0 offen
+; WAVE32-OPT-NEXT:    s_endpgm
+;
+; WAVE64-OPT-LABEL: kernel_store_stacksave_nocall:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_getpc_b64 s[4:5]
+; WAVE64-OPT-NEXT:    s_mov_b32 s4, s0
+; WAVE64-OPT-NEXT:    v_mov_b32_e32 v0, 0
+; WAVE64-OPT-NEXT:    s_load_dwordx4 s[4:7], s[4:5], 0x0
+; WAVE64-OPT-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_add_u32 s4, s4, s1
+; WAVE64-OPT-NEXT:    s_addc_u32 s5, s5, 0
+; WAVE64-OPT-NEXT:    s_lshr_b32 s0, s32, 6
+; WAVE64-OPT-NEXT:    v_mov_b32_e32 v1, s0
+; WAVE64-OPT-NEXT:    buffer_store_dword v0, v1, s[4:7], 0 offen
+; WAVE64-OPT-NEXT:    s_endpgm
+;
+; WAVE32-O0-LABEL: kernel_store_stacksave_nocall:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_getpc_b64 s[12:13]
+; WAVE32-O0-NEXT:    s_mov_b32 s12, s0
+; WAVE32-O0-NEXT:    s_load_dwordx4 s[12:15], s[12:13], 0x0
+; WAVE32-O0-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_bitset0_b32 s15, 21
+; WAVE32-O0-NEXT:    s_add_u32 s12, s12, s11
+; WAVE32-O0-NEXT:    s_addc_u32 s13, s13, 0
+; WAVE32-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s0, s0, 5
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v0, 0
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v1, s0
+; WAVE32-O0-NEXT:    buffer_store_dword v0, v1, s[12:15], 0 offen
+; WAVE32-O0-NEXT:    s_endpgm
+;
+; WAVE64-O0-LABEL: kernel_store_stacksave_nocall:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_getpc_b64 s[12:13]
+; WAVE64-O0-NEXT:    s_mov_b32 s12, s0
+; WAVE64-O0-NEXT:    s_load_dwordx4 s[12:15], s[12:13], 0x0
+; WAVE64-O0-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_add_u32 s12, s12, s11
+; WAVE64-O0-NEXT:    s_addc_u32 s13, s13, 0
+; WAVE64-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s0, s0, 6
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v0, 0
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v1, s0
+; WAVE64-O0-NEXT:    buffer_store_dword v0, v1, s[12:15], 0 offen
+; WAVE64-O0-NEXT:    s_endpgm
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  store i32 0, ptr addrspace(5) %stacksave
+  ret void
+}
+
+define void @func_stacksave_nonentry_block(i1 %cond) {
+; WAVE32-OPT-LABEL: func_stacksave_nonentry_block:
+; WAVE32-OPT:       ; %bb.0: ; %bb0
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    v_and_b32_e32 v0, 1, v0
+; WAVE32-OPT-NEXT:    s_mov_b32 s4, exec_lo
+; WAVE32-OPT-NEXT:    v_cmpx_eq_u32_e32 1, v0
+; WAVE32-OPT-NEXT:    s_cbranch_execz .LBB4_2
+; WAVE32-OPT-NEXT:  ; %bb.1: ; %bb1
+; WAVE32-OPT-NEXT:    s_lshr_b32 s5, s32, 5
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s5
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:  .LBB4_2: ; %bb2
+; WAVE32-OPT-NEXT:    s_or_b32 exec_lo, exec_lo, s4
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_nonentry_block:
+; WAVE64-OPT:       ; %bb.0: ; %bb0
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    v_and_b32_e32 v0, 1, v0
+; WAVE64-OPT-NEXT:    s_mov_b64 s[4:5], exec
+; WAVE64-OPT-NEXT:    v_cmpx_eq_u32_e32 1, v0
+; WAVE64-OPT-NEXT:    s_cbranch_execz .LBB4_2
+; WAVE64-OPT-NEXT:  ; %bb.1: ; %bb1
+; WAVE64-OPT-NEXT:    s_lshr_b32 s6, s32, 6
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s6
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:  .LBB4_2: ; %bb2
+; WAVE64-OPT-NEXT:    s_or_b64 exec, exec, s[4:5]
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_nonentry_block:
+; WAVE32-O0:       ; %bb.0: ; %bb0
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_xor_saveexec_b32 s4, -1
+; WAVE32-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s4
+; WAVE32-O0-NEXT:    ; implicit-def: $vgpr1
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v1, v0
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s7, -1
+; WAVE32-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s7
+; WAVE32-O0-NEXT:    v_and_b32_e64 v1, 1, v1
+; WAVE32-O0-NEXT:    v_cmp_eq_u32_e64 s5, v1, 1
+; WAVE32-O0-NEXT:    s_mov_b32 s4, exec_lo
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-O0-NEXT:    v_writelane_b32 v0, s4, 0
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s7, -1
+; WAVE32-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s32 ; 4-byte Folded Spill
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s7
+; WAVE32-O0-NEXT:    s_and_b32 s4, s4, s5
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s4
+; WAVE32-O0-NEXT:    s_cbranch_execz .LBB4_2
+; WAVE32-O0-NEXT:  ; %bb.1: ; %bb1
+; WAVE32-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s4
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:  .LBB4_2: ; %bb2
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s7, -1
+; WAVE32-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s7
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-O0-NEXT:    v_readlane_b32 s4, v0, 0
+; WAVE32-O0-NEXT:    s_or_b32 exec_lo, exec_lo, s4
+; WAVE32-O0-NEXT:    ; kill: killed $vgpr0
+; WAVE32-O0-NEXT:    s_xor_saveexec_b32 s4, -1
+; WAVE32-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s4
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_nonentry_block:
+; WAVE64-O0:       ; %bb.0: ; %bb0
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_xor_saveexec_b64 s[4:5], -1
+; WAVE64-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[4:5]
+; WAVE64-O0-NEXT:    ; implicit-def: $vgpr1
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v1, v0
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[10:11], -1
+; WAVE64-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[10:11]
+; WAVE64-O0-NEXT:    v_and_b32_e64 v1, 1, v1
+; WAVE64-O0-NEXT:    v_cmp_eq_u32_e64 s[6:7], v1, 1
+; WAVE64-O0-NEXT:    s_mov_b64 s[4:5], exec
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-O0-NEXT:    v_writelane_b32 v0, s4, 0
+; WAVE64-O0-NEXT:    v_writelane_b32 v0, s5, 1
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[10:11], -1
+; WAVE64-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s32 ; 4-byte Folded Spill
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[10:11]
+; WAVE64-O0-NEXT:    s_and_b64 s[4:5], s[4:5], s[6:7]
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[4:5]
+; WAVE64-O0-NEXT:    s_cbranch_execz .LBB4_2
+; WAVE64-O0-NEXT:  ; %bb.1: ; %bb1
+; WAVE64-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s4
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:  .LBB4_2: ; %bb2
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[10:11], -1
+; WAVE64-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s32 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[10:11]
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-O0-NEXT:    v_readlane_b32 s4, v0, 0
+; WAVE64-O0-NEXT:    v_readlane_b32 s5, v0, 1
+; WAVE64-O0-NEXT:    s_or_b64 exec, exec, s[4:5]
+; WAVE64-O0-NEXT:    ; kill: killed $vgpr0
+; WAVE64-O0-NEXT:    s_xor_saveexec_b64 s[4:5], -1
+; WAVE64-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[4:5]
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+bb0:
+  br i1 %cond, label %bb1, label %bb2
+
+bb1:
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  br label %bb2
+
+bb2:
+  ret void
+}
+
+define void @func_stackrestore_poison() {
+; WAVE32-OPT-LABEL: func_stackrestore_poison:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_lshl_b32 s32, s4, 5
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stackrestore_poison:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_lshl_b32 s32, s4, 6
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stackrestore_poison:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr4
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_lshl_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stackrestore_poison:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr4
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_lshl_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  call void @llvm.stackrestore.p5(ptr addrspace(5) poison)
+  ret void
+}
+
+define void @func_stackrestore_null() {
+; WAVE32-OPT-LABEL: func_stackrestore_null:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_mov_b32 s32, 0
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stackrestore_null:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_mov_b32 s32, 0
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stackrestore_null:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, 0
+; WAVE32-O0-NEXT:    s_lshl_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stackrestore_null:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, 0
+; WAVE64-O0-NEXT:    s_lshl_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  call void @llvm.stackrestore.p5(ptr addrspace(5) null)
+  ret void
+}
+
+define void @func_stackrestore_neg1() {
+; WAVE32-OPT-LABEL: func_stackrestore_neg1:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_movk_i32 s32, 0xffe0
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stackrestore_neg1:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_movk_i32 s32, 0xffc0
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stackrestore_neg1:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, -1
+; WAVE32-O0-NEXT:    s_lshl_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stackrestore_neg1:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, -1
+; WAVE64-O0-NEXT:    s_lshl_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  call void @llvm.stackrestore.p5(ptr addrspace(5) inttoptr (i32 -1 to ptr addrspace(5)))
+  ret void
+}
+
+define void @func_stackrestore_42() {
+; WAVE32-OPT-LABEL: func_stackrestore_42:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_movk_i32 s32, 0x540
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stackrestore_42:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_movk_i32 s32, 0xa80
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stackrestore_42:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, 42
+; WAVE32-O0-NEXT:    s_lshl_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stackrestore_42:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, 42
+; WAVE64-O0-NEXT:    s_lshl_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  call void @llvm.stackrestore.p5(ptr addrspace(5) inttoptr (i32 42 to ptr addrspace(5)))
+  ret void
+}
+
+define void @func_stacksave_stackrestore() {
+; WAVE32-OPT-LABEL: func_stacksave_stackrestore:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_stackrestore:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_stackrestore:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_stackrestore:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stacksave)
+  ret void
+}
+
+define void @func_stacksave_stackrestore_use() {
+; WAVE32-OPT-LABEL: func_stacksave_stackrestore_use:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_lshr_b32 s4, s32, 5
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s4
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_stackrestore_use:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_lshr_b32 s4, s32, 6
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s4
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_stackrestore_use:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s5, s4, 5
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s5
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_stackrestore_use:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s5, s4, 6
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s5
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stacksave)
+  ret void
+}
+
+define amdgpu_kernel void @kernel_stacksave_stackrestore_use() {
+; WAVE32-OPT-LABEL: kernel_stacksave_stackrestore_use:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_lshr_b32 s0, s32, 5
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s0
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_endpgm
+;
+; WAVE64-OPT-LABEL: kernel_stacksave_stackrestore_use:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_lshr_b32 s0, s32, 6
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s0
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_endpgm
+;
+; WAVE32-O0-LABEL: kernel_stacksave_stackrestore_use:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s1, s0, 5
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s1
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s0
+; WAVE32-O0-NEXT:    s_endpgm
+;
+; WAVE64-O0-LABEL: kernel_stacksave_stackrestore_use:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s1, s0, 6
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s1
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s0
+; WAVE64-O0-NEXT:    s_endpgm
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stacksave)
+  ret void
+}
+
+define void @func_stacksave_stackrestore_voffset(i32 %offset) {
+; WAVE32-OPT-LABEL: func_stacksave_stackrestore_voffset:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_lshr_b32 s4, s32, 5
+; WAVE32-OPT-NEXT:    v_add_nc_u32_e32 v0, s4, v0
+; WAVE32-OPT-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE32-OPT-NEXT:    s_lshl_b32 s32, s4, 5
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_stackrestore_voffset:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_lshr_b32 s4, s32, 6
+; WAVE64-OPT-NEXT:    v_add_nc_u32_e32 v0, s4, v0
+; WAVE64-OPT-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE64-OPT-NEXT:    s_lshl_b32 s32, s4, 6
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_stackrestore_voffset:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE32-O0-NEXT:    s_lshr_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    v_add_nc_u32_e64 v0, s4, v0
+; WAVE32-O0-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE32-O0-NEXT:    s_lshl_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_stackrestore_voffset:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s4, s32
+; WAVE64-O0-NEXT:    s_lshr_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    v_add_nc_u32_e64 v0, s4, v0
+; WAVE64-O0-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE64-O0-NEXT:    s_lshl_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  %gep = getelementptr i8, ptr addrspace(5) %stacksave, i32 %offset
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %gep)
+  ret void
+}
+
+define void @func_stacksave_vgpr(ptr addrspace(5) %stack) {
+; WAVE32-OPT-LABEL: func_stacksave_vgpr:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE32-OPT-NEXT:    s_lshl_b32 s32, s4, 5
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_vgpr:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE64-OPT-NEXT:    s_lshl_b32 s32, s4, 6
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_vgpr:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE32-O0-NEXT:    s_lshl_b32 s4, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_vgpr:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    v_readfirstlane_b32 s4, v0
+; WAVE64-O0-NEXT:    s_lshl_b32 s4, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stack)
+  ret void
+}
+
+define amdgpu_gfx void @func_stacksave_sgpr(ptr addrspace(5) inreg %stack) {
+; WAVE32-OPT-LABEL: func_stacksave_sgpr:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_lshl_b32 s32, s4, 5
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_sgpr:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_lshl_b32 s32, s4, 6
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_sgpr:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_lshl_b32 s34, s4, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s34
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_sgpr:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_lshl_b32 s34, s4, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s34
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stack)
+  ret void
+}
+
+define amdgpu_kernel void @kernel_stacksave_sgpr(ptr addrspace(5) %stack) {
+; WAVE32-OPT-LABEL: kernel_stacksave_sgpr:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_load_dword s0, s[0:1], 0x0
+; WAVE32-OPT-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s0
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_endpgm
+;
+; WAVE64-OPT-LABEL: kernel_stacksave_sgpr:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_load_dword s0, s[0:1], 0x0
+; WAVE64-OPT-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s0
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_endpgm
+;
+; WAVE32-O0-LABEL: kernel_stacksave_sgpr:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_load_dword s0, s[4:5], 0x0
+; WAVE32-O0-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s1, s0
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s1
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_lshl_b32 s0, s0, 5
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s0
+; WAVE32-O0-NEXT:    s_endpgm
+;
+; WAVE64-O0-LABEL: kernel_stacksave_sgpr:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_load_dword s0, s[4:5], 0x0
+; WAVE64-O0-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s1, s0
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s1
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_lshl_b32 s0, s0, 6
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s0
+; WAVE64-O0-NEXT:    s_endpgm
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stack)
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stack)
+  ret void
+}
+
+define amdgpu_kernel void @kernel_stacksave_stackrestore_call_with_stack_objects() {
+; WAVE32-OPT-LABEL: kernel_stacksave_stackrestore_call_with_stack_objects:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_getpc_b64 s[8:9]
+; WAVE32-OPT-NEXT:    s_mov_b32 s8, s0
+; WAVE32-OPT-NEXT:    s_movk_i32 s32, 0x200
+; WAVE32-OPT-NEXT:    s_load_dwordx4 s[8:11], s[8:9], 0x0
+; WAVE32-OPT-NEXT:    s_mov_b32 s0, s32
+; WAVE32-OPT-NEXT:    v_mov_b32_e32 v0, 42
+; WAVE32-OPT-NEXT:    v_mov_b32_e32 v1, 17
+; WAVE32-OPT-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_bitset0_b32 s11, 21
+; WAVE32-OPT-NEXT:    s_add_u32 s8, s8, s1
+; WAVE32-OPT-NEXT:    s_addc_u32 s9, s9, 0
+; WAVE32-OPT-NEXT:    s_getpc_b64 s[4:5]
+; WAVE32-OPT-NEXT:    s_add_u32 s4, s4, stack_passed_argument@rel32@lo+4
+; WAVE32-OPT-NEXT:    s_addc_u32 s5, s5, stack_passed_argument@rel32@hi+12
+; WAVE32-OPT-NEXT:    s_lshr_b32 s6, s0, 5
+; WAVE32-OPT-NEXT:    s_mov_b64 s[0:1], s[8:9]
+; WAVE32-OPT-NEXT:    s_mov_b64 s[2:3], s[10:11]
+; WAVE32-OPT-NEXT:    buffer_store_dword v0, off, s[8:11], 0 offset:4
+; WAVE32-OPT-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE32-OPT-NEXT:    buffer_store_dword v1, off, s[8:11], s32 offset:4
+; WAVE32-OPT-NEXT:    s_swappc_b64 s[30:31], s[4:5]
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s6
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_endpgm
+;
+; WAVE64-OPT-LABEL: kernel_stacksave_stackrestore_call_with_stack_objects:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_getpc_b64 s[8:9]
+; WAVE64-OPT-NEXT:    s_mov_b32 s8, s0
+; WAVE64-OPT-NEXT:    s_movk_i32 s32, 0x400
+; WAVE64-OPT-NEXT:    s_load_dwordx4 s[8:11], s[8:9], 0x0
+; WAVE64-OPT-NEXT:    s_mov_b32 s0, s32
+; WAVE64-OPT-NEXT:    v_mov_b32_e32 v0, 42
+; WAVE64-OPT-NEXT:    v_mov_b32_e32 v1, 17
+; WAVE64-OPT-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_add_u32 s8, s8, s1
+; WAVE64-OPT-NEXT:    s_addc_u32 s9, s9, 0
+; WAVE64-OPT-NEXT:    s_getpc_b64 s[4:5]
+; WAVE64-OPT-NEXT:    s_add_u32 s4, s4, stack_passed_argument@rel32@lo+4
+; WAVE64-OPT-NEXT:    s_addc_u32 s5, s5, stack_passed_argument@rel32@hi+12
+; WAVE64-OPT-NEXT:    s_lshr_b32 s6, s0, 6
+; WAVE64-OPT-NEXT:    s_mov_b64 s[0:1], s[8:9]
+; WAVE64-OPT-NEXT:    s_mov_b64 s[2:3], s[10:11]
+; WAVE64-OPT-NEXT:    buffer_store_dword v0, off, s[8:11], 0 offset:4
+; WAVE64-OPT-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE64-OPT-NEXT:    buffer_store_dword v1, off, s[8:11], s32 offset:4
+; WAVE64-OPT-NEXT:    s_swappc_b64 s[30:31], s[4:5]
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s6
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_endpgm
+;
+; WAVE32-O0-LABEL: kernel_stacksave_stackrestore_call_with_stack_objects:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_mov_b32 s32, 0x1200
+; WAVE32-O0-NEXT:    s_getpc_b64 s[20:21]
+; WAVE32-O0-NEXT:    s_mov_b32 s20, s0
+; WAVE32-O0-NEXT:    s_load_dwordx4 s[20:23], s[20:21], 0x0
+; WAVE32-O0-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_bitset0_b32 s23, 21
+; WAVE32-O0-NEXT:    s_add_u32 s20, s20, s11
+; WAVE32-O0-NEXT:    s_addc_u32 s21, s21, 0
+; WAVE32-O0-NEXT:    ; implicit-def: $vgpr3
+; WAVE32-O0-NEXT:    s_mov_b32 s14, s10
+; WAVE32-O0-NEXT:    s_mov_b32 s13, s9
+; WAVE32-O0-NEXT:    s_mov_b32 s12, s8
+; WAVE32-O0-NEXT:    s_mov_b64 s[10:11], s[6:7]
+; WAVE32-O0-NEXT:    s_mov_b64 s[8:9], s[4:5]
+; WAVE32-O0-NEXT:    s_mov_b64 s[6:7], s[2:3]
+; WAVE32-O0-NEXT:    s_mov_b64 s[4:5], s[0:1]
+; WAVE32-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE32-O0-NEXT:    v_writelane_b32 v3, s0, 0
+; WAVE32-O0-NEXT:    s_lshr_b32 s0, s0, 5
+; WAVE32-O0-NEXT:    v_writelane_b32 v3, s0, 1
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s19, -1
+; WAVE32-O0-NEXT:    buffer_store_dword v3, off, s[20:23], 0 offset:132 ; 4-byte Folded Spill
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s19
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v3, 42
+; WAVE32-O0-NEXT:    buffer_store_dword v3, off, s[20:23], 0 offset:4
+; WAVE32-O0-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE32-O0-NEXT:    s_mov_b64 s[0:1], s[20:21]
+; WAVE32-O0-NEXT:    s_mov_b64 s[2:3], s[22:23]
+; WAVE32-O0-NEXT:    s_mov_b32 s15, s32
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v3, 17
+; WAVE32-O0-NEXT:    buffer_store_dword v3, off, s[20:23], s15 offset:4
+; WAVE32-O0-NEXT:    s_getpc_b64 s[16:17]
+; WAVE32-O0-NEXT:    s_add_u32 s16, s16, stack_passed_argument@rel32@lo+4
+; WAVE32-O0-NEXT:    s_addc_u32 s17, s17, stack_passed_argument@rel32@hi+12
+; WAVE32-O0-NEXT:    s_mov_b32 s15, 20
+; WAVE32-O0-NEXT:    v_lshlrev_b32_e64 v2, s15, v2
+; WAVE32-O0-NEXT:    s_mov_b32 s15, 10
+; WAVE32-O0-NEXT:    v_lshlrev_b32_e64 v1, s15, v1
+; WAVE32-O0-NEXT:    v_or3_b32 v31, v0, v1, v2
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr15
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v0, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v1, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v2, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v3, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v4, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v5, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v6, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v7, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v8, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v9, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v10, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v11, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v12, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v13, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v14, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v15, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v16, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v17, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v18, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v19, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v20, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v21, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v22, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v23, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v24, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v25, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v26, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v27, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v28, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v29, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v30, s18
+; WAVE32-O0-NEXT:    s_swappc_b64 s[30:31], s[16:17]
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s19, -1
+; WAVE32-O0-NEXT:    buffer_load_dword v0, off, s[20:23], 0 offset:132 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s19
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-O0-NEXT:    v_readlane_b32 s1, v0, 1
+; WAVE32-O0-NEXT:    v_readlane_b32 s0, v0, 0
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s1
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s0
+; WAVE32-O0-NEXT:    ; kill: killed $vgpr0
+; WAVE32-O0-NEXT:    s_endpgm
+;
+; WAVE64-O0-LABEL: kernel_stacksave_stackrestore_call_with_stack_objects:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_mov_b32 s32, 0x2400
+; WAVE64-O0-NEXT:    s_getpc_b64 s[24:25]
+; WAVE64-O0-NEXT:    s_mov_b32 s24, s0
+; WAVE64-O0-NEXT:    s_load_dwordx4 s[24:27], s[24:25], 0x0
+; WAVE64-O0-NEXT:    s_waitcnt lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_add_u32 s24, s24, s11
+; WAVE64-O0-NEXT:    s_addc_u32 s25, s25, 0
+; WAVE64-O0-NEXT:    ; implicit-def: $vgpr3
+; WAVE64-O0-NEXT:    s_mov_b32 s14, s10
+; WAVE64-O0-NEXT:    s_mov_b32 s13, s9
+; WAVE64-O0-NEXT:    s_mov_b32 s12, s8
+; WAVE64-O0-NEXT:    s_mov_b64 s[10:11], s[6:7]
+; WAVE64-O0-NEXT:    s_mov_b64 s[8:9], s[4:5]
+; WAVE64-O0-NEXT:    s_mov_b64 s[6:7], s[2:3]
+; WAVE64-O0-NEXT:    s_mov_b64 s[4:5], s[0:1]
+; WAVE64-O0-NEXT:    s_mov_b32 s0, s32
+; WAVE64-O0-NEXT:    v_writelane_b32 v3, s0, 0
+; WAVE64-O0-NEXT:    s_lshr_b32 s0, s0, 6
+; WAVE64-O0-NEXT:    v_writelane_b32 v3, s0, 1
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[20:21], -1
+; WAVE64-O0-NEXT:    buffer_store_dword v3, off, s[24:27], 0 offset:132 ; 4-byte Folded Spill
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[20:21]
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v3, 42
+; WAVE64-O0-NEXT:    buffer_store_dword v3, off, s[24:27], 0 offset:4
+; WAVE64-O0-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE64-O0-NEXT:    s_mov_b64 s[0:1], s[24:25]
+; WAVE64-O0-NEXT:    s_mov_b64 s[2:3], s[26:27]
+; WAVE64-O0-NEXT:    s_mov_b32 s15, s32
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v3, 17
+; WAVE64-O0-NEXT:    buffer_store_dword v3, off, s[24:27], s15 offset:4
+; WAVE64-O0-NEXT:    s_getpc_b64 s[16:17]
+; WAVE64-O0-NEXT:    s_add_u32 s16, s16, stack_passed_argument@rel32@lo+4
+; WAVE64-O0-NEXT:    s_addc_u32 s17, s17, stack_passed_argument@rel32@hi+12
+; WAVE64-O0-NEXT:    s_mov_b32 s15, 20
+; WAVE64-O0-NEXT:    v_lshlrev_b32_e64 v2, s15, v2
+; WAVE64-O0-NEXT:    s_mov_b32 s15, 10
+; WAVE64-O0-NEXT:    v_lshlrev_b32_e64 v1, s15, v1
+; WAVE64-O0-NEXT:    v_or3_b32 v31, v0, v1, v2
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr15
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v0, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v1, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v2, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v3, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v4, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v5, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v6, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v7, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v8, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v9, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v10, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v11, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v12, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v13, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v14, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v15, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v16, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v17, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v18, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v19, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v20, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v21, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v22, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v23, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v24, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v25, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v26, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v27, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v28, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v29, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v30, s18
+; WAVE64-O0-NEXT:    s_swappc_b64 s[30:31], s[16:17]
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[20:21], -1
+; WAVE64-O0-NEXT:    buffer_load_dword v0, off, s[24:27], 0 offset:132 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[20:21]
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-O0-NEXT:    v_readlane_b32 s1, v0, 1
+; WAVE64-O0-NEXT:    v_readlane_b32 s0, v0, 0
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s1
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s0
+; WAVE64-O0-NEXT:    ; kill: killed $vgpr0
+; WAVE64-O0-NEXT:    s_endpgm
+  %alloca = alloca [32 x i32], addrspace(5)
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  store volatile i32 42, ptr addrspace(5) %alloca
+  call void @stack_passed_argument([32 x i32] poison, i32 17)
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stacksave)
+  ret void
+}
+
+define void @func_stacksave_stackrestore_call_with_stack_objects() {
+; WAVE32-OPT-LABEL: func_stacksave_stackrestore_call_with_stack_objects:
+; WAVE32-OPT:       ; %bb.0:
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-OPT-NEXT:    s_mov_b32 s8, s33
+; WAVE32-OPT-NEXT:    s_mov_b32 s33, s32
+; WAVE32-OPT-NEXT:    s_xor_saveexec_b32 s4, -1
+; WAVE32-OPT-NEXT:    buffer_store_dword v31, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
+; WAVE32-OPT-NEXT:    s_mov_b32 exec_lo, s4
+; WAVE32-OPT-NEXT:    v_writelane_b32 v31, s30, 0
+; WAVE32-OPT-NEXT:    v_mov_b32_e32 v0, 42
+; WAVE32-OPT-NEXT:    v_mov_b32_e32 v1, 17
+; WAVE32-OPT-NEXT:    s_addk_i32 s32, 0x200
+; WAVE32-OPT-NEXT:    s_getpc_b64 s[4:5]
+; WAVE32-OPT-NEXT:    s_add_u32 s4, s4, stack_passed_argument@rel32@lo+4
+; WAVE32-OPT-NEXT:    s_addc_u32 s5, s5, stack_passed_argument@rel32@hi+12
+; WAVE32-OPT-NEXT:    s_mov_b32 s6, s32
+; WAVE32-OPT-NEXT:    v_writelane_b32 v31, s31, 1
+; WAVE32-OPT-NEXT:    s_lshr_b32 s7, s6, 5
+; WAVE32-OPT-NEXT:    buffer_store_dword v0, off, s[0:3], s33
+; WAVE32-OPT-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE32-OPT-NEXT:    buffer_store_dword v1, off, s[0:3], s32 offset:4
+; WAVE32-OPT-NEXT:    s_swappc_b64 s[30:31], s[4:5]
+; WAVE32-OPT-NEXT:    ;;#ASMSTART
+; WAVE32-OPT-NEXT:    ; use s7
+; WAVE32-OPT-NEXT:    ;;#ASMEND
+; WAVE32-OPT-NEXT:    s_mov_b32 s32, s6
+; WAVE32-OPT-NEXT:    v_readlane_b32 s31, v31, 1
+; WAVE32-OPT-NEXT:    v_readlane_b32 s30, v31, 0
+; WAVE32-OPT-NEXT:    s_xor_saveexec_b32 s4, -1
+; WAVE32-OPT-NEXT:    buffer_load_dword v31, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
+; WAVE32-OPT-NEXT:    s_mov_b32 exec_lo, s4
+; WAVE32-OPT-NEXT:    s_addk_i32 s32, 0xfe00
+; WAVE32-OPT-NEXT:    s_mov_b32 s33, s8
+; WAVE32-OPT-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-OPT-LABEL: func_stacksave_stackrestore_call_with_stack_objects:
+; WAVE64-OPT:       ; %bb.0:
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-OPT-NEXT:    s_mov_b32 s8, s33
+; WAVE64-OPT-NEXT:    s_mov_b32 s33, s32
+; WAVE64-OPT-NEXT:    s_xor_saveexec_b64 s[4:5], -1
+; WAVE64-OPT-NEXT:    buffer_store_dword v31, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
+; WAVE64-OPT-NEXT:    s_mov_b64 exec, s[4:5]
+; WAVE64-OPT-NEXT:    v_writelane_b32 v31, s30, 0
+; WAVE64-OPT-NEXT:    v_mov_b32_e32 v0, 42
+; WAVE64-OPT-NEXT:    v_mov_b32_e32 v1, 17
+; WAVE64-OPT-NEXT:    s_addk_i32 s32, 0x400
+; WAVE64-OPT-NEXT:    s_getpc_b64 s[4:5]
+; WAVE64-OPT-NEXT:    s_add_u32 s4, s4, stack_passed_argument@rel32@lo+4
+; WAVE64-OPT-NEXT:    s_addc_u32 s5, s5, stack_passed_argument@rel32@hi+12
+; WAVE64-OPT-NEXT:    s_mov_b32 s6, s32
+; WAVE64-OPT-NEXT:    v_writelane_b32 v31, s31, 1
+; WAVE64-OPT-NEXT:    s_lshr_b32 s7, s6, 6
+; WAVE64-OPT-NEXT:    buffer_store_dword v0, off, s[0:3], s33
+; WAVE64-OPT-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE64-OPT-NEXT:    buffer_store_dword v1, off, s[0:3], s32 offset:4
+; WAVE64-OPT-NEXT:    s_swappc_b64 s[30:31], s[4:5]
+; WAVE64-OPT-NEXT:    ;;#ASMSTART
+; WAVE64-OPT-NEXT:    ; use s7
+; WAVE64-OPT-NEXT:    ;;#ASMEND
+; WAVE64-OPT-NEXT:    s_mov_b32 s32, s6
+; WAVE64-OPT-NEXT:    v_readlane_b32 s31, v31, 1
+; WAVE64-OPT-NEXT:    v_readlane_b32 s30, v31, 0
+; WAVE64-OPT-NEXT:    s_xor_saveexec_b64 s[4:5], -1
+; WAVE64-OPT-NEXT:    buffer_load_dword v31, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
+; WAVE64-OPT-NEXT:    s_mov_b64 exec, s[4:5]
+; WAVE64-OPT-NEXT:    s_addk_i32 s32, 0xfc00
+; WAVE64-OPT-NEXT:    s_mov_b32 s33, s8
+; WAVE64-OPT-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-OPT-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE32-O0-LABEL: func_stacksave_stackrestore_call_with_stack_objects:
+; WAVE32-O0:       ; %bb.0:
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE32-O0-NEXT:    s_mov_b32 s25, s33
+; WAVE32-O0-NEXT:    s_mov_b32 s33, s32
+; WAVE32-O0-NEXT:    s_xor_saveexec_b32 s16, -1
+; WAVE32-O0-NEXT:    buffer_store_dword v32, off, s[0:3], s33 offset:128 ; 4-byte Folded Spill
+; WAVE32-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 offset:136 ; 4-byte Folded Spill
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s16
+; WAVE32-O0-NEXT:    s_add_i32 s32, s32, 0x1200
+; WAVE32-O0-NEXT:    ; implicit-def: $vgpr0
+; WAVE32-O0-NEXT:    v_writelane_b32 v32, s30, 0
+; WAVE32-O0-NEXT:    v_writelane_b32 v32, s31, 1
+; WAVE32-O0-NEXT:    s_mov_b32 s16, s32
+; WAVE32-O0-NEXT:    v_writelane_b32 v0, s16, 0
+; WAVE32-O0-NEXT:    s_lshr_b32 s16, s16, 5
+; WAVE32-O0-NEXT:    v_writelane_b32 v0, s16, 1
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s24, -1
+; WAVE32-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 offset:132 ; 4-byte Folded Spill
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s24
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v0, 42
+; WAVE32-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33
+; WAVE32-O0-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE32-O0-NEXT:    s_mov_b64 s[22:23], s[2:3]
+; WAVE32-O0-NEXT:    s_mov_b64 s[20:21], s[0:1]
+; WAVE32-O0-NEXT:    s_mov_b32 s16, s32
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v0, 17
+; WAVE32-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s16 offset:4
+; WAVE32-O0-NEXT:    s_getpc_b64 s[16:17]
+; WAVE32-O0-NEXT:    s_add_u32 s16, s16, stack_passed_argument@rel32@lo+4
+; WAVE32-O0-NEXT:    s_addc_u32 s17, s17, stack_passed_argument@rel32@hi+12
+; WAVE32-O0-NEXT:    s_mov_b64 s[0:1], s[20:21]
+; WAVE32-O0-NEXT:    s_mov_b64 s[2:3], s[22:23]
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v0, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v1, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v2, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v3, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v4, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v5, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v6, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v7, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v8, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v9, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v10, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v11, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v12, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v13, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v14, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v15, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v16, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v17, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v18, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v19, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v20, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v21, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v22, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v23, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v24, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v25, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v26, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v27, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v28, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v29, s18
+; WAVE32-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE32-O0-NEXT:    v_mov_b32_e32 v30, s18
+; WAVE32-O0-NEXT:    s_swappc_b64 s[30:31], s[16:17]
+; WAVE32-O0-NEXT:    s_or_saveexec_b32 s24, -1
+; WAVE32-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 offset:132 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s24
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-O0-NEXT:    v_readlane_b32 s5, v0, 1
+; WAVE32-O0-NEXT:    v_readlane_b32 s4, v0, 0
+; WAVE32-O0-NEXT:    ;;#ASMSTART
+; WAVE32-O0-NEXT:    ; use s5
+; WAVE32-O0-NEXT:    ;;#ASMEND
+; WAVE32-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE32-O0-NEXT:    v_readlane_b32 s31, v32, 1
+; WAVE32-O0-NEXT:    v_readlane_b32 s30, v32, 0
+; WAVE32-O0-NEXT:    ; kill: killed $vgpr0
+; WAVE32-O0-NEXT:    s_xor_saveexec_b32 s4, -1
+; WAVE32-O0-NEXT:    buffer_load_dword v32, off, s[0:3], s33 offset:128 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 offset:136 ; 4-byte Folded Reload
+; WAVE32-O0-NEXT:    s_mov_b32 exec_lo, s4
+; WAVE32-O0-NEXT:    s_add_i32 s32, s32, 0xffffee00
+; WAVE32-O0-NEXT:    s_mov_b32 s33, s25
+; WAVE32-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE32-O0-NEXT:    s_setpc_b64 s[30:31]
+;
+; WAVE64-O0-LABEL: func_stacksave_stackrestore_call_with_stack_objects:
+; WAVE64-O0:       ; %bb.0:
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; WAVE64-O0-NEXT:    s_mov_b32 s19, s33
+; WAVE64-O0-NEXT:    s_mov_b32 s33, s32
+; WAVE64-O0-NEXT:    s_xor_saveexec_b64 s[16:17], -1
+; WAVE64-O0-NEXT:    buffer_store_dword v32, off, s[0:3], s33 offset:128 ; 4-byte Folded Spill
+; WAVE64-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 offset:136 ; 4-byte Folded Spill
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[16:17]
+; WAVE64-O0-NEXT:    s_add_i32 s32, s32, 0x2400
+; WAVE64-O0-NEXT:    ; implicit-def: $vgpr0
+; WAVE64-O0-NEXT:    v_writelane_b32 v32, s30, 0
+; WAVE64-O0-NEXT:    v_writelane_b32 v32, s31, 1
+; WAVE64-O0-NEXT:    s_mov_b32 s16, s32
+; WAVE64-O0-NEXT:    v_writelane_b32 v0, s16, 0
+; WAVE64-O0-NEXT:    s_lshr_b32 s16, s16, 6
+; WAVE64-O0-NEXT:    v_writelane_b32 v0, s16, 1
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[26:27], -1
+; WAVE64-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 offset:132 ; 4-byte Folded Spill
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[26:27]
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v0, 42
+; WAVE64-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33
+; WAVE64-O0-NEXT:    s_waitcnt_vscnt null, 0x0
+; WAVE64-O0-NEXT:    s_mov_b64 s[22:23], s[2:3]
+; WAVE64-O0-NEXT:    s_mov_b64 s[20:21], s[0:1]
+; WAVE64-O0-NEXT:    s_mov_b32 s16, s32
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v0, 17
+; WAVE64-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s16 offset:4
+; WAVE64-O0-NEXT:    s_getpc_b64 s[16:17]
+; WAVE64-O0-NEXT:    s_add_u32 s16, s16, stack_passed_argument@rel32@lo+4
+; WAVE64-O0-NEXT:    s_addc_u32 s17, s17, stack_passed_argument@rel32@hi+12
+; WAVE64-O0-NEXT:    s_mov_b64 s[0:1], s[20:21]
+; WAVE64-O0-NEXT:    s_mov_b64 s[2:3], s[22:23]
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v0, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v1, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v2, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v3, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v4, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v5, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v6, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v7, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v8, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v9, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v10, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v11, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v12, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v13, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v14, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v15, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v16, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v17, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v18, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v19, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v20, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v21, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v22, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v23, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v24, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v25, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v26, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v27, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v28, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v29, s18
+; WAVE64-O0-NEXT:    ; implicit-def: $sgpr18
+; WAVE64-O0-NEXT:    v_mov_b32_e32 v30, s18
+; WAVE64-O0-NEXT:    s_swappc_b64 s[30:31], s[16:17]
+; WAVE64-O0-NEXT:    s_or_saveexec_b64 s[26:27], -1
+; WAVE64-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 offset:132 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[26:27]
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-O0-NEXT:    v_readlane_b32 s5, v0, 1
+; WAVE64-O0-NEXT:    v_readlane_b32 s4, v0, 0
+; WAVE64-O0-NEXT:    ;;#ASMSTART
+; WAVE64-O0-NEXT:    ; use s5
+; WAVE64-O0-NEXT:    ;;#ASMEND
+; WAVE64-O0-NEXT:    s_mov_b32 s32, s4
+; WAVE64-O0-NEXT:    v_readlane_b32 s31, v32, 1
+; WAVE64-O0-NEXT:    v_readlane_b32 s30, v32, 0
+; WAVE64-O0-NEXT:    ; kill: killed $vgpr0
+; WAVE64-O0-NEXT:    s_xor_saveexec_b64 s[4:5], -1
+; WAVE64-O0-NEXT:    buffer_load_dword v32, off, s[0:3], s33 offset:128 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 offset:136 ; 4-byte Folded Reload
+; WAVE64-O0-NEXT:    s_mov_b64 exec, s[4:5]
+; WAVE64-O0-NEXT:    s_add_i32 s32, s32, 0xffffdc00
+; WAVE64-O0-NEXT:    s_mov_b32 s33, s19
+; WAVE64-O0-NEXT:    s_waitcnt vmcnt(0)
+; WAVE64-O0-NEXT:    s_setpc_b64 s[30:31]
+  %alloca = alloca [32 x i32], addrspace(5)
+  %stacksave = call ptr addrspace(5) @llvm.stacksave.p5()
+  store volatile i32 42, ptr addrspace(5) %alloca
+  call void @stack_passed_argument([32 x i32] poison, i32 17)
+  call void asm sideeffect "; use $0", "s"(ptr addrspace(5) %stacksave)
+  call void @llvm.stackrestore.p5(ptr addrspace(5) %stacksave)
+  ret void
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; WAVE32: {{.*}}
+; WAVE64: {{.*}}