[AMDGPU] Emit debugger prologue and emit the rest of the debugger fields in the kernel code header

[kzhuravl]

Summary

Debugger prologue is emitted if -mattr=+amdgpu-debugger-emit-prologue.

Debugger prologue writes work group IDs and work item IDs to scratch memory at fixed location in the following format:

• offset 0: work group ID x
• offset 4: work group ID y
• offset 8: work group ID z
• offset 16: work item ID x
• offset 20: work item ID y
• offset 24: work item ID z

Set

• amd_kernel_code_t::debug_wavefront_private_segment_offset_sgpr to scratch wave offset reg
• amd_kernel_code_t::debug_private_segment_buffer_sgpr to scratch rsrc reg
• amd_kernel_code_t::is_debug_supported to true if all debugger features are enabled

[• tstellarAMD]

lib/Target/AMDGPU/SIInstructions.td

1904–1920 ↗

(On Diff #57509)

Why can't we use the existing spill pseudos for this?

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

93–103

I think it would be better to have AMDGPUAnnotateKernelFeatures check ST.debuggerEmitPrologue() and add the attribute in that pass.

111

We are creating stack objects during ISel for the debugger prologue, so doesn't that make ST.debuggerEmitPrologue() redundant?

[kzhuravl]

Update based on Tom's review feedback

[kzhuravl]

Ping

[arsenm]

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

93–103

I don't agree. I don't think it makes sense to add a attribute based on a subtarget feature. If the feature is attached to the subtarget already, this is just adding more clutter to the attribute list. We should just implement the per-function subtarget stuff since a subtarget is basically a collection of subtarget features.

[arsenm]

lib/Target/AMDGPU/AMDGPUAsmPrinter.h

80–86

Why are these using the SGPR number and special invalid register instead of the phys reg ID and NoReg?

lib/Target/AMDGPU/SIISelLowering.cpp

690

Why not create a single object and use offsets into it?

[kzhuravl]

Rebase to tot

[kzhuravl]

lib/Target/AMDGPU/AMDGPUAsmPrinter.h

80–86

Because we have to be compliant with our existing spec, which says to put SGPR number or (uint16_t)-1.

lib/Target/AMDGPU/SIISelLowering.cpp

690

Is there any advantage to having a single object?

The reason for having multiple objects is so we can use storeRegToStackSlot, which creates spills.

[kzhuravl]

Address review feedback

[kzhuravl]

Ping

[kzhuravl]

Ping

[arsenm]

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

201–202

This willl print the wrong thing if it isn't set. Should it print something else if not valid?

lib/Target/AMDGPU/AMDGPUAsmPrinter.h

80–86

Oh, OK. I didn't realize this was in the printer

lib/Target/AMDGPU/AMDGPUSubtarget.h

321

&& on previous line

lib/Target/AMDGPU/SIFrameLowering.cpp

42

Since my patch yesterday this should be SISubtarget, and ST can be re-used below

295–296

SISubtarget and then you can remove the casts

lib/Target/AMDGPU/SIISelLowering.cpp

686–687

Can you move the body of the loop into a helper function as well as the comment

test/CodeGen/AMDGPU/debugger-emit-prologue.ll

19–20

Should also check these in a test where these aren't set

[kzhuravl]

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

201–202

Thanks for catching this one. I have changed it so it only gets printed if amdgpu-debugger-emit-prologue attribute was specified.

[kzhuravl]

Address review comments

[kzhuravl]

Upload correct diff

[arsenm]

LGTM

[aaboud]

This commit is causing build to fail on windows configuration.

llvm/trunk/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

57 ↗

(On Diff #61883)

These lines caused a build failure on windows BuildBot.

error C2536: 'llvm::SIMachineFunctionInfo::llvm::SIMachineFunctionInfo::DebuggerWorkGroupIDStackObjectIndices' : cannot specify explicit initializer for arrays

One option to fix this is to use "std::array<int, 3>" as type of these two fields instead of "int[3]".
Please, fix the issue as soon as possible.

[kzhuravl]

Sorry, I was travelling until now. It seems like it is fixed in rL273860

lib/Target/AMDGPU/AMDGPU.h

 FunctionPass *createSIFixControlFlowLiveIntervalsPass();
 FunctionPass *createSIFixSGPRCopiesPass();
 FunctionPass *createSICodeEmitterPass(formatted_raw_ostream &OS);
 FunctionPass *createSIDebuggerInsertNopsPass();
 FunctionPass *createSIInsertWaitsPass();
 
 ScheduleDAGInstrs *createSIMachineScheduler(MachineSchedContext *C);
 
-ModulePass *createAMDGPUAnnotateKernelFeaturesPass();
+ModulePass *createAMDGPUAnnotateKernelFeaturesPass(const TargetMachine *TM = nullptr);
 void initializeAMDGPUAnnotateKernelFeaturesPass(PassRegistry &);
 extern char &AMDGPUAnnotateKernelFeaturesID;
 
 void initializeSIFoldOperandsPass(PassRegistry &);
 extern char &SIFoldOperandsID;
 
 void initializeSIFixSGPRCopiesPass(PassRegistry &);
 extern char &SIFixSGPRCopiesID;

lib/Target/AMDGPU/AMDGPU.td

 
 def FeatureDebuggerReserveTrapRegs : SubtargetFeature<
   "amdgpu-debugger-reserve-trap-regs",
   "DebuggerReserveTrapVGPRs",
   "true",
   "Reserve VGPRs for trap handler usage"
 >;
 
+def FeatureDebuggerEmitPrologue : SubtargetFeature<
+  "amdgpu-debugger-emit-prologue",
+  "DebuggerEmitPrologue",
+  "true",
+  "Emit debugger prologue"
+>;
+
 //===----------------------------------------------------------------------===//
 
 def AMDGPUInstrInfo : InstrInfo {
   let guessInstructionProperties = 1;
   let noNamedPositionallyEncodedOperands = 1;
 }
 
 def AMDGPUAsmParser : AsmParser {

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

 //===-- AMDGPUAnnotateKernelFeaturesPass.cpp ------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 /// \file This pass adds target attributes to functions which use intrinsics
 /// which will impact calling convention lowering.
 //
 //===----------------------------------------------------------------------===//
 
 #include "AMDGPU.h"
+#include "AMDGPUSubtarget.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Module.h"
 
 #define DEBUG_TYPE "amdgpu-annotate-kernel-features"
 
 using namespace llvm;
 
 namespace {
 
 class AMDGPUAnnotateKernelFeatures : public ModulePass {
 private:
+  const TargetMachine *TM;
   static bool hasAddrSpaceCast(const Function &F);
 
   void addAttrToCallers(Function *Intrin, StringRef AttrName);
   bool addAttrsForIntrinsics(Module &M, ArrayRef<StringRef[2]>);
+  bool addAttrsForFeatures(Module &M);
 
 public:
   static char ID;
 
-  AMDGPUAnnotateKernelFeatures() : ModulePass(ID) { }
+  AMDGPUAnnotateKernelFeatures(const TargetMachine *TM_ = nullptr) :
+    ModulePass(ID),
+    TM(TM_) { }
   bool runOnModule(Module &M) override;
   const char *getPassName() const override {
     return "AMDGPU Annotate Kernel Features";
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesAll();
     ModulePass::getAnalysisUsage(AU);
if (Function *Fn = M.getFunction(Arr[0])) {
       addAttrToCallers(Fn, Arr[1]);
       Changed = true;
     }
   }
 
   return Changed;
 }
 
+bool AMDGPUAnnotateKernelFeatures::addAttrsForFeatures(Module &M) {
+  if (!TM)
+    return false;
+
+  static const StringRef DebuggerEmitPrologueAttrs[] = {
+    "amdgpu-work-item-id-y", "amdgpu-work-item-id-z",
+    "amdgpu-work-group-id-y", "amdgpu-work-group-id-z"
+  };
+
+  bool Changed = false;
+  for (auto &F : M) {
+    const AMDGPUSubtarget &ST = TM->getSubtarget<AMDGPUSubtarget>(F);
+    if (!ST.debuggerEmitPrologue())
+      continue;
+    if (F.isIntrinsic())
+      continue;
+
+    for (auto &Attr : DebuggerEmitPrologueAttrs) {
+      if (!F.hasFnAttribute(Attr)) {
+        F.addFnAttr(Attr);
+        Changed = true;
+      }
+    }
+  }
+
+  return Changed;
+}
+
 bool AMDGPUAnnotateKernelFeatures::runOnModule(Module &M) {
   Triple TT(M.getTargetTriple());
 
   static const StringRef IntrinsicToAttr[][2] = {
     // .x omitted
     { "llvm.amdgcn.workitem.id.y", "amdgpu-work-item-id-y" },
     { "llvm.amdgcn.workitem.id.z", "amdgpu-work-item-id-z" },
 
if (TT.getOS() == Triple::AMDHSA) {
     for (Function &F : M) {
       if (F.hasFnAttribute("amdgpu-queue-ptr"))
         continue;
 
       if (hasAddrSpaceCast(F))
         F.addFnAttr("amdgpu-queue-ptr");
     }
   }
+  Changed |= addAttrsForFeatures(M);
 
   return Changed;
 }
 
-ModulePass *llvm::createAMDGPUAnnotateKernelFeaturesPass() {
-  return new AMDGPUAnnotateKernelFeatures();
+ModulePass *llvm::createAMDGPUAnnotateKernelFeaturesPass(const TargetMachine *TM) {
+  return new AMDGPUAnnotateKernelFeatures(TM);
 }

lib/Target/AMDGPU/AMDGPUAsmPrinter.h

SIProgramInfo() :
       LDSBlocks(0),
       ScratchBlocks(0),
       ComputePGMRSrc2(0),
       NumVGPR(0),
       NumSGPR(0),
       FlatUsed(false),
       ReservedVGPRFirst(0),
       ReservedVGPRCount(0),
+      DebuggerWavefrontPrivateSegmentOffsetSGPR((uint16_t)-1),
+      DebuggerPrivateSegmentBufferSGPR((uint16_t)-1),
       VCCUsed(false),
       CodeLen(0) {}
 
     // Fields set in PGM_RSRC1 pm4 packet.
     uint32_t VGPRBlocks;
     uint32_t SGPRBlocks;
     uint32_t Priority;
     uint32_t FloatMode;
struct SIProgramInfo {
     bool FlatUsed;
 
     // If ReservedVGPRCount is 0 then must be 0. Otherwise, this is the first
     // fixed VGPR number reserved.
     uint16_t ReservedVGPRFirst;
     // The number of consecutive VGPRs reserved.
     uint16_t ReservedVGPRCount;
 
+    // Fixed SGPR number used to hold wave scratch offset for entire kernel
+    // execution, or uint16_t(-1) if the register is not used or not known.
+    uint16_t DebuggerWavefrontPrivateSegmentOffsetSGPR;
+    // Fixed SGPR number of the first 4 SGPRs used to hold scratch V# for entire
+    // kernel execution, or uint16_t(-1) if the register is not used or not
+    // known.
+    uint16_t DebuggerPrivateSegmentBufferSGPR;
+
     // Bonus information for debugging.
     bool VCCUsed;
     uint64_t CodeLen;
   };
 
   void getSIProgramInfo(SIProgramInfo &Out, const MachineFunction &MF) const;
   void findNumUsedRegistersSI(const MachineFunction &MF,
                               unsigned &NumSGPR,

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

if (STM.getGeneration() >= AMDGPUSubtarget::SOUTHERN_ISLANDS) {
       OutStreamer->emitRawComment(" LDSByteSize: " + Twine(KernelInfo.LDSSize) +
                                   " bytes/workgroup (compile time only)", false);
 
       OutStreamer->emitRawComment(" ReservedVGPRFirst: " + Twine(KernelInfo.ReservedVGPRFirst),
                                   false);
       OutStreamer->emitRawComment(" ReservedVGPRCount: " + Twine(KernelInfo.ReservedVGPRCount),
                                   false);
 
+      OutStreamer->emitRawComment(" DebuggerWavefrontPrivateSegmentOffsetSGPR: s" +
+                                  Twine(KernelInfo.DebuggerWavefrontPrivateSegmentOffsetSGPR), false);
+      OutStreamer->emitRawComment(" DebuggerPrivateSegmentBufferSGPR: s" +
+                                  Twine(KernelInfo.DebuggerPrivateSegmentBufferSGPR), false);
+
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:USER_SGPR: " +
                                   Twine(G_00B84C_USER_SGPR(KernelInfo.ComputePGMRSrc2)),
                                   false);
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:TGID_X_EN: " +
                                   Twine(G_00B84C_TGID_X_EN(KernelInfo.ComputePGMRSrc2)),
                                   false);
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:TGID_Y_EN: " +
                                   Twine(G_00B84C_TGID_Y_EN(KernelInfo.ComputePGMRSrc2)),
void AMDGPUAsmPrinter::getSIProgramInfo(SIProgramInfo &ProgInfo,
   // Update necessary Reserved* fields and max VGPRs used if
   // "amdgpu-debugger-reserve-trap-regs" attribute was specified.
   if (STM.debuggerReserveTrapVGPRs()) {
     ProgInfo.ReservedVGPRFirst = MaxVGPR + 1;
     ProgInfo.ReservedVGPRCount = MFI->getDebuggerReserveTrapVGPRCount();
     MaxVGPR += MFI->getDebuggerReserveTrapVGPRCount();
   }
 
+  // Update DebuggerWavefrontPrivateSegmentOffsetSGPR and
+  // DebuggerPrivateSegmentBufferSGPR fields if "amdgpu-debugger-emit-prologue"
+  // attribute was specified.
+  if (STM.debuggerEmitPrologue()) {
+    ProgInfo.DebuggerWavefrontPrivateSegmentOffsetSGPR =
+      RI->getHWRegIndex(MFI->getScratchWaveOffsetReg());
+    ProgInfo.DebuggerPrivateSegmentBufferSGPR =
+      RI->getHWRegIndex(MFI->getScratchRSrcReg());
+  }
+
   // We found the maximum register index. They start at 0, so add one to get the
   // number of registers.
   ProgInfo.NumVGPR = MaxVGPR + 1;
   ProgInfo.NumSGPR = MaxSGPR + 1;
 
   if (STM.hasSGPRInitBug()) {
     if (ProgInfo.NumSGPR > AMDGPUSubtarget::FIXED_SGPR_COUNT_FOR_INIT_BUG) {
       LLVMContext &Ctx = MF.getFunction()->getContext();
void AMDGPUAsmPrinter::EmitAmdKernelCodeT(const MachineFunction &MF,
   if (MFI->hasGridWorkgroupCountZ()) {
     header.code_properties |=
       AMD_CODE_PROPERTY_ENABLE_SGPR_GRID_WORKGROUP_COUNT_Z;
   }
 
   if (MFI->hasDispatchPtr())
     header.code_properties |= AMD_CODE_PROPERTY_ENABLE_SGPR_DISPATCH_PTR;
 
+  if (STM.debuggerSupported())
+    header.code_properties |= AMD_CODE_PROPERTY_IS_DEBUG_SUPPORTED;
+
   if (STM.isXNACKEnabled())
     header.code_properties |= AMD_CODE_PROPERTY_IS_XNACK_SUPPORTED;
 
   header.kernarg_segment_byte_size = MFI->ABIArgOffset;
   header.wavefront_sgpr_count = KernelInfo.NumSGPR;
   header.workitem_vgpr_count = KernelInfo.NumVGPR;
   header.workitem_private_segment_byte_size = KernelInfo.ScratchSize;
   header.workgroup_group_segment_byte_size = KernelInfo.LDSSize;
   header.reserved_vgpr_first = KernelInfo.ReservedVGPRFirst;
   header.reserved_vgpr_count = KernelInfo.ReservedVGPRCount;
+  header.debug_wavefront_private_segment_offset_sgpr =
+    KernelInfo.DebuggerWavefrontPrivateSegmentOffsetSGPR;
+  header.debug_private_segment_buffer_sgpr =
+    KernelInfo.DebuggerPrivateSegmentBufferSGPR;
 
   AMDGPUTargetStreamer *TS =
       static_cast<AMDGPUTargetStreamer *>(OutStreamer->getTargetStreamer());
 
   OutStreamer->SwitchSection(getObjFileLowering().getTextSection());
   TS->EmitAMDKernelCodeT(header);
 }
 

lib/Target/AMDGPU/AMDGPUSubtarget.h

private:
   bool HasSMemRealTime;
   bool Has16BitInsts;
   bool FeatureDisable;
   int LDSBankCount;
   unsigned IsaVersion;
   bool EnableSIScheduler;
   bool DebuggerInsertNops;
   bool DebuggerReserveTrapVGPRs;
+  bool DebuggerEmitPrologue;
 
   std::unique_ptr<AMDGPUFrameLowering> FrameLowering;
   std::unique_ptr<AMDGPUTargetLowering> TLInfo;
   std::unique_ptr<AMDGPUInstrInfo> InstrInfo;
   std::unique_ptr<GISelAccessor> GISel;
   InstrItineraryData InstrItins;
   Triple TargetTriple;
 
public:
   bool isTargetELF() const {
     return false;
   }
 
   bool enableSIScheduler() const {
     return EnableSIScheduler;
   }
 
+  bool debuggerSupported() const {
+    return debuggerInsertNops() && debuggerReserveTrapVGPRs()
+             && debuggerEmitPrologue();
+  }
+
   bool debuggerInsertNops() const {
     return DebuggerInsertNops;
   }
 
   bool debuggerReserveTrapVGPRs() const {
     return DebuggerReserveTrapVGPRs;
   }
 
+  bool debuggerEmitPrologue() const {
+    return DebuggerEmitPrologue;
+  }
+
   bool dumpCode() const {
     return DumpCode;
   }
   bool r600ALUEncoding() const {
     return R600ALUInst;
   }
   bool isAmdHsaOS() const {
     return TargetTriple.getOS() == Triple::AMDHSA;

lib/Target/AMDGPU/AMDGPUSubtarget.cpp

: AMDGPUGenSubtargetInfo(TT, GPU, FS),
       LocalMemorySize(0), MaxPrivateElementSize(0),
       EnableVGPRSpilling(false), SGPRInitBug(false), IsGCN(false),
       GCN1Encoding(false), GCN3Encoding(false), CIInsts(false),
       HasSMemRealTime(false), Has16BitInsts(false),
       LDSBankCount(0),
       IsaVersion(ISAVersion0_0_0),
       EnableSIScheduler(false),
       DebuggerInsertNops(false), DebuggerReserveTrapVGPRs(false),
+      DebuggerEmitPrologue(false),
       FrameLowering(nullptr),
       GISel(),
       InstrItins(getInstrItineraryForCPU(GPU)), TargetTriple(TT) {
 
   initializeSubtargetDependencies(TT, GPU, FS);
 
   const unsigned MaxStackAlign = 64 * 16; // Maximum stack alignment (long16)
 

lib/Target/AMDGPU/AMDGPUTargetMachine.cpp

 
 //===----------------------------------------------------------------------===//
 // GCN Pass Setup
 //===----------------------------------------------------------------------===//
 
 bool GCNPassConfig::addPreISel() {
   AMDGPUPassConfig::addPreISel();
 
+  const AMDGPUTargetMachine &TM = getAMDGPUTargetMachine();
   // FIXME: We need to run a pass to propagate the attributes when calls are
   // supported.
-  addPass(&AMDGPUAnnotateKernelFeaturesID);
+  addPass(createAMDGPUAnnotateKernelFeaturesPass(&TM));
   addPass(createStructurizeCFGPass(true)); // true -> SkipUniformRegions
   addPass(createSinkingPass());
   addPass(createSITypeRewriter());
   addPass(createAMDGPUAnnotateUniformValues());
   addPass(createSIAnnotateControlFlowPass());
 
   return false;
 }

lib/Target/AMDGPU/SIFrameLowering.h

public:
   ~SIFrameLowering() override {}
 
   void emitPrologue(MachineFunction &MF,
                     MachineBasicBlock &MBB) const override;
 
   void processFunctionBeforeFrameFinalized(
     MachineFunction &MF,
     RegScavenger *RS = nullptr) const override;
+
+private:
+  /// \brief Emits debugger prologue.
+  void emitDebuggerPrologue(MachineFunction &MF, MachineBasicBlock &MBB) const;
 };
 
 }
 
 #endif

lib/Target/AMDGPU/SIFrameLowering.cpp

 
 static ArrayRef<MCPhysReg> getAllSGPRs() {
   return makeArrayRef(AMDGPU::SGPR_32RegClass.begin(),
                       AMDGPU::SGPR_32RegClass.getNumRegs());
 }
 
 void SIFrameLowering::emitPrologue(MachineFunction &MF,
                                    MachineBasicBlock &MBB) const {
+  // Emit debugger prologue if "amdgpu-debugger-emit-prologue" attribute was
+  // specified.
+  const AMDGPUSubtarget &ST = MF.getSubtarget<AMDGPUSubtarget>();
+  if (ST.debuggerEmitPrologue())
+    emitDebuggerPrologue(MF, MBB);
+
   if (!MF.getFrameInfo()->hasStackObjects())
     return;
 
   assert(&MF.front() == &MBB && "Shrink-wrapping not yet supported");
 
   SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();
 
   // If we only have SGPR spills, we won't actually be using scratch memory
   // since these spill to VGPRs.
   //
   // FIXME: We should be cleaning up these unused SGPR spill frame indices
   // somewhere.
   if (hasOnlySGPRSpills(MFI, MF.getFrameInfo()))
     return;
 
   const SIInstrInfo *TII =
       static_cast<const SIInstrInfo *>(MF.getSubtarget().getInstrInfo());
   const SIRegisterInfo *TRI = &TII->getRegisterInfo();
-  const AMDGPUSubtarget &ST = MF.getSubtarget<AMDGPUSubtarget>();
   MachineRegisterInfo &MRI = MF.getRegInfo();
   MachineBasicBlock::iterator I = MBB.begin();
 
   // We need to insert initialization of the scratch resource descriptor.
   unsigned ScratchRsrcReg = MFI->getScratchRSrcReg();
   assert(ScratchRsrcReg != AMDGPU::NoRegister);
 
   unsigned ScratchWaveOffsetReg = MFI->getScratchWaveOffsetReg();
if (MFI->hasFlatScratchInit()) {
 
     // TODO: If we know we don't have flat instructions earlier, we can omit
     // this from the input registers.
     //
     // TODO: We only need to know if we access scratch space through a flat
     // pointer. Because we only detect if flat instructions are used at all,
     // this will be used more often than necessary on VI.
 
+    // Debug location must be unknown since the first debug location is used to
+    // determine the end of the prologue.
     DebugLoc DL;
 
     unsigned FlatScratchInitReg
       = TRI->getPreloadedValue(MF, SIRegisterInfo::FLAT_SCRATCH_INIT);
 
     MRI.addLiveIn(FlatScratchInitReg);
     MBB.addLiveIn(FlatScratchInitReg);
 
void SIFrameLowering::processFunctionBeforeFrameFinalized(
 
   if (MayNeedScavengingEmergencySlot) {
     int ScavengeFI = MFI->CreateSpillStackObject(
       AMDGPU::SGPR_32RegClass.getSize(),
       AMDGPU::SGPR_32RegClass.getAlignment());
     RS->addScavengingFrameIndex(ScavengeFI);
   }
 }
+
+void SIFrameLowering::emitDebuggerPrologue(MachineFunction &MF,
+                                           MachineBasicBlock &MBB) const {
+  const AMDGPUSubtarget &ST = MF.getSubtarget<AMDGPUSubtarget>();
+  const SIInstrInfo *TII = static_cast<const SIInstrInfo*>(ST.getInstrInfo());
+  const SIRegisterInfo *TRI = &TII->getRegisterInfo();
+  const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();
+
+  MachineBasicBlock::iterator I = MBB.begin();
+  DebugLoc DL;
+
+  // For each dimension:
+  for (unsigned i = 0; i < 3; ++i) {
+    // Get work group ID SGPR, and make it live-in again.
+    unsigned WorkGroupIDSGPR = MFI->getWorkGroupIDSGPR(i);
+    MF.getRegInfo().addLiveIn(WorkGroupIDSGPR);
+    MBB.addLiveIn(WorkGroupIDSGPR);
+
+    // Since SGPRs are spilled into VGPRs, copy work group ID SGPR to VGPR in
+    // order to spill it to scratch.
+    unsigned WorkGroupIDVGPR =
+      MF.getRegInfo().createVirtualRegister(&AMDGPU::VGPR_32RegClass);
+    BuildMI(MBB, I, DL, TII->get(AMDGPU::V_MOV_B32_e32), WorkGroupIDVGPR)
+      .addReg(WorkGroupIDSGPR);
+
+    // Spill work group ID.
+    int WorkGroupIDObjectIdx = MFI->getDebuggerWorkGroupIDStackObjectIndex(i);
+    TII->storeRegToStackSlot(MBB, I, WorkGroupIDVGPR, false,
+      WorkGroupIDObjectIdx, &AMDGPU::VGPR_32RegClass, TRI);
+
+    // Get work item ID VGPR, and make it live-in again.
+    unsigned WorkItemIDVGPR = MFI->getWorkItemIDVGPR(i);
+    MF.getRegInfo().addLiveIn(WorkItemIDVGPR);
+    MBB.addLiveIn(WorkItemIDVGPR);
+
+    // Spill work item ID.
+    int WorkItemIDObjectIdx = MFI->getDebuggerWorkItemIDStackObjectIndex(i);
+    TII->storeRegToStackSlot(MBB, I, WorkItemIDVGPR, false,
+      WorkItemIDObjectIdx, &AMDGPU::VGPR_32RegClass, TRI);
+  }
+}

lib/Target/AMDGPU/SIISelLowering.cpp

SDValue SITargetLowering::LowerFormalArguments(
   if (Subtarget->isAmdHsaOS() && AMDGPU::isShader(CallConv)) {
     const Function *Fn = MF.getFunction();
     DiagnosticInfoUnsupported NoGraphicsHSA(
         *Fn, "unsupported non-compute shaders with HSA", DL.getDebugLoc());
     DAG.getContext()->diagnose(NoGraphicsHSA);
     return SDValue();
   }
 
+  // Create stack objects that are used for emitting debugger prologue if
+  // "amdgpu-debugger-emit-prologue" attribute was specified.
+  //
+  // Debugger prologue writes work group IDs and work item IDs to scratch memory
+  // at fixed location in the following format:
+  //   offset 0:  work group ID x
+  //   offset 4:  work group ID y
+  //   offset 8:  work group ID z
+  //   offset 16: work item ID x
+  //   offset 20: work item ID y
+  //   offset 24: work item ID z
+  if (ST.debuggerEmitPrologue()) {
+    int ObjectIdx = 0;
+    // For each dimension:
+    for (unsigned i = 0; i < 3; ++i) {
+      // Create fixed stack object for work group ID.
+      ObjectIdx = MF.getFrameInfo()->CreateFixedObject(4, i * 4, true);
+      Info->setDebuggerWorkGroupIDStackObjectIndex(i, ObjectIdx);
+      // Create fixed stack object for work item ID.
+      ObjectIdx = MF.getFrameInfo()->CreateFixedObject(4, i * 4 + 16, true);
+      Info->setDebuggerWorkItemIDStackObjectIndex(i, ObjectIdx);
+    }
+  }
+
   SmallVector<ISD::InputArg, 16> Splits;
   BitVector Skipped(Ins.size());
 
   for (unsigned i = 0, e = Ins.size(), PSInputNum = 0; i != e; ++i) {
     const ISD::InputArg &Arg = Ins[i];
 
     // First check if it's a PS input addr
     if (CallConv == CallingConv::AMDGPU_PS && !Arg.Flags.isInReg() &&

lib/Target/AMDGPU/SIMachineFunctionInfo.h

class SIMachineFunctionInfo final : public AMDGPUMachineFunction {
   // Graphics info.
   unsigned PSInputAddr;
   bool ReturnsVoid;
 
   unsigned MaximumWorkGroupSize;
 
   // Number of reserved VGPRs for trap handler usage.
   unsigned DebuggerReserveTrapVGPRCount;
+  // Stack object indices for work group IDs.
+  int DebuggerWorkGroupIDStackObjectIndices[3];
+  // Stack object indices for work item IDs.
+  int DebuggerWorkItemIDStackObjectIndices[3];
 
 public:
   // FIXME: Make private
   unsigned LDSWaveSpillSize;
   unsigned PSInputEna;
   std::map<unsigned, unsigned> LaneVGPRs;
   unsigned ScratchOffsetReg;
   unsigned NumUserSGPRs;
public:
   void setIfReturnsVoid(bool Value) {
     ReturnsVoid = Value;
   }
 
   unsigned getDebuggerReserveTrapVGPRCount() const {
     return DebuggerReserveTrapVGPRCount;
   }
 
+  /// \returns Stack object index for \p Dim's work group ID.
+  int getDebuggerWorkGroupIDStackObjectIndex(unsigned Dim) const {
+    assert(Dim < 3);
+    return DebuggerWorkGroupIDStackObjectIndices[Dim];
+  }
+
+  /// \brief Sets stack object index for \p Dim's work group ID to \p ObjectIdx.
+  void setDebuggerWorkGroupIDStackObjectIndex(unsigned Dim, int ObjectIdx) {
+    assert(Dim < 3);
+    DebuggerWorkGroupIDStackObjectIndices[Dim] = ObjectIdx;
+  }
+
+  /// \returns Stack object index for \p Dim's work item ID.
+  int getDebuggerWorkItemIDStackObjectIndex(unsigned Dim) const {
+    assert(Dim < 3);
+    return DebuggerWorkItemIDStackObjectIndices[Dim];
+  }
+
+  /// \brief Sets stack object index for \p Dim's work item ID to \p ObjectIdx.
+  void setDebuggerWorkItemIDStackObjectIndex(unsigned Dim, int ObjectIdx) {
+    assert(Dim < 3);
+    DebuggerWorkItemIDStackObjectIndices[Dim] = ObjectIdx;
+  }
+
+  /// \returns SGPR used for \p Dim's work group ID.
+  unsigned getWorkGroupIDSGPR(unsigned Dim) const {
+    switch (Dim) {
+    case 0:
+      assert(hasWorkGroupIDX());
+      return WorkGroupIDXSystemSGPR;
+    case 1:
+      assert(hasWorkGroupIDY());
+      return WorkGroupIDYSystemSGPR;
+    case 2:
+      assert(hasWorkGroupIDZ());
+      return WorkGroupIDZSystemSGPR;
+    }
+    llvm_unreachable("unexpected dimension");
+  }
+
+  /// \returns VGPR used for \p Dim' work item ID.
+  unsigned getWorkItemIDVGPR(unsigned Dim) const {
+    switch (Dim) {
+    case 0:
+      assert(hasWorkItemIDX());
+      return AMDGPU::VGPR0;
+    case 1:
+      assert(hasWorkItemIDY());
+      return AMDGPU::VGPR1;
+    case 2:
+      assert(hasWorkItemIDZ());
+      return AMDGPU::VGPR2;
+    }
+    llvm_unreachable("unexpected dimension");
+  }
+
   unsigned getMaximumWorkGroupSize(const MachineFunction &MF) const;
 };
 
 } // End namespace llvm
 
 #endif

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

: AMDGPUMachineFunction(MF),
     WorkGroupIDYSystemSGPR(AMDGPU::NoRegister),
     WorkGroupIDZSystemSGPR(AMDGPU::NoRegister),
     WorkGroupInfoSystemSGPR(AMDGPU::NoRegister),
     PrivateSegmentWaveByteOffsetSystemSGPR(AMDGPU::NoRegister),
     PSInputAddr(0),
     ReturnsVoid(true),
     MaximumWorkGroupSize(0),
     DebuggerReserveTrapVGPRCount(0),
+    DebuggerWorkGroupIDStackObjectIndices{0, 0, 0},
+    DebuggerWorkItemIDStackObjectIndices{0, 0, 0},
     LDSWaveSpillSize(0),
     PSInputEna(0),
     NumUserSGPRs(0),
     NumSystemSGPRs(0),
     HasSpilledSGPRs(false),
     HasSpilledVGPRs(false),
     HasNonSpillStackObjects(false),
     HasFlatInstructions(false),
SIMachineFunctionInfo::SIMachineFunctionInfo(const MachineFunction &MF)
   const MachineFrameInfo *FrameInfo = MF.getFrameInfo();
 
   if (!AMDGPU::isShader(F->getCallingConv())) {
     KernargSegmentPtr = true;
     WorkGroupIDX = true;
     WorkItemIDX = true;
   }
 
   if (F->hasFnAttribute("amdgpu-work-group-id-y"))
     WorkGroupIDY = true;
 
   if (F->hasFnAttribute("amdgpu-work-group-id-z"))
     WorkGroupIDZ = true;
 
   if (F->hasFnAttribute("amdgpu-work-item-id-y"))
     WorkItemIDY = true;
 
   if (F->hasFnAttribute("amdgpu-work-item-id-z"))
     WorkItemIDZ = true;
 
   // X, XY, and XYZ are the only supported combinations, so make sure Y is
   // enabled if Z is.
   if (WorkItemIDZ)
     WorkItemIDY = true;
 
   bool MaySpill = ST.isVGPRSpillingEnabled(*F);
   bool HasStackObjects = FrameInfo->hasStackObjects();
 
   if (HasStackObjects || MaySpill)
     PrivateSegmentWaveByteOffset = true;
 
   if (ST.isAmdHsaOS()) {
     if (HasStackObjects || MaySpill)
       PrivateSegmentBuffer = true;
 

test/CodeGen/AMDGPU/debugger-emit-prologue.ll

+; RUN: llc -O0 -mtriple=amdgcn--amdhsa -mcpu=fiji -mattr=+amdgpu-debugger-emit-prologue -verify-machineinstrs < %s | FileCheck %s
+
+; CHECK: debug_wavefront_private_segment_offset_sgpr = [[SOFF:[0-9]+]]
+; CHECK: debug_private_segment_buffer_sgpr = [[SREG:[0-9]+]]
+
+; CHECK: v_mov_b32_e32 [[WGIDX:v[0-9]+]], s{{[0-9]+}}
+; CHECK: buffer_store_dword [[WGIDX]], off, s[{{[0-9]+:[0-9]+}}], s[[SOFF]]
+; CHECK: buffer_store_dword v0, off, s[{{[0-9]+:[0-9]+}}], s[[SOFF]] offset:16
+
+; CHECK: v_mov_b32_e32 [[WGIDY:v[0-9]+]], s{{[0-9]+}}
+; CHECK: buffer_store_dword [[WGIDY]], off, s[{{[0-9]+:[0-9]+}}], s[[SOFF]] offset:4
+; CHECK: buffer_store_dword v1, off, s[{{[0-9]+:[0-9]+}}], s[[SOFF]] offset:20
+
+; CHECK: v_mov_b32_e32 [[WGIDZ:v[0-9]+]], s{{[0-9]+}}
+; CHECK: buffer_store_dword [[WGIDZ]], off, s[{{[0-9]+:[0-9]+}}], s[[SOFF]] offset:8
+; CHECK: buffer_store_dword v2, off, s[{{[0-9]+:[0-9]+}}], s[[SOFF]] offset:24
+
+; CHECK: DebuggerWavefrontPrivateSegmentOffsetSGPR: s[[SOFF]]
+; CHECK: DebuggerPrivateSegmentBufferSGPR: s[[SREG]]
+
+; Function Attrs: nounwind
+define void @test(i32 addrspace(1)* %A) #0 !dbg !12 {
+entry:
+  %A.addr = alloca i32 addrspace(1)*, align 4
+  store i32 addrspace(1)* %A, i32 addrspace(1)** %A.addr, align 4
+  call void @llvm.dbg.declare(metadata i32 addrspace(1)** %A.addr, metadata !17, metadata !18), !dbg !19
+  %0 = load i32 addrspace(1)*, i32 addrspace(1)** %A.addr, align 4, !dbg !20
+  %arrayidx = getelementptr inbounds i32, i32 addrspace(1)* %0, i32 0, !dbg !20
+  store i32 1, i32 addrspace(1)* %arrayidx, align 4, !dbg !21
+  %1 = load i32 addrspace(1)*, i32 addrspace(1)** %A.addr, align 4, !dbg !22
+  %arrayidx1 = getelementptr inbounds i32, i32 addrspace(1)* %1, i32 1, !dbg !22
+  store i32 2, i32 addrspace(1)* %arrayidx1, align 4, !dbg !23
+  %2 = load i32 addrspace(1)*, i32 addrspace(1)** %A.addr, align 4, !dbg !24
+  %arrayidx2 = getelementptr inbounds i32, i32 addrspace(1)* %2, i32 2, !dbg !24
+  store i32 3, i32 addrspace(1)* %arrayidx2, align 4, !dbg !25
+  ret void, !dbg !26
+}
+
+; Function Attrs: nounwind readnone
+declare void @llvm.dbg.declare(metadata, metadata, metadata) #1
+
+attributes #0 = { nounwind "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="fiji" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { nounwind readnone }
+
+!llvm.dbg.cu = !{!0}
+!opencl.kernels = !{!3}
+!llvm.module.flags = !{!9, !10}
+!llvm.ident = !{!11}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.9.0 (trunk 269772)", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
+!1 = !DIFile(filename: "test01.cl", directory: "/home/kzhuravl/Lightning/testing")
+!2 = !{}
+!3 = !{void (i32 addrspace(1)*)* @test, !4, !5, !6, !7, !8}
+!4 = !{!"kernel_arg_addr_space", i32 1}
+!5 = !{!"kernel_arg_access_qual", !"none"}
+!6 = !{!"kernel_arg_type", !"int*"}
+!7 = !{!"kernel_arg_base_type", !"int*"}
+!8 = !{!"kernel_arg_type_qual", !""}
+!9 = !{i32 2, !"Dwarf Version", i32 2}
+!10 = !{i32 2, !"Debug Info Version", i32 3}
+!11 = !{!"clang version 3.9.0 (trunk 269772)"}
+!12 = distinct !DISubprogram(name: "test", scope: !1, file: !1, line: 1, type: !13, isLocal: false, isDefinition: true, scopeLine: 1, flags: DIFlagPrototyped, isOptimized: false, unit: !0, variables: !2)
+!13 = !DISubroutineType(types: !14)
+!14 = !{null, !15}
+!15 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !16, size: 64, align: 32)
+!16 = !DIBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed)
+!17 = !DILocalVariable(name: "A", arg: 1, scope: !12, file: !1, line: 1, type: !15)
+!18 = !DIExpression()
+!19 = !DILocation(line: 1, column: 30, scope: !12)
+!20 = !DILocation(line: 2, column: 3, scope: !12)
+!21 = !DILocation(line: 2, column: 8, scope: !12)
+!22 = !DILocation(line: 3, column: 3, scope: !12)
+!23 = !DILocation(line: 3, column: 8, scope: !12)
+!24 = !DILocation(line: 4, column: 3, scope: !12)
+!25 = !DILocation(line: 4, column: 8, scope: !12)
+!26 = !DILocation(line: 5, column: 1, scope: !12)