[AMDGPU] Add `.workgroup_processor_mode` to v5 MD

[Pierre-vh]

Summary

Adds Workgroup Processor Mode (WGP) to the HSA Metadata for Code Object v5/GFX10+.
The field is already present as an asm directive and in the compute program resource register but is also needed in the MD.

[foad]

Adds Workgroup Processor Mode (WGM)

Is that a typo for "WGP"?

[Pierre-vh]

In D139931#3991751, @foad wrote:

Adds Workgroup Processor Mode (WGM)

Is that a typo for "WGP"?

Indeed
(note the patch is still a WIP - tests are lacking,I should have included it in the title)

[kzhuravl]

Also needs a test.

llvm/include/llvm/Support/AMDHSAKernelDescriptor.h

165–166 ↗

(On Diff #482445)

These changes are not needed. WGP_MODE is recorded in COMPUTE_PGM_RSRC1.

llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

417–419 ↗

(On Diff #482445)

These changes are not needed.

[arsenm]

llvm/lib/Target/AMDGPU/AMDGPUHSAMetadataStreamer.cpp

881

Can you hide this generation check in a supportsWGP predicate in the subtarget

[Pierre-vh]

Address comments, add test

[kzhuravl]

llvm/test/CodeGen/AMDGPU/hsa-metadata-workgroup-processor-mode-v5.ll

7–8

Can you also add checks for amdhsa_workgroup_processor_mode ASM directive?

[Pierre-vh]

Add extra test check

[kzhuravl]

lgtm, thanks

llvm/docs/AMDGPUUsage.rst

"amdhsa.version"  sequence of    Required  - The first integer is the major
                                                   version. Currently 2.
      ================= ============== ========= =======================================
 
 ..
 
   .. table:: AMDHSA Code Object V5 Kernel Metadata Map Additions
      :name: amdgpu-amdhsa-code-object-kernel-metadata-map-table-v5
 
-     ===================== ============= ========== =======================================
-     String Key            Value Type     Required? Description
-     ===================== ============= ========== =======================================
-     ".uses_dynamic_stack" boolean                  Indicates if the generated machine code
+     ============================= ============= ========== =======================================
+     String Key                    Value Type     Required? Description
+     ============================= ============= ========== =======================================
+     ".uses_dynamic_stack"         boolean                  Indicates if the generated machine code
                                                             is using a dynamically sized stack.
-     ===================== ============= ========== =======================================
+     ".workgroup_processor_mode"   boolean                  (GFX10+) Controls ENABLE_WGP_MODE in
+                                                            :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
+     ============================= ============= ========== =======================================
 
 ..
 
   .. table:: AMDHSA Code Object V5 Kernel Argument Metadata Map Additions and Changes
      :name: amdgpu-amdhsa-code-object-kernel-argument-metadata-map-table-v5
 
      ====================== ============== ========= ================================
      String Key             Value Type     Required? Description

llvm/lib/BinaryFormat/AMDGPUMetadataVerifier.cpp

if (!verifyIntegerEntry(KernelMap, ".kernarg_segment_size", true))
     return false;
   if (!verifyIntegerEntry(KernelMap, ".group_segment_fixed_size", true))
     return false;
   if (!verifyIntegerEntry(KernelMap, ".private_segment_fixed_size", true))
     return false;
   if (!verifyScalarEntry(KernelMap, ".uses_dynamic_stack", false,
                          msgpack::Type::Boolean))
     return false;
+  if (!verifyIntegerEntry(KernelMap, ".workgroup_processor_mode", false))
+    return false;
   if (!verifyIntegerEntry(KernelMap, ".kernarg_segment_align", true))
     return false;
   if (!verifyIntegerEntry(KernelMap, ".wavefront_size", true))
     return false;
   if (!verifyIntegerEntry(KernelMap, ".sgpr_count", true))
     return false;
   if (!verifyIntegerEntry(KernelMap, ".vgpr_count", true))
     return false;

llvm/lib/Target/AMDGPU/AMDGPUHSAMetadataStreamer.cpp

Kern[".kernarg_segment_size"] = Kern.getDocument()->getNode(
       STM.getKernArgSegmentSize(F, MaxKernArgAlign));
   Kern[".group_segment_fixed_size"] =
       Kern.getDocument()->getNode(ProgramInfo.LDSSize);
   Kern[".private_segment_fixed_size"] =
       Kern.getDocument()->getNode(ProgramInfo.ScratchSize);
   if (AMDGPU::getAmdhsaCodeObjectVersion() >= 5)
     Kern[".uses_dynamic_stack"] =
         Kern.getDocument()->getNode(ProgramInfo.DynamicCallStack);
+  if (AMDGPU::getAmdhsaCodeObjectVersion() >= 5 && STM.supportsWGP())
+    Kern[".workgroup_processor_mode"] =
+        Kern.getDocument()->getNode(ProgramInfo.WgpMode);
 
   // FIXME: The metadata treats the minimum as 16?
   Kern[".kernarg_segment_align"] =
       Kern.getDocument()->getNode(std::max(Align(4), MaxKernArgAlign).value());
   Kern[".wavefront_size"] =
       Kern.getDocument()->getNode(STM.getWavefrontSize());
   Kern[".sgpr_count"] = Kern.getDocument()->getNode(ProgramInfo.NumSGPR);
   Kern[".vgpr_count"] = Kern.getDocument()->getNode(ProgramInfo.NumVGPR);

llvm/lib/Target/AMDGPU/GCNSubtarget.h

public:
 
   unsigned getConstantBusLimit(unsigned Opcode) const;
 
   /// Returns if the result of this instruction with a 16-bit result returned in
   /// a 32-bit register implicitly zeroes the high 16-bits, rather than preserve
   /// the original value.
   bool zeroesHigh16BitsOfDest(unsigned Opcode) const;
 
+  bool supportsWGP() const { return getGeneration() >= GFX10; }
+
   bool hasIntClamp() const {
     return HasIntClamp;
   }
 
   bool hasFP64() const {
     return FP64;
   }
 

llvm/test/CodeGen/AMDGPU/hsa-metadata-workgroup-processor-mode-v5.ll

+; RUN: llc -mtriple=amdgcn-amd-amdhsa --amdhsa-code-object-version=5 -mcpu=gfx1010 -mattr=+cumode < %s | FileCheck -check-prefix=GFX10 %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa --amdhsa-code-object-version=5 -mcpu=gfx1010 < %s | FileCheck -check-prefix=GFX10-CU %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa --amdhsa-code-object-version=5 -mcpu=gfx1100 -mattr=+cumode < %s | FileCheck -check-prefix=GFX10 %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa --amdhsa-code-object-version=5 -mcpu=gfx1100 < %s | FileCheck -check-prefix=GFX10-CU %s
+
+; GFX10:    .amdhsa_workgroup_processor_mode 0
+; GFX10:    .workgroup_processor_mode: 0
+; GFX10-CU: .amdhsa_workgroup_processor_mode 1
+; GFX10-CU: .workgroup_processor_mode: 1
+
+define amdgpu_kernel void @wavefrontsize() {
+entry:
+  ret void
+}