[NFC][AMDGPU] Refactor AMDGPUDisassembler

[scott.linder]

Summary

Clean up ahead of a patch to fix bugs in the AMDGPUDisassembler.

Use split-file to simplify and extend existing kernel-descriptor
disassembly tests.

Add a comment to AMDHSAKernelDescriptor.h, as at least one small set
towards keeping all kernel-descriptor sensitive code in sync.

[arsenm]

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

1915

What are these comments?

[scott.linder]

llvm/test/tools/llvm-objdump/ELF/AMDGPU/lit.local.cfg

5

I'm not certain I struck the best balance here between "hiding mostly irrelevant details" and "obfuscating the RUN: lines of the KD tests", but I found it hard to tell at a glance what each test case was doing without at least some of the factoring I ended up with here (and the RUN lines were getting long, even with continuations across lines)

Suggestions on this are welcome!

[scott.linder]

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

1915

They suppress clang-tidy diagnostics, with the idea here being that it is worth using the COMPUTE_PGM_RSRC3 identifier from the KD specification directly in the method name, even though it breaks the normal LLVM style rules

[arsenm]

llvm/test/tools/llvm-objdump/ELF/AMDGPU/kd-vgpr.s

3

Do you really need to rm first? I don't remember seeing any other test do that

[kzhuravl]

LGTM, thanks!

[scott.linder]

llvm/test/tools/llvm-objdump/ELF/AMDGPU/kd-vgpr.s

3

I'm also curious where this originates, as it seems around 40% of the uses of split-file are preceded by rm -rf:

$ rg --count-matches split-file | awk -F: '{ sum+=$2 } END { print sum }'
519
$ rg --count-matches 'rm -rf.*split-file' | awk -F: '{ sum+=$2 } END { print sum }'
202

I will remove the rm from the version I land, but it would be nice to understand why it is sometimes present.

[cmtice]

This commit appears to be causing our bootstrap builds to fail, while running these tests. The error we get for kd-vgpr.s is
[ 5] ;--- 1.s
[ 6] ; RUN: %assemble -mcpu=gfx908 <1.s >1.o [FAIL]
%assemble -mcpu=gfx908 <1.s >1.o
bash: line 0: fg: no job control
Command failed: exit status 1

The error we get for kd-sgpr.s is
[ 6] ;; Only set next_free_sgpr.
[ 7] ; RUN: %assemble -mcpu=gfx908 <1.s >1.o [FAIL]
%assemble -mcpu=gfx908 <1.s >1.o
bash: line 0: fg: no job control
Command failed: exit status 1

Could you revert this commit until these issues are fixed?

[scott.linder]

Reverted in 552539bdaccc187ce1781f579a8076e4c917fa28

[scott.linder]

I am not sure I understand the exact root cause of the failures, but I do know
that bash sometimes interprets % as starting a "job specifier", so I tried
removing it from the substitution names.

I also removed two substitutions that actually belong in D128014.

Does anyone have a better idea of why this might be failing? Is there a way
for me to reproduce the bootstrap build locally to test?

[foad]

In D130105#3804814, @cmtice wrote:

[ 6] ; RUN: %assemble -mcpu=gfx908 <1.s >1.o [FAIL]
%assemble -mcpu=gfx908 <1.s >1.o
bash: line 0: fg: no job control

This seems pretty clear that lit has not applied the substitution for %assemble before passing the command line to the shell. But I don't know how that could have happened. @cmtice do you have a link to your actual build? Is there anything unusual about it?

[arsenm]

Was this ever re-committed?

[scott.linder]

In D130105#3864667, @foad wrote:

In D130105#3804814, @cmtice wrote:

[ 6] ; RUN: %assemble -mcpu=gfx908 <1.s >1.o [FAIL]
%assemble -mcpu=gfx908 <1.s >1.o
bash: line 0: fg: no job control

This seems pretty clear that lit has not applied the substitution for %assemble before passing the command line to the shell. But I don't know how that could have happened. @cmtice do you have a link to your actual build? Is there anything unusual about it?

I didn't make the connection that the lit substitutions would apply before bash sees the result, somehow. I am also curious why the substitution would not apply correctly,

In D130105#4441463, @arsenm wrote:

Was this ever re-committed?

No, I never reproduced the issue and didn't have the time to keep chasing it down

[scott.linder]

Remove the lit substitutions as I can't work out why they were failing

[MaskRay]

llvm/test/tools/llvm-objdump/ELF/AMDGPU/kd-sgpr.s

7–49

<1.s => < 1.s. The same applies to >1.o

10

Use cmp for binary blobs.

[scott.linder]

• Add spaces around bash redirection operators
• Use cmp for binary files
• Update commit message to remove mention of lit substitutions

llvm/include/llvm/Support/AMDHSAKernelDescriptor.h

 //===--- AMDHSAKernelDescriptor.h -----------------------------*- C++ -*---===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 /// \file
 /// AMDHSA kernel descriptor definitions. For more information, visit
 /// https://llvm.org/docs/AMDGPUUsage.html#kernel-descriptor
+///
+/// \warning
+/// Any changes to this file should also be audited for corresponding changes
+/// needed in both the assembler and disassembler, namely:
+/// * AMDGPUAsmPrinter.{cpp,h}
+/// * AMDGPUTargetStreamer.{cpp,h}
+/// * AMDGPUDisassembler.{cpp,h}
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_SUPPORT_AMDHSAKERNELDESCRIPTOR_H
 #define LLVM_SUPPORT_AMDHSAKERNELDESCRIPTOR_H
 
 #include <cstddef>
 #include <cstdint>

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.h

public:
 
   /// Decode as directives that handle COMPUTE_PGM_RSRC2.
   /// \param FourByteBuffer - Bytes holding contents of COMPUTE_PGM_RSRC2.
   /// \param KdStream       - Stream to write the disassembled directives to.
   // NOLINTNEXTLINE(readability-identifier-naming)
   DecodeStatus decodeCOMPUTE_PGM_RSRC2(uint32_t FourByteBuffer,
                                        raw_string_ostream &KdStream) const;
 
+  /// Decode as directives that handle COMPUTE_PGM_RSRC3.
+  /// \param FourByteBuffer - Bytes holding contents of COMPUTE_PGM_RSRC3.
+  /// \param KdStream       - Stream to write the disassembled directives to.
+  // NOLINTNEXTLINE(readability-identifier-naming)
+  DecodeStatus decodeCOMPUTE_PGM_RSRC3(uint32_t FourByteBuffer,
+                                       raw_string_ostream &KdStream) const;
+
   DecodeStatus convertEXPInst(MCInst &MI) const;
   DecodeStatus convertVINTERPInst(MCInst &MI) const;
   DecodeStatus convertFMAanyK(MCInst &MI, int ImmLitIdx) const;
   DecodeStatus convertSDWAInst(MCInst &MI) const;
   DecodeStatus convertDPP8Inst(MCInst &MI) const;
   DecodeStatus convertMIMGInst(MCInst &MI) const;
   DecodeStatus convertVOP3DPPInst(MCInst &MI) const;
   DecodeStatus convertVOP3PDPPInst(MCInst &MI) const;

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

 
 bool AMDGPUDisassembler::hasArchitectedFlatScratch() const {
   return STI.getFeatureBits()[AMDGPU::FeatureArchitectedFlatScratch];
 }
 
 //===----------------------------------------------------------------------===//
 // AMDGPU specific symbol handling
 //===----------------------------------------------------------------------===//
+#define GET_FIELD(MASK) (AMDHSA_BITS_GET(FourByteBuffer, MASK))
 #define PRINT_DIRECTIVE(DIRECTIVE, MASK)                                       \
   do {                                                                         \
-    KdStream << Indent << DIRECTIVE " "                                        \
-             << ((FourByteBuffer & MASK) >> (MASK##_SHIFT)) << '\n';           \
+    KdStream << Indent << DIRECTIVE " " << GET_FIELD(MASK) << '\n';            \
   } while (0)
 
 // NOLINTNEXTLINE(readability-identifier-naming)
 MCDisassembler::DecodeStatus AMDGPUDisassembler::decodeCOMPUTE_PGM_RSRC1(
     uint32_t FourByteBuffer, raw_string_ostream &KdStream) const {
   using namespace amdhsa;
   StringRef Indent = "\t";
 
   // We cannot accurately backward compute #VGPRs used from
   // GRANULATED_WORKITEM_VGPR_COUNT. But we are concerned with getting the same
   // value of GRANULATED_WORKITEM_VGPR_COUNT in the reassembled binary. So we
   // simply calculate the inverse of what the assembler does.
 
   uint32_t GranulatedWorkitemVGPRCount =
-      (FourByteBuffer & COMPUTE_PGM_RSRC1_GRANULATED_WORKITEM_VGPR_COUNT) >>
-      COMPUTE_PGM_RSRC1_GRANULATED_WORKITEM_VGPR_COUNT_SHIFT;
+      GET_FIELD(COMPUTE_PGM_RSRC1_GRANULATED_WORKITEM_VGPR_COUNT);
 
   uint32_t NextFreeVGPR = (GranulatedWorkitemVGPRCount + 1) *
                           AMDGPU::IsaInfo::getVGPREncodingGranule(&STI);
 
   KdStream << Indent << ".amdhsa_next_free_vgpr " << NextFreeVGPR << '\n';
 
   // We cannot backward compute values used to calculate
   // GRANULATED_WAVEFRONT_SGPR_COUNT. Hence the original values for following
MCDisassembler::DecodeStatus AMDGPUDisassembler::decodeCOMPUTE_PGM_RSRC1(
   // are set to 0. So while disassembling we consider that:
   //
   // GRANULATED_WAVEFRONT_SGPR_COUNT
   //    = f(NEXT_FREE_SGPR + 0 + 0 + 0)
   //
   // The disassembler cannot recover the original values of those 3 directives.
 
   uint32_t GranulatedWavefrontSGPRCount =
-      (FourByteBuffer & COMPUTE_PGM_RSRC1_GRANULATED_WAVEFRONT_SGPR_COUNT) >>
-      COMPUTE_PGM_RSRC1_GRANULATED_WAVEFRONT_SGPR_COUNT_SHIFT;
+      GET_FIELD(COMPUTE_PGM_RSRC1_GRANULATED_WAVEFRONT_SGPR_COUNT);
 
   if (isGFX10Plus() && GranulatedWavefrontSGPRCount)
     return MCDisassembler::Fail;
 
   uint32_t NextFreeSGPR = (GranulatedWavefrontSGPRCount + 1) *
                           AMDGPU::IsaInfo::getSGPREncodingGranule(&STI);
 
   KdStream << Indent << ".amdhsa_reserve_vcc " << 0 << '\n';
PRINT_DIRECTIVE(".amdhsa_exception_int_div_zero",
                   COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_INT_DIVIDE_BY_ZERO);
 
   if (FourByteBuffer & COMPUTE_PGM_RSRC2_RESERVED0)
     return MCDisassembler::Fail;
 
   return MCDisassembler::Success;
 }
 
+// NOLINTNEXTLINE(readability-identifier-naming)
+MCDisassembler::DecodeStatus AMDGPUDisassembler::decodeCOMPUTE_PGM_RSRC3(
+    uint32_t FourByteBuffer, raw_string_ostream &KdStream) const {
+  using namespace amdhsa;
+  if (!isGFX10Plus() && FourByteBuffer) {
+    return MCDisassembler::Fail;
+  }
+  return MCDisassembler::Success;
+}
 #undef PRINT_DIRECTIVE
+#undef GET_FIELD
 
 MCDisassembler::DecodeStatus
 AMDGPUDisassembler::decodeKernelDescriptorDirective(
     DataExtractor::Cursor &Cursor, ArrayRef<uint8_t> Bytes,
     raw_string_ostream &KdStream) const {
 #define PRINT_DIRECTIVE(DIRECTIVE, MASK)                                       \
   do {                                                                         \
     KdStream << Indent << DIRECTIVE " "                                        \
case amdhsa::RESERVED1_OFFSET:
     for (int I = 0; I < 20; ++I) {
       if (ReservedBytes[I] != 0) {
         return MCDisassembler::Fail;
       }
     }
     return MCDisassembler::Success;
 
   case amdhsa::COMPUTE_PGM_RSRC3_OFFSET:
-    // COMPUTE_PGM_RSRC3
-    //  - Only set for GFX10, GFX6-9 have this to be 0.
-    //  - Currently no directives directly control this.
     FourByteBuffer = DE.getU32(Cursor);
-    if (!isGFX10Plus() && FourByteBuffer) {
-      return MCDisassembler::Fail;
-    }
-    return MCDisassembler::Success;
+    return decodeCOMPUTE_PGM_RSRC3(FourByteBuffer, KdStream);
 
   case amdhsa::COMPUTE_PGM_RSRC1_OFFSET:
     FourByteBuffer = DE.getU32(Cursor);
-    if (decodeCOMPUTE_PGM_RSRC1(FourByteBuffer, KdStream) ==
-        MCDisassembler::Fail) {
-      return MCDisassembler::Fail;
-    }
-    return MCDisassembler::Success;
+    return decodeCOMPUTE_PGM_RSRC1(FourByteBuffer, KdStream);
 
   case amdhsa::COMPUTE_PGM_RSRC2_OFFSET:
     FourByteBuffer = DE.getU32(Cursor);
-    if (decodeCOMPUTE_PGM_RSRC2(FourByteBuffer, KdStream) ==
-        MCDisassembler::Fail) {
-      return MCDisassembler::Fail;
-    }
-    return MCDisassembler::Success;
+    return decodeCOMPUTE_PGM_RSRC2(FourByteBuffer, KdStream);
 
   case amdhsa::KERNEL_CODE_PROPERTIES_OFFSET:
     using namespace amdhsa;
     TwoByteBuffer = DE.getU16(Cursor);
 
     if (!hasArchitectedFlatScratch())
       PRINT_DIRECTIVE(".amdhsa_user_sgpr_private_segment_buffer",
                       KERNEL_CODE_PROPERTY_ENABLE_SGPR_PRIVATE_SEGMENT_BUFFER);

llvm/test/tools/llvm-objdump/ELF/AMDGPU/kd-sgpr.s

 ;; Test disassembly for GRANULATED_WAVEFRONT_SGPR_COUNT in the kernel descriptor.
 
-; RUN: split-file %s %t.dir
-
-; RUN: llvm-mc %t.dir/1.s --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t1
-; RUN: llvm-objdump --disassemble-symbols=my_kernel_1.kd %t1 | tail -n +7 \
-; RUN: | llvm-mc --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t1-re-assemble
-; RUN: diff %t1 %t1-re-assemble
-
-; RUN: llvm-mc %t.dir/2.s --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t2
-; RUN: llvm-objdump --disassemble-symbols=my_kernel_2.kd %t2 | tail -n +7 \
-; RUN: | llvm-mc --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t2-re-assemble
-; RUN: diff %t2 %t2-re-assemble
-
-; RUN: llvm-mc %t.dir/3.s --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t3
-; RUN: llvm-objdump --disassemble-symbols=my_kernel_3.kd %t3 | tail -n +7 \
-; RUN: | llvm-mc --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t3-re-assemble
-; RUN: diff %t3 %t3-re-assemble
-
+; RUN: rm -rf %t && split-file %s %t && cd %t
 
 ;--- 1.s
 ;; Only set next_free_sgpr.
-.amdhsa_kernel my_kernel_1
+; RUN: %assemble -mcpu=gfx908 <1.s >1.o
+; RUN: %disassemble_kd 1.o | %tee_kd 1-disasm.s | FileCheck 1.s
+; RUN: %assemble -mcpu=gfx908 <1-disasm.s >1-disasm.o
+; RUN: diff 1.o 1-disasm.o
+; CHECK: .amdhsa_kernel kernel
+; CHECK-NEXT: .amdhsa_group_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_private_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_kernarg_size 0
+; CHECK-NEXT: .amdhsa_next_free_vgpr 4
+; CHECK-NEXT: .amdhsa_reserve_vcc 0
+; CHECK-NEXT: .amdhsa_reserve_flat_scratch 0
+; CHECK-NEXT: .amdhsa_reserve_xnack_mask 0
+; CHECK-NEXT: .amdhsa_next_free_sgpr 48
+; CHECK-NEXT: .amdhsa_float_round_mode_32 0
+; CHECK-NEXT: .amdhsa_float_round_mode_16_64 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_32 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_16_64 3
+; CHECK-NEXT: .amdhsa_dx10_clamp 1
+; CHECK-NEXT: .amdhsa_ieee_mode 1
+; CHECK-NEXT: .amdhsa_fp16_overflow 0
+; CHECK-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_x 1
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_y 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_z 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_info 0
+; CHECK-NEXT: .amdhsa_system_vgpr_workitem_id 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_invalid_op 0
+; CHECK-NEXT: .amdhsa_exception_fp_denorm_src 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_div_zero 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_overflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_underflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_inexact 0
+; CHECK-NEXT: .amdhsa_exception_int_div_zero 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_buffer 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_queue_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_id 0
+; CHECK-NEXT: .amdhsa_user_sgpr_flat_scratch_init 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_size 0
+; CHECK-NEXT: .amdhsa_uses_dynamic_stack 0
+; CHECK-NEXT: .end_amdhsa_kernel
+.amdhsa_kernel kernel
   .amdhsa_next_free_vgpr 0
   .amdhsa_next_free_sgpr 42
   .amdhsa_reserve_flat_scratch 0
   .amdhsa_reserve_xnack_mask 0
   .amdhsa_reserve_vcc 0
 .end_amdhsa_kernel
 
 ;--- 2.s
 ;; Only set other directives.
-.amdhsa_kernel my_kernel_2
+; RUN: %assemble -mcpu=gfx908 <2.s >2.o
+; RUN: %disassemble_kd 2.o | %tee_kd 2-disasm.s | FileCheck 2.s
+; RUN: %assemble -mcpu=gfx908 <2-disasm.s >2-disasm.o
+; RUN: diff 2.o 2-disasm.o
+; CHECK: .amdhsa_kernel kernel
+; CHECK-NEXT: .amdhsa_group_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_private_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_kernarg_size 0
+; CHECK-NEXT: .amdhsa_next_free_vgpr 4
+; CHECK-NEXT: .amdhsa_reserve_vcc 0
+; CHECK-NEXT: .amdhsa_reserve_flat_scratch 0
+; CHECK-NEXT: .amdhsa_reserve_xnack_mask 0
+; CHECK-NEXT: .amdhsa_next_free_sgpr 8
+; CHECK-NEXT: .amdhsa_float_round_mode_32 0
+; CHECK-NEXT: .amdhsa_float_round_mode_16_64 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_32 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_16_64 3
+; CHECK-NEXT: .amdhsa_dx10_clamp 1
+; CHECK-NEXT: .amdhsa_ieee_mode 1
+; CHECK-NEXT: .amdhsa_fp16_overflow 0
+; CHECK-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_x 1
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_y 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_z 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_info 0
+; CHECK-NEXT: .amdhsa_system_vgpr_workitem_id 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_invalid_op 0
+; CHECK-NEXT: .amdhsa_exception_fp_denorm_src 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_div_zero 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_overflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_underflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_inexact 0
+; CHECK-NEXT: .amdhsa_exception_int_div_zero 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_buffer 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_queue_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_id 0
+; CHECK-NEXT: .amdhsa_user_sgpr_flat_scratch_init 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_size 0
+; CHECK-NEXT: .amdhsa_uses_dynamic_stack 0
+; CHECK-NEXT: .end_amdhsa_kernel
+.amdhsa_kernel kernel
   .amdhsa_next_free_vgpr 0
   .amdhsa_next_free_sgpr 0
   .amdhsa_reserve_flat_scratch 1
   .amdhsa_reserve_xnack_mask 0
   .amdhsa_reserve_vcc 1
 .end_amdhsa_kernel
 
 ;--- 3.s
 ;; Set all affecting directives.
-.amdhsa_kernel my_kernel_3
+; RUN: %assemble -mcpu=gfx908 <3.s >3.o
+; RUN: %disassemble_kd 3.o | %tee_kd 3-disasm.s | FileCheck 3.s
+; RUN: %assemble -mcpu=gfx908 <3-disasm.s >3-disasm.o
+; RUN: diff 3.o 3-disasm.o
+; CHECK: .amdhsa_kernel kernel
+; CHECK-NEXT: .amdhsa_group_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_private_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_kernarg_size 0
+; CHECK-NEXT: .amdhsa_next_free_vgpr 4
+; CHECK-NEXT: .amdhsa_reserve_vcc 0
+; CHECK-NEXT: .amdhsa_reserve_flat_scratch 0
+; CHECK-NEXT: .amdhsa_reserve_xnack_mask 0
+; CHECK-NEXT: .amdhsa_next_free_sgpr 48
+; CHECK-NEXT: .amdhsa_float_round_mode_32 0
+; CHECK-NEXT: .amdhsa_float_round_mode_16_64 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_32 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_16_64 3
+; CHECK-NEXT: .amdhsa_dx10_clamp 1
+; CHECK-NEXT: .amdhsa_ieee_mode 1
+; CHECK-NEXT: .amdhsa_fp16_overflow 0
+; CHECK-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_x 1
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_y 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_z 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_info 0
+; CHECK-NEXT: .amdhsa_system_vgpr_workitem_id 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_invalid_op 0
+; CHECK-NEXT: .amdhsa_exception_fp_denorm_src 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_div_zero 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_overflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_underflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_inexact 0
+; CHECK-NEXT: .amdhsa_exception_int_div_zero 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_buffer 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_queue_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_id 0
+; CHECK-NEXT: .amdhsa_user_sgpr_flat_scratch_init 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_size 0
+; CHECK-NEXT: .amdhsa_uses_dynamic_stack 0
+; CHECK-NEXT: .end_amdhsa_kernel
+.amdhsa_kernel kernel
   .amdhsa_next_free_vgpr 0
   .amdhsa_next_free_sgpr 35
   .amdhsa_reserve_flat_scratch 1
   .amdhsa_reserve_xnack_mask 0
   .amdhsa_reserve_vcc 1
 .end_amdhsa_kernel

llvm/test/tools/llvm-objdump/ELF/AMDGPU/kd-vgpr.s

 ;; Test disassembly for GRANULATED_WORKITEM_VGPR_COUNT in the kernel descriptor.
 
-; RUN: split-file %s %t.dir
-
-; RUN: llvm-mc %t.dir/1.s --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t1
-; RUN: llvm-objdump --disassemble-symbols=my_kernel_1.kd %t1 | tail -n +7 \
-; RUN: | llvm-mc --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t1-re-assemble
-; RUN: diff %t1 %t1-re-assemble
-
-; RUN: llvm-mc %t.dir/2.s --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t2
-; RUN: llvm-objdump --disassemble-symbols=my_kernel_2.kd %t2 | tail -n +7 \
-; RUN: | llvm-mc --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t2-re-assemble
-; RUN: diff %t2 %t2-re-assemble
-
-; RUN: llvm-mc %t.dir/3.s --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t3
-; RUN: llvm-objdump --disassemble-symbols=my_kernel_3.kd %t3 | tail -n +7 \
-; RUN: | llvm-mc --triple=amdgcn-amd-amdhsa -mcpu=gfx908 -mattr=-xnack -filetype=obj -o %t3-re-assemble
-; RUN: diff %t3 %t3-re-assemble
+; RUN: rm -rf %t && split-file %s %t && cd %t
 
 ;--- 1.s
-.amdhsa_kernel my_kernel_1
+; RUN: %assemble -mcpu=gfx908 <1.s >1.o
+; RUN: %disassemble_kd 1.o | %tee_kd 1-disasm.s | FileCheck 1.s
+; RUN: %assemble -mcpu=gfx908 <1-disasm.s >1-disasm.o
+; RUN: diff 1.o 1-disasm.o
+; CHECK: .amdhsa_kernel kernel
+; CHECK-NEXT: .amdhsa_group_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_private_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_kernarg_size 0
+; CHECK-NEXT: .amdhsa_next_free_vgpr 24
+; CHECK-NEXT: .amdhsa_reserve_vcc 0
+; CHECK-NEXT: .amdhsa_reserve_flat_scratch 0
+; CHECK-NEXT: .amdhsa_reserve_xnack_mask 0
+; CHECK-NEXT: .amdhsa_next_free_sgpr 8
+; CHECK-NEXT: .amdhsa_float_round_mode_32 0
+; CHECK-NEXT: .amdhsa_float_round_mode_16_64 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_32 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_16_64 3
+; CHECK-NEXT: .amdhsa_dx10_clamp 1
+; CHECK-NEXT: .amdhsa_ieee_mode 1
+; CHECK-NEXT: .amdhsa_fp16_overflow 0
+; CHECK-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_x 1
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_y 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_z 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_info 0
+; CHECK-NEXT: .amdhsa_system_vgpr_workitem_id 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_invalid_op 0
+; CHECK-NEXT: .amdhsa_exception_fp_denorm_src 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_div_zero 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_overflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_underflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_inexact 0
+; CHECK-NEXT: .amdhsa_exception_int_div_zero 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_buffer 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_queue_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_id 0
+; CHECK-NEXT: .amdhsa_user_sgpr_flat_scratch_init 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_size 0
+; CHECK-NEXT: .amdhsa_uses_dynamic_stack 0
+; CHECK-NEXT: .end_amdhsa_kernel
+.amdhsa_kernel kernel
   .amdhsa_next_free_vgpr 23
   .amdhsa_next_free_sgpr 0
 .end_amdhsa_kernel
 
 ;--- 2.s
-.amdhsa_kernel my_kernel_2
+; RUN: %assemble -mcpu=gfx908 <2.s >2.o
+; RUN: %disassemble_kd 2.o | %tee_kd 2-disasm.s | FileCheck 2.s
+; RUN: %assemble -mcpu=gfx908 <2-disasm.s >2-disasm.o
+; RUN: diff 2.o 2-disasm.o
+; CHECK: .amdhsa_kernel kernel
+; CHECK-NEXT: .amdhsa_group_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_private_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_kernarg_size 0
+; CHECK-NEXT: .amdhsa_next_free_vgpr 16
+; CHECK-NEXT: .amdhsa_reserve_vcc 0
+; CHECK-NEXT: .amdhsa_reserve_flat_scratch 0
+; CHECK-NEXT: .amdhsa_reserve_xnack_mask 0
+; CHECK-NEXT: .amdhsa_next_free_sgpr 8
+; CHECK-NEXT: .amdhsa_float_round_mode_32 0
+; CHECK-NEXT: .amdhsa_float_round_mode_16_64 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_32 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_16_64 3
+; CHECK-NEXT: .amdhsa_dx10_clamp 1
+; CHECK-NEXT: .amdhsa_ieee_mode 1
+; CHECK-NEXT: .amdhsa_fp16_overflow 0
+; CHECK-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_x 1
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_y 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_z 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_info 0
+; CHECK-NEXT: .amdhsa_system_vgpr_workitem_id 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_invalid_op 0
+; CHECK-NEXT: .amdhsa_exception_fp_denorm_src 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_div_zero 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_overflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_underflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_inexact 0
+; CHECK-NEXT: .amdhsa_exception_int_div_zero 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_buffer 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_queue_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_id 0
+; CHECK-NEXT: .amdhsa_user_sgpr_flat_scratch_init 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_size 0
+; CHECK-NEXT: .amdhsa_uses_dynamic_stack 0
+; CHECK-NEXT: .end_amdhsa_kernel
+.amdhsa_kernel kernel
   .amdhsa_next_free_vgpr 14
   .amdhsa_next_free_sgpr 0
 .end_amdhsa_kernel
 
 ;--- 3.s
-.amdhsa_kernel my_kernel_3
+; RUN: %assemble -mcpu=gfx908 <3.s >3.o
+; RUN: %disassemble_kd 3.o | %tee_kd 3-disasm.s | FileCheck 3.s
+; RUN: %assemble -mcpu=gfx908 <3-disasm.s >3-disasm.o
+; RUN: diff 3.o 3-disasm.o
+; CHECK: .amdhsa_kernel kernel
+; CHECK-NEXT: .amdhsa_group_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_private_segment_fixed_size 0
+; CHECK-NEXT: .amdhsa_kernarg_size 0
+; CHECK-NEXT: .amdhsa_next_free_vgpr 32
+; CHECK-NEXT: .amdhsa_reserve_vcc 0
+; CHECK-NEXT: .amdhsa_reserve_flat_scratch 0
+; CHECK-NEXT: .amdhsa_reserve_xnack_mask 0
+; CHECK-NEXT: .amdhsa_next_free_sgpr 8
+; CHECK-NEXT: .amdhsa_float_round_mode_32 0
+; CHECK-NEXT: .amdhsa_float_round_mode_16_64 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_32 0
+; CHECK-NEXT: .amdhsa_float_denorm_mode_16_64 3
+; CHECK-NEXT: .amdhsa_dx10_clamp 1
+; CHECK-NEXT: .amdhsa_ieee_mode 1
+; CHECK-NEXT: .amdhsa_fp16_overflow 0
+; CHECK-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_x 1
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_y 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_id_z 0
+; CHECK-NEXT: .amdhsa_system_sgpr_workgroup_info 0
+; CHECK-NEXT: .amdhsa_system_vgpr_workitem_id 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_invalid_op 0
+; CHECK-NEXT: .amdhsa_exception_fp_denorm_src 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_div_zero 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_overflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_underflow 0
+; CHECK-NEXT: .amdhsa_exception_fp_ieee_inexact 0
+; CHECK-NEXT: .amdhsa_exception_int_div_zero 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_buffer 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_queue_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 0
+; CHECK-NEXT: .amdhsa_user_sgpr_dispatch_id 0
+; CHECK-NEXT: .amdhsa_user_sgpr_flat_scratch_init 0
+; CHECK-NEXT: .amdhsa_user_sgpr_private_segment_size 0
+; CHECK-NEXT: .amdhsa_uses_dynamic_stack 0
+; CHECK-NEXT: .end_amdhsa_kernel
+.amdhsa_kernel kernel
   .amdhsa_next_free_vgpr 32
   .amdhsa_next_free_sgpr 0
 .end_amdhsa_kernel

llvm/test/tools/llvm-objdump/ELF/AMDGPU/lit.local.cfg

 if not 'AMDGPU' in config.root.targets:
     config.unsupported = True
+
+config.substitutions.append(('%disassemble_kd', 'llvm-objdump --disassemble-symbols=kernel.kd'))
+config.substitutions.append(('%tee_kd', 'tail -n +7 | tee'))
+config.substitutions.append(('%assemble_wave32', 'llvm-mc --triple=amdgcn-amd-amdhsa -mattr=-xnack,+wavefrontsize32,-wavefrontsize64 -filetype=obj'))
+config.substitutions.append(('%assemble_wave64', 'llvm-mc --triple=amdgcn-amd-amdhsa -mattr=-xnack,+wavefrontsize64,-wavefrontsize32 -filetype=obj'))
+config.substitutions.append(('%assemble', 'llvm-mc --triple=amdgcn-amd-amdhsa -mattr=-xnack -filetype=obj'))