[amdgpu] Default to code object v3

[JonChesterfield]

Summary

[amdgpu] Default to code object v3
v4 is not yet readily available, and doesn't appear
to be implemented in the back end

[JonChesterfield]

• update docs

[JonChesterfield]

• another part of docs

[t-tye]

llvm/docs/AMDGPUUsage.rst

2873–2874

Move this to the "Code Object V4 Metadata" section.

[t-tye]

LGTM

[thakis]

This seems to break check-clang everywhere: http://45.33.8.238/linux/35295/step_7.txt

PTAL, and if it takes a while to investigate, please revert for now.

[thakis]

reverted in c9ede6f3367a627baeef78f30d18078af9a4ffca for now

[yaxunl]

@JonChesterfield Did this patch pass ePSDB in gerrit? Better do that before committing it to trunk since we don't know if math libs are compatible with this patch. Also you need to fix lit tests.

[JonChesterfield]

In D93258#2453724, @thakis wrote:

reverted in c9ede6f3367a627baeef78f30d18078af9a4ffca for now

Thanks! Just saw the CI emails come through. I didn't realise code object version was hardcoded across various tests. That's a weird thing to do when it has no effect on the generated IR.

@yaxunl This would never pass the gerrit testing. AMD internal has moved to code object v4 so changing the default to v3 will break internal tests that assume that. The hsa runtime that can load v4 objects hasn't shipped, either in rocm or as source in github, and the backend that generates v4 object files also hasn't shipped, so defaulting to v4 in the trunk front end is a poor choice.

[yaxunl]

In D93258#2453815, @JonChesterfield wrote:

In D93258#2453724, @thakis wrote:

reverted in c9ede6f3367a627baeef78f30d18078af9a4ffca for now

Thanks! Just saw the CI emails come through. I didn't realise code object version was hardcoded across various tests. That's a weird thing to do when it has no effect on the generated IR.

@yaxunl This would never pass the gerrit testing. AMD internal has moved to code object v4 so changing the default to v3 will break internal tests that assume that. The hsa runtime that can load v4 objects hasn't shipped, either in rocm or as source in github, and the backend that generates v4 object files also hasn't shipped, so defaulting to v4 in the trunk front end is a poor choice.

Can they use rocm release branches of llvm/clang with the corresponding rocm release? As llvm/clang trunk is like the development branch, it is understandable that they may contain new changes that may temporarily not working with previous rocm releases.

[JonChesterfield]

In D93258#2453828, @yaxunl wrote:

Can they use rocm release branches of llvm/clang with the corresponding rocm release? As llvm/clang trunk is like the development branch, it is understandable that they may contain new changes that may temporarily not working with previous rocm releases.

Code object v4 does not work with any rocm release, or with code published that is not in a release. As far as open source goes, it exists only as some documentation and the change to clang that decided it was the right default to use.

That change turned out to be largely harmless as the back end uses v3 despite the docs and clang -### claiming it will use v4.

Depending on a library that doesn't exist is absolutely not 'understandable'. I believe this patch undoes a mistaken step in that direction and that the hip tests are also in error.

[JonChesterfield]

• Remove hard coded version numbers in tests

[JonChesterfield]

@yaxunl Please see Gerrit 456139 for a close approximation to the test changes here. I don't think hip should be hard coding a version number in tests that don't care about it, so would like to move trunk and internal to a regex.

[yaxunl]

Thanks for fixing the lit tests. Using regex is the right choice.

Do we have a plan about how to merge this to amd-stg-open? Will it cause ePSDB to fail? Do you have a follow up patch to make amd-stg-open happy?

Thanks.

[JonChesterfield]

In D93258#2457905, @yaxunl wrote:

Thanks for fixing the lit tests. Using regex is the right choice.

Do we have a plan about how to merge this to amd-stg-open? Will it cause ePSDB to fail? Do you have a follow up patch to make amd-stg-open happy?

Thanks.

Extracted as D93398. Suggest we land that, then it flows into gerrit (where it won't break anything), and rebase this patch to reduce the noise.

This will need to be merged carefully into amd-stg-open. No follow up patch necessary, the trick would be to not land this on amd-stg-open if we want to keep v4 as the default.

Note that none of this would be necessary if trunk clang hadn't been prematurely updated to default to a code object version that llvm trunk doesn't use.

[JonChesterfield]

• rebase

[yaxunl]

LGTM. Thanks.

clang/include/clang/Driver/Options.td

 def mno_tail_call : Flag<["-"], "mno-tail-call">, Group<m_wasm_Features_Group>;
 def mreference_types : Flag<["-"], "mreference-types">, Group<m_wasm_Features_Group>;
 def mno_reference_types : Flag<["-"], "mno-reference-types">, Group<m_wasm_Features_Group>;
 def mexec_model_EQ : Joined<["-"], "mexec-model=">, Group<m_wasm_Features_Driver_Group>,
                      Values<"command,reactor">,
                      HelpText<"Execution model (WebAssembly only)">;
 
 def mcode_object_version_EQ : Joined<["-"], "mcode-object-version=">, Group<m_Group>,
-  HelpText<"Specify code object ABI version. Defaults to 4. (AMDGPU only)">,
+  HelpText<"Specify code object ABI version. Defaults to 3. (AMDGPU only)">,
   MetaVarName<"<version>">, Values<"2,3,4">;
 
 def mcode_object_v3_legacy : Flag<["-"], "mcode-object-v3">, Group<m_Group>,
   HelpText<"Legacy option to specify code object ABI V2 (-mnocode-object-v3) or V3 (-mcode-object-v3) (AMDGPU only)">;
 def mno_code_object_v3_legacy : Flag<["-"], "mno-code-object-v3">, Group<m_Group>;
 
 def mcumode : Flag<["-"], "mcumode">, Group<m_amdgpu_Features_Group>,
   HelpText<"Specify CU (-mcumode) or WGP (-mno-cumode) wavefront execution mode (AMDGPU only)">;

clang/lib/Driver/ToolChains/CommonArgs.cpp

if (const Arg *A = Args.getLastArg(options::OPT_mpad_max_prefix_size_EQ)) {
     }
   }
 }
 
 unsigned tools::getOrCheckAMDGPUCodeObjectVersion(
     const Driver &D, const llvm::opt::ArgList &Args, bool Diagnose) {
   const unsigned MinCodeObjVer = 2;
   const unsigned MaxCodeObjVer = 4;
-  unsigned CodeObjVer = 4;
+  unsigned CodeObjVer = 3;
 
   // Emit warnings for legacy options even if they are overridden.
   if (Diagnose) {
     if (Args.hasArg(options::OPT_mno_code_object_v3_legacy))
       D.Diag(diag::warn_drv_deprecated_arg) << "-mno-code-object-v3"
                                             << "-mcode-object-version=2";
 
     if (Args.hasArg(options::OPT_mcode_object_v3_legacy))

clang/test/Driver/hip-code-object-version.hip

 // V4: "-targets=host-x86_64-unknown-linux,hip-amdgcn-amd-amdhsa--gfx906"
 
 // Check bundle ID for code object version default
 
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --offload-arch=gfx906 -nogpulib \
 // RUN:   %s 2>&1 | FileCheck -check-prefix=VD %s
 
-// VD: "-mllvm" "--amdhsa-code-object-version=4"
+// VD: "-mllvm" "--amdhsa-code-object-version=3"
 // VD: "-targets=host-x86_64-unknown-linux,hip-amdgcn-amd-amdhsa--gfx906"
 
 // Check invalid code object version option.
 
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   -mcode-object-version=1 \
 // RUN:   --offload-arch=gfx906 -nogpulib \
 // RUN:   %s 2>&1 | FileCheck -check-prefix=INVALID %s

llvm/docs/AMDGPUUsage.rst

``e_ident[EI_ABIVERSION]``
   object conforms:
 
   * ``ELFABIVERSION_AMDGPU_HSA_V2`` is used to specify the version of AMD HSA
     runtime ABI for code object V2. Specify using the Clang option
     ``-mcode-object-version=2``.
 
   * ``ELFABIVERSION_AMDGPU_HSA_V3`` is used to specify the version of AMD HSA
     runtime ABI for code object V3. Specify using the Clang option
-    ``-mcode-object-version=3``.
+    ``-mcode-object-version=3``. This is the default code object
+    version if not specified.
 
   * ``ELFABIVERSION_AMDGPU_HSA_V4`` is used to specify the version of AMD HSA
     runtime ABI for code object V4. Specify using the Clang option
-    ``-mcode-object-version=4``. This is the default code object
-    version if not specified.
+    ``-mcode-object-version=4``.
 
   * ``ELFABIVERSION_AMDGPU_PAL`` is used to specify the version of AMD PAL
     runtime ABI.
 
   * ``ELFABIVERSION_AMDGPU_MESA3D`` is used to specify the version of AMD MESA
     3D runtime ABI.
 
 ``e_type``
"NumSpilledVGPRs"            integer                  Number of stores from
                                                            created spill
                                                            location.
      ============================ ============== ========= =====================
 
 .. _amdgpu-amdhsa-code-object-metadata-v3:
 
 Code Object V3 Metadata
 +++++++++++++++++++++++
 
-.. warning::
-  Code object V3 is not the default code object version emitted by this version
-  of LLVM.
-
 Code object V3 to V4 metadata is specified by the ``NT_AMDGPU_METADATA`` note
 record (see :ref:`amdgpu-note-records-v3-v4`).
 
 The metadata is represented as Message Pack formatted binary data (see
 [MsgPack]_). The top level is a Message Pack map that includes the
 keys defined in table
 :ref:`amdgpu-amdhsa-code-object-metadata-map-table-v3` and referenced
 tables.
 
.. table:: AMDHSA Code Object V3 Kernel Argument Metadata Map
 
      ====================== ============== ========= ================================
 
 .. _amdgpu-amdhsa-code-object-metadata-v4:
 
 Code Object V4 Metadata
 +++++++++++++++++++++++
 
+.. warning::
+  Code object V4 is not the default code object version emitted by this version
+  of LLVM.
+
 Code object V4 metadata is the same as
 :ref:`amdgpu-amdhsa-code-object-metadata-v3` with the changes and additions
 defined in table :ref:`amdgpu-amdhsa-code-object-metadata-map-table-v3`.
 
   .. table:: AMDHSA Code Object V4 Metadata Map Changes from :ref:`amdgpu-amdhsa-code-object-metadata-v3`
      :name: amdgpu-amdhsa-code-object-metadata-map-table-v4
 
      ================= ============== ========= =======================================