[NFC][AMDGPU] Document the AMDGPU target feature defaults

[t-tye]

Summary

Document the default for the XNACK and SRAMECC target features for code object V2-V3 and V4.

[kzhuravl]

LGTM

[foad]

llvm/docs/AMDGPUUsage.rst

509–528

Isn't your third paragraph here just explaining that the first two are identical? I.e. "generate code that works with xnack replay enabled" and "generate code that works whether xnack replay is enabled or not" are the same thing.

[t-tye]

llvm/docs/AMDGPUUsage.rst

509–528

Code object v2/v3 does not support the concept of ANY and so the default has to be ether ON or OFF. The second paragraph specifies that ON is used.

Code object V4 has the concept of ANY and the third paragraph states that the default is ANY.

The fourth paragraph describes the consequences of these defaults.

[foad]

llvm/docs/AMDGPUUsage.rst

509–528

But ANY and ON are synonyms, aren't they? They both seem to tell the compiler to do exactly the same thing, so the default behaviour hasn't changed in any observable way.

[t-tye]

llvm/docs/AMDGPUUsage.rst

509–528

No they are not. Code object v2-v3 only supports ON and OFF. Code object supports ON, OFF, and ANY. ANY tells the loader that it is safe to load the code on a device configured with either ON or OFF. ON tells the loader it can only load onto a device with the feature ON.

It may be that the compiler generates the same code for ON and ANY but that is distinct from how the loader acts on them.

This is explained more in the TargetID section of the clang offload bundler.

[foad]

llvm/docs/AMDGPUUsage.rst

509–528

ANY tells the loader that it is safe to load the code on a device configured with either ON or OFF. ON tells the loader it can only load onto a device with the feature ON.

Thanks. But doesn't that contradict what you wrote: "Executing code that was generated with XNACK replay enabled [...] on a device that does not have XNACK replay enabled will execute correctly". What does it mean to say it "will execute correctly" if the loader would refuse to load it?

In any case, if the loader treats ANY and ON differently, then I think you should mention that here. Otherwise there is nothing to explain that ANY and ON are different, which makes changing the default from ON (v2/v3) to ANY (v4) sound rather pointless :-)

[t-tye]

llvm/docs/AMDGPUUsage.rst

509–528

ANY tells the loader that it is safe to load the code on a device configured with either ON or OFF. ON tells the loader it can only load onto a device with the feature ON.

Thanks. But doesn't that contradict what you wrote: "Executing code that was generated with XNACK replay enabled [...] on a device that does not have XNACK replay enabled will execute correctly". What does it mean to say it "will execute correctly" if the loader would refuse to load it?

It is not a contradiction. It explains that the code is the same, even though the loader does the enforcement described. Some users like to know this, and the original text said this so I left it here:-)

In any case, if the loader treats ANY and ON differently, then I think you should mention that here. Otherwise there is nothing to explain that ANY and ON are different, which makes changing the default from ON (v2/v3) to ANY (v4) sound rather pointless :-)

The documentation on ANY and ON appears in the following section on Target ID which links to the Clang Offload Bundler documentation.

[tra]

llvm/docs/AMDGPUUsage.rst

484–485

@t-tye: What does it mean for the process to have SRAMECC (or XNACK) setting ?

The description of the target features implies that SRAMECC/XNACK are properties of the hardware, so I suspect it's a typo.
Is it supposed to be executed on a processor instead?

If the features are indeed something that's specific for particular GPU,
are there any tools/or docs which would tell me which particular set of features is needed for the GPU I have?
What determines the specific on/off values for the features? GPU's BIOS?

E.g. gfx908 lists sramecc and xnack as supported, but does not tell me which specific combination of them I need to use to build for a MI100 GPU.
For older GPUs, the fixed feature set described for object V2 provided useful list of the features, but it does not include newer GPUs.

llvm/docs/AMDGPUUsage.rst

.. table:: AMDGPU Target Features
      =============== ============================ ==================================================
      cumode          - ``-m[no-]cumode``          Control the wavefront execution mode used
                                                   when generating code for kernels. When disabled
                                                   native WGP wavefront execution mode is used,
                                                   when enabled CU wavefront execution mode is used
                                                   (see :ref:`amdgpu-amdhsa-memory-model`).
 
      sramecc         - ``-mcpu``                  If specified, generate code that can only be
                      - ``--offload-arch``         loaded and executed in a process that has a
                                                   matching setting for SRAMECC.
 
-                                                  If not specified, generate code that can be
-                                                  loaded and executed in a process with either
-                                                  setting of SRAMECC.
+                                                  If not specified for code object V2 to V3, generate
+                                                  code that can be loaded and executed in a process
+                                                  with SRAMECC enabled.
+
+                                                  If not specified for code object V4, generate
+                                                  code that can be loaded and executed in a process
+                                                  with either setting of SRAMECC.
 
      tgsplit           ``-m[no-]tgsplit``         Enable/disable generating code that assumes
                                                   work-groups are launched in threadgroup split mode.
                                                   When enabled the waves of a work-group may be
                                                   launched in different CUs.
 
      wavefrontsize64 - ``-m[no-]wavefrontsize64`` Control the wavefront size used when
                                                   generating code for kernels. When disabled
                                                   native wavefront size 32 is used, when enabled
                                                   wavefront size 64 is used.
 
      xnack           - ``-mcpu``                  If specified, generate code that can only be
                      - ``--offload-arch``         loaded and executed in a process that has a
                                                   matching setting for XNACK replay.
 
-                                                  If not specified, generate code that can be
-                                                  loaded and executed in a process with either
-                                                  setting of XNACK replay.
-
-                                                  This is used for demand paging and page
-                                                  migration. If XNACK replay is enabled in
-                                                  the device, then if a page fault occurs
-                                                  the code may execute incorrectly if the
-                                                  ``xnack`` feature is not enabled. Executing
-                                                  code that has the feature enabled on a
-                                                  device that does not have XNACK replay
-                                                  enabled will execute correctly but may
-                                                  be less performant than code with the
-                                                  feature disabled.
+                                                  If not specified for code object V2 to V3, generate
+                                                  code that can be loaded and executed in a process
+                                                  with XNACK replay enabled.
+
+                                                  If not specified for code object V4, generate
+                                                  code that can be loaded and executed in a process
+                                                  with either setting of XNACK replay.
+
+                                                  XNACK replay can be used for demand paging and
+                                                  page migration. If enabled in the device, then if
+                                                  a page fault occurs the code may execute
+                                                  incorrectly unless generated with XNACK replay
+                                                  enabled, or generated for code object V4 without
+                                                  specifying XNACK replay. Executing code that was
+                                                  generated with XNACK replay enabled, or generated
+                                                  for code object V4 without specifying XNACK replay,
+                                                  on a device that does not have XNACK replay
+                                                  enabled will execute correctly but may be less
+                                                  performant than code generated for XNACK replay
+                                                  disabled.
      =============== ============================ ==================================================
 
 .. _amdgpu-target-id:
 
 Target ID
 ---------
 
 AMDGPU supports target IDs. See `Clang Offload Bundler