This is an archive of the discontinued LLVM Phabricator instance.

Differential D92434

[NFC][AMDGPU] AMDGPU code object V4 ABI documentation
ClosedPublic

Authored by t-tye on Dec 1 2020, 3:31 PM.

Details

Reviewers
kzhuravl
scott.linder
b-sumner
tpr
rampitec
yaxunl
kerbowa
jdoerfert
Commits
rG04424c69bc7b: [NFC][AMDGPU] AMDGPU code object V4 ABI documentation
Summary
  • Documantation for AMDGPU code object V4.
  • Documentation clarification for code object V2 and V3.
  • Documentation for the clang-offload-bundler.
  • Numerous other documentation clarifications.

Change-Id: I338b327cc9e75da6c987b7e081b496402a5a020e

Diff Detail

Event Timeline

t-tye created this revision.Dec 1 2020, 3:31 PM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptDec 1 2020, 3:31 PM
Herald added subscribers: llvm-commits, cfe-commits, jfb and 3 others. · View Herald Transcript
t-tye requested review of this revision.Dec 1 2020, 3:31 PM
Herald added a reviewer: jdoerfert. · View Herald TranscriptDec 1 2020, 3:31 PM
Herald added subscribers: sstefan1, wdng. · View Herald Transcript
kzhuravl accepted this revision.Dec 1 2020, 3:36 PM

LGTM

This revision is now accepted and ready to land.Dec 1 2020, 3:36 PM
This revision was landed with ongoing or failed builds.Dec 1 2020, 3:39 PM
Closed by commit rG04424c69bc7b: [NFC][AMDGPU] AMDGPU code object V4 ABI documentation (authored by t-tye). · Explain Why
This revision was automatically updated to reflect the committed changes.
t-tye added a commit: rG04424c69bc7b: [NFC][AMDGPU] AMDGPU code object V4 ABI documentation.
Harbormaster completed remote builds in B80731: Diff 308798.Dec 1 2020, 3:53 PM
tra added a subscriber: tra.Jan 5 2021, 2:20 PM
tra added inline comments.
llvm/docs/AMDGPUUsage.rst
959

Where do these flags live?
I do not see them in https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/BinaryFormat/ELF.h#L738

Do these doc changes reflect a patch that did not land yet?

Revision Contents

PathSize
clang/
docs/
211 lines
llvm/
docs/
1579 lines

Diff 308801

clang/docs/ClangOffloadBundler.rst

  • This file was added.
=====================
Clang Offload Bundler
=====================
.. contents::
:local:
.. _clang-offload-bundler:
Introduction
============
For heterogeneous single source programming languages, use one or more
``--offload-arch=<target-id>`` Clang options to specify the target IDs of the
code to generate for the offload code regions.
The tool chain may perform multiple compilations of a translation unit to
produce separate code objects for the host and potentially multiple offloaded
devices. The ``clang-offload-bundler`` tool may be used as part of the tool
chain to combine these multiple code objects into a single bundled code object.
The tool chain may use a bundled code object as an intermediate step so that
each tool chain step consumes and produces a single file as in traditional
non-heterogeneous tool chains. The bundled code object contains the code objects
for the host and all the offload devices.
A bundled code object may also be used to bundle just the offloaded code
objects, and embedded as data into the host code object. The host compilation
includes an ``init`` function that will use the runtime corresponding to the
offload kind (see :ref:`clang-offload-kind-table`) to load the offload code
objects appropriate to the devices present when the host program is executed.
.. _clang-bundled-code-object-layout:
Bundled Code Object Layout
==========================
The layout of a bundled code object is defined by the following table:
.. table:: Bundled Code Object Layout
:name: bundled-code-object-layout-table
=================================== ======= ================ ===============================
Field Type Size in Bytes Description
=================================== ======= ================ ===============================
Magic String string 24 ``__CLANG_OFFLOAD_BUNDLE__``
Number Of Code Objects integer 8 Number od bundled code objects.
1st Bundle Entry Code Object Offset integer 8 Byte offset from beginning of
bundled code object to 1st code
object.
1st Bundle Entry Code Object Size integer 8 Byte size of 1st code object.
1st Bundle Entry ID Length integer 8 Character length of bundle
entry ID of 1st code object.
1st Bundle Entry ID string 1st Bundle Entry Bundle entry ID of 1st code
ID Length object. This is not NUL
terminated. See
:ref:`clang-bundle-entry-id`.
\...
Nth Bundle Entry Code Object Offset integer 8
Nth Bundle Entry Code Object Size integer 8
Nth Bundle Entry ID Length integer 8
Nth Bundle Entry ID string 1st Bundle Entry
ID Length
1st Bundle Entry Code Object bytes 1st Bundle Entry
Code Object Size
\...
Nth Bundle Entry Code Object bytes Nth Bundle Entry
Code Object Size
=================================== ======= ================ ===============================
.. _clang-bundle-entry-id:
Bundle Entry ID
===============
Each entry in a bundled code object (see
:ref:`clang-bundled-code-object-layout`) has a bundle entry ID that indicates
the kind of the entry's code object and the runtime that manages it.
Bundle entry ID syntax is defined by the following BNF syntax:
.. code::
<bundle-entry-id> ::== <offload-kind> "-" <target-triple> [ "-" <target-id> ]
Where:
**offload-kind**
The runtime responsible for managing the bundled entry code object. See
:ref:`clang-offload-kind-table`.
.. table:: Bundled Code Object Offload Kind
:name: clang-offload-kind-table
============= ==============================================================
Offload Kind Description
============= ==============================================================
host Host code object. ``clang-offload-bundler`` always includes
this entry as the first bundled code object entry. For an
embedded bundled code object this entry is not used by the
runtime and so is generally an empty code object.
hip Offload code object for the HIP language. Used for all
HIP language offload code objects when the
``clang-offload-bundler`` is used to bundle code objects as
intermediate steps of the tool chain. Also used for AMD GPU
code objects before ABI version V4 when the
``clang-offload-bundler`` is used to create a *fat binary*
to be loaded by the HIP runtime. The fat binary can be
loaded directly from a file, or be embedded in the host code
object as a data section with the name ``.hip_fatbin``.
hipv4 Offload code object for the HIP language. Used for AMD GPU
code objects with at least ABI version V4 when the
``clang-offload-bundler`` is used to create a *fat binary*
to be loaded by the HIP runtime. The fat binary can be
loaded directly from a file, or be embedded in the host code
object as a data section with the name ``.hip_fatbin``.
openmp Offload code object for the OpenMP language extension.
============= ==============================================================
**target-triple**
The target triple of the code object.
**target-id**
The canonical target ID of the code object. Present only if the target
supports a target ID. See :ref:`clang-target-id`.
Each entry of a bundled code object must have a different bundle entry ID. There
can be multiple entries for the same processor provided they differ in target
feature settings. If there is an entry with a target feature specified as *Any*,
then all entries must specify that target feature as *Any* for the same
processor. There may be additional target specific restrictions.
.. _clang-target-id:
Target ID
=========
A target ID is used to indicate the processor and optionally its configuration,
expressed by a set of target features, that affect ISA generation. It is target
specific if a target ID is supported, or if the target triple alone is
sufficient to specify the ISA generation.
It is used with the ``-mcpu=<target-id>`` and ``--offload-arch=<target-id>``
Clang compilation options to specify the kind of code to generate.
It is also used as part of the bundle entry ID to identify the code object. See
:ref:`clang-bundle-entry-id`.
Target ID syntax is defined by the following BNF syntax:
.. code::
<target-id> ::== <processor> ( ":" <target-feature> ( "+" | "-" ) )*
Where:
**processor**
Is a the target specific processor or any alternative processor name.
**target-feature**
Is a target feature name that is supported by the processor. Each target
feature must appear at most once in a target ID and can have one of three
values:
*Any*
Specified by omitting the target feature from the target ID.
A code object compiled with a target ID specifying the default
value of a target feature can be loaded and executed on a processor
configured with the target feature on or off.
*On*
Specified by ``+``, indicating the target feature is enabled. A code
object compiled with a target ID specifying a target feature on
can only be loaded on a processor configured with the target feature on.
*Off*
specified by ``-``, indicating the target feature is disabled. A code
object compiled with a target ID specifying a target feature off
can only be loaded on a processor configured with the target feature off.
There are two forms of target ID:
*Non-Canonical Form*
The non-canonical form is used as the input to user commands to allow the user
greater convenience. It allows both the primary and alternative processor name
to be used and the target features may be specified in any order.
*Canonical Form*
The canonical form is used for all generated output to allow greater
convenience for tools that consume the information. It is also used for
internal passing of information between tools. Only the primary and not
alternative processor name is used and the target features are specified in
alphabetic order. Command line tools convert non-canonical form to canonical
form.
Target Specific information
===========================
Target specific information is available for the following:
*AMD GPU*
AMD GPU supports target ID and target features. See `User Guide for AMDGPU Backend
<https://llvm.org/docs/AMDGPUUsage.html>`_ which defines the `processors
<https://llvm.org/docs/AMDGPUUsage.html#amdgpu-processors>`_ and `target
features <https://llvm.org/docs/AMDGPUUsage.html#amdgpu-target-features>`_
supported.
Most other targets do not support target IDs.
No newline at end of file

llvm/docs/AMDGPUUsage.rst

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show All 32 Lines
LLVM LLVM
==== ====
.. _amdgpu-target-triples: .. _amdgpu-target-triples:
Target Triples Target Triples
-------------- --------------
Use the ``clang -target <Architecture>-<Vendor>-<OS>-<Environment>`` option to Use the Clang option ``-target <Architecture>-<Vendor>-<OS>-<Environment>``
specify the target triple: to specify the target triple:
.. table:: AMDGPU Architectures .. table:: AMDGPU Architectures
:name: amdgpu-architecture-table :name: amdgpu-architecture-table
============ ============================================================== ============ ==============================================================
Architecture Description Architecture Description
============ ============================================================== ============ ==============================================================
``r600`` AMD GPUs HD2XXX-HD6XXX for graphics and compute shaders. ``r600`` AMD GPUs HD2XXX-HD6XXX for graphics and compute shaders.
``amdgcn`` AMD GPUs GCN GFX6 onwards for graphics and compute shaders. ``amdgcn`` AMD GPUs GCN GFX6 onwards for graphics and compute shaders.
============ ============================================================== ============ ==============================================================
.. table:: AMDGPU Vendors .. table:: AMDGPU Vendors
:name: amdgpu-vendor-table :name: amdgpu-vendor-table
============ ============================================================== ============ ==============================================================
Vendor Description Vendor Description
============ ============================================================== ============ ==============================================================
``amd`` Can be used for all AMD GPU usage. ``amd`` Can be used for all AMD GPU usage.
``mesa3d`` Can be used if the OS is ``mesa3d``. ``mesa3d`` Can be used if the OS is ``mesa3d``.
============ ============================================================== ============ ==============================================================
.. table:: AMDGPU Operating Systems .. table:: AMDGPU Operating Systems
:name: amdgpu-os-table :name: amdgpu-os
============== ============================================================ ============== ============================================================
OS Description OS Description
============== ============================================================ ============== ============================================================
*<empty>* Defaults to the *unknown* OS. *<empty>* Defaults to the *unknown* OS.
``amdhsa`` Compute kernels executed on HSA [HSA]_ compatible runtimes ``amdhsa`` Compute kernels executed on HSA [HSA]_ compatible runtimes
such as AMD's ROCm [AMD-ROCm]_. such as:
``amdpal`` Graphic shaders and compute kernels executed on AMD PAL
runtime. - AMD's ROCm runtime [AMD-ROCm]_ on Linux. See *AMD ROCm
Release Notes* [AMD-ROCm-Release-Notes]_ for supported
hardware and software.
- AMD's PAL runtime using the *amdhsa* loader on Windows.
``amdpal`` Graphic shaders and compute kernels executed on AMD's PAL
runtime using the *amdpal* loader on Windows.
``mesa3d`` Graphic shaders and compute kernels executed on Mesa 3D ``mesa3d`` Graphic shaders and compute kernels executed on Mesa 3D
runtime. runtime.
============== ============================================================ ============== ============================================================
.. table:: AMDGPU Environments .. table:: AMDGPU Environments
:name: amdgpu-environment-table :name: amdgpu-environment-table
============ ============================================================== ============ ==============================================================
Environment Description Environment Description
============ ============================================================== ============ ==============================================================
*<empty>* Default. *<empty>* Default.
============ ============================================================== ============ ==============================================================
.. _amdgpu-processors: .. _amdgpu-processors:
Processors Processors
---------- ----------
Use the ``clang -mcpu <Processor>`` option to specify the AMDGPU processor. The Use the Clang options ``-mcpu=<target-id>`` or ``--offload-arch=<target-id>`` to
names from both the *Processor* and *Alternative Processor* can be used. specify the AMDGPU processor together with optional target features. See
:ref:`amdgpu-target-id` and :ref:`amdgpu-target-features` for AMD GPU target
specific information.
.. table:: AMDGPU Processors .. table:: AMDGPU Processors
:name: amdgpu-processor-table :name: amdgpu-processor-table
=========== =============== ============ ===== ============================= ======= ====================== =========== =============== ============ ===== ================= =========== ============== ======================
Processor Alternative Target dGPU/ Target ROCm Example Processor Alternative Target dGPU/ Target Target OS Example
Processor Triple APU Features Support Products Processor Triple APU Features Properties Support Products
Architecture Supported Architecture Supported ()
[Default] *(see*
=========== =============== ============ ===== ============================= ======= ====================== `amdgpu-os`_
*and
corresponding
runtime
release notes
for current
information
and level of
support)*
=========== =============== ============ ===== ================= =========== ============== ======================
**Radeon HD 2000/3000 Series (R600)** [AMD-RADEON-HD-2000-3000]_ **Radeon HD 2000/3000 Series (R600)** [AMD-RADEON-HD-2000-3000]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``r600`` ``r600`` dGPU ``r600`` ``r600`` dGPU - Does not
``r630`` ``r600`` dGPU support
``rs880`` ``r600`` dGPU generic
``rv670`` ``r600`` dGPU address
space
``r630`` ``r600`` dGPU - Does not
support
generic
address
space
``rs880`` ``r600`` dGPU - Does not
support
generic
address
space
``rv670`` ``r600`` dGPU - Does not
support
generic
address
space
**Radeon HD 4000 Series (R700)** [AMD-RADEON-HD-4000]_ **Radeon HD 4000 Series (R700)** [AMD-RADEON-HD-4000]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``rv710`` ``r600`` dGPU ``rv710`` ``r600`` dGPU - Does not
``rv730`` ``r600`` dGPU support
``rv770`` ``r600`` dGPU generic
address
space
``rv730`` ``r600`` dGPU - Does not
support
generic
address
space
``rv770`` ``r600`` dGPU - Does not
support
generic
address
space
**Radeon HD 5000 Series (Evergreen)** [AMD-RADEON-HD-5000]_ **Radeon HD 5000 Series (Evergreen)** [AMD-RADEON-HD-5000]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``cedar`` ``r600`` dGPU ``cedar`` ``r600`` dGPU - Does not
``cypress`` ``r600`` dGPU support
``juniper`` ``r600`` dGPU generic
``redwood`` ``r600`` dGPU address
``sumo`` ``r600`` dGPU space
``cypress`` ``r600`` dGPU - Does not
support
generic
address
space
``juniper`` ``r600`` dGPU - Does not
support
generic
address
space
``redwood`` ``r600`` dGPU - Does not
support
generic
address
space
``sumo`` ``r600`` dGPU - Does not
support
generic
address
space
**Radeon HD 6000 Series (Northern Islands)** [AMD-RADEON-HD-6000]_ **Radeon HD 6000 Series (Northern Islands)** [AMD-RADEON-HD-6000]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``barts`` ``r600`` dGPU ``barts`` ``r600`` dGPU - Does not
``caicos`` ``r600`` dGPU support
``cayman`` ``r600`` dGPU generic
``turks`` ``r600`` dGPU address
space
``caicos`` ``r600`` dGPU - Does not
support
generic
address
space
``cayman`` ``r600`` dGPU - Does not
support
generic
address
space
``turks`` ``r600`` dGPU - Does not
support
generic
address
space
**GCN GFX6 (Southern Islands (SI))** [AMD-GCN-GFX6]_ **GCN GFX6 (Southern Islands (SI))** [AMD-GCN-GFX6]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``gfx600`` - ``tahiti`` ``amdgcn`` dGPU ``gfx600`` - ``tahiti`` ``amdgcn`` dGPU - Does not - AMD PAL
``gfx601`` - ``pitcairn`` ``amdgcn`` dGPU support
- ``verde`` generic
``gfx602`` - ``hainan`` ``amdgcn`` dGPU address
- ``oland`` space
``gfx601`` - ``pitcairn`` ``amdgcn`` dGPU - Does not - AMD PAL
- ``verde`` support
generic
address
space
``gfx602`` - ``hainan`` ``amdgcn`` dGPU - Does not - AMD PAL
- ``oland`` support
generic
address
space
**GCN GFX7 (Sea Islands (CI))** [AMD-GCN-GFX7]_ **GCN GFX7 (Sea Islands (CI))** [AMD-GCN-GFX7]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``gfx700`` - ``kaveri`` ``amdgcn`` APU - A6-7000 ``gfx700`` - ``kaveri`` ``amdgcn`` APU - AMD ROCm - A6-7000
- A6 Pro-7050B - AMD PAL - A6 Pro-7050B
- A8-7100 - A8-7100
- A8 Pro-7150B - A8 Pro-7150B
- A10-7300 - A10-7300
- A10 Pro-7350B - A10 Pro-7350B
- FX-7500 - FX-7500
- A8-7200P - A8-7200P
- A10-7400P - A10-7400P
- FX-7600P - FX-7600P
``gfx701`` - ``hawaii`` ``amdgcn`` dGPU ROCm - FirePro W8100 ``gfx701`` - ``hawaii`` ``amdgcn`` dGPU - AMD ROCm - FirePro W8100
- FirePro W9100 - AMD PAL - FirePro W9100
- FirePro S9150 - FirePro S9150
- FirePro S9170 - FirePro S9170
``gfx702`` ``amdgcn`` dGPU ROCm - Radeon R9 290 ``gfx702`` ``amdgcn`` dGPU - AMD ROCm - Radeon R9 290
- Radeon R9 290x - AMD PAL - Radeon R9 290x
- Radeon R390 - Radeon R390
- Radeon R390x - Radeon R390x
``gfx703`` - ``kabini`` ``amdgcn`` APU - E1-2100 ``gfx703`` - ``kabini`` ``amdgcn`` APU - AMD PAL - E1-2100
- ``mullins`` - E1-2200 - ``mullins`` - E1-2200
- E1-2500 - E1-2500
- E2-3000 - E2-3000
- E2-3800 - E2-3800
- A4-5000 - A4-5000
- A4-5100 - A4-5100
- A6-5200 - A6-5200
- A4 Pro-3340B - A4 Pro-3340B
``gfx704`` - ``bonaire`` ``amdgcn`` dGPU - Radeon HD 7790 ``gfx704`` - ``bonaire`` ``amdgcn`` dGPU - AMD PAL - Radeon HD 7790
- Radeon HD 8770 - Radeon HD 8770
- R7 260 - R7 260
- R7 260X - R7 260X
``gfx705`` ``amdgcn`` APU *TBA* ``gfx705`` ``amdgcn`` APU - AMD PAL *TBA*
.. TODO:: .. TODO::
Add product Add product
names. names.
**GCN GFX8 (Volcanic Islands (VI))** [AMD-GCN-GFX8]_ **GCN GFX8 (Volcanic Islands (VI))** [AMD-GCN-GFX8]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``gfx801`` - ``carrizo`` ``amdgcn`` APU - xnack - A6-8500P ``gfx801`` - ``carrizo`` ``amdgcn`` APU - xnack - AMD ROCm - A6-8500P
[on] - Pro A6-8500B - AMD PAL - Pro A6-8500B
- A8-8600P - A8-8600P
- Pro A8-8600B - Pro A8-8600B
- FX-8800P - FX-8800P
- Pro A12-8800B - Pro A12-8800B
\ ``amdgcn`` APU - xnack ROCm - A10-8700P - A10-8700P
[on] - Pro A10-8700B - Pro A10-8700B
- A10-8780P - A10-8780P
\ ``amdgcn`` APU - xnack - A10-9600P - A10-9600P
[on] - A10-9630P - A10-9630P
- A12-9700P - A12-9700P
- A12-9730P - A12-9730P
- FX-9800P - FX-9800P
- FX-9830P - FX-9830P
\ ``amdgcn`` APU - xnack - E2-9010 - E2-9010
[on] - A6-9210 - A6-9210
- A9-9410 - A9-9410
``gfx802`` - ``iceland`` ``amdgcn`` dGPU - xnack ROCm - Radeon R285 ``gfx802`` - ``iceland`` ``amdgcn`` dGPU - AMD ROCm - Radeon R9 285
- ``tonga`` [off] - Radeon R9 380 - ``tonga`` - AMD PAL - Radeon R9 380
- Radeon R9 385 - Radeon R9 385
``gfx803`` - ``fiji`` ``amdgcn`` dGPU - xnack ROCm - Radeon R9 Nano ``gfx803`` - ``fiji`` ``amdgcn`` dGPU - AMD ROCm - Radeon R9 Nano
[off] - Radeon R9 Fury - AMD PAL - Radeon R9 Fury
- Radeon R9 FuryX - Radeon R9 FuryX
- Radeon Pro Duo - Radeon Pro Duo
- FirePro S9300x2 - FirePro S9300x2
- Radeon Instinct MI8 - Radeon Instinct MI8
\ - ``polaris10`` ``amdgcn`` dGPU - xnack ROCm - Radeon RX 470 \ - ``polaris10`` ``amdgcn`` dGPU - AMD ROCm - Radeon RX 470
[off] - Radeon RX 480 - AMD PAL - Radeon RX 480
- Radeon Instinct MI6 - Radeon Instinct MI6
\ - ``polaris11`` ``amdgcn`` dGPU - xnack ROCm - Radeon RX 460 \ - ``polaris11`` ``amdgcn`` dGPU - AMD ROCm - Radeon RX 460
[off] - AMD PAL
``gfx805`` - ``tongapro`` ``amdgcn`` dGPU - xnack ROCm - FirePro S7150 ``gfx805`` - ``tongapro`` ``amdgcn`` dGPU - AMD ROCm - FirePro S7150
[off] - FirePro S7100 - AMD PAL - FirePro S7100
- FirePro W7100 - FirePro W7100
- Mobile FirePro - Mobile FirePro
M7170 M7170
``gfx810`` - ``stoney`` ``amdgcn`` APU - xnack *TBA* ``gfx810`` - ``stoney`` ``amdgcn`` APU - xnack - AMD ROCm *TBA*
[on] - AMD PAL
.. TODO:: .. TODO::
Add product Add product
names. names.
**GCN GFX9** [AMD-GCN-GFX9]_ **GCN GFX9** [AMD-GCN-GFX9]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``gfx900`` ``amdgcn`` dGPU - xnack ROCm - Radeon Vega ``gfx900`` ``amdgcn`` dGPU - xnack - AMD ROCm - Radeon Vega
[off] Frontier Edition - AMD PAL Frontier Edition
- Radeon RX Vega 56 - Radeon RX Vega 56
- Radeon RX Vega 64 - Radeon RX Vega 64
- Radeon RX Vega 64 - Radeon RX Vega 64
Liquid Liquid
- Radeon Instinct MI25 - Radeon Instinct MI25
``gfx902`` ``amdgcn`` APU - xnack - Ryzen 3 2200G ``gfx902`` ``amdgcn`` APU - xnack - AMD ROCm - Ryzen 3 2200G
[on] - Ryzen 5 2400G - AMD PAL - Ryzen 5 2400G
``gfx904`` ``amdgcn`` dGPU - xnack *TBA* ``gfx904`` ``amdgcn`` dGPU - xnack - AMD ROCm *TBA*
[off] - AMD PAL
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx906`` ``amdgcn`` dGPU - xnack - Radeon Instinct MI50 ``gfx906`` ``amdgcn`` dGPU - sramecc - AMD ROCm - Radeon Instinct MI50
[off] - Radeon Instinct MI60 - xnack - AMD PAL - Radeon Instinct MI60
- sram-ecc - Radeon VII - Radeon VII
[off] - Radeon Pro VII - Radeon Pro VII
``gfx908`` ``amdgcn`` dGPU - xnack *TBA* ``gfx908`` ``amdgcn`` dGPU - sramecc - AMD ROCm *TBA*
[off] - xnack
- sram-ecc
[on]
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx909`` ``amdgcn`` APU - xnack *TBA* ``gfx909`` ``amdgcn`` APU - xnack - AMD PAL *TBA*
[off]
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx90c`` ``amdgcn`` APU - xnack - Ryzen 7 4700G ``gfx90c`` ``amdgcn`` APU - xnack - AMD PAL - Ryzen 7 4700G
[on] - Ryzen 7 4700GE
- Ryzen 7 4700G
- Ryzen 7 4700GE - Ryzen 7 4700GE
- Ryzen 5 4600G - Ryzen 5 4600G
- Ryzen 5 4600GE - Ryzen 5 4600GE
- Ryzen 3 4300G - Ryzen 3 4300G
- Ryzen 3 4300GE - Ryzen 3 4300GE
- Ryzen Pro 4000G - Ryzen Pro 4000G
- Ryzen 7 Pro 4700G - Ryzen 7 Pro 4700G
- Ryzen 7 Pro 4750GE - Ryzen 7 Pro 4750GE
- Ryzen 5 Pro 4650G - Ryzen 5 Pro 4650G
- Ryzen 5 Pro 4650GE - Ryzen 5 Pro 4650GE
- Ryzen 3 Pro 4350G - Ryzen 3 Pro 4350G
- Ryzen 3 Pro 4350GE - Ryzen 3 Pro 4350GE
**GCN GFX10** [AMD-GCN-GFX10]_ **GCN GFX10** [AMD-GCN-GFX10]_
----------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------
``gfx1010`` ``amdgcn`` dGPU - xnack - Radeon RX 5700 ``gfx1010`` ``amdgcn`` dGPU - cumode - AMD ROCm - Radeon RX 5700
[off] - Radeon RX 5700 XT - wavefrontsize64 - AMD PAL - Radeon RX 5700 XT
- wavefrontsize64 - Radeon Pro 5600 XT - xnack - Radeon Pro 5600 XT
[off] - Radeon Pro 5600M - Radeon Pro 5600M
- cumode ``gfx1011`` ``amdgcn`` dGPU - cumode - AMD ROCm *TBA*
[off] - wavefrontsize64 - AMD PAL
``gfx1011`` ``amdgcn`` dGPU - xnack *TBA* - xnack
[off]
- wavefrontsize64
[off]
- cumode
[off]
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx1012`` ``amdgcn`` dGPU - xnack - Radeon RX 5500 ``gfx1012`` ``amdgcn`` dGPU - cumode - AMD ROCm - Radeon RX 5500
[off] - Radeon RX 5500 XT - wavefrontsize64 - AMD PAL - Radeon RX 5500 XT
- wavefrontsize64 - xnack
[off]
- cumode ``gfx1030`` ``amdgcn`` dGPU - cumode - AMD ROCm *TBA*
[off] - wavefrontsize64 - AMD PAL
``gfx1030`` ``amdgcn`` dGPU - wavefrontsize64 *TBA*
[off]
- cumode
[off]
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx1031`` ``amdgcn`` dGPU - wavefrontsize64 *TBA* ``gfx1031`` ``amdgcn`` dGPU - cumode - AMD ROCm *TBA*
[off] - wavefrontsize64 - AMD PAL
- cumode
[off]
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx1032`` ``amdgcn`` dGPU - wavefrontsize64 *TBA* ``gfx1032`` ``amdgcn`` dGPU - cumode - AMD PAL *TBA*
[off] - wavefrontsize64
- cumode
[off]
.. TODO:: .. TODO::
Add product Add product
names. names.
``gfx1033`` ``amdgcn`` APU - wavefrontsize64 *TBA*
[off] ``gfx1033`` ``amdgcn`` APU - cumode - AMD PAL *TBA*
- cumode - wavefrontsize64
[off]
.. TODO:: .. TODO::
Add product Add product
names. names.
=========== =============== ============ ===== ============================= ======= ====================== =========== =============== ============ ===== ================= =========== ============== ======================
.. _amdgpu-target-features: .. _amdgpu-target-features:
Target Features Target Features
--------------- ---------------
Target features control how code is generated to support certain Target features control how code is generated to support certain
processor specific features. Not all target features are supported by processor specific features. Not all target features are supported by
all processors. The runtime must ensure that the features supported by all processors. The runtime must ensure that the features supported by
the device used to execute the code match the features enabled when the device used to execute the code match the features enabled when
generating the code. A mismatch of features may result in incorrect generating the code. A mismatch of features may result in incorrect
execution, or a reduction in performance. execution, or a reduction in performance.
The target features supported by each processor, and the default value The target features supported by each processor is listed in
used if not specified explicitly, is listed in
:ref:`amdgpu-processor-table`. :ref:`amdgpu-processor-table`.
Use the ``clang -m[no-]<TargetFeature>`` option to specify the AMDGPU Target features are controlled by exactly one of the following Clang
target features. options:
``-mcpu=<target-id>`` or ``--offload-arch=<target-id>``
The ``-mcpu`` and ``--offload-arch`` can specify the target feature as
optional components of the target ID. If omitted, the target feature has the
``any`` value. See :ref:`amdgpu-target-id`.
``-m[no-]<target-feature>``
Target features not specified by the target ID are specified using a
separate option. These target features can have an ``on`` or ``off``
value. ``on`` is specified by omitting the ``no-`` prefix, and
``off`` is specified by including the ``no-`` prefix. The default
if not specified is ``off``.
For example: For example:
``-mxnack`` ``-mcpu=gfx908:xnack+``
Enable the ``xnack`` feature. Enable the ``xnack`` feature.
``-mno-xnack`` ``-mcpu=gfx908:xnack-``
Disable the ``xnack`` feature. Disable the ``xnack`` feature.
``-mcumode``
Enable the ``cumode`` feature.
``-mno-cumode``
Disable the ``cumode`` feature.
.. table:: AMDGPU Target Features .. table:: AMDGPU Target Features
:name: amdgpu-target-feature-table :name: amdgpu-target-features-table
====================== ================================================== =============== ============================ ==================================================
Target Feature Description Target Feature Clang Option to Control Description
====================== ================================================== Name
-m[no-]xnack Enable/disable generating code that has =============== ============================ ==================================================
memory clauses that are compatible with cumode - ``-m[no-]cumode`` Control the wavefront execution mode used
having XNACK replay enabled. when generating code for kernels. When disabled
native WGP wavefront execution mode is used,
when enabled CU wavefront execution mode is used
(see :ref:`amdgpu-amdhsa-memory-model`).
sramecc - ``-mcpu`` If specified, generate code that can only be
- ``--offload-arch`` loaded and executed in a process that has a
matching setting for SRAMECC.
If not specified, generate code that can be
loaded and executed in a process with either
setting of SRAMECC.
wavefrontsize64 - ``-m[no-]wavefrontsize64`` Control the wavefront size used when
generating code for kernels. When disabled
native wavefront size 32 is used, when enabled
wavefront size 64 is used.
xnack - ``-mcpu`` If specified, generate code that can only be
- ``--offload-arch`` loaded and executed in a process that has a
matching setting for XNACK replay.
If not specified, generate code that can be
loaded and executed in a process with either
setting of XNACK replay.
This is used for demand paging and page This is used for demand paging and page
migration. If XNACK replay is enabled in migration. If XNACK replay is enabled in
the device, then if a page fault occurs the device, then if a page fault occurs
the code may execute incorrectly if the the code may execute incorrectly if the
``xnack`` feature is not enabled. Executing ``xnack`` feature is not enabled. Executing
code that has the feature enabled on a code that has the feature enabled on a
device that does not have XNACK replay device that does not have XNACK replay
enabled will execute correctly but may enabled will execute correctly but may
be less performant than code with the be less performant than code with the
feature disabled. feature disabled.
=============== ============================ ==================================================
-m[no-]sram-ecc Enable/disable generating code that assumes SRAM .. _amdgpu-target-id:
ECC is enabled/disabled.
-m[no-]wavefrontsize64 Control the default wavefront size used when Target ID
generating code for kernels. When disabled ---------
native wavefront size 32 is used, when enabled
wavefront size 64 is used.
-m[no-]cumode Control the default wavefront execution mode used AMDGPU supports target IDs. See `Clang Offload Bundler
when generating code for kernels. When disabled <https://clang.llvm.org/docs/ClangOffload.html>`_ for a general description. The
native WGP wavefront execution mode is used, AMDGPU target specific information is:
when enabled CU wavefront execution mode is used
(see :ref:`amdgpu-amdhsa-memory-model`). **processor**
====================== ================================================== Is a AMDGPU processor or alternative processor name specified in
:ref:`amdgpu-processor-table`. The non-canonical form target ID allows both
the primary processor and alternative processor names. The canonical form
target ID only allow the primary processor name.
**target-feature**
Is a target feature name specified in :ref:`amdgpu-target-features-table` that
is supported by the processor. The target features supported by each processor
is specified in :ref:`amdgpu-processor-table`. Those that can be specifeid in
a target ID are marked as being controlled by ``-mcpu`` and
``--offload-arch``. Each target feature must appear at most once in a target
ID. The non-canonical form target ID allows the target features to be
specified in any order. The canonical form target ID requires the target
features to be specified in alphabetic order.
.. _amdgpu-embedding-bundled-objects:
Embedding Bundled Code Objects
------------------------------
AMDGPU supports the HIP and OpenMP languages that perform code object embedding
as described in `Clang Offload Bundler
<https://clang.llvm.org/docs/ClangOffload.html>`_.
.. _amdgpu-address-spaces: .. _amdgpu-address-spaces:
Address Spaces Address Spaces
-------------- --------------
The AMDGPU architecture supports a number of memory address spaces. The address The AMDGPU architecture supports a number of memory address spaces. The address
space names use the OpenCL standard names, with some additions. space names use the OpenCL standard names, with some additions.
Show All 20 Lines .. table:: AMDGPU Address Spaces
Local 3 group LDS 32 0xFFFFFFFF Local 3 group LDS 32 0xFFFFFFFF
Constant 4 constant *same as global* 64 0x0000000000000000 Constant 4 constant *same as global* 64 0x0000000000000000
Private 5 private scratch 32 0xFFFFFFFF Private 5 private scratch 32 0xFFFFFFFF
Constant 32-bit 6 *TODO* 0x00000000 Constant 32-bit 6 *TODO* 0x00000000
Buffer Fat Pointer (experimental) 7 *TODO* Buffer Fat Pointer (experimental) 7 *TODO*
================================= =============== =========== ================ ======= ============================ ================================= =============== =========== ================ ======= ============================
**Generic** **Generic**
The generic address space uses the hardware flat address support available in The generic address space is supported unless the *Target Properties* column
GFX7-GFX10. This uses two fixed ranges of virtual addresses (the private and of :ref:`amdgpu-processor-table` specifies *Does not support generic address
local apertures), that are outside the range of addressable global memory, to space*.
map from a flat address to a private or local address.
The generic address space uses the hardware flat address support for two fixed
FLAT instructions can take a flat address and access global, private ranges of virtual addresses (the private and local apertures), that are
(scratch), and group (LDS) memory depending on if the address is within one outside the range of addressable global memory, to map from a flat address to
of the aperture ranges. Flat access to scratch requires hardware aperture a private or local address. This uses FLAT instructions that can take a flat
setup and setup in the kernel prologue (see address and access global, private (scratch), and group (LDS) memory depending
:ref:`amdgpu-amdhsa-kernel-prolog-flat-scratch`). Flat access to LDS requires on if the address is within one of the aperture ranges.
hardware aperture setup and M0 (GFX7-GFX8) register setup (see
:ref:`amdgpu-amdhsa-kernel-prolog-m0`). Flat access to scratch requires hardware aperture setup and setup in the
kernel prologue (see :ref:`amdgpu-amdhsa-kernel-prolog-flat-scratch`). Flat
access to LDS requires hardware aperture setup and M0 (GFX7-GFX8) register
setup (see :ref:`amdgpu-amdhsa-kernel-prolog-m0`).
To convert between a private or group address space address (termed a segment To convert between a private or group address space address (termed a segment
address) and a flat address the base address of the corresponding aperture address) and a flat address the base address of the corresponding aperture
can be used. For GFX7-GFX8 these are available in the can be used. For GFX7-GFX8 these are available in the
:ref:`amdgpu-amdhsa-hsa-aql-queue` the address of which can be obtained with :ref:`amdgpu-amdhsa-hsa-aql-queue` the address of which can be obtained with
Queue Ptr SGPR (see :ref:`amdgpu-amdhsa-initial-kernel-execution-state`). For Queue Ptr SGPR (see :ref:`amdgpu-amdhsa-initial-kernel-execution-state`). For
GFX9-GFX10 the aperture base addresses are directly available as inline GFX9-GFX10 the aperture base addresses are directly available as inline
constant registers ``SRC_SHARED_BASE/LIMIT`` and ``SRC_PRIVATE_BASE/LIMIT``. constant registers ``SRC_SHARED_BASE/LIMIT`` and ``SRC_PRIVATE_BASE/LIMIT``.
▲ Show 20 Lines • Show All 243 Lines • ▼ Show 20 Lines .. table:: AMDGPU ELF Header
Field Value Field Value
========================== =============================== ========================== ===============================
``e_ident[EI_CLASS]`` ``ELFCLASS64`` ``e_ident[EI_CLASS]`` ``ELFCLASS64``
``e_ident[EI_DATA]`` ``ELFDATA2LSB`` ``e_ident[EI_DATA]`` ``ELFDATA2LSB``
``e_ident[EI_OSABI]`` - ``ELFOSABI_NONE`` ``e_ident[EI_OSABI]`` - ``ELFOSABI_NONE``
- ``ELFOSABI_AMDGPU_HSA`` - ``ELFOSABI_AMDGPU_HSA``
- ``ELFOSABI_AMDGPU_PAL`` - ``ELFOSABI_AMDGPU_PAL``
- ``ELFOSABI_AMDGPU_MESA3D`` - ``ELFOSABI_AMDGPU_MESA3D``
``e_ident[EI_ABIVERSION]`` - ``ELFABIVERSION_AMDGPU_HSA`` ``e_ident[EI_ABIVERSION]`` - ``ELFABIVERSION_AMDGPU_HSA_V2``
- ``ELFABIVERSION_AMDGPU_HSA_V3``
- ``ELFABIVERSION_AMDGPU_HSA_V4``
- ``ELFABIVERSION_AMDGPU_PAL`` - ``ELFABIVERSION_AMDGPU_PAL``
- ``ELFABIVERSION_AMDGPU_MESA3D`` - ``ELFABIVERSION_AMDGPU_MESA3D``
``e_type`` - ``ET_REL`` ``e_type`` - ``ET_REL``
- ``ET_DYN`` - ``ET_DYN``
``e_machine`` ``EM_AMDGPU`` ``e_machine`` ``EM_AMDGPU``
``e_entry`` 0 ``e_entry`` 0
``e_flags`` See :ref:`amdgpu-elf-header-e_flags-table` ``e_flags`` See :ref:`amdgpu-elf-header-e_flags-v2-table`,
:ref:`amdgpu-elf-header-e_flags-table-v3`,
and :ref:`amdgpu-elf-header-e_flags-table-v4`
========================== =============================== ========================== ===============================
.. ..
.. table:: AMDGPU ELF Header Enumeration Values .. table:: AMDGPU ELF Header Enumeration Values
:name: amdgpu-elf-header-enumeration-values-table :name: amdgpu-elf-header-enumeration-values-table
=============================== ===== =============================== =====
Name Value Name Value
=============================== ===== =============================== =====
``EM_AMDGPU`` 224 ``EM_AMDGPU`` 224
``ELFOSABI_NONE`` 0 ``ELFOSABI_NONE`` 0
``ELFOSABI_AMDGPU_HSA`` 64 ``ELFOSABI_AMDGPU_HSA`` 64
``ELFOSABI_AMDGPU_PAL`` 65 ``ELFOSABI_AMDGPU_PAL`` 65
``ELFOSABI_AMDGPU_MESA3D`` 66 ``ELFOSABI_AMDGPU_MESA3D`` 66
``ELFABIVERSION_AMDGPU_HSA`` 1 ``ELFABIVERSION_AMDGPU_HSA_V2`` 0
``ELFABIVERSION_AMDGPU_HSA_V3`` 1
``ELFABIVERSION_AMDGPU_HSA_V4`` 2
``ELFABIVERSION_AMDGPU_PAL`` 0 ``ELFABIVERSION_AMDGPU_PAL`` 0
``ELFABIVERSION_AMDGPU_MESA3D`` 0 ``ELFABIVERSION_AMDGPU_MESA3D`` 0
=============================== ===== =============================== =====
``e_ident[EI_CLASS]`` ``e_ident[EI_CLASS]``
The ELF class is: The ELF class is:
* ``ELFCLASS32`` for ``r600`` architecture. * ``ELFCLASS32`` for ``r600`` architecture.
* ``ELFCLASS64`` for ``amdgcn`` architecture which only supports 64-bit * ``ELFCLASS64`` for ``amdgcn`` architecture which only supports 64-bit
process address space applications. process address space applications.
``e_ident[EI_DATA]`` ``e_ident[EI_DATA]``
All AMDGPU targets use ``ELFDATA2LSB`` for little-endian byte ordering. All AMDGPU targets use ``ELFDATA2LSB`` for little-endian byte ordering.
``e_ident[EI_OSABI]`` ``e_ident[EI_OSABI]``
One of the following AMDGPU target architecture specific OS ABIs One of the following AMDGPU target architecture specific OS ABIs
(see :ref:`amdgpu-os-table`): (see :ref:`amdgpu-os`):
* ``ELFOSABI_NONE`` for *unknown* OS. * ``ELFOSABI_NONE`` for *unknown* OS.
* ``ELFOSABI_AMDGPU_HSA`` for ``amdhsa`` OS. * ``ELFOSABI_AMDGPU_HSA`` for ``amdhsa`` OS.
* ``ELFOSABI_AMDGPU_PAL`` for ``amdpal`` OS. * ``ELFOSABI_AMDGPU_PAL`` for ``amdpal`` OS.
* ``ELFOSABI_AMDGPU_MESA3D`` for ``mesa3D`` OS. * ``ELFOSABI_AMDGPU_MESA3D`` for ``mesa3D`` OS.
``e_ident[EI_ABIVERSION]`` ``e_ident[EI_ABIVERSION]``
The ABI version of the AMDGPU target architecture specific OS ABI to which the code The ABI version of the AMDGPU target architecture specific OS ABI to which the code
object conforms: object conforms:
* ``ELFABIVERSION_AMDGPU_HSA`` is used to specify the version of AMD HSA * ``ELFABIVERSION_AMDGPU_HSA_V2`` is used to specify the version of AMD HSA
runtime ABI. runtime ABI for code object V2. Specify using the Clang option
``-mcode-object-version=2``.
* ``ELFABIVERSION_AMDGPU_HSA_V3`` is used to specify the version of AMD HSA
runtime ABI for code object V3. Specify using the Clang option
``-mcode-object-version=3``.
* ``ELFABIVERSION_AMDGPU_HSA_V4`` is used to specify the version of AMD HSA
runtime ABI for code object V4. Specify using the Clang option
``-mcode-object-version=4``. This is the default code object
version if not specified.
* ``ELFABIVERSION_AMDGPU_PAL`` is used to specify the version of AMD PAL * ``ELFABIVERSION_AMDGPU_PAL`` is used to specify the version of AMD PAL
runtime ABI. runtime ABI.
* ``ELFABIVERSION_AMDGPU_MESA3D`` is used to specify the version of AMD MESA * ``ELFABIVERSION_AMDGPU_MESA3D`` is used to specify the version of AMD MESA
3D runtime ABI. 3D runtime ABI.
``e_type`` ``e_type``
Can be one of the following values: Can be one of the following values:
``ET_REL`` ``ET_REL``
The type produced by the AMDGPU backend compiler as it is relocatable code The type produced by the AMDGPU backend compiler as it is relocatable code
object. object.
``ET_DYN`` ``ET_DYN``
The type produced by the linker as it is a shared code object. The type produced by the linker as it is a shared code object.
The AMD HSA runtime loader requires a ``ET_DYN`` code object. The AMD HSA runtime loader requires a ``ET_DYN`` code object.
``e_machine`` ``e_machine``
The value ``EM_AMDGPU`` is used for the machine for all processors supported The value ``EM_AMDGPU`` is used for the machine for all processors supported
by the ``r600`` and ``amdgcn`` architectures (see by the ``r600`` and ``amdgcn`` architectures (see
:ref:`amdgpu-processor-table`). The specific processor is specified in the :ref:`amdgpu-processor-table`). The specific processor is specified in the
``EF_AMDGPU_MACH`` bit field of the ``e_flags`` (see ``NT_AMD_HSA_ISA_VERSION`` note record for code object V2 (see
:ref:`amdgpu-elf-header-e_flags-table`). :ref:`amdgpu-note-records-v2`) and in the ``EF_AMDGPU_MACH`` bit field of the
``e_flags`` for code object V3 to V4 (see
:ref:`amdgpu-elf-header-e_flags-table-v3` and
:ref:`amdgpu-elf-header-e_flags-table-v4`).
``e_entry`` ``e_entry``
The entry point is 0 as the entry points for individual kernels must be The entry point is 0 as the entry points for individual kernels must be
selected in order to invoke them through AQL packets. selected in order to invoke them through AQL packets.
``e_flags`` ``e_flags``
The AMDGPU backend uses the following ELF header flags: The AMDGPU backend uses the following ELF header flags:
.. table:: AMDGPU ELF Header ``e_flags`` .. table:: AMDGPU ELF Header ``e_flags`` for Code Object V2
:name: amdgpu-elf-header-e_flags-table :name: amdgpu-elf-header-e_flags-v2-table
================================= ========== ============================= ===================================== ===== =============================
Name Value Description Name Value Description
================================= ========== ============================= ===================================== ===== =============================
**AMDGPU Processor Flag** See :ref:`amdgpu-processor-table`. ``EF_AMDGPU_FEATURE_XNACK_V2`` 0x01 Indicates if the ``xnack``
traUnsubmitted
Not Done
ReplyInline Actions

Where do these flags live?
I do not see them in https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/BinaryFormat/ELF.h#L738

Do these doc changes reflect a patch that did not land yet?

tra: Where do these flags live? I do not see them in https://github.com/llvm/llvm…
-------------------------------------------- ----------------------------- target feature is
``EF_AMDGPU_MACH`` 0x000000ff AMDGPU processor selection enabled for all code
contained in the code object.
If the processor
does not support the
``xnack`` target
feature then must
be 0.
See
:ref:`amdgpu-target-features`.
``EF_AMDGPU_FEATURE_TRAP_HANDLER_V2`` 0x02 Indicates if the trap
handler is enabled for all
code contained in the code
object. If the processor
does not support a trap
handler then must be 0.
See
:ref:`amdgpu-target-features`.
===================================== ===== =============================
.. table:: AMDGPU ELF Header ``e_flags`` for Code Object V3
:name: amdgpu-elf-header-e_flags-table-v3
================================= ===== =============================
Name Value Description
================================= ===== =============================
``EF_AMDGPU_MACH`` 0x0ff AMDGPU processor selection
mask for mask for
``EF_AMDGPU_MACH_xxx`` values ``EF_AMDGPU_MACH_xxx`` values
defined in defined in
:ref:`amdgpu-ef-amdgpu-mach-table`. :ref:`amdgpu-ef-amdgpu-mach-table`.
``EF_AMDGPU_XNACK`` 0x00000100 Indicates if the ``xnack`` ``EF_AMDGPU_FEATURE_XNACK_V3`` 0x100 Indicates if the ``xnack``
target feature is target feature is
enabled for all code enabled for all code
contained in the code object. contained in the code object.
If the processor If the processor
does not support the does not support the
``xnack`` target ``xnack`` target
feature then must feature then must
be 0. be 0.
See See
:ref:`amdgpu-target-features`. :ref:`amdgpu-target-features`.
``EF_AMDGPU_SRAM_ECC`` 0x00000200 Indicates if the ``sram-ecc`` ``EF_AMDGPU_FEATURE_SRAMECC_V3`` 0x200 Indicates if the ``sramecc``
target feature is target feature is
enabled for all code enabled for all code
contained in the code object. contained in the code object.
If the processor If the processor
does not support the does not support the
``sram-ecc`` target ``sramecc`` target
feature then must feature then must
be 0. be 0.
See See
:ref:`amdgpu-target-features`. :ref:`amdgpu-target-features`.
================================= ========== ============================= ================================= ===== =============================
.. table:: AMDGPU ELF Header ``e_flags`` for Code Object V4
:name: amdgpu-elf-header-e_flags-table-v4
============================================ ===== ===================================
Name Value Description
============================================ ===== ===================================
``EF_AMDGPU_MACH`` 0x0ff AMDGPU processor selection
mask for
``EF_AMDGPU_MACH_xxx`` values
defined in
:ref:`amdgpu-ef-amdgpu-mach-table`.
``EF_AMDGPU_FEATURE_XNACK_V4`` 0x300 XNACK selection mask for
``EF_AMDGPU_FEATURE_XNACK_*_V4``
values.
``EF_AMDGPU_FEATURE_XNACK_UNSUPPORTED_V4`` 0x000 XNACK unsuppored.
``EF_AMDGPU_FEATURE_XNACK_ANY_V4`` 0x100 XNACK can have any value.
``EF_AMDGPU_FEATURE_XNACK_OFF_V4`` 0x200 XNACK disabled.
``EF_AMDGPU_FEATURE_XNACK_ON_V4`` 0x300 XNACK enabled.
``EF_AMDGPU_FEATURE_SRAMECC_V4`` 0xc00 SRAMECC selection mask for
``EF_AMDGPU_FEATURE_SRAMECC_*_V4``
values.
``EF_AMDGPU_FEATURE_SRAMECC_UNSUPPORTED_V4`` 0x000 SRAMECC unsuppored.
``EF_AMDGPU_FEATURE_SRAMECC_ANY_V4`` 0x400 SRAMECC can have any value.
``EF_AMDGPU_FEATURE_SRAMECC_OFF_V4`` 0x800 SRAMECC disabled,
``EF_AMDGPU_FEATURE_SRAMECC_ON_V4`` 0xc00 SRAMECC enabled.
============================================ ===== ===================================
.. table:: AMDGPU ``EF_AMDGPU_MACH`` Values .. table:: AMDGPU ``EF_AMDGPU_MACH`` Values
:name: amdgpu-ef-amdgpu-mach-table :name: amdgpu-ef-amdgpu-mach-table
==================================== ========== ============================= ==================================== ========== =============================
Name Value Description (see Name Value Description (see
:ref:`amdgpu-processor-table`) :ref:`amdgpu-processor-table`)
==================================== ========== ============================= ==================================== ========== =============================
▲ Show 20 Lines • Show All 109 Lines • ▼ Show 20 Lines
.. _amdgpu-note-records: .. _amdgpu-note-records:
Note Records Note Records
------------ ------------
The AMDGPU backend code object contains ELF note records in the ``.note`` The AMDGPU backend code object contains ELF note records in the ``.note``
section. The set of generated notes and their semantics depend on the code section. The set of generated notes and their semantics depend on the code
object version; see :ref:`amdgpu-note-records-v2` and object version; see :ref:`amdgpu-note-records-v2` and
:ref:`amdgpu-note-records-v3`. :ref:`amdgpu-note-records-v3-v4`.
As required by ``ELFCLASS32`` and ``ELFCLASS64``, minimal zero-byte padding As required by ``ELFCLASS32`` and ``ELFCLASS64``, minimal zero-byte padding
must be generated after the ``name`` field to ensure the ``desc`` field is 4 must be generated after the ``name`` field to ensure the ``desc`` field is 4
byte aligned. In addition, minimal zero-byte padding must be generated to byte aligned. In addition, minimal zero-byte padding must be generated to
ensure the ``desc`` field size is a multiple of 4 bytes. The ``sh_addralign`` ensure the ``desc`` field size is a multiple of 4 bytes. The ``sh_addralign``
field of the ``.note`` section must be at least 4 to indicate at least 8 byte field of the ``.note`` section must be at least 4 to indicate at least 8 byte
alignment. alignment.
.. _amdgpu-note-records-v2: .. _amdgpu-note-records-v2:
Code Object V2 Note Records (--amdhsa-code-object-version=2) Code Object V2 Note Records
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. warning:: Code Object V2 is not the default code object version emitted by .. warning::
this version of LLVM. For a description of the notes generated with the Code object V2 is not the default code object version emitted by
default configuration (Code Object V3) see :ref:`amdgpu-note-records-v3`. this version of LLVM.
The AMDGPU backend code object uses the following ELF note record in the The AMDGPU backend code object uses the following ELF note record in the
``.note`` section when compiling for Code Object V2 ``.note`` section when compiling for code object V2.
(--amdhsa-code-object-version=2).
The note record vendor field is "AMD".
Additional note records may be present, but any which are not documented here Additional note records may be present, but any which are not documented here
are deprecated and should not be used. are deprecated and should not be used.
.. table:: AMDGPU Code Object V2 ELF Note Records .. table:: AMDGPU Code Object V2 ELF Note Records
:name: amdgpu-elf-note-records-table-v2 :name: amdgpu-elf-note-records-v2-table
===== ============================== ====================================== ===== ===================================== ======================================
Name Type Description Name Type Description
===== ============================== ====================================== ===== ===================================== ======================================
"AMD" ``NT_AMD_AMDGPU_HSA_METADATA`` <metadata null terminated string> "AMD" ``NT_AMD_HSA_CODE_OBJECT_VERSION`` Code object version.
===== ============================== ====================================== "AMD" ``NT_AMD_HSA_HSAIL`` HSAIL properties generated by the HSAIL
Finalizer and not the LLVM compiler.
"AMD" ``NT_AMD_HSA_ISA_VERSION`` Target ISA version.
"AMD" ``NT_AMD_HSA_METADATA`` Metadata null terminated string in
YAML [YAML]_ textual format.
"AMD" ``NT_AMD_HSA_ISA_NAME`` Target ISA name.
===== ===================================== ======================================
.. ..
.. table:: AMDGPU Code Object V2 ELF Note Record Enumeration Values .. table:: AMDGPU Code Object V2 ELF Note Record Enumeration Values
:name: amdgpu-elf-note-record-enumeration-values-table-v2 :name: amdgpu-elf-note-record-enumeration-values-v2-table
============================== ===== ===================================== =====
Name Value Name Value
============================== ===== ===================================== =====
*reserved* 0-9 ``NT_AMD_HSA_CODE_OBJECT_VERSION`` 1
``NT_AMD_AMDGPU_HSA_METADATA`` 10 ``NT_AMD_HSA_HSAIL`` 2
*reserved* 11 ``NT_AMD_HSA_ISA_VERSION`` 3
============================== ===== *reserved* 4-9
``NT_AMD_HSA_METADATA`` 10
``NT_AMD_HSA_ISA_NAME`` 11
===================================== =====
``NT_AMD_HSA_CODE_OBJECT_VERSION``
Specifies the code object version number. The description field has the
following layout:
``NT_AMD_AMDGPU_HSA_METADATA`` .. code::
struct amdgpu_hsa_note_code_object_version_s {
uint32_t major_version;
uint32_t minor_version;
};
The ``major_version`` has a value less than or equal to 2.
``NT_AMD_HSA_HSAIL``
Specifies the HSAIL properties used by the HSAIL Finalizer. The description
field has the following layout:
.. code::
struct amdgpu_hsa_note_hsail_s {
uint32_t hsail_major_version;
uint32_t hsail_minor_version;
uint8_t profile;
uint8_t machine_model;
uint8_t default_float_round;
};
``NT_AMD_HSA_ISA_VERSION``
Specifies the target ISA version. The description field has the following layout:
.. code::
struct amdgpu_hsa_note_isa_s {
uint16_t vendor_name_size;
uint16_t architecture_name_size;
uint32_t major;
uint32_t minor;
uint32_t stepping;
char vendor_and_architecture_name[1];
};
``vendor_name_size`` and ``architecture_name_size`` are the length of the
vendor and architecture names respectively, including the NUL character.
``vendor_and_architecture_name`` contains the NUL terminates string for the
vendor, immediately followed by the NUL terminated string for the
architecture.
This note record is used by the HSA runtime loader.
Code object V2 only supports a limited number of processors and has fixed
settings for target features. See
:ref:`amdgpu-elf-note-record-supported_processors-v2-table` for a list of
processors and the corresponding target ID. In the table the note record ISA
name is a concatenation of the vendor name, architecture name, major, minor,
and stepping separated by a ":".
The target ID column shows the processor name and fixed target features used
by the LLVM compiler. The LLVM compiler does not generate a
``NT_AMD_HSA_HSAIL`` note record.
A code object generated by the Finalizer also uses code object V2 and always
generates a ``NT_AMD_HSA_HSAIL`` note record. The processor name and
``sramecc`` target feature is as shown in
:ref:`amdgpu-elf-note-record-supported_processors-v2-table` but the ``xnack``
target feature is specified by the ``EF_AMDGPU_FEATURE_XNACK_V2`` ``e_flags``
bit.
``NT_AMD_HSA_ISA_NAME``
Specifies the target ISA name as a non-NUL terminated string.
This note record is not used by the HSA runtime loader.
See the ``NT_AMD_HSA_ISA_VERSION`` note record description of the code object
V2's limited support of processors and fixed settings for target features.
See :ref:`amdgpu-elf-note-record-supported_processors-v2-table` for a mapping
from the string to the corresponding target ID. If the ``xnack`` target
feature is supported and enabled, the string produced by the LLVM compiler
will may have a ``+xnack`` appended. The Finlizer did not do the appending and
instead used the ``EF_AMDGPU_FEATURE_XNACK_V2`` ``e_flags`` bit.
``NT_AMD_HSA_METADATA``
Specifies extensible metadata associated with the code objects executed on HSA Specifies extensible metadata associated with the code objects executed on HSA
[HSA]_ compatible runtimes such as AMD's ROCm [AMD-ROCm]_. It is required when [HSA]_ compatible runtimes (see :ref:`amdgpu-os`). It is required when the
the target triple OS is ``amdhsa`` (see :ref:`amdgpu-target-triples`). See target triple OS is ``amdhsa`` (see :ref:`amdgpu-target-triples`). See
:ref:`amdgpu-amdhsa-code-object-metadata-v2` for the syntax of the code :ref:`amdgpu-amdhsa-code-object-metadata-v2` for the syntax of the code object
object metadata string. metadata string.
.. table:: AMDGPU Code Object V2 Supported Processors and Fixed Target Feature Settings
:name: amdgpu-elf-note-record-supported_processors-v2-table
==================== ==========================
Note Record ISA Name Target ID
==================== ==========================
``AMD:AMDGPU:6:0:0`` ``gfx600``
``AMD:AMDGPU:6:0:1`` ``gfx601``
``AMD:AMDGPU:6:0:2`` ``gfx602``
``AMD:AMDGPU:7:0:0`` ``gfx700``
``AMD:AMDGPU:7:0:1`` ``gfx701``
``AMD:AMDGPU:7:0:2`` ``gfx702``
``AMD:AMDGPU:7:0:3`` ``gfx703``
``AMD:AMDGPU:7:0:4`` ``gfx704``
``AMD:AMDGPU:7:0:5`` ``gfx705``
``AMD:AMDGPU:8:0:1`` ``gfx801:xnack+``
``AMD:AMDGPU:8:0:2`` ``gfx802``
``AMD:AMDGPU:8:0:3`` ``gfx803``
``AMD:AMDGPU:8:0:5`` ``gfx805``
``AMD:AMDGPU:8:1:0`` ``gfx810:xnack+``
``AMD:AMDGPU:9:0:0`` ``gfx900:xnack-``
``AMD:AMDGPU:9:0:1`` ``gfx900:xnack+``
``AMD:AMDGPU:9:0:2`` ``gfx902:xnack-``
``AMD:AMDGPU:9:0:3`` ``gfx902:xnack+``
``AMD:AMDGPU:9:0:4`` ``gfx904:xnack-``
``AMD:AMDGPU:9:0:5`` ``gfx904:xnack+``
``AMD:AMDGPU:9:0:6`` ``gfx906:sramecc-:xnack-``
``AMD:AMDGPU:9:0:7`` ``gfx906:sramecc-:xnack+``
==================== ==========================
.. _amdgpu-note-records-v3: .. _amdgpu-note-records-v3-v4:
Code Object V3 Note Records (--amdhsa-code-object-version=3) Code Object V3 to V4 Note Records
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The AMDGPU backend code object uses the following ELF note record in the The AMDGPU backend code object uses the following ELF note record in the
``.note`` section when compiling for Code Object V3 ``.note`` section when compiling for code object V3 to V4.
(--amdhsa-code-object-version=3).
The note record vendor field is "AMDGPU".
Additional note records may be present, but any which are not documented here Additional note records may be present, but any which are not documented here
are deprecated and should not be used. are deprecated and should not be used.
.. table:: AMDGPU Code Object V3 ELF Note Records .. table:: AMDGPU Code Object V3 to V4 ELF Note Records
:name: amdgpu-elf-note-records-table-v3 :name: amdgpu-elf-note-records-table-v3-v4
======== ============================== ====================================== ======== ============================== ======================================
Name Type Description Name Type Description
======== ============================== ====================================== ======== ============================== ======================================
"AMDGPU" ``NT_AMDGPU_METADATA`` Metadata in Message Pack [MsgPack]_ "AMDGPU" ``NT_AMDGPU_METADATA`` Metadata in Message Pack [MsgPack]_
binary format. binary format.
======== ============================== ====================================== ======== ============================== ======================================
.. ..
.. table:: AMDGPU Code Object V3 ELF Note Record Enumeration Values .. table:: AMDGPU Code Object V3 to V4 ELF Note Record Enumeration Values
:name: amdgpu-elf-note-record-enumeration-values-table-v3 :name: amdgpu-elf-note-record-enumeration-values-table-v3-v4
============================== ===== ============================== =====
Name Value Name Value
============================== ===== ============================== =====
*reserved* 0-31 *reserved* 0-31
``NT_AMDGPU_METADATA`` 32 ``NT_AMDGPU_METADATA`` 32
============================== ===== ============================== =====
``NT_AMDGPU_METADATA`` ``NT_AMDGPU_METADATA``
Specifies extensible metadata associated with an AMDGPU code Specifies extensible metadata associated with an AMDGPU code object. It is
object. It is encoded as a map in the Message Pack [MsgPack]_ binary encoded as a map in the Message Pack [MsgPack]_ binary data format. See
data format. See :ref:`amdgpu-amdhsa-code-object-metadata-v3` for the :ref:`amdgpu-amdhsa-code-object-metadata-v3` and
map keys defined for the ``amdhsa`` OS. :ref:`amdgpu-amdhsa-code-object-metadata-v4` for the map keys defined for the
``amdhsa`` OS.
.. _amdgpu-symbols: .. _amdgpu-symbols:
Symbols Symbols
------- -------
Symbols include the following: Symbols include the following:
▲ Show 20 Lines • Show All 1,018 Lines • ▼ Show 20 Lines
(see :ref:`amdgpu-target-triples`). (see :ref:`amdgpu-target-triples`).
AMDHSA AMDHSA
------ ------
This section provides code conventions used when the target triple OS is This section provides code conventions used when the target triple OS is
``amdhsa`` (see :ref:`amdgpu-target-triples`). ``amdhsa`` (see :ref:`amdgpu-target-triples`).
.. _amdgpu-amdhsa-code-object-target-identification:
Code Object Target Identification
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The AMDHSA OS uses the following syntax to specify the code object
target as a single string:
``<Architecture>-<Vendor>-<OS>-<Environment>-<Processor><Target Features>``
Where:
- ``<Architecture>``, ``<Vendor>``, ``<OS>`` and ``<Environment>``
are the same as the *Target Triple* (see
:ref:`amdgpu-target-triples`).
- ``<Processor>`` is the same as the *Processor* (see
:ref:`amdgpu-processors`).
- ``<Target Features>`` is a list of the enabled *Target Features*
(see :ref:`amdgpu-target-features`), each prefixed by a plus, that
apply to *Processor*. The list must be in the same order as listed
in the table :ref:`amdgpu-target-feature-table`. Note that *Target
Features* must be included in the list if they are enabled even if
that is the default for *Processor*.
For example:
``"amdgcn-amd-amdhsa--gfx902+xnack"``
.. _amdgpu-amdhsa-code-object-metadata: .. _amdgpu-amdhsa-code-object-metadata:
Code Object Metadata Code Object Metadata
~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
The code object metadata specifies extensible metadata associated with the code The code object metadata specifies extensible metadata associated with the code
objects executed on HSA [HSA]_ compatible runtimes such as AMD's ROCm objects executed on HSA [HSA]_ compatible runtimes (see :ref:`amdgpu-os`). The
[AMD-ROCm]_. The encoding and semantics of this metadata depends on the code encoding and semantics of this metadata depends on the code object version; see
object version; see :ref:`amdgpu-amdhsa-code-object-metadata-v2` and :ref:`amdgpu-amdhsa-code-object-metadata-v2`,
:ref:`amdgpu-amdhsa-code-object-metadata-v3`. :ref:`amdgpu-amdhsa-code-object-metadata-v3`, and
:ref:`amdgpu-amdhsa-code-object-metadata-v4`.
Code object metadata is specified in a note record (see Code object metadata is specified in a note record (see
:ref:`amdgpu-note-records`) and is required when the target triple OS is :ref:`amdgpu-note-records`) and is required when the target triple OS is
``amdhsa`` (see :ref:`amdgpu-target-triples`). It must contain the minimum ``amdhsa`` (see :ref:`amdgpu-target-triples`). It must contain the minimum
information necessary to support the ROCM kernel queries. For example, the information necessary to support the HSA compatible runtime kernel queries. For
segment sizes needed in a dispatch packet. In addition, a high-level language example, the segment sizes needed in a dispatch packet. In addition, a
runtime may require other information to be included. For example, the AMD high-level language runtime may require other information to be included. For
OpenCL runtime records kernel argument information. example, the AMD OpenCL runtime records kernel argument information.
.. _amdgpu-amdhsa-code-object-metadata-v2: .. _amdgpu-amdhsa-code-object-metadata-v2:
Code Object V2 Metadata (--amdhsa-code-object-version=2) Code Object V2 Metadata
++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++
.. warning:: Code Object V2 is not the default code object version emitted by .. warning::
this version of LLVM. For a description of the metadata generated with the Code object V2 is not the default code object version emitted by this version
default configuration (Code Object V3) see of LLVM.
:ref:`amdgpu-amdhsa-code-object-metadata-v3`.
Code object V2 metadata is specified by the ``NT_AMD_AMDGPU_METADATA`` note Code object V2 metadata is specified by the ``NT_AMD_HSA_METADATA`` note record
record (see :ref:`amdgpu-note-records-v2`). (see :ref:`amdgpu-note-records-v2`).
The metadata is specified as a YAML formatted string (see [YAML]_ and The metadata is specified as a YAML formatted string (see [YAML]_ and
:doc:`YamlIO`). :doc:`YamlIO`).
.. TODO:: .. TODO::
Is the string null terminated? It probably should not if YAML allows it to Is the string null terminated? It probably should not if YAML allows it to
contain null characters, otherwise it should be. contain null characters, otherwise it should be.
The metadata is represented as a single YAML document comprised of the mapping The metadata is represented as a single YAML document comprised of the mapping
defined in table :ref:`amdgpu-amdhsa-code-object-metadata-map-table-v2` and defined in table :ref:`amdgpu-amdhsa-code-object-metadata-map-v2-table` and
referenced tables. referenced tables.
For boolean values, the string values of ``false`` and ``true`` are used for For boolean values, the string values of ``false`` and ``true`` are used for
false and true respectively. false and true respectively.
Additional information can be added to the mappings. To avoid conflicts, any Additional information can be added to the mappings. To avoid conflicts, any
non-AMD key names should be prefixed by "*vendor-name*.". non-AMD key names should be prefixed by "*vendor-name*.".
.. table:: AMDHSA Code Object V2 Metadata Map .. table:: AMDHSA Code Object V2 Metadata Map
:name: amdgpu-amdhsa-code-object-metadata-map-table-v2 :name: amdgpu-amdhsa-code-object-metadata-map-v2-table
========== ============== ========= ======================================= ========== ============== ========= =======================================
String Key Value Type Required? Description String Key Value Type Required? Description
========== ============== ========= ======================================= ========== ============== ========= =======================================
"Version" sequence of Required - The first integer is the major "Version" sequence of Required - The first integer is the major
2 integers version. Currently 1. 2 integers version. Currently 1.
- The second integer is the minor - The second integer is the minor
version. Currently 0. version. Currently 0.
Show All 20 Lines "Printf" sequence of Each string is encoded information
of the i-th FormatString argument of of the i-th FormatString argument of
the printf function call the printf function call
FormatString FormatString
The format string passed to the The format string passed to the
printf function call. printf function call.
"Kernels" sequence of Required Sequence of the mappings for each "Kernels" sequence of Required Sequence of the mappings for each
mapping kernel in the code object. See mapping kernel in the code object. See
:ref:`amdgpu-amdhsa-code-object-kernel-metadata-map-table-v2` :ref:`amdgpu-amdhsa-code-object-kernel-metadata-map-v2-table`
for the definition of the mapping. for the definition of the mapping.
========== ============== ========= ======================================= ========== ============== ========= =======================================
.. ..
.. table:: AMDHSA Code Object V2 Kernel Metadata Map .. table:: AMDHSA Code Object V2 Kernel Metadata Map
:name: amdgpu-amdhsa-code-object-kernel-metadata-map-table-v2 :name: amdgpu-amdhsa-code-object-kernel-metadata-map-v2-table
================= ============== ========= ================================ ================= ============== ========= ================================
String Key Value Type Required? Description String Key Value Type Required? Description
================= ============== ========= ================================ ================= ============== ========= ================================
"Name" string Required Source name of the kernel. "Name" string Required Source name of the kernel.
"SymbolName" string Required Name of the kernel "SymbolName" string Required Name of the kernel
descriptor ELF symbol. descriptor ELF symbol.
"Language" string Source language of the kernel. "Language" string Source language of the kernel.
Values include: Values include:
- "OpenCL C" - "OpenCL C"
- "OpenCL C++" - "OpenCL C++"
- "HCC" - "HCC"
- "OpenMP" - "OpenMP"
"LanguageVersion" sequence of - The first integer is the major "LanguageVersion" sequence of - The first integer is the major
2 integers version. 2 integers version.
- The second integer is the - The second integer is the
minor version. minor version.
"Attrs" mapping Mapping of kernel attributes. "Attrs" mapping Mapping of kernel attributes.
See See
:ref:`amdgpu-amdhsa-code-object-kernel-attribute-metadata-map-table-v2` :ref:`amdgpu-amdhsa-code-object-kernel-attribute-metadata-map-v2-table`
for the mapping definition. for the mapping definition.
"Args" sequence of Sequence of mappings of the "Args" sequence of Sequence of mappings of the
mapping kernel arguments. See mapping kernel arguments. See
:ref:`amdgpu-amdhsa-code-object-kernel-argument-metadata-map-table-v2` :ref:`amdgpu-amdhsa-code-object-kernel-argument-metadata-map-v2-table`
for the definition of the mapping. for the definition of the mapping.
"CodeProps" mapping Mapping of properties related to "CodeProps" mapping Mapping of properties related to
the kernel code. See the kernel code. See
:ref:`amdgpu-amdhsa-code-object-kernel-code-properties-metadata-map-table-v2` :ref:`amdgpu-amdhsa-code-object-kernel-code-properties-metadata-map-v2-table`
for the mapping definition. for the mapping definition.
================= ============== ========= ================================ ================= ============== ========= ================================
.. ..
.. table:: AMDHSA Code Object V2 Kernel Attribute Metadata Map .. table:: AMDHSA Code Object V2 Kernel Attribute Metadata Map
:name: amdgpu-amdhsa-code-object-kernel-attribute-metadata-map-table-v2 :name: amdgpu-amdhsa-code-object-kernel-attribute-metadata-map-v2-table
=================== ============== ========= ============================== =================== ============== ========= ==============================
String Key Value Type Required? Description String Key Value Type Required? Description
=================== ============== ========= ============================== =================== ============== ========= ==============================
"ReqdWorkGroupSize" sequence of If not 0, 0, 0 then all values "ReqdWorkGroupSize" sequence of If not 0, 0, 0 then all values
3 integers must be >=1 and the dispatch 3 integers must be >=1 and the dispatch
work-group size X, Y, Z must work-group size X, Y, Z must
correspond to the specified correspond to the specified
Show All 24 Lines "RuntimeHandle" string The external symbol name
device side enqueueing. Only device side enqueueing. Only
available for device side available for device side
enqueued kernels. enqueued kernels.
=================== ============== ========= ============================== =================== ============== ========= ==============================
.. ..
.. table:: AMDHSA Code Object V2 Kernel Argument Metadata Map .. table:: AMDHSA Code Object V2 Kernel Argument Metadata Map
:name: amdgpu-amdhsa-code-object-kernel-argument-metadata-map-table-v2 :name: amdgpu-amdhsa-code-object-kernel-argument-metadata-map-v2-table
================= ============== ========= ================================ ================= ============== ========= ================================
String Key Value Type Required? Description String Key Value Type Required? Description
================= ============== ========= ================================ ================= ============== ========= ================================
"Name" string Kernel argument name. "Name" string Kernel argument name.
"TypeName" string Kernel argument type name. "TypeName" string Kernel argument type name.
"Size" integer Required Kernel argument size in bytes. "Size" integer Required Kernel argument size in bytes.
"Align" integer Required Kernel argument alignment in "Align" integer Required Kernel argument alignment in
▲ Show 20 Lines • Show All 178 Lines • ▼ Show 20 Lines "IsPipe" boolean Indicates if the kernel argument
Can GlobalBuffer be pipe Can GlobalBuffer be pipe
qualified? qualified?
================= ============== ========= ================================ ================= ============== ========= ================================
.. ..
.. table:: AMDHSA Code Object V2 Kernel Code Properties Metadata Map .. table:: AMDHSA Code Object V2 Kernel Code Properties Metadata Map
:name: amdgpu-amdhsa-code-object-kernel-code-properties-metadata-map-table-v2 :name: amdgpu-amdhsa-code-object-kernel-code-properties-metadata-map-v2-table
============================ ============== ========= ===================== ============================ ============== ========= =====================
String Key Value Type Required? Description String Key Value Type Required? Description
============================ ============== ========= ===================== ============================ ============== ========= =====================
"KernargSegmentSize" integer Required The size in bytes of "KernargSegmentSize" integer Required The size in bytes of
the kernarg segment the kernarg segment
that holds the values that holds the values
of the arguments to of the arguments to
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Lines "NumSpilledVGPRs" integer Number of stores from
a vector register to a vector register to
a register allocator a register allocator
created spill created spill
location. location.
============================ ============== ========= ===================== ============================ ============== ========= =====================
.. _amdgpu-amdhsa-code-object-metadata-v3: .. _amdgpu-amdhsa-code-object-metadata-v3:
Code Object V3 Metadata (--amdhsa-code-object-version=3) Code Object V3 Metadata
++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++
.. warning::
Code object V3 is not the default code object version emitted by this version
of LLVM.
Code object V3 metadata is specified by the ``NT_AMDGPU_METADATA`` note record Code object V3 to V4 metadata is specified by the ``NT_AMDGPU_METADATA`` note
(see :ref:`amdgpu-note-records-v3`). record (see :ref:`amdgpu-note-records-v3-v4`).
The metadata is represented as Message Pack formatted binary data (see The metadata is represented as Message Pack formatted binary data (see
[MsgPack]_). The top level is a Message Pack map that includes the [MsgPack]_). The top level is a Message Pack map that includes the
keys defined in table keys defined in table
:ref:`amdgpu-amdhsa-code-object-metadata-map-table-v3` and referenced :ref:`amdgpu-amdhsa-code-object-metadata-map-table-v3` and referenced
tables. tables.
Additional information can be added to the maps. To avoid conflicts, Additional information can be added to the maps. To avoid conflicts,
▲ Show 20 Lines • Show All 381 Lines • ▼ Show 20 Lines ".is_pipe" boolean Indicates if the kernel argument
.. TODO:: .. TODO::
Can "global_buffer" be pipe Can "global_buffer" be pipe
qualified? qualified?
====================== ============== ========= ================================ ====================== ============== ========= ================================
.. _amdgpu-amdhsa-code-object-metadata-v4:
Code Object V4 Metadata
+++++++++++++++++++++++
Code object V4 metadata is the same as
:ref:`amdgpu-amdhsa-code-object-metadata-v3` with the changes and additions
defined in table :ref:`amdgpu-amdhsa-code-object-metadata-map-table-v3`.
.. table:: AMDHSA Code Object V4 Metadata Map Changes from :ref:`amdgpu-amdhsa-code-object-metadata-v3`
:name: amdgpu-amdhsa-code-object-metadata-map-table-v4
================= ============== ========= =======================================
String Key Value Type Required? Description
================= ============== ========= =======================================
"amdhsa.version" sequence of Required - The first integer is the major
2 integers version. Currently 1.
- The second integer is the minor
version. Currently 1.
"amdhsa.target" string Required The target name of the code using the syntax:
.. code::
<target-triple> [ "-" <target-id> ]
A canonical target ID must be
used. See :ref:`amdgpu-target-triples`
and :ref:`amdgpu-target-id`.
================= ============== ========= =======================================
.. ..
Kernel Dispatch Kernel Dispatch
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
The HSA architected queuing language (AQL) defines a user space memory interface The HSA architected queuing language (AQL) defines a user space memory interface
that can be used to control the dispatch of kernels, in an agent independent that can be used to control the dispatch of kernels, in an agent independent
way. An agent can have zero or more AQL queues created for it using the ROCm way. An agent can have zero or more AQL queues created for it using an HSA
runtime, in which AQL packets (all of which are 64 bytes) can be placed. See the compatible runtime (see :ref:`amdgpu-os`), in which AQL packets (all of which
*HSA Platform System Architecture Specification* [HSA]_ for the AQL queue are 64 bytes) can be placed. See the *HSA Platform System Architecture
mechanics and packet layouts. Specification* [HSA]_ for the AQL queue mechanics and packet layouts.
The packet processor of a kernel agent is responsible for detecting and The packet processor of a kernel agent is responsible for detecting and
dispatching HSA kernels from the AQL queues associated with it. For AMD GPUs the dispatching HSA kernels from the AQL queues associated with it. For AMD GPUs the
packet processor is implemented by the hardware command processor (CP), packet processor is implemented by the hardware command processor (CP),
asynchronous dispatch controller (ADC) and shader processor input controller asynchronous dispatch controller (ADC) and shader processor input controller
(SPI). (SPI).
The ROCm runtime can be used to allocate an AQL queue object. It uses the kernel An HSA compatible runtime can be used to allocate an AQL queue object. It uses
mode driver to initialize and register the AQL queue with CP. the kernel mode driver to initialize and register the AQL queue with CP.
To dispatch a kernel the following actions are performed. This can occur in the To dispatch a kernel the following actions are performed. This can occur in the
CPU host program, or from an HSA kernel executing on a GPU. CPU host program, or from an HSA kernel executing on a GPU.
1. A pointer to an AQL queue for the kernel agent on which the kernel is to be 1. A pointer to an AQL queue for the kernel agent on which the kernel is to be
executed is obtained. executed is obtained.
2. A pointer to the kernel descriptor (see 2. A pointer to the kernel descriptor (see
:ref:`amdgpu-amdhsa-kernel-descriptor`) of the kernel to execute is obtained. :ref:`amdgpu-amdhsa-kernel-descriptor`) of the kernel to execute is obtained.
It must be for a kernel that is contained in a code object that that was It must be for a kernel that is contained in a code object that that was
loaded by the ROCm runtime on the kernel agent with which the AQL queue is loaded by an HSA compatible runtime on the kernel agent with which the AQL
associated. queue is associated.
3. Space is allocated for the kernel arguments using the ROCm runtime allocator 3. Space is allocated for the kernel arguments using the HSA compatible runtime
for a memory region with the kernarg property for the kernel agent that will allocator for a memory region with the kernarg property for the kernel agent
execute the kernel. It must be at least 16-byte aligned. that will execute the kernel. It must be at least 16-byte aligned.
4. Kernel argument values are assigned to the kernel argument memory 4. Kernel argument values are assigned to the kernel argument memory
allocation. The layout is defined in the *HSA Programmer's Language allocation. The layout is defined in the *HSA Programmer's Language
Reference* [HSA]_. For AMDGPU the kernel execution directly accesses the Reference* [HSA]_. For AMDGPU the kernel execution directly accesses the
kernel argument memory in the same way constant memory is accessed. (Note kernel argument memory in the same way constant memory is accessed. (Note
that the HSA specification allows an implementation to copy the kernel that the HSA specification allows an implementation to copy the kernel
argument contents to another location that is accessed by the kernel.) argument contents to another location that is accessed by the kernel.)
5. An AQL kernel dispatch packet is created on the AQL queue. The ROCm runtime 5. An AQL kernel dispatch packet is created on the AQL queue. The HSA compatible
api uses 64-bit atomic operations to reserve space in the AQL queue for the runtime api uses 64-bit atomic operations to reserve space in the AQL queue
packet. The packet must be set up, and the final write must use an atomic for the packet. The packet must be set up, and the final write must use an
store release to set the packet kind to ensure the packet contents are atomic store release to set the packet kind to ensure the packet contents are
visible to the kernel agent. AQL defines a doorbell signal mechanism to visible to the kernel agent. AQL defines a doorbell signal mechanism to
notify the kernel agent that the AQL queue has been updated. These rules, and notify the kernel agent that the AQL queue has been updated. These rules, and
the layout of the AQL queue and kernel dispatch packet is defined in the *HSA the layout of the AQL queue and kernel dispatch packet is defined in the *HSA
System Architecture Specification* [HSA]_. System Architecture Specification* [HSA]_.
6. A kernel dispatch packet includes information about the actual dispatch, 6. A kernel dispatch packet includes information about the actual dispatch,
such as grid and work-group size, together with information from the code such as grid and work-group size, together with information from the code
object about the kernel, such as segment sizes. The ROCm runtime queries on object about the kernel, such as segment sizes. The HSA compatible runtime
the kernel symbol can be used to obtain the code object values which are queries on the kernel symbol can be used to obtain the code object values
recorded in the :ref:`amdgpu-amdhsa-code-object-metadata`. which are recorded in the :ref:`amdgpu-amdhsa-code-object-metadata`.
7. CP executes micro-code and is responsible for detecting and setting up the 7. CP executes micro-code and is responsible for detecting and setting up the
GPU to execute the wavefronts of a kernel dispatch. GPU to execute the wavefronts of a kernel dispatch.
8. CP ensures that when the a wavefront starts executing the kernel machine 8. CP ensures that when the a wavefront starts executing the kernel machine
code, the scalar general purpose registers (SGPR) and vector general purpose code, the scalar general purpose registers (SGPR) and vector general purpose
registers (VGPR) are set up as required by the machine code. The required registers (VGPR) are set up as required by the machine code. The required
setup is defined in the :ref:`amdgpu-amdhsa-kernel-descriptor`. The initial setup is defined in the :ref:`amdgpu-amdhsa-kernel-descriptor`. The initial
register state is defined in register state is defined in
:ref:`amdgpu-amdhsa-initial-kernel-execution-state`. :ref:`amdgpu-amdhsa-initial-kernel-execution-state`.
▲ Show 20 Lines • Show All 81 Lines • ▼ Show 20 Lines
GFX9-GFX10 the aperture base addresses are directly available as inline constant GFX9-GFX10 the aperture base addresses are directly available as inline constant
registers ``SRC_SHARED_BASE/LIMIT`` and ``SRC_PRIVATE_BASE/LIMIT``. In 64 bit registers ``SRC_SHARED_BASE/LIMIT`` and ``SRC_PRIVATE_BASE/LIMIT``. In 64 bit
address mode the aperture sizes are 2^32 bytes and the base is aligned to 2^32 address mode the aperture sizes are 2^32 bytes and the base is aligned to 2^32
which makes it easier to convert from flat to segment or segment to flat. which makes it easier to convert from flat to segment or segment to flat.
Image and Samplers Image and Samplers
~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~
Image and sample handles created by the ROCm runtime are 64-bit addresses of a Image and sample handles created by an HSA compatible runtime (see
hardware 32-byte V# and 48 byte S# object respectively. In order to support the :ref:`amdgpu-os`) are 64-bit addresses of a hardware 32-byte V# and 48 byte S#
HSA ``query_sampler`` operations two extra dwords are used to store the HSA BRIG object respectively. In order to support the HSA ``query_sampler`` operations
enumeration values for the queries that are not trivially deducible from the S# two extra dwords are used to store the HSA BRIG enumeration values for the
representation. queries that are not trivially deducible from the S# representation.
HSA Signals HSA Signals
~~~~~~~~~~~ ~~~~~~~~~~~
HSA signal handles created by the ROCm runtime are 64-bit addresses of a HSA signal handles created by an HSA compatible runtime (see :ref:`amdgpu-os`)
structure allocated in memory accessible from both the CPU and GPU. The are 64-bit addresses of a structure allocated in memory accessible from both the
structure is defined by the ROCm runtime and subject to change between releases CPU and GPU. The structure is defined by the runtime and subject to change
(see [AMD-ROCm-github]_). between releases. For example, see [AMD-ROCm-github]_.
.. _amdgpu-amdhsa-hsa-aql-queue: .. _amdgpu-amdhsa-hsa-aql-queue:
HSA AQL Queue HSA AQL Queue
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
The HSA AQL queue structure is defined by the ROCm runtime and subject to change The HSA AQL queue structure is defined by an HSA compatible runtime (see
between releases (see [AMD-ROCm-github]_). For some processors it contains :ref:`amdgpu-os`) and subject to change between releases. For example, see
fields needed to implement certain language features such as the flat address [AMD-ROCm-github]_. For some processors it contains fields needed to implement
aperture bases. It also contains fields used by CP such as managing the certain language features such as the flat address aperture bases. It also
allocation of scratch memory. contains fields used by CP such as managing the allocation of scratch memory.
.. _amdgpu-amdhsa-kernel-descriptor: .. _amdgpu-amdhsa-kernel-descriptor:
Kernel Descriptor Kernel Descriptor
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
A kernel descriptor consists of the information needed by CP to initiate the A kernel descriptor consists of the information needed by CP to initiate the
execution of a kernel, including the entry point address of the machine code execution of a kernel, including the entry point address of the machine code
that implements the kernel. that implements the kernel.
Code Object V3 Kernel Descriptor for GFX6-GFX10 (--amdhsa-code-object-version=3) Code Object V3 Kernel Descriptor
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=+++++++++ ++++++++++++++++++++++++++++++++
CP microcode requires the Kernel descriptor to be allocated on 64-byte CP microcode requires the Kernel descriptor to be allocated on 64-byte
alignment. alignment.
The fields used by CP for code objects before V3 also match those specified in The fields used by CP for code objects before V3 also match those specified in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
.. table:: Code Object V3 Kernel Descriptor for GFX6-GFX10 .. table:: Code Object V3 Kernel Descriptor
:name: amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3 :name: amdgpu-amdhsa-kernel-descriptor-v3-table
======= ======= =============================== ============================ ======= ======= =============================== ============================
Bits Size Field Name Description Bits Size Field Name Description
======= ======= =============================== ============================ ======= ======= =============================== ============================
31:0 4 bytes GROUP_SEGMENT_FIXED_SIZE The amount of fixed local 31:0 4 bytes GROUP_SEGMENT_FIXED_SIZE The amount of fixed local
address space memory address space memory
required for a work-group required for a work-group
in bytes. This does not in bytes. This does not
include any dynamically include any dynamically
allocated local address allocated local address
space memory that may be space memory that may be
added when the kernel is added when the kernel is
dispatched. dispatched.
63:32 4 bytes PRIVATE_SEGMENT_FIXED_SIZE The amount of fixed 63:32 4 bytes PRIVATE_SEGMENT_FIXED_SIZE The amount of fixed
private address space private address space
memory required for a memory required for a
work-item in bytes. If work-item in bytes.
is_dynamic_callstack is 1 Additional space may need to
then additional space must be added to this value if
be added to this value for the call stack has
the call stack. non-inlined function calls.
127:64 8 bytes Reserved, must be 0. 95:64 4 bytes KERNARG_SIZE The size of the kernarg
memory pointed to by the
AQL dispatch packet. The
kernarg memory is used to
pass arguments to the
kernel.
* If the kernarg pointer in
the dispatch packet is NULL
then there are no kernel
arguments.
* If the kernarg pointer in
the dispatch packet is
not NULL and this value
is 0 then the kernarg
memory size is
unspecified.
* If the kernarg pointer in
the dispatch packet is
not NULL and this value
is not 0 then the value
specifies the kernarg
memory size in bytes. It
is recommended to provide
a value as it may be used
by CP to optimize making
the kernarg memory
visible to the kernel
code.
127:96 4 bytes Reserved, must be 0.
191:128 8 bytes KERNEL_CODE_ENTRY_BYTE_OFFSET Byte offset (possibly 191:128 8 bytes KERNEL_CODE_ENTRY_BYTE_OFFSET Byte offset (possibly
negative) from base negative) from base
address of kernel address of kernel
descriptor to kernel's descriptor to kernel's
entry point instruction entry point instruction
which must be 256 byte which must be 256 byte
aligned. aligned.
351:272 20 Reserved, must be 0. 351:272 20 Reserved, must be 0.
▲ Show 20 Lines • Show All 4,361 Lines • ▼ Show 20 Lines fence seq_cst - singlethread *none* *Same as corresponding
- workgroup except must generated - workgroup except must generated
- agent all instructions even - agent all instructions even
- system for OpenCL.* - system for OpenCL.*
============ ============ ============== ========== ================================ ============ ============ ============== ========== ================================
Trap Handler ABI Trap Handler ABI
~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~
For code objects generated by AMDGPU backend for HSA [HSA]_ compatible runtimes For code objects generated by the AMDGPU backend for HSA [HSA]_ compatible
(such as ROCm [AMD-ROCm]_), the runtime installs a trap handler that supports runtimes (see :ref:`amdgpu-os`), the runtime installs a trap handler that
the ``s_trap`` instruction with the following usage: supports the ``s_trap`` instruction. For usage see:
- :ref:`amdgpu-trap-handler-for-amdhsa-os-v2-table`
- :ref:`amdgpu-trap-handler-for-amdhsa-os-v3-table`
- :ref:`amdgpu-trap-handler-for-amdhsa-os-v4-table`
.. table:: AMDGPU Trap Handler for AMDHSA OS .. table:: AMDGPU Trap Handler for AMDHSA OS Code Object V2
:name: amdgpu-trap-handler-for-amdhsa-os-table :name: amdgpu-trap-handler-for-amdhsa-os-v2-table
=================== =============== =============== ======================= =================== =============== =============== =======================================
Usage Code Sequence Trap Handler Description Usage Code Sequence Trap Handler Description
Inputs Inputs
=================== =============== =============== ======================= =================== =============== =============== =======================================
reserved ``s_trap 0x00`` Reserved by hardware. reserved ``s_trap 0x00`` Reserved by hardware.
``debugtrap(arg)`` ``s_trap 0x01`` ``SGPR0-1``: Reserved for HSA ``debugtrap(arg)`` ``s_trap 0x01`` ``SGPR0-1``: Reserved for Finalizer HSA ``debugtrap``
``queue_ptr`` ``debugtrap`` ``queue_ptr`` intrinsic (not implemented).
``VGPR0``: intrinsic (not ``VGPR0``:
``arg`` implemented). ``arg``
``llvm.trap`` ``s_trap 0x02`` ``SGPR0-1``: Causes dispatch to be ``llvm.trap`` ``s_trap 0x02`` ``SGPR0-1``: Causes wave to be halted with the PC at
``queue_ptr`` terminated and its ``queue_ptr`` the trap instruction. The associated
associated queue put queue is signalled to put it into the
into the error state. error state. When the queue is put in
``llvm.debugtrap`` ``s_trap 0x03`` - If debugger not the error state, the waves executing
installed then dispatches on the queue will be
behaves as a terminated.
no-operation. The ``llvm.debugtrap`` ``s_trap 0x03`` *none* - If debugger not enabled then behaves
trap handler is as a no-operation. The trap handler
entered and is entered and immediately returns to
immediately returns continue execution of the wavefront.
to continue - If the debugger is enabled, causes
execution of the the debug trap to be reported by the
wavefront. debugger and the wavefront is put in
- If the debugger is the halt state with the PC at the
installed, causes instruction. The debugger must
the debug trap to be increment the PC and resume the wave.
reported by the
debugger and the
wavefront is put in
the halt state until
resumed by the
debugger.
reserved ``s_trap 0x04`` Reserved. reserved ``s_trap 0x04`` Reserved.
reserved ``s_trap 0x05`` Reserved. reserved ``s_trap 0x05`` Reserved.
reserved ``s_trap 0x06`` Reserved. reserved ``s_trap 0x06`` Reserved.
debugger breakpoint ``s_trap 0x07`` Reserved for debugger reserved ``s_trap 0x07`` Reserved.
breakpoints.
reserved ``s_trap 0x08`` Reserved. reserved ``s_trap 0x08`` Reserved.
reserved ``s_trap 0xfe`` Reserved. reserved ``s_trap 0xfe`` Reserved.
reserved ``s_trap 0xff`` Reserved. reserved ``s_trap 0xff`` Reserved.
=================== =============== =============== ======================= =================== =============== =============== =======================================
..
.. table:: AMDGPU Trap Handler for AMDHSA OS Code Object V3
:name: amdgpu-trap-handler-for-amdhsa-os-v3-table
=================== =============== =============== =======================================
Usage Code Sequence Trap Handler Description
Inputs
=================== =============== =============== =======================================
reserved ``s_trap 0x00`` Reserved by hardware.
debugger breakpoint ``s_trap 0x01`` *none* Reserved for debugger to use for
breakpoints. Causes wave to be halted
with the PC at the trap instruction.
The debugger is responsible to resume
the wave, including the instruction
that the breakpoint overwrote.
``llvm.trap`` ``s_trap 0x02`` ``SGPR0-1``: Causes wave to be halted with the PC at
``queue_ptr`` the trap instruction. The associated
queue is signalled to put it into the
error state. When the queue is put in
the error state, the waves executing
dispatches on the queue will be
terminated.
``llvm.debugtrap`` ``s_trap 0x03`` *none* - If debugger not enabled then behaves
as a no-operation. The trap handler
is entered and immediately returns to
continue execution of the wavefront.
- If the debugger is enabled, causes
the debug trap to be reported by the
debugger and the wavefront is put in
the halt state with the PC at the
instruction. The debugger must
increment the PC and resume the wave.
reserved ``s_trap 0x04`` Reserved.
reserved ``s_trap 0x05`` Reserved.
reserved ``s_trap 0x06`` Reserved.
reserved ``s_trap 0x07`` Reserved.
reserved ``s_trap 0x08`` Reserved.
reserved ``s_trap 0xfe`` Reserved.
reserved ``s_trap 0xff`` Reserved.
=================== =============== =============== =======================================
..
.. table:: AMDGPU Trap Handler for AMDHSA OS Code Object V4
:name: amdgpu-trap-handler-for-amdhsa-os-v4-table
=================== =============== =============== ============== =======================================
Usage Code Sequence GFX6-8 Inputs GFX9-10 Inputs Description
=================== =============== =============== ============== =======================================
reserved ``s_trap 0x00`` Reserved by hardware.
debugger breakpoint ``s_trap 0x01`` *none* *none* Reserved for debugger to use for
breakpoints. Causes wave to be halted
with the PC at the trap instruction.
The debugger is responsible to resume
the wave, including the instruction
that the breakpoint overwrote.
``llvm.trap`` ``s_trap 0x02`` ``SGPR0-1``: *none* Causes wave to be halted with the PC at
``queue_ptr`` the trap instruction. The associated
queue is signalled to put it into the
error state. When the queue is put in
the error state, the waves executing
dispatches on the queue will be
terminated.
``llvm.debugtrap`` ``s_trap 0x03`` *none* *none* - If debugger not enabled then behaves
as a no-operation. The trap handler
is entered and immediately returns to
continue execution of the wavefront.
- If the debugger is enabled, causes
the debug trap to be reported by the
debugger and the wavefront is put in
the halt state with the PC at the
instruction. The debugger must
increment the PC and resume the wave.
reserved ``s_trap 0x04`` Reserved.
reserved ``s_trap 0x05`` Reserved.
reserved ``s_trap 0x06`` Reserved.
reserved ``s_trap 0x07`` Reserved.
reserved ``s_trap 0x08`` Reserved.
reserved ``s_trap 0xfe`` Reserved.
reserved ``s_trap 0xff`` Reserved.
=================== =============== =============== ============== =======================================
.. _amdgpu-amdhsa-function-call-convention: .. _amdgpu-amdhsa-function-call-convention:
Call Convention Call Convention
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
.. note:: .. note::
▲ Show 20 Lines • Show All 219 Lines • ▼ Show 20 Lines
1. Work-Item ID (1 VGPR) 1. Work-Item ID (1 VGPR)
The X, Y and Z work-item ID are packed into a single VGRP with the following The X, Y and Z work-item ID are packed into a single VGRP with the following
layout. Only fields actually used by the function are set. The other bits layout. Only fields actually used by the function are set. The other bits
are undefined. are undefined.
The values come from the initial kernel execution state. See The values come from the initial kernel execution state. See
:ref:`amdgpu-amdhsa-vgpr-register-set-up-order-table`. :ref:`amdgpu-amdhsa-initial-kernel-execution-state`.
.. table:: Work-item implicit argument layout .. table:: Work-item implicit argument layout
:name: amdgpu-amdhsa-workitem-implicit-argument-layout-table :name: amdgpu-amdhsa-workitem-implicit-argument-layout-table
======= ======= ============== ======= ======= ==============
Bits Size Field Name Bits Size Field Name
======= ======= ============== ======= ======= ==============
9:0 10 bits X Work-Item ID 9:0 10 bits X Work-Item ID
▲ Show 20 Lines • Show All 46 Lines • ▼ Show 20 Lines
.. note:: .. note::
There are likely some errors and omissions in the following description that There are likely some errors and omissions in the following description that
need correction. need correction.
.. TODO:: .. TODO::
Check the clang source code to decipher how function arguments and return Check the Clang source code to decipher how function arguments and return
results are handled. Also see the AMDGPU specific values used. results are handled. Also see the AMDGPU specific values used.
* VGPR arguments are assigned to consecutive VGPRs starting at VGPR0 up to * VGPR arguments are assigned to consecutive VGPRs starting at VGPR0 up to
VGPR31. VGPR31.
If there are more arguments than will fit in these registers, the remaining If there are more arguments than will fit in these registers, the remaining
arguments are allocated on the stack in order on naturally aligned arguments are allocated on the stack in order on naturally aligned
addresses. addresses.
▲ Show 20 Lines • Show All 546 Lines • ▼ Show 20 Lines.. code-block:: nasm
v_mov_b32 v1, v2 dst_sel:BYTE_0 dst_unused:UNUSED_PRESERVE src0_sel:DWORD v_mov_b32 v1, v2 dst_sel:BYTE_0 dst_unused:UNUSED_PRESERVE src0_sel:DWORD
v_min_u32 v200, v200, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:DWORD v_min_u32 v200, v200, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:DWORD
v_sin_f32 v0, v0 dst_unused:UNUSED_PAD src0_sel:WORD_1 v_sin_f32 v0, v0 dst_unused:UNUSED_PAD src0_sel:WORD_1
v_fract_f32 v0, |v0| dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 v_fract_f32 v0, |v0| dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
v_cmpx_le_u32 vcc, v1, v2 src0_sel:BYTE_2 src1_sel:WORD_0 v_cmpx_le_u32 vcc, v1, v2 src0_sel:BYTE_2 src1_sel:WORD_0
For full list of supported instructions, refer to "Vector ALU instructions". For full list of supported instructions, refer to "Vector ALU instructions".
.. TODO::
Remove once we switch to code object v3 by default.
.. _amdgpu-amdhsa-assembler-predefined-symbols-v2: .. _amdgpu-amdhsa-assembler-predefined-symbols-v2:
Code Object V2 Predefined Symbols (--amdhsa-code-object-version=2) Code Object V2 Predefined Symbols
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. warning:: Code Object V2 is not the default code object version emitted by .. warning::
this version of LLVM. For a description of the predefined symbols available Code object V2 is not the default code object version emitted by
with the default configuration (Code Object V3) see this version of LLVM.
:ref:`amdgpu-amdhsa-assembler-predefined-symbols-v3`.
The AMDGPU assembler defines and updates some symbols automatically. These The AMDGPU assembler defines and updates some symbols automatically. These
symbols do not affect code generation. symbols do not affect code generation.
.option.machine_version_major .option.machine_version_major
+++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++
Set to the GFX major generation number of the target being assembled for. For Set to the GFX major generation number of the target being assembled for. For
Show All 34 Lines
:ref:`amdgpu-amdhsa-assembler-directive-amdgpu_hsa_kernel` directive is :ref:`amdgpu-amdhsa-assembler-directive-amdgpu_hsa_kernel` directive is
encountered. At each instruction, if the current value of this symbol is less encountered. At each instruction, if the current value of this symbol is less
than or equal to the maximum VGPR number explicitly referenced within that than or equal to the maximum VGPR number explicitly referenced within that
instruction then the symbol value is updated to equal that SGPR number plus instruction then the symbol value is updated to equal that SGPR number plus
one. one.
.. _amdgpu-amdhsa-assembler-directives-v2: .. _amdgpu-amdhsa-assembler-directives-v2:
Code Object V2 Directives (--amdhsa-code-object-version=2) Code Object V2 Directives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
.. warning:: Code Object V2 is not the default code object version emitted by .. warning::
this version of LLVM. For a description of the directives supported with Code object V2 is not the default code object version emitted by
the default configuration (Code Object V3) see this version of LLVM.
:ref:`amdgpu-amdhsa-assembler-directives-v3`.
AMDGPU ABI defines auxiliary data in output code object. In assembly source, AMDGPU ABI defines auxiliary data in output code object. In assembly source,
one can specify them with assembler directives. one can specify them with assembler directives.
.hsa_code_object_version major, minor .hsa_code_object_version major, minor
+++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++
*major* and *minor* are integers that specify the version of the HSA code *major* and *minor* are integers that specify the version of the HSA code
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
The *.amd_kernel_code_t* directive must be placed immediately after the The *.amd_kernel_code_t* directive must be placed immediately after the
function label and before any instructions. function label and before any instructions.
For a full list of amd_kernel_code_t keys, refer to AMDGPU ABI document, For a full list of amd_kernel_code_t keys, refer to AMDGPU ABI document,
comments in lib/Target/AMDGPU/AmdKernelCodeT.h and test/CodeGen/AMDGPU/hsa.s. comments in lib/Target/AMDGPU/AmdKernelCodeT.h and test/CodeGen/AMDGPU/hsa.s.
.. _amdgpu-amdhsa-assembler-example-v2: .. _amdgpu-amdhsa-assembler-example-v2:
Code Object V2 Example Source Code (--amdhsa-code-object-version=2) Code Object V2 Example Source Code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. warning:: Code Object V2 is not the default code object version emitted by .. warning::
this version of LLVM. For a description of the directives supported with Code Object V2 is not the default code object version emitted by
the default configuration (Code Object V3) see this version of LLVM.
:ref:`amdgpu-amdhsa-assembler-example-v3`.
Here is an example of a minimal assembly source file, defining one HSA kernel: Here is an example of a minimal assembly source file, defining one HSA kernel:
.. code:: .. code::
:number-lines: :number-lines:
.hsa_code_object_version 1,0 .hsa_code_object_version 1,0
.hsa_code_object_isa .hsa_code_object_isa
Show All 21 Lines hello_world:
s_waitcnt lgkmcnt(0) s_waitcnt lgkmcnt(0)
v_mov_b32 v1, s0 v_mov_b32 v1, s0
v_mov_b32 v2, s1 v_mov_b32 v2, s1
flat_store_dword v[1:2], v0 flat_store_dword v[1:2], v0
s_endpgm s_endpgm
.Lfunc_end0: .Lfunc_end0:
.size hello_world, .Lfunc_end0-hello_world .size hello_world, .Lfunc_end0-hello_world
.. _amdgpu-amdhsa-assembler-predefined-symbols-v3: .. _amdgpu-amdhsa-assembler-predefined-symbols-v3-v4:
Code Object V3 Predefined Symbols (--amdhsa-code-object-version=3) Code Object V3 to V4 Predefined Symbols
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The AMDGPU assembler defines and updates some symbols automatically. These The AMDGPU assembler defines and updates some symbols automatically. These
symbols do not affect code generation. symbols do not affect code generation.
.amdgcn.gfx_generation_number .amdgcn.gfx_generation_number
+++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++
Set to the GFX major generation number of the target being assembled for. For Set to the GFX major generation number of the target being assembled for. For
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
referenced within that instruction then the symbol value is updated to equal referenced within that instruction then the symbol value is updated to equal
that SGPR number plus one. that SGPR number plus one.
May be used to set the `.amdhsa_next_free_spgr` directive in May be used to set the `.amdhsa_next_free_spgr` directive in
:ref:`amdhsa-kernel-directives-table`. :ref:`amdhsa-kernel-directives-table`.
May be set at any time, e.g. manually set to zero at the start of each kernel. May be set at any time, e.g. manually set to zero at the start of each kernel.
.. _amdgpu-amdhsa-assembler-directives-v3: .. _amdgpu-amdhsa-assembler-directives-v3-v4:
Code Object V3 Directives (--amdhsa-code-object-version=3) Code Object V3 to V4 Directives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Directives which begin with ``.amdgcn`` are valid for all ``amdgcn`` Directives which begin with ``.amdgcn`` are valid for all ``amdgcn``
architecture processors, and are not OS-specific. Directives which begin with architecture processors, and are not OS-specific. Directives which begin with
``.amdhsa`` are specific to ``amdgcn`` architecture processors when the ``.amdhsa`` are specific to ``amdgcn`` architecture processors when the
``amdhsa`` OS is specified. See :ref:`amdgpu-target-triples` and ``amdhsa`` OS is specified. See :ref:`amdgpu-target-triples` and
:ref:`amdgpu-processors`. :ref:`amdgpu-processors`.
.amdgcn_target <target> .amdgcn_target <target-triple> "-" <target-id>
+++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++
Optional directive which declares the target supported by the containing Optional directive which declares the ``<target-triple>-<target-id>`` supported
assembler source file. Valid values are described in by the containing assembler source file. Used by the assembler to validate
:ref:`amdgpu-amdhsa-code-object-target-identification`. Used by the assembler command-line options such as ``-triple``, ``-mcpu``, and
to validate command-line options such as ``-triple``, ``-mcpu``, and those ``--offload-arch=<target-id>``. A non-canonical target ID is allowed. See
which specify target features. :ref:`amdgpu-target-triples` and :ref:`amdgpu-target-id`.
.amdhsa_kernel <name> .amdhsa_kernel <name>
+++++++++++++++++++++ +++++++++++++++++++++
Creates a correctly aligned AMDHSA kernel descriptor and a symbol, Creates a correctly aligned AMDHSA kernel descriptor and a symbol,
``<name>.kd``, in the current location of the current section. Only valid when ``<name>.kd``, in the current location of the current section. Only valid when
the OS is ``amdhsa``. ``<name>`` must be a symbol that labels the first the OS is ``amdhsa``. ``<name>`` must be a symbol that labels the first
instruction to execute, and does not need to be previously defined. instruction to execute, and does not need to be previously defined.
Show All 11 Linesterminated by an ``.end_amdhsa_kernel`` directive.
.. table:: AMDHSA Kernel Assembler Directives .. table:: AMDHSA Kernel Assembler Directives
:name: amdhsa-kernel-directives-table :name: amdhsa-kernel-directives-table
======================================================== =================== ============ =================== ======================================================== =================== ============ ===================
Directive Default Supported On Description Directive Default Supported On Description
======================================================== =================== ============ =================== ======================================================== =================== ============ ===================
``.amdhsa_group_segment_fixed_size`` 0 GFX6-GFX10 Controls GROUP_SEGMENT_FIXED_SIZE in ``.amdhsa_group_segment_fixed_size`` 0 GFX6-GFX10 Controls GROUP_SEGMENT_FIXED_SIZE in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_private_segment_fixed_size`` 0 GFX6-GFX10 Controls PRIVATE_SEGMENT_FIXED_SIZE in ``.amdhsa_private_segment_fixed_size`` 0 GFX6-GFX10 Controls PRIVATE_SEGMENT_FIXED_SIZE in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_kernarg_size`` 0 GFX6-GFX10 Controls KERNARG_SIZE in
:ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_private_segment_buffer`` 0 GFX6-GFX10 Controls ENABLE_SGPR_PRIVATE_SEGMENT_BUFFER in ``.amdhsa_user_sgpr_private_segment_buffer`` 0 GFX6-GFX10 Controls ENABLE_SGPR_PRIVATE_SEGMENT_BUFFER in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_dispatch_ptr`` 0 GFX6-GFX10 Controls ENABLE_SGPR_DISPATCH_PTR in ``.amdhsa_user_sgpr_dispatch_ptr`` 0 GFX6-GFX10 Controls ENABLE_SGPR_DISPATCH_PTR in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_queue_ptr`` 0 GFX6-GFX10 Controls ENABLE_SGPR_QUEUE_PTR in ``.amdhsa_user_sgpr_queue_ptr`` 0 GFX6-GFX10 Controls ENABLE_SGPR_QUEUE_PTR in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_kernarg_segment_ptr`` 0 GFX6-GFX10 Controls ENABLE_SGPR_KERNARG_SEGMENT_PTR in ``.amdhsa_user_sgpr_kernarg_segment_ptr`` 0 GFX6-GFX10 Controls ENABLE_SGPR_KERNARG_SEGMENT_PTR in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_dispatch_id`` 0 GFX6-GFX10 Controls ENABLE_SGPR_DISPATCH_ID in ``.amdhsa_user_sgpr_dispatch_id`` 0 GFX6-GFX10 Controls ENABLE_SGPR_DISPATCH_ID in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_flat_scratch_init`` 0 GFX6-GFX10 Controls ENABLE_SGPR_FLAT_SCRATCH_INIT in ``.amdhsa_user_sgpr_flat_scratch_init`` 0 GFX6-GFX10 Controls ENABLE_SGPR_FLAT_SCRATCH_INIT in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_user_sgpr_private_segment_size`` 0 GFX6-GFX10 Controls ENABLE_SGPR_PRIVATE_SEGMENT_SIZE in ``.amdhsa_user_sgpr_private_segment_size`` 0 GFX6-GFX10 Controls ENABLE_SGPR_PRIVATE_SEGMENT_SIZE in
:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
``.amdhsa_wavefront_size32`` Target GFX10 Controls ENABLE_WAVEFRONT_SIZE32 in ``.amdhsa_wavefront_size32`` Target GFX10 Controls ENABLE_WAVEFRONT_SIZE32 in
Feature :ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. Feature :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
Specific Specific
(-wavefrontsize64) (wavefrontsize64)
``.amdhsa_system_sgpr_private_segment_wavefront_offset`` 0 GFX6-GFX10 Controls ENABLE_SGPR_PRIVATE_SEGMENT_WAVEFRONT_OFFSET in ``.amdhsa_system_sgpr_private_segment_wavefront_offset`` 0 GFX6-GFX10 Controls ENABLE_SGPR_PRIVATE_SEGMENT_WAVEFRONT_OFFSET in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_system_sgpr_workgroup_id_x`` 1 GFX6-GFX10 Controls ENABLE_SGPR_WORKGROUP_ID_X in ``.amdhsa_system_sgpr_workgroup_id_x`` 1 GFX6-GFX10 Controls ENABLE_SGPR_WORKGROUP_ID_X in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_system_sgpr_workgroup_id_y`` 0 GFX6-GFX10 Controls ENABLE_SGPR_WORKGROUP_ID_Y in ``.amdhsa_system_sgpr_workgroup_id_y`` 0 GFX6-GFX10 Controls ENABLE_SGPR_WORKGROUP_ID_Y in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_system_sgpr_workgroup_id_z`` 0 GFX6-GFX10 Controls ENABLE_SGPR_WORKGROUP_ID_Z in ``.amdhsa_system_sgpr_workgroup_id_z`` 0 GFX6-GFX10 Controls ENABLE_SGPR_WORKGROUP_ID_Z in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
Show All 14 Lines ``.amdhsa_reserve_vcc`` 1 GFX6-GFX10 Whether the kernel may use the special VCC SGPR.
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_reserve_flat_scratch`` 1 GFX7-GFX10 Whether the kernel may use flat instructions to access ``.amdhsa_reserve_flat_scratch`` 1 GFX7-GFX10 Whether the kernel may use flat instructions to access
scratch memory. Used to calculate scratch memory. Used to calculate
GRANULATED_WAVEFRONT_SGPR_COUNT in GRANULATED_WAVEFRONT_SGPR_COUNT in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_reserve_xnack_mask`` Target GFX8-GFX10 Whether the kernel may trigger XNACK replay. ``.amdhsa_reserve_xnack_mask`` Target GFX8-GFX10 Whether the kernel may trigger XNACK replay.
Feature Used to calculate GRANULATED_WAVEFRONT_SGPR_COUNT in Feature Used to calculate GRANULATED_WAVEFRONT_SGPR_COUNT in
Specific :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. Specific :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
(+xnack) (xnack)
``.amdhsa_float_round_mode_32`` 0 GFX6-GFX10 Controls FLOAT_ROUND_MODE_32 in ``.amdhsa_float_round_mode_32`` 0 GFX6-GFX10 Controls FLOAT_ROUND_MODE_32 in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
Possible values are defined in Possible values are defined in
:ref:`amdgpu-amdhsa-floating-point-rounding-mode-enumeration-values-table`. :ref:`amdgpu-amdhsa-floating-point-rounding-mode-enumeration-values-table`.
``.amdhsa_float_round_mode_16_64`` 0 GFX6-GFX10 Controls FLOAT_ROUND_MODE_16_64 in ``.amdhsa_float_round_mode_16_64`` 0 GFX6-GFX10 Controls FLOAT_ROUND_MODE_16_64 in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
Possible values are defined in Possible values are defined in
:ref:`amdgpu-amdhsa-floating-point-rounding-mode-enumeration-values-table`. :ref:`amdgpu-amdhsa-floating-point-rounding-mode-enumeration-values-table`.
``.amdhsa_float_denorm_mode_32`` 0 GFX6-GFX10 Controls FLOAT_DENORM_MODE_32 in ``.amdhsa_float_denorm_mode_32`` 0 GFX6-GFX10 Controls FLOAT_DENORM_MODE_32 in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
Possible values are defined in Possible values are defined in
:ref:`amdgpu-amdhsa-floating-point-denorm-mode-enumeration-values-table`. :ref:`amdgpu-amdhsa-floating-point-denorm-mode-enumeration-values-table`.
``.amdhsa_float_denorm_mode_16_64`` 3 GFX6-GFX10 Controls FLOAT_DENORM_MODE_16_64 in ``.amdhsa_float_denorm_mode_16_64`` 3 GFX6-GFX10 Controls FLOAT_DENORM_MODE_16_64 in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
Possible values are defined in Possible values are defined in
:ref:`amdgpu-amdhsa-floating-point-denorm-mode-enumeration-values-table`. :ref:`amdgpu-amdhsa-floating-point-denorm-mode-enumeration-values-table`.
``.amdhsa_dx10_clamp`` 1 GFX6-GFX10 Controls ENABLE_DX10_CLAMP in ``.amdhsa_dx10_clamp`` 1 GFX6-GFX10 Controls ENABLE_DX10_CLAMP in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_ieee_mode`` 1 GFX6-GFX10 Controls ENABLE_IEEE_MODE in ``.amdhsa_ieee_mode`` 1 GFX6-GFX10 Controls ENABLE_IEEE_MODE in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_fp16_overflow`` 0 GFX9-GFX10 Controls FP16_OVFL in ``.amdhsa_fp16_overflow`` 0 GFX9-GFX10 Controls FP16_OVFL in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_workgroup_processor_mode`` Target GFX10 Controls ENABLE_WGP_MODE in ``.amdhsa_workgroup_processor_mode`` Target GFX10 Controls ENABLE_WGP_MODE in
Feature :ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx10-table-v3`. Feature :ref:`amdgpu-amdhsa-kernel-descriptor-v3-table`.
Specific Specific
(-cumode) (cumode)
``.amdhsa_memory_ordered`` 1 GFX10 Controls MEM_ORDERED in ``.amdhsa_memory_ordered`` 1 GFX10 Controls MEM_ORDERED in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_forward_progress`` 0 GFX10 Controls FWD_PROGRESS in ``.amdhsa_forward_progress`` 0 GFX10 Controls FWD_PROGRESS in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx10-table`.
``.amdhsa_exception_fp_ieee_invalid_op`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_INVALID_OPERATION in ``.amdhsa_exception_fp_ieee_invalid_op`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_INVALID_OPERATION in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_exception_fp_denorm_src`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_FP_DENORMAL_SOURCE in ``.amdhsa_exception_fp_denorm_src`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_FP_DENORMAL_SOURCE in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_exception_fp_ieee_div_zero`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_DIVISION_BY_ZERO in ``.amdhsa_exception_fp_ieee_div_zero`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_DIVISION_BY_ZERO in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_exception_fp_ieee_overflow`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_OVERFLOW in ``.amdhsa_exception_fp_ieee_overflow`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_OVERFLOW in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_exception_fp_ieee_underflow`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_UNDERFLOW in ``.amdhsa_exception_fp_ieee_underflow`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_UNDERFLOW in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_exception_fp_ieee_inexact`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_INEXACT in ``.amdhsa_exception_fp_ieee_inexact`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_IEEE_754_FP_INEXACT in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
``.amdhsa_exception_int_div_zero`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_INT_DIVIDE_BY_ZERO in ``.amdhsa_exception_int_div_zero`` 0 GFX6-GFX10 Controls ENABLE_EXCEPTION_INT_DIVIDE_BY_ZERO in
:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`. :ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx10-table`.
======================================================== =================== ============ =================== ======================================================== =================== ============ ===================
.amdgpu_metadata .amdgpu_metadata
++++++++++++++++ ++++++++++++++++
Optional directive which declares the contents of the ``NT_AMDGPU_METADATA`` Optional directive which declares the contents of the ``NT_AMDGPU_METADATA``
note record (see :ref:`amdgpu-elf-note-records-table-v3`). note record (see :ref:`amdgpu-elf-note-records-table-v3-v4`).
The contents must be in the [YAML]_ markup format, with the same structure and The contents must be in the [YAML]_ markup format, with the same structure and
semantics described in :ref:`amdgpu-amdhsa-code-object-metadata-v3`. semantics described in :ref:`amdgpu-amdhsa-code-object-metadata-v3` or
:ref:`amdgpu-amdhsa-code-object-metadata-v4`.
This directive is terminated by an ``.end_amdgpu_metadata`` directive. This directive is terminated by an ``.end_amdgpu_metadata`` directive.
.. _amdgpu-amdhsa-assembler-example-v3: .. _amdgpu-amdhsa-assembler-example-v3-v4:
Code Object V3 Example Source Code (--amdhsa-code-object-version=3) Code Object V3 to V4 Example Source Code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here is an example of a minimal assembly source file, defining one HSA kernel: Here is an example of a minimal assembly source file, defining one HSA kernel:
.. code:: .. code::
:number-lines: :number-lines:
.amdgcn_target "amdgcn-amd-amdhsa--gfx900+xnack" // optional .amdgcn_target "amdgcn-amd-amdhsa--gfx900+xnack" // optional
▲ Show 20 Lines • Show All 121 Lines • ▼ Show 20 Lines
.. [AMD-GCN-GFX7] `AMD Sea Islands Series ISA <http://developer.amd.com/wordpress/media/2013/07/AMD_Sea_Islands_Instruction_Set_Architecture.pdf>`_ .. [AMD-GCN-GFX7] `AMD Sea Islands Series ISA <http://developer.amd.com/wordpress/media/2013/07/AMD_Sea_Islands_Instruction_Set_Architecture.pdf>`_
.. [AMD-GCN-GFX8] `AMD GCN3 Instruction Set Architecture <http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_GCN3_Instruction_Set_Architecture_rev1.1.pdf>`__ .. [AMD-GCN-GFX8] `AMD GCN3 Instruction Set Architecture <http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_GCN3_Instruction_Set_Architecture_rev1.1.pdf>`__
.. [AMD-GCN-GFX9] `AMD "Vega" Instruction Set Architecture <http://developer.amd.com/wordpress/media/2013/12/Vega_Shader_ISA_28July2017.pdf>`__ .. [AMD-GCN-GFX9] `AMD "Vega" Instruction Set Architecture <http://developer.amd.com/wordpress/media/2013/12/Vega_Shader_ISA_28July2017.pdf>`__
.. [AMD-GCN-GFX10] `AMD "RDNA 1.0" Instruction Set Architecture <https://gpuopen.com/wp-content/uploads/2019/08/RDNA_Shader_ISA_5August2019.pdf>`__ .. [AMD-GCN-GFX10] `AMD "RDNA 1.0" Instruction Set Architecture <https://gpuopen.com/wp-content/uploads/2019/08/RDNA_Shader_ISA_5August2019.pdf>`__
.. [AMD-RADEON-HD-2000-3000] `AMD R6xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R600_Instruction_Set_Architecture.pdf>`__ .. [AMD-RADEON-HD-2000-3000] `AMD R6xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R600_Instruction_Set_Architecture.pdf>`__
.. [AMD-RADEON-HD-4000] `AMD R7xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R700-Family_Instruction_Set_Architecture.pdf>`__ .. [AMD-RADEON-HD-4000] `AMD R7xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R700-Family_Instruction_Set_Architecture.pdf>`__
.. [AMD-RADEON-HD-5000] `AMD Evergreen shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_Evergreen-Family_Instruction_Set_Architecture.pdf>`__ .. [AMD-RADEON-HD-5000] `AMD Evergreen shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_Evergreen-Family_Instruction_Set_Architecture.pdf>`__
.. [AMD-RADEON-HD-6000] `AMD Cayman/Trinity shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_HD_6900_Series_Instruction_Set_Architecture.pdf>`__ .. [AMD-RADEON-HD-6000] `AMD Cayman/Trinity shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_HD_6900_Series_Instruction_Set_Architecture.pdf>`__
.. [AMD-ROCm] `AMD ROCm Platform <https://rocm-documentation.readthedocs.io>`__ .. [AMD-ROCm] `AMD ROCm Platform <https://rocmdocs.amd.com/>`__
.. [AMD-ROCm-github] `ROCm github <http://github.com/RadeonOpenCompute>`__ .. [AMD-ROCm-github] `AMD ROCm github <http://github.com/RadeonOpenCompute>`__
.. [AMD-ROCm-Release-Notes] `AMD ROCm Release Notes <https://github.com/RadeonOpenCompute/ROCm>`__
.. [CLANG-ATTR] `Attributes in Clang <https://clang.llvm.org/docs/AttributeReference.html>`__ .. [CLANG-ATTR] `Attributes in Clang <https://clang.llvm.org/docs/AttributeReference.html>`__
.. [DWARF] `DWARF Debugging Information Format <http://dwarfstd.org/>`__ .. [DWARF] `DWARF Debugging Information Format <http://dwarfstd.org/>`__
.. [ELF] `Executable and Linkable Format (ELF) <http://www.sco.com/developers/gabi/>`__ .. [ELF] `Executable and Linkable Format (ELF) <http://www.sco.com/developers/gabi/>`__
.. [HRF] `Heterogeneous-race-free Memory Models <http://benedictgaster.org/wp-content/uploads/2014/01/asplos269-FINAL.pdf>`__ .. [HRF] `Heterogeneous-race-free Memory Models <http://benedictgaster.org/wp-content/uploads/2014/01/asplos269-FINAL.pdf>`__
.. [HSA] `Heterogeneous System Architecture (HSA) Foundation <http://www.hsafoundation.com/>`__ .. [HSA] `Heterogeneous System Architecture (HSA) Foundation <http://www.hsafoundation.com/>`__
.. [MsgPack] `Message Pack <http://www.msgpack.org/>`__ .. [MsgPack] `Message Pack <http://www.msgpack.org/>`__
.. [OpenCL] `The OpenCL Specification Version 2.0 <http://www.khronos.org/registry/cl/specs/opencl-2.0.pdf>`__ .. [OpenCL] `The OpenCL Specification Version 2.0 <http://www.khronos.org/registry/cl/specs/opencl-2.0.pdf>`__
.. [SEMVER] `Semantic Versioning <https://semver.org/>`__ .. [SEMVER] `Semantic Versioning <https://semver.org/>`__
.. [YAML] `YAML Ain't Markup Language (YAML™) Version 1.2 <http://www.yaml.org/spec/1.2/spec.html>`__ .. [YAML] `YAML Ain't Markup Language (YAML™) Version 1.2 <http://www.yaml.org/spec/1.2/spec.html>`__