Diff 152360

llvm/trunk/docs/AMDGPUUsage.rst

This file is larger than 256 KB, so syntax highlighting is disabled by default.

Show First 20 Lines • Show All 690 Lines • ▼ Show 20 Lines	.. table:: AMDGPU ELF Note Record Enumeration Values
``NT_AMD_AMDGPU_HSA_METADATA`` 10		``NT_AMD_AMDGPU_HSA_METADATA`` 10
reserved 11		reserved 11
============================== =====		============================== =====

``NT_AMD_AMDGPU_HSA_METADATA``		``NT_AMD_AMDGPU_HSA_METADATA``
Specifies extensible metadata associated with the code objects executed on HSA		Specifies extensible metadata associated with the code objects executed on HSA
[HSA]_ compatible runtimes such as AMD's ROCm [AMD-ROCm]_. It is required when		[HSA]_ compatible runtimes such as AMD's ROCm [AMD-ROCm]_. It is required when
the target triple OS is ``amdhsa`` (see :ref:`amdgpu-target-triples`). See		the target triple OS is ``amdhsa`` (see :ref:`amdgpu-target-triples`). See
:ref:`amdgpu-amdhsa-hsa-code-object-metadata` for the syntax of the code		:ref:`amdgpu-amdhsa-code-object-metadata` for the syntax of the code
object metadata string.		object metadata string.

.. _amdgpu-symbols:		.. _amdgpu-symbols:

Symbols		Symbols
-------		-------

Symbols include the following:		Symbols include the following:
▲ Show 20 Lines • Show All 241 Lines • ▼ Show 20 Lines
(see :ref:`amdgpu-target-triples`).		(see :ref:`amdgpu-target-triples`).

AMDHSA		AMDHSA
------		------

This section provides code conventions used when the target triple OS is		This section provides code conventions used when the target triple OS is
``amdhsa`` (see :ref:`amdgpu-target-triples`).		``amdhsa`` (see :ref:`amdgpu-target-triples`).

.. _amdgpu-amdhsa-hsa-code-object-metadata:		.. _amdgpu-amdhsa-code-object-target-identification:

Code Object Target Identification		Code Object Target Identification
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~		~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The AMDHSA OS uses the following syntax to specify the code object		The AMDHSA OS uses the following syntax to specify the code object
target as a single string:		target as a single string:

``<Architecture>-<Vendor>-<OS>-<Environment>-<Processor><Target Features>``		``<Architecture>-<Vendor>-<OS>-<Environment>-<Processor><Target Features>``
Show All 13 Lines	- ``<Target Features>`` is a list of the enabled Target Features
in the table :ref:`amdgpu-target-feature-table`. Note that *Target		in the table :ref:`amdgpu-target-feature-table`. Note that *Target
Features* must be included in the list if they are enabled even if		Features* must be included in the list if they are enabled even if
that is the default for Processor.		that is the default for Processor.

For example:		For example:

``"amdgcn-amd-amdhsa--gfx902+xnack"``		``"amdgcn-amd-amdhsa--gfx902+xnack"``

		.. _amdgpu-amdhsa-code-object-metadata:

Code Object Metadata		Code Object Metadata
~~~~~~~~~~~~~~~~~~~~		~~~~~~~~~~~~~~~~~~~~

The code object metadata specifies extensible metadata associated with the code		The code object metadata specifies extensible metadata associated with the code
objects executed on HSA [HSA]_ compatible runtimes such as AMD's ROCm		objects executed on HSA [HSA]_ compatible runtimes such as AMD's ROCm
[AMD-ROCm]_. It is specified by the ``NT_AMD_AMDGPU_HSA_METADATA`` note record		[AMD-ROCm]_. It is specified by the ``NT_AMD_AMDGPU_HSA_METADATA`` note record
(see :ref:`amdgpu-note-records`) and is required when the target triple OS is		(see :ref:`amdgpu-note-records`) and is required when the target triple OS is
``amdhsa`` (see :ref:`amdgpu-target-triples`). It must contain the minimum		``amdhsa`` (see :ref:`amdgpu-target-triples`). It must contain the minimum
▲ Show 20 Lines • Show All 465 Lines • ▼ Show 20 Lines	5. An AQL kernel dispatch packet is created on the AQL queue. The ROCm runtime
visible to the kernel agent. AQL defines a doorbell signal mechanism to		visible to the kernel agent. AQL defines a doorbell signal mechanism to
notify the kernel agent that the AQL queue has been updated. These rules, and		notify the kernel agent that the AQL queue has been updated. These rules, and
the layout of the AQL queue and kernel dispatch packet is defined in the *HSA		the layout of the AQL queue and kernel dispatch packet is defined in the *HSA
System Architecture Specification* [HSA]_.		System Architecture Specification* [HSA]_.
6. A kernel dispatch packet includes information about the actual dispatch,		6. A kernel dispatch packet includes information about the actual dispatch,
such as grid and work-group size, together with information from the code		such as grid and work-group size, together with information from the code
object about the kernel, such as segment sizes. The ROCm runtime queries on		object about the kernel, such as segment sizes. The ROCm runtime queries on
the kernel symbol can be used to obtain the code object values which are		the kernel symbol can be used to obtain the code object values which are
recorded in the :ref:`amdgpu-amdhsa-hsa-code-object-metadata`.		recorded in the :ref:`amdgpu-amdhsa-code-object-metadata`.
7. CP executes micro-code and is responsible for detecting and setting up the		7. CP executes micro-code and is responsible for detecting and setting up the
GPU to execute the wavefronts of a kernel dispatch.		GPU to execute the wavefronts of a kernel dispatch.
8. CP ensures that when the a wavefront starts executing the kernel machine		8. CP ensures that when the a wavefront starts executing the kernel machine
code, the scalar general purpose registers (SGPR) and vector general purpose		code, the scalar general purpose registers (SGPR) and vector general purpose
registers (VGPR) are set up as required by the machine code. The required		registers (VGPR) are set up as required by the machine code. The required
setup is defined in the :ref:`amdgpu-amdhsa-kernel-descriptor`. The initial		setup is defined in the :ref:`amdgpu-amdhsa-kernel-descriptor`. The initial
register state is defined in		register state is defined in
:ref:`amdgpu-amdhsa-initial-kernel-execution-state`.		:ref:`amdgpu-amdhsa-initial-kernel-execution-state`.
▲ Show 20 Lines • Show All 117 Lines • ▼ Show 20 Lines

A kernel descriptor consists of the information needed by CP to initiate the		A kernel descriptor consists of the information needed by CP to initiate the
execution of a kernel, including the entry point address of the machine code		execution of a kernel, including the entry point address of the machine code
that implements the kernel.		that implements the kernel.

Kernel Descriptor for GFX6-GFX9		Kernel Descriptor for GFX6-GFX9
+++++++++++++++++++++++++++++++		+++++++++++++++++++++++++++++++

CP microcode requires the Kernel descritor to be allocated on 64 byte alignment.		CP microcode requires the Kernel descriptor to be allocated on 64 byte
		alignment.

.. table:: Kernel Descriptor for GFX6-GFX9		.. table:: Kernel Descriptor for GFX6-GFX9
:name: amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table		:name: amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table

======= ======= =============================== ============================		======= ======= =============================== ============================
Bits Size Field Name Description		Bits Size Field Name Description
======= ======= =============================== ============================		======= ======= =============================== ============================
31:0 4 bytes GROUP_SEGMENT_FIXED_SIZE The amount of fixed local		31:0 4 bytes GROUP_SEGMENT_FIXED_SIZE The amount of fixed local
▲ Show 20 Lines • Show All 64 Lines • ▼ Show 20 Lines
..		..

.. table:: compute_pgm_rsrc1 for GFX6-GFX9		.. table:: compute_pgm_rsrc1 for GFX6-GFX9
:name: amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table		:name: amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table

======= ======= =============================== ===========================================================================		======= ======= =============================== ===========================================================================
Bits Size Field Name Description		Bits Size Field Name Description
======= ======= =============================== ===========================================================================		======= ======= =============================== ===========================================================================
5:0 6 bits GRANULATED_WORKITEM_VGPR_COUNT Number of vector registers		5:0 6 bits GRANULATED_WORKITEM_VGPR_COUNT Number of vector register
used by each work-item,		blocks used by each work-item;
granularity is device		granularity is device
specific:		specific:

GFX6-GFX9		GFX6-GFX9
- max_vgpr 1..256		- vgprs_used 0..256
- roundup((max_vgpg + 1)		- max(0, ceil(vgprs_used / 4) - 1)
/ 4) - 1
		Where vgprs_used is defined
		as the highest VGPR number
		explicitly referenced plus
		one.

Used by CP to set up		Used by CP to set up
``COMPUTE_PGM_RSRC1.VGPRS``.		``COMPUTE_PGM_RSRC1.VGPRS``.
9:6 4 bits GRANULATED_WAVEFRONT_SGPR_COUNT Number of scalar registers
used by a wavefront,		The
		:ref:`amdgpu-assembler`
		calculates this
		automatically for the
		selected processor from
		values provided to the
		`.amdhsa_kernel` directive
		by the
		`.amdhsa_next_free_vgpr`
		nested directive (see
		:ref:`amdhsa-kernel-directives-table`).
		9:6 4 bits GRANULATED_WAVEFRONT_SGPR_COUNT Number of scalar register
		blocks used by a wavefront;
granularity is device		granularity is device
specific:		specific:

GFX6-GFX8		GFX6-GFX8
- max_sgpr 1..112		- sgprs_used 0..112
- roundup((max_sgpg + 1)		- max(0, ceil(sgprs_used / 8) - 1)
/ 8) - 1
GFX9		GFX9
- max_sgpr 1..112		- sgprs_used 0..112
- roundup((max_sgpg + 1)		- 2 * max(0, ceil(sgprs_used / 16) - 1)
/ 16) - 1
		Where sgprs_used is
Includes the special SGPRs		defined as the highest
for VCC, Flat Scratch (for		SGPR number explicitly
GFX7 onwards) and XNACK		referenced plus one, plus
(for GFX8 onwards). It does		a target-specific number
not include the 16 SGPR		of additional special
added if a trap handler is		SGPRs for VCC,
		FLAT_SCRATCH (GFX7+) and
		XNACK_MASK (GFX8+), and
		any additional
		target-specific
		limitations. It does not
		include the 16 SGPRs added
		if a trap handler is
enabled.		enabled.

		The target-specific
		limitations and special
		SGPR layout are defined in
		the hardware
		documentation, which can
		be found in the
		:ref:`amdgpu-processors`
		table.

Used by CP to set up		Used by CP to set up
``COMPUTE_PGM_RSRC1.SGPRS``.		``COMPUTE_PGM_RSRC1.SGPRS``.

		The
		:ref:`amdgpu-assembler`
		calculates this
		automatically for the
		selected processor from
		values provided to the
		`.amdhsa_kernel` directive
		by the
		`.amdhsa_next_free_sgpr`
		and `.amdhsa_reserve_*`
		nested directives (see
		:ref:`amdhsa-kernel-directives-table`).
11:10 2 bits PRIORITY Must be 0.		11:10 2 bits PRIORITY Must be 0.

Start executing wavefront		Start executing wavefront
at the specified priority.		at the specified priority.

CP is responsible for		CP is responsible for
filling in		filling in
``COMPUTE_PGM_RSRC1.PRIORITY``.		``COMPUTE_PGM_RSRC1.PRIORITY``.
▲ Show 20 Lines • Show All 2,262 Lines • ▼ Show 20 Lines

When the language is OpenCL the following differences occur:		When the language is OpenCL the following differences occur:

1. The OpenCL memory model is used (see :ref:`amdgpu-amdhsa-memory-model`).		1. The OpenCL memory model is used (see :ref:`amdgpu-amdhsa-memory-model`).
2. The AMDGPU backend appends additional arguments to the kernel's explicit		2. The AMDGPU backend appends additional arguments to the kernel's explicit
arguments for the AMDHSA OS (see		arguments for the AMDHSA OS (see
:ref:`opencl-kernel-implicit-arguments-appended-for-amdhsa-os-table`).		:ref:`opencl-kernel-implicit-arguments-appended-for-amdhsa-os-table`).
3. Additional metadata is generated		3. Additional metadata is generated
(see :ref:`amdgpu-amdhsa-hsa-code-object-metadata`).		(see :ref:`amdgpu-amdhsa-code-object-metadata`).

.. table:: OpenCL kernel implicit arguments appended for AMDHSA OS		.. table:: OpenCL kernel implicit arguments appended for AMDHSA OS
:name: opencl-kernel-implicit-arguments-appended-for-amdhsa-os-table		:name: opencl-kernel-implicit-arguments-appended-for-amdhsa-os-table

======== ==== ========= ===========================================		======== ==== ========= ===========================================
Position Byte Byte Description		Position Byte Byte Description
Size Alignment		Size Alignment
======== ==== ========= ===========================================		======== ==== ========= ===========================================
Show All 11 Lines

HCC		HCC
---		---

When the language is HCC the following differences occur:		When the language is HCC the following differences occur:

1. The HSA memory model is used (see :ref:`amdgpu-amdhsa-memory-model`).		1. The HSA memory model is used (see :ref:`amdgpu-amdhsa-memory-model`).

		.. _amdgpu-assembler:

Assembler		Assembler
---------		---------

AMDGPU backend has LLVM-MC based assembler which is currently in development.		AMDGPU backend has LLVM-MC based assembler which is currently in development.
It supports AMDGCN GFX6-GFX9.		It supports AMDGCN GFX6-GFX9.

This section describes general syntax for instructions and operands.		This section describes general syntax for instructions and operands.

▲ Show 20 Lines • Show All 216 Lines • ▼ Show 20 Lines	.. code-block:: nasm
v_mov_b32 v1, v2 dst_sel:BYTE_0 dst_unused:UNUSED_PRESERVE src0_sel:DWORD		v_mov_b32 v1, v2 dst_sel:BYTE_0 dst_unused:UNUSED_PRESERVE src0_sel:DWORD
v_min_u32 v200, v200, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:DWORD		v_min_u32 v200, v200, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:DWORD
v_sin_f32 v0, v0 dst_unused:UNUSED_PAD src0_sel:WORD_1		v_sin_f32 v0, v0 dst_unused:UNUSED_PAD src0_sel:WORD_1
v_fract_f32 v0, \|v0\| dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1		v_fract_f32 v0, \|v0\| dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
v_cmpx_le_u32 vcc, v1, v2 src0_sel:BYTE_2 src1_sel:WORD_0		v_cmpx_le_u32 vcc, v1, v2 src0_sel:BYTE_2 src1_sel:WORD_0

For full list of supported instructions, refer to "Vector ALU instructions".		For full list of supported instructions, refer to "Vector ALU instructions".

HSA Code Object Directives		Predefined Symbols
~~~~~~~~~~~~~~~~~~~~~~~~~~		~~~~~~~~~~~~~~~~~~

AMDGPU ABI defines auxiliary data in output code object. In assembly source,
one can specify them with assembler directives.

.hsa_code_object_version major, minor
+++++++++++++++++++++++++++++++++++++

major and minor are integers that specify the version of the HSA code
object that will be generated by the assembler.

.hsa_code_object_isa [major, minor, stepping, vendor, arch]
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


major, minor, and stepping are all integers that describe the instruction
set architecture (ISA) version of the assembly program.

vendor and arch are quoted strings. vendor should always be equal to
"AMD" and arch should always be equal to "AMDGPU".

By default, the assembler will derive the ISA version, vendor, and arch
from the value of the -mcpu option that is passed to the assembler.

.amdgpu_hsa_kernel (name)
+++++++++++++++++++++++++

This directives specifies that the symbol with given name is a kernel entry point
(label) and the object should contain corresponding symbol of type STT_AMDGPU_HSA_KERNEL.

.amd_kernel_code_t
++++++++++++++++++

This directive marks the beginning of a list of key / value pairs that are used		The AMDGPU assembler defines and updates some symbols automatically. These
to specify the amd_kernel_code_t object that will be emitted by the assembler.		symbols do not affect code generation.
The list must be terminated by the .end_amd_kernel_code_t directive. For
any amd_kernel_code_t values that are unspecified a default value will be
used. The default value for all keys is 0, with the following exceptions:

- kernel_code_version_major defaults to 1.		.amdgcn.gfx_generation_number
- machine_kind defaults to 1.		+++++++++++++++++++++++++++++
- machine_version_major, machine_version_minor, and
machine_version_stepping are derived from the value of the -mcpu option
that is passed to the assembler.
- kernel_code_entry_byte_offset defaults to 256.
- wavefront_size defaults to 6.
- kernarg_segment_alignment, group_segment_alignment, and
private_segment_alignment default to 4. Note that alignments are specified
as a power of two, so a value of n means an alignment of 2^ n.

The .amd_kernel_code_t directive must be placed immediately after the		Set to the GFX generation number of the target being assembled for. For
function label and before any instructions.		example, when assembling for a "GFX9" target this will be set to the integer
		value "9". The possible GFX generation numbers are presented in
		:ref:`amdgpu-processors`.

		.amdgcn.next_free_vgpr
		++++++++++++++++++++++

		Set to zero before assembly begins. At each instruction, if the current value
		of this symbol is less than or equal to the maximum VGPR number explicitly
		referenced within that instruction then the symbol value is updated to equal
		that VGPR number plus one.

		May be used to set the `.amdhsa_next_free_vpgr` directive in
		:ref:`amdhsa-kernel-directives-table`.

		May be set at any time, e.g. manually set to zero at the start of each kernel.

		.amdgcn.next_free_sgpr
		++++++++++++++++++++++

		Set to zero before assembly begins. At each instruction, if the current value
		of this symbol is less than or equal the maximum SGPR number explicitly
		referenced within that instruction then the symbol value is updated to equal
		that SGPR number plus one.

		May be used to set the `.amdhsa_next_free_spgr` directive in
		:ref:`amdhsa-kernel-directives-table`.

		May be set at any time, e.g. manually set to zero at the start of each kernel.

		Code Object Directives
		~~~~~~~~~~~~~~~~~~~~~~

		Directives which begin with ``.amdgcn`` are valid for all ``amdgcn``
		architecture processors, and are not OS-specific. Directives which begin with
		``.amdhsa`` are specific to ``amdgcn`` architecture processors when the
		``amdhsa`` OS is specified. See :ref:`amdgpu-target-triples` and
		:ref:`amdgpu-processors`.

		.amdgcn_target <target>
		+++++++++++++++++++++++

		Optional directive which declares the target supported by the containing
		assembler source file. Valid values are described in
		:ref:`amdgpu-amdhsa-code-object-target-identification`. Used by the assembler
		to validate command-line options such as ``-triple``, ``-mcpu``, and those
		which specify target features.

		.amdhsa_kernel <name>
		+++++++++++++++++++++

		Creates a correctly aligned AMDHSA kernel descriptor and a symbol,
		``<name>.kd``, in the current location of the current section. Only valid when
		the OS is ``amdhsa``. ``<name>`` must be a symbol that labels the first
		instruction to execute, and does not need to be previously defined.

		Marks the beginning of a list of directives used to generate the bytes of a
		kernel descriptor, as described in :ref:`amdgpu-amdhsa-kernel-descriptor`.
		Directives which may appear in this list are described in
		:ref:`amdhsa-kernel-directives-table`. Directives may appear in any order, must
		be valid for the target being assembled for, and cannot be repeated. Directives
		support the range of values specified by the field they reference in
		:ref:`amdgpu-amdhsa-kernel-descriptor`. If a directive is not specified, it is
		assumed to have its default value, unless it is marked as "Required", in which
		case it is an error to omit the directive. This list of directives is
		terminated by an ``.end_amdhsa_kernel`` directive.

		.. table:: AMDHSA Kernel Assembler Directives
		:name: amdhsa-kernel-directives-table

		======================================================== ================ ============ ===================
		Directive Default Supported On Description
		======================================================== ================ ============ ===================
		``.amdhsa_group_segment_fixed_size`` 0 GFX6-GFX9 Controls GROUP_SEGMENT_FIXED_SIZE in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_private_segment_fixed_size`` 0 GFX6-GFX9 Controls PRIVATE_SEGMENT_FIXED_SIZE in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_private_segment_buffer`` 0 GFX6-GFX9 Controls ENABLE_SGPR_PRIVATE_SEGMENT_BUFFER in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_dispatch_ptr`` 0 GFX6-GFX9 Controls ENABLE_SGPR_DISPATCH_PTR in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_queue_ptr`` 0 GFX6-GFX9 Controls ENABLE_SGPR_QUEUE_PTR in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_kernarg_segment_ptr`` 0 GFX6-GFX9 Controls ENABLE_SGPR_KERNARG_SEGMENT_PTR in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_dispatch_id`` 0 GFX6-GFX9 Controls ENABLE_SGPR_DISPATCH_ID in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_flat_scratch_init`` 0 GFX6-GFX9 Controls ENABLE_SGPR_FLAT_SCRATCH_INIT in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_user_sgpr_private_segment_size`` 0 GFX6-GFX9 Controls ENABLE_SGPR_PRIVATE_SEGMENT_SIZE in
		:ref:`amdgpu-amdhsa-kernel-descriptor-gfx6-gfx9-table`.
		``.amdhsa_system_sgpr_private_segment_wavefront_offset`` 0 GFX6-GFX9 Controls ENABLE_SGPR_PRIVATE_SEGMENT_WAVEFRONT_OFFSET in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_system_sgpr_workgroup_id_x`` 1 GFX6-GFX9 Controls ENABLE_SGPR_WORKGROUP_ID_X in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_system_sgpr_workgroup_id_y`` 0 GFX6-GFX9 Controls ENABLE_SGPR_WORKGROUP_ID_Y in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_system_sgpr_workgroup_id_z`` 0 GFX6-GFX9 Controls ENABLE_SGPR_WORKGROUP_ID_Z in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_system_sgpr_workgroup_info`` 0 GFX6-GFX9 Controls ENABLE_SGPR_WORKGROUP_INFO in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_system_vgpr_workitem_id`` 0 GFX6-GFX9 Controls ENABLE_VGPR_WORKITEM_ID in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		Possible values are defined in
		:ref:`amdgpu-amdhsa-system-vgpr-work-item-id-enumeration-values-table`.
		``.amdhsa_next_free_vgpr`` Required GFX6-GFX9 Maximum VGPR number explicitly referenced, plus one.
		Used to calculate GRANULATED_WORKITEM_VGPR_COUNT in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_next_free_sgpr`` Required GFX6-GFX9 Maximum SGPR number explicitly referenced, plus one.
		Used to calculate GRANULATED_WAVEFRONT_SGPR_COUNT in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_reserve_vcc`` 1 GFX6-GFX9 Whether the kernel may use the special VCC SGPR.
		Used to calculate GRANULATED_WAVEFRONT_SGPR_COUNT in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_reserve_flat_scratch`` 1 GFX7-GFX9 Whether the kernel may use flat instructions to access
		scratch memory. Used to calculate
		GRANULATED_WAVEFRONT_SGPR_COUNT in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_reserve_xnack_mask`` Target GFX8-GFX9 Whether the kernel may trigger XNACK replay.
		Feature Used to calculate GRANULATED_WAVEFRONT_SGPR_COUNT in
		Specific :ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		(+xnack)
		``.amdhsa_float_round_mode_32`` 0 GFX6-GFX9 Controls FLOAT_ROUND_MODE_32 in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		Possible values are defined in
		:ref:`amdgpu-amdhsa-floating-point-rounding-mode-enumeration-values-table`.
		``.amdhsa_float_round_mode_16_64`` 0 GFX6-GFX9 Controls FLOAT_ROUND_MODE_16_64 in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		Possible values are defined in
		:ref:`amdgpu-amdhsa-floating-point-rounding-mode-enumeration-values-table`.
		``.amdhsa_float_denorm_mode_32`` 0 GFX6-GFX9 Controls FLOAT_DENORM_MODE_32 in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		Possible values are defined in
		:ref:`amdgpu-amdhsa-floating-point-denorm-mode-enumeration-values-table`.
		``.amdhsa_float_denorm_mode_16_64`` 3 GFX6-GFX9 Controls FLOAT_DENORM_MODE_16_64 in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		Possible values are defined in
		:ref:`amdgpu-amdhsa-floating-point-denorm-mode-enumeration-values-table`.
		``.amdhsa_dx10_clamp`` 1 GFX6-GFX9 Controls ENABLE_DX10_CLAMP in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_ieee_mode`` 1 GFX6-GFX9 Controls ENABLE_IEEE_MODE in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_fp16_overflow`` 0 GFX9 Controls FP16_OVFL in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc1-gfx6-gfx9-table`.
		``.amdhsa_exception_fp_ieee_invalid_op`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_IEEE_754_FP_INVALID_OPERATION in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_exception_fp_denorm_src`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_FP_DENORMAL_SOURCE in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_exception_fp_ieee_div_zero`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_IEEE_754_FP_DIVISION_BY_ZERO in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_exception_fp_ieee_overflow`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_IEEE_754_FP_OVERFLOW in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_exception_fp_ieee_underflow`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_IEEE_754_FP_UNDERFLOW in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_exception_fp_ieee_inexact`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_IEEE_754_FP_INEXACT in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		``.amdhsa_exception_int_div_zero`` 0 GFX6-GFX9 Controls ENABLE_EXCEPTION_INT_DIVIDE_BY_ZERO in
		:ref:`amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table`.
		======================================================== ================ ============ ===================

For a full list of amd_kernel_code_t keys, refer to AMDGPU ABI document,		Example HSA Source Code
comments in lib/Target/AMDGPU/AmdKernelCodeT.h and test/CodeGen/AMDGPU/hsa.s.		~~~~~~~~~~~~~~~~~~~~~~~

Here is an example of a minimal amd_kernel_code_t specification:		Here is an example of a minimal assembly source file, defining one HSA kernel:

.. code-block:: none		.. code-block:: nasm

.hsa_code_object_version 1,0		.amdgcn_target "amdgcn-amd-amdhsa--gfx900+xnack" // optional
.hsa_code_object_isa

.hsatext		.text
.globl hello_world		.globl hello_world
.p2align 8		.p2align 8
.amdgpu_hsa_kernel hello_world		.type hello_world,@function

hello_world:		hello_world:

.amd_kernel_code_t
enable_sgpr_kernarg_segment_ptr = 1
is_ptr64 = 1
compute_pgm_rsrc1_vgprs = 0
compute_pgm_rsrc1_sgprs = 0
compute_pgm_rsrc2_user_sgpr = 2
kernarg_segment_byte_size = 8
wavefront_sgpr_count = 2
workitem_vgpr_count = 3
.end_amd_kernel_code_t

s_load_dwordx2 s[0:1], s[0:1] 0x0		s_load_dwordx2 s[0:1], s[0:1] 0x0
v_mov_b32 v0, 3.14159		v_mov_b32 v0, 3.14159
s_waitcnt lgkmcnt(0)		s_waitcnt lgkmcnt(0)
v_mov_b32 v1, s0		v_mov_b32 v1, s0
v_mov_b32 v2, s1		v_mov_b32 v2, s1
flat_store_dword v[1:2], v0		flat_store_dword v[1:2], v0
s_endpgm		s_endpgm
.Lfunc_end0:		.Lfunc_end0:
.size hello_world, .Lfunc_end0-hello_world		.size hello_world, .Lfunc_end0-hello_world

		.rodata
		.p2align 6
		.amdhsa_kernel hello_world
		.amdhsa_user_sgpr_kernarg_segment_ptr 1
		.amdhsa_next_free_vgpr .amdgcn.next_free_vgpr
		.amdhsa_next_free_sgpr .amdgcn.next_free_sgpr
		.end_amdhsa_kernel


Additional Documentation		Additional Documentation
========================		========================

.. [AMD-RADEON-HD-2000-3000] `AMD R6xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R600_Instruction_Set_Architecture.pdf>`__		.. [AMD-RADEON-HD-2000-3000] `AMD R6xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R600_Instruction_Set_Architecture.pdf>`__
.. [AMD-RADEON-HD-4000] `AMD R7xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R700-Family_Instruction_Set_Architecture.pdf>`__		.. [AMD-RADEON-HD-4000] `AMD R7xx shader ISA <http://developer.amd.com/wordpress/media/2012/10/R700-Family_Instruction_Set_Architecture.pdf>`__
.. [AMD-RADEON-HD-5000] `AMD Evergreen shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_Evergreen-Family_Instruction_Set_Architecture.pdf>`__		.. [AMD-RADEON-HD-5000] `AMD Evergreen shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_Evergreen-Family_Instruction_Set_Architecture.pdf>`__
.. [AMD-RADEON-HD-6000] `AMD Cayman/Trinity shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_HD_6900_Series_Instruction_Set_Architecture.pdf>`__		.. [AMD-RADEON-HD-6000] `AMD Cayman/Trinity shader ISA <http://developer.amd.com/wordpress/media/2012/10/AMD_HD_6900_Series_Instruction_Set_Architecture.pdf>`__
.. [AMD-GCN-GFX6] `AMD Southern Islands Series ISA <http://developer.amd.com/wordpress/media/2012/12/AMD_Southern_Islands_Instruction_Set_Architecture.pdf>`__		.. [AMD-GCN-GFX6] `AMD Southern Islands Series ISA <http://developer.amd.com/wordpress/media/2012/12/AMD_Southern_Islands_Instruction_Set_Architecture.pdf>`__
Show All 12 Lines

llvm/trunk/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

Show First 20 Lines • Show All 231 Lines • ▼ Show 20 Lines	void AMDGPUAsmPrinter::EmitFunctionBodyEnd() {
// alignment.		// alignment.
Streamer.EmitValueToAlignment(64, 0, 1, 0);		Streamer.EmitValueToAlignment(64, 0, 1, 0);
if (ReadOnlySection.getAlignment() < 64)		if (ReadOnlySection.getAlignment() < 64)
ReadOnlySection.setAlignment(64);		ReadOnlySection.setAlignment(64);

SmallString<128> KernelName;		SmallString<128> KernelName;
getNameWithPrefix(KernelName, &MF->getFunction());		getNameWithPrefix(KernelName, &MF->getFunction());
getTargetStreamer()->EmitAmdhsaKernelDescriptor(		getTargetStreamer()->EmitAmdhsaKernelDescriptor(
KernelName, getAmdhsaKernelDescriptor(*MF, CurrentProgramInfo));		getSTI(), KernelName, getAmdhsaKernelDescriptor(MF, CurrentProgramInfo),
		CurrentProgramInfo.NumVGPRsForWavesPerEU,
		CurrentProgramInfo.NumSGPRsForWavesPerEU -
		IsaInfo::getNumExtraSGPRs(getSTI()->getFeatureBits(),
		CurrentProgramInfo.VCCUsed,
		CurrentProgramInfo.FlatUsed),
		CurrentProgramInfo.VCCUsed, CurrentProgramInfo.FlatUsed,
		hasXNACK(*getSTI()));

Streamer.PopSection();		Streamer.PopSection();
}		}

void AMDGPUAsmPrinter::EmitFunctionEntryLabel() {		void AMDGPUAsmPrinter::EmitFunctionEntryLabel() {
if (IsaInfo::hasCodeObjectV3(getSTI()) &&		if (IsaInfo::hasCodeObjectV3(getSTI()) &&
TM.getTargetTriple().getOS() == Triple::AMDHSA) {		TM.getTargetTriple().getOS() == Triple::AMDHSA) {
AsmPrinter::EmitFunctionEntryLabel();		AsmPrinter::EmitFunctionEntryLabel();
▲ Show 20 Lines • Show All 305 Lines • ▼ Show 20 Lines	static bool hasAnyNonFlatUseOfReg(const MachineRegisterInfo &MRI,
for (const MachineOperand &UseOp : MRI.reg_operands(Reg)) {		for (const MachineOperand &UseOp : MRI.reg_operands(Reg)) {
if (!UseOp.isImplicit() \|\| !TII.isFLAT(*UseOp.getParent()))		if (!UseOp.isImplicit() \|\| !TII.isFLAT(*UseOp.getParent()))
return true;		return true;
}		}

return false;		return false;
}		}

static unsigned getNumExtraSGPRs(const SISubtarget &ST,
bool VCCUsed,
bool FlatScrUsed) {
unsigned ExtraSGPRs = 0;
if (VCCUsed)
ExtraSGPRs = 2;

if (ST.getGeneration() < SISubtarget::VOLCANIC_ISLANDS) {
if (FlatScrUsed)
ExtraSGPRs = 4;
} else {
if (ST.isXNACKEnabled())
ExtraSGPRs = 4;

if (FlatScrUsed)
ExtraSGPRs = 6;
}

return ExtraSGPRs;
}

int32_t AMDGPUAsmPrinter::SIFunctionResourceInfo::getTotalNumSGPRs(		int32_t AMDGPUAsmPrinter::SIFunctionResourceInfo::getTotalNumSGPRs(
const SISubtarget &ST) const {		const SISubtarget &ST) const {
return NumExplicitSGPR + getNumExtraSGPRs(ST, UsesVCC, UsesFlatScratch);		return NumExplicitSGPR + IsaInfo::getNumExtraSGPRs(ST.getFeatureBits(),
		UsesVCC, UsesFlatScratch);
}		}

AMDGPUAsmPrinter::SIFunctionResourceInfo AMDGPUAsmPrinter::analyzeResourceUsage(		AMDGPUAsmPrinter::SIFunctionResourceInfo AMDGPUAsmPrinter::analyzeResourceUsage(
const MachineFunction &MF) const {		const MachineFunction &MF) const {
SIFunctionResourceInfo Info;		SIFunctionResourceInfo Info;

const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();		const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();
const SISubtarget &ST = MF.getSubtarget<SISubtarget>();		const SISubtarget &ST = MF.getSubtarget<SISubtarget>();
▲ Show 20 Lines • Show All 178 Lines • ▼ Show 20 Lines	for (const MachineInstr &MI : MBB) {
const MachineOperand *CalleeOp		const MachineOperand *CalleeOp
= TII->getNamedOperand(MI, AMDGPU::OpName::callee);		= TII->getNamedOperand(MI, AMDGPU::OpName::callee);
const Function *Callee = cast<Function>(CalleeOp->getGlobal());		const Function *Callee = cast<Function>(CalleeOp->getGlobal());
if (Callee->isDeclaration()) {		if (Callee->isDeclaration()) {
// If this is a call to an external function, we can't do much. Make		// If this is a call to an external function, we can't do much. Make
// conservative guesses.		// conservative guesses.

// 48 SGPRs - vcc, - flat_scr, -xnack		// 48 SGPRs - vcc, - flat_scr, -xnack
int MaxSGPRGuess = 47 - getNumExtraSGPRs(ST, true,		int MaxSGPRGuess =
		47 - IsaInfo::getNumExtraSGPRs(ST.getFeatureBits(), true,
ST.hasFlatAddressSpace());		ST.hasFlatAddressSpace());
MaxSGPR = std::max(MaxSGPR, MaxSGPRGuess);		MaxSGPR = std::max(MaxSGPR, MaxSGPRGuess);
MaxVGPR = std::max(MaxVGPR, 23);		MaxVGPR = std::max(MaxVGPR, 23);

CalleeFrameSize = std::max(CalleeFrameSize, UINT64_C(16384));		CalleeFrameSize = std::max(CalleeFrameSize, UINT64_C(16384));
Info.UsesVCC = true;		Info.UsesVCC = true;
Info.UsesFlatScratch = ST.hasFlatAddressSpace();		Info.UsesFlatScratch = ST.hasFlatAddressSpace();
Info.HasDynamicallySizedStack = true;		Info.HasDynamicallySizedStack = true;
} else {		} else {
▲ Show 20 Lines • Show All 43 Lines • ▼ Show 20 Lines	if (!isUInt<32>(ProgInfo.ScratchSize)) {
MF.getFunction().getContext().diagnose(DiagStackSize);		MF.getFunction().getContext().diagnose(DiagStackSize);
}		}

const SISubtarget &STM = MF.getSubtarget<SISubtarget>();		const SISubtarget &STM = MF.getSubtarget<SISubtarget>();
const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();		const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();
const SIInstrInfo *TII = STM.getInstrInfo();		const SIInstrInfo *TII = STM.getInstrInfo();
const SIRegisterInfo *RI = &TII->getRegisterInfo();		const SIRegisterInfo *RI = &TII->getRegisterInfo();

unsigned ExtraSGPRs = getNumExtraSGPRs(STM,		// TODO(scott.linder): The calculations related to SGPR/VGPR blocks are
ProgInfo.VCCUsed,		// duplicated in part in AMDGPUAsmParser::calculateGPRBlocks, and could be
ProgInfo.FlatUsed);		// unified.
		unsigned ExtraSGPRs = IsaInfo::getNumExtraSGPRs(
		STM.getFeatureBits(), ProgInfo.VCCUsed, ProgInfo.FlatUsed);
unsigned ExtraVGPRs = STM.getReservedNumVGPRs(MF);		unsigned ExtraVGPRs = STM.getReservedNumVGPRs(MF);

// Check the addressable register limit before we add ExtraSGPRs.		// Check the addressable register limit before we add ExtraSGPRs.
if (STM.getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS &&		if (STM.getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS &&
!STM.hasSGPRInitBug()) {		!STM.hasSGPRInitBug()) {
unsigned MaxAddressableNumSGPRs = STM.getAddressableNumSGPRs();		unsigned MaxAddressableNumSGPRs = STM.getAddressableNumSGPRs();
if (ProgInfo.NumSGPR > MaxAddressableNumSGPRs) {		if (ProgInfo.NumSGPR > MaxAddressableNumSGPRs) {
// This can happen due to a compiler bug or when using inline asm.		// This can happen due to a compiler bug or when using inline asm.
▲ Show 20 Lines • Show All 66 Lines • ▼ Show 20 Lines	void AMDGPUAsmPrinter::getSIProgramInfo(SIProgramInfo &ProgInfo,

if (MFI->getLDSSize() > static_cast<unsigned>(STM.getLocalMemorySize())) {		if (MFI->getLDSSize() > static_cast<unsigned>(STM.getLocalMemorySize())) {
LLVMContext &Ctx = MF.getFunction().getContext();		LLVMContext &Ctx = MF.getFunction().getContext();
DiagnosticInfoResourceLimit Diag(MF.getFunction(), "local memory",		DiagnosticInfoResourceLimit Diag(MF.getFunction(), "local memory",
MFI->getLDSSize(), DS_Error);		MFI->getLDSSize(), DS_Error);
Ctx.diagnose(Diag);		Ctx.diagnose(Diag);
}		}

// SGPRBlocks is actual number of SGPR blocks minus 1.		ProgInfo.SGPRBlocks = IsaInfo::getNumSGPRBlocks(
ProgInfo.SGPRBlocks = alignTo(ProgInfo.NumSGPRsForWavesPerEU,		STM.getFeatureBits(), ProgInfo.NumSGPRsForWavesPerEU);
STM.getSGPREncodingGranule());		ProgInfo.VGPRBlocks = IsaInfo::getNumVGPRBlocks(
ProgInfo.SGPRBlocks = ProgInfo.SGPRBlocks / STM.getSGPREncodingGranule() - 1;		STM.getFeatureBits(), ProgInfo.NumVGPRsForWavesPerEU);

// VGPRBlocks is actual number of VGPR blocks minus 1.
ProgInfo.VGPRBlocks = alignTo(ProgInfo.NumVGPRsForWavesPerEU,
STM.getVGPREncodingGranule());
ProgInfo.VGPRBlocks = ProgInfo.VGPRBlocks / STM.getVGPREncodingGranule() - 1;

// Record first reserved VGPR and number of reserved VGPRs.		// Record first reserved VGPR and number of reserved VGPRs.
ProgInfo.ReservedVGPRFirst = STM.debuggerReserveRegs() ? ProgInfo.NumVGPR : 0;		ProgInfo.ReservedVGPRFirst = STM.debuggerReserveRegs() ? ProgInfo.NumVGPR : 0;
ProgInfo.ReservedVGPRCount = STM.getReservedNumVGPRs(MF);		ProgInfo.ReservedVGPRCount = STM.getReservedNumVGPRs(MF);

// Update DebuggerWavefrontPrivateSegmentOffsetSGPR and		// Update DebuggerWavefrontPrivateSegmentOffsetSGPR and
// DebuggerPrivateSegmentBufferSGPR fields if "amdgpu-debugger-emit-prologue"		// DebuggerPrivateSegmentBufferSGPR fields if "amdgpu-debugger-emit-prologue"
// attribute was requested.		// attribute was requested.
▲ Show 20 Lines • Show All 365 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

Show All 36 Lines
#include "llvm/MC/MCParser/MCAsmParserExtension.h"		#include "llvm/MC/MCParser/MCAsmParserExtension.h"
#include "llvm/MC/MCParser/MCParsedAsmOperand.h"		#include "llvm/MC/MCParser/MCParsedAsmOperand.h"
#include "llvm/MC/MCParser/MCTargetAsmParser.h"		#include "llvm/MC/MCParser/MCTargetAsmParser.h"
#include "llvm/MC/MCRegisterInfo.h"		#include "llvm/MC/MCRegisterInfo.h"
#include "llvm/MC/MCStreamer.h"		#include "llvm/MC/MCStreamer.h"
#include "llvm/MC/MCSubtargetInfo.h"		#include "llvm/MC/MCSubtargetInfo.h"
#include "llvm/MC/MCSymbol.h"		#include "llvm/MC/MCSymbol.h"
#include "llvm/Support/AMDGPUMetadata.h"		#include "llvm/Support/AMDGPUMetadata.h"
		#include "llvm/Support/AMDHSAKernelDescriptor.h"
#include "llvm/Support/Casting.h"		#include "llvm/Support/Casting.h"
#include "llvm/Support/Compiler.h"		#include "llvm/Support/Compiler.h"
#include "llvm/Support/ErrorHandling.h"		#include "llvm/Support/ErrorHandling.h"
#include "llvm/Support/MachineValueType.h"		#include "llvm/Support/MachineValueType.h"
#include "llvm/Support/MathExtras.h"		#include "llvm/Support/MathExtras.h"
#include "llvm/Support/SMLoc.h"		#include "llvm/Support/SMLoc.h"
#include "llvm/Support/TargetRegistry.h"		#include "llvm/Support/TargetRegistry.h"
#include "llvm/Support/raw_ostream.h"		#include "llvm/Support/raw_ostream.h"
#include <algorithm>		#include <algorithm>
#include <cassert>		#include <cassert>
#include <cstdint>		#include <cstdint>
#include <cstring>		#include <cstring>
#include <iterator>		#include <iterator>
#include <map>		#include <map>
#include <memory>		#include <memory>
#include <string>		#include <string>

using namespace llvm;		using namespace llvm;
using namespace llvm::AMDGPU;		using namespace llvm::AMDGPU;
		using namespace llvm::amdhsa;

namespace {		namespace {

class AMDGPUAsmParser;		class AMDGPUAsmParser;

enum RegisterKind { IS_UNKNOWN, IS_VGPR, IS_SGPR, IS_TTMP, IS_SPECIAL };		enum RegisterKind { IS_UNKNOWN, IS_VGPR, IS_SGPR, IS_TTMP, IS_SPECIAL };

//===----------------------------------------------------------------------===//		//===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 768 Lines • ▼ Show 20 Lines

#define GET_ASSEMBLER_HEADER		#define GET_ASSEMBLER_HEADER
#include "AMDGPUGenAsmMatcher.inc"		#include "AMDGPUGenAsmMatcher.inc"

/// }		/// }

private:		private:
bool ParseAsAbsoluteExpression(uint32_t &Ret);		bool ParseAsAbsoluteExpression(uint32_t &Ret);
		bool OutOfRangeError(SMRange Range);
		/// Calculate VGPR/SGPR blocks required for given target, reserved
		/// registers, and user-specified NextFreeXGPR values.
		///
		/// \param Features [in] Target features, used for bug corrections.
		/// \param VCCUsed [in] Whether VCC special SGPR is reserved.
		/// \param FlatScrUsed [in] Whether FLAT_SCRATCH special SGPR is reserved.
		/// \param XNACKUsed [in] Whether XNACK_MASK special SGPR is reserved.
		/// \param NextFreeVGPR [in] Max VGPR number referenced, plus one.
		/// \param VGPRRange [in] Token range, used for VGPR diagnostics.
		/// \param NextFreeSGPR [in] Max SGPR number referenced, plus one.
		/// \param SGPRRange [in] Token range, used for SGPR diagnostics.
		/// \param VGPRBlocks [out] Result VGPR block count.
		/// \param SGPRBlocks [out] Result SGPR block count.
		bool calculateGPRBlocks(const FeatureBitset &Features, bool VCCUsed,
		bool FlatScrUsed, bool XNACKUsed,
		unsigned NextFreeVGPR, SMRange VGPRRange,
		unsigned NextFreeSGPR, SMRange SGPRRange,
		unsigned &VGPRBlocks, unsigned &SGPRBlocks);
		bool ParseDirectiveAMDGCNTarget();
		bool ParseDirectiveAMDHSAKernel();
bool ParseDirectiveMajorMinor(uint32_t &Major, uint32_t &Minor);		bool ParseDirectiveMajorMinor(uint32_t &Major, uint32_t &Minor);
bool ParseDirectiveHSACodeObjectVersion();		bool ParseDirectiveHSACodeObjectVersion();
bool ParseDirectiveHSACodeObjectISA();		bool ParseDirectiveHSACodeObjectISA();
bool ParseAMDKernelCodeTValue(StringRef ID, amd_kernel_code_t &Header);		bool ParseAMDKernelCodeTValue(StringRef ID, amd_kernel_code_t &Header);
bool ParseDirectiveAMDKernelCodeT();		bool ParseDirectiveAMDKernelCodeT();
bool subtargetHasRegister(const MCRegisterInfo &MRI, unsigned RegNo) const;		bool subtargetHasRegister(const MCRegisterInfo &MRI, unsigned RegNo) const;
bool ParseDirectiveAMDGPUHsaKernel();		bool ParseDirectiveAMDGPUHsaKernel();

bool ParseDirectiveISAVersion();		bool ParseDirectiveISAVersion();
bool ParseDirectiveHSAMetadata();		bool ParseDirectiveHSAMetadata();
bool ParseDirectivePALMetadata();		bool ParseDirectivePALMetadata();

bool AddNextRegisterToList(unsigned& Reg, unsigned& RegWidth,		bool AddNextRegisterToList(unsigned& Reg, unsigned& RegWidth,
RegisterKind RegKind, unsigned Reg1,		RegisterKind RegKind, unsigned Reg1,
unsigned RegNum);		unsigned RegNum);
bool ParseAMDGPURegister(RegisterKind& RegKind, unsigned& Reg,		bool ParseAMDGPURegister(RegisterKind& RegKind, unsigned& Reg,
unsigned& RegNum, unsigned& RegWidth,		unsigned& RegNum, unsigned& RegWidth,
unsigned *DwordRegIndex);		unsigned *DwordRegIndex);
		Optional<StringRef> getGprCountSymbolName(RegisterKind RegKind);
		void initializeGprCountSymbol(RegisterKind RegKind);
		bool updateGprCountSymbols(RegisterKind RegKind, unsigned DwordRegIndex,
		unsigned RegWidth);
void cvtMubufImpl(MCInst &Inst, const OperandVector &Operands,		void cvtMubufImpl(MCInst &Inst, const OperandVector &Operands,
bool IsAtomic, bool IsAtomicReturn, bool IsLds = false);		bool IsAtomic, bool IsAtomicReturn, bool IsLds = false);
void cvtDSImpl(MCInst &Inst, const OperandVector &Operands,		void cvtDSImpl(MCInst &Inst, const OperandVector &Operands,
bool IsGdsHardcoded);		bool IsGdsHardcoded);

public:		public:
enum AMDGPUMatchResultTy {		enum AMDGPUMatchResultTy {
Match_PreferE32 = FIRST_TARGET_MATCH_RESULT_TY		Match_PreferE32 = FIRST_TARGET_MATCH_RESULT_TY
Show All 17 Lines	AMDGPUAsmParser(const MCSubtargetInfo &STI, MCAsmParser &_Parser,
{		{
// TODO: make those pre-defined variables read-only.		// TODO: make those pre-defined variables read-only.
// Currently there is none suitable machinery in the core llvm-mc for this.		// Currently there is none suitable machinery in the core llvm-mc for this.
// MCSymbol::isRedefinable is intended for another purpose, and		// MCSymbol::isRedefinable is intended for another purpose, and
// AsmParser::parseDirectiveSet() cannot be specialized for specific target.		// AsmParser::parseDirectiveSet() cannot be specialized for specific target.
AMDGPU::IsaInfo::IsaVersion ISA =		AMDGPU::IsaInfo::IsaVersion ISA =
AMDGPU::IsaInfo::getIsaVersion(getFeatureBits());		AMDGPU::IsaInfo::getIsaVersion(getFeatureBits());
MCContext &Ctx = getContext();		MCContext &Ctx = getContext();
		if (ISA.Major >= 6 && AMDGPU::IsaInfo::hasCodeObjectV3(&getSTI())) {
		MCSymbol *Sym =
		Ctx.getOrCreateSymbol(Twine(".amdgcn.gfx_generation_number"));
		Sym->setVariableValue(MCConstantExpr::create(ISA.Major, Ctx));
		} else {
MCSymbol *Sym =		MCSymbol *Sym =
Ctx.getOrCreateSymbol(Twine(".option.machine_version_major"));		Ctx.getOrCreateSymbol(Twine(".option.machine_version_major"));
Sym->setVariableValue(MCConstantExpr::create(ISA.Major, Ctx));		Sym->setVariableValue(MCConstantExpr::create(ISA.Major, Ctx));
Sym = Ctx.getOrCreateSymbol(Twine(".option.machine_version_minor"));		Sym = Ctx.getOrCreateSymbol(Twine(".option.machine_version_minor"));
Sym->setVariableValue(MCConstantExpr::create(ISA.Minor, Ctx));		Sym->setVariableValue(MCConstantExpr::create(ISA.Minor, Ctx));
Sym = Ctx.getOrCreateSymbol(Twine(".option.machine_version_stepping"));		Sym = Ctx.getOrCreateSymbol(Twine(".option.machine_version_stepping"));
Sym->setVariableValue(MCConstantExpr::create(ISA.Stepping, Ctx));		Sym->setVariableValue(MCConstantExpr::create(ISA.Stepping, Ctx));
}		}
		if (ISA.Major >= 6 && AMDGPU::IsaInfo::hasCodeObjectV3(&getSTI())) {
		initializeGprCountSymbol(IS_VGPR);
		initializeGprCountSymbol(IS_SGPR);
		} else
KernelScope.initialize(getContext());		KernelScope.initialize(getContext());
}		}
		}

bool hasXNACK() const {		bool hasXNACK() const {
return AMDGPU::hasXNACK(getSTI());		return AMDGPU::hasXNACK(getSTI());
}		}

bool hasMIMG_R128() const {		bool hasMIMG_R128() const {
return AMDGPU::hasMIMG_R128(getSTI());		return AMDGPU::hasMIMG_R128(getSTI());
}		}
▲ Show 20 Lines • Show All 847 Lines • ▼ Show 20 Lines	default:
llvm_unreachable("unexpected register kind");		llvm_unreachable("unexpected register kind");
}		}

if (!subtargetHasRegister(*TRI, Reg))		if (!subtargetHasRegister(*TRI, Reg))
return false;		return false;
return true;		return true;
}		}

		Optional<StringRef>
		AMDGPUAsmParser::getGprCountSymbolName(RegisterKind RegKind) {
		switch (RegKind) {
		case IS_VGPR:
		return StringRef(".amdgcn.next_free_vgpr");
		case IS_SGPR:
		return StringRef(".amdgcn.next_free_sgpr");
		default:
		return None;
		}
		}

		void AMDGPUAsmParser::initializeGprCountSymbol(RegisterKind RegKind) {
		auto SymbolName = getGprCountSymbolName(RegKind);
		assert(SymbolName && "initializing invalid register kind");
		MCSymbol Sym = getContext().getOrCreateSymbol(SymbolName);
		Sym->setVariableValue(MCConstantExpr::create(0, getContext()));
		}

		bool AMDGPUAsmParser::updateGprCountSymbols(RegisterKind RegKind,
		unsigned DwordRegIndex,
		unsigned RegWidth) {
		// Symbols are only defined for GCN targets
		if (AMDGPU::IsaInfo::getIsaVersion(getFeatureBits()).Major < 6)
		return true;

		auto SymbolName = getGprCountSymbolName(RegKind);
		if (!SymbolName)
		return true;
		MCSymbol Sym = getContext().getOrCreateSymbol(SymbolName);

		int64_t NewMax = DwordRegIndex + RegWidth - 1;
		int64_t OldCount;

		if (!Sym->isVariable())
		return !Error(getParser().getTok().getLoc(),
		".amdgcn.next_free_{v,s}gpr symbols must be variable");
		if (!Sym->getVariableValue(false)->evaluateAsAbsolute(OldCount))
		return !Error(
		getParser().getTok().getLoc(),
		".amdgcn.next_free_{v,s}gpr symbols must be absolute expressions");

		if (OldCount <= NewMax)
		Sym->setVariableValue(MCConstantExpr::create(NewMax + 1, getContext()));

		return true;
		}

std::unique_ptr<AMDGPUOperand> AMDGPUAsmParser::parseRegister() {		std::unique_ptr<AMDGPUOperand> AMDGPUAsmParser::parseRegister() {
const auto &Tok = Parser.getTok();		const auto &Tok = Parser.getTok();
SMLoc StartLoc = Tok.getLoc();		SMLoc StartLoc = Tok.getLoc();
SMLoc EndLoc = Tok.getEndLoc();		SMLoc EndLoc = Tok.getEndLoc();
RegisterKind RegKind;		RegisterKind RegKind;
unsigned Reg, RegNum, RegWidth, DwordRegIndex;		unsigned Reg, RegNum, RegWidth, DwordRegIndex;

if (!ParseAMDGPURegister(RegKind, Reg, RegNum, RegWidth, &DwordRegIndex)) {		if (!ParseAMDGPURegister(RegKind, Reg, RegNum, RegWidth, &DwordRegIndex)) {
return nullptr;		return nullptr;
}		}
		if (AMDGPU::IsaInfo::hasCodeObjectV3(&getSTI())) {
		if (!updateGprCountSymbols(RegKind, DwordRegIndex, RegWidth))
		return nullptr;
		} else
KernelScope.usesRegister(RegKind, DwordRegIndex, RegWidth);		KernelScope.usesRegister(RegKind, DwordRegIndex, RegWidth);
return AMDGPUOperand::CreateReg(this, Reg, StartLoc, EndLoc, false);		return AMDGPUOperand::CreateReg(this, Reg, StartLoc, EndLoc, false);
}		}

bool		bool
AMDGPUAsmParser::parseAbsoluteExpr(int64_t &Val, bool AbsMod) {		AMDGPUAsmParser::parseAbsoluteExpr(int64_t &Val, bool AbsMod) {
if (AbsMod && getLexer().peekTok().is(AsmToken::Pipe) &&		if (AbsMod && getLexer().peekTok().is(AsmToken::Pipe) &&
(getLexer().getKind() == AsmToken::Integer \|\|		(getLexer().getKind() == AsmToken::Integer \|\|
getLexer().getKind() == AsmToken::Real)) {		getLexer().getKind() == AsmToken::Real)) {
▲ Show 20 Lines • Show All 742 Lines • ▼ Show 20 Lines	bool AMDGPUAsmParser::ParseDirectiveMajorMinor(uint32_t &Major,
Lex();		Lex();

if (ParseAsAbsoluteExpression(Minor))		if (ParseAsAbsoluteExpression(Minor))
return TokError("invalid minor version");		return TokError("invalid minor version");

return false;		return false;
}		}

		bool AMDGPUAsmParser::ParseDirectiveAMDGCNTarget() {
		if (getSTI().getTargetTriple().getArch() != Triple::amdgcn)
		return TokError("directive only supported for amdgcn architecture");

		std::string Target;

		SMLoc TargetStart = getTok().getLoc();
		if (getParser().parseEscapedString(Target))
		return true;
		SMRange TargetRange = SMRange(TargetStart, getTok().getLoc());

		std::string ExpectedTarget;
		raw_string_ostream ExpectedTargetOS(ExpectedTarget);
		IsaInfo::streamIsaVersion(&getSTI(), ExpectedTargetOS);

		if (Target != ExpectedTargetOS.str())
		return getParser().Error(TargetRange.Start, "target must match options",
		TargetRange);

		getTargetStreamer().EmitDirectiveAMDGCNTarget(Target);
		return false;
		}

		bool AMDGPUAsmParser::OutOfRangeError(SMRange Range) {
		return getParser().Error(Range.Start, "value out of range", Range);
		}

		bool AMDGPUAsmParser::calculateGPRBlocks(
		const FeatureBitset &Features, bool VCCUsed, bool FlatScrUsed,
		bool XNACKUsed, unsigned NextFreeVGPR, SMRange VGPRRange,
		unsigned NextFreeSGPR, SMRange SGPRRange, unsigned &VGPRBlocks,
		unsigned &SGPRBlocks) {
		// TODO(scott.linder): These calculations are duplicated from
		// AMDGPUAsmPrinter::getSIProgramInfo and could be unified.
		IsaInfo::IsaVersion Version = IsaInfo::getIsaVersion(Features);

		unsigned NumVGPRs = NextFreeVGPR;
		unsigned NumSGPRs = NextFreeSGPR;
		unsigned MaxAddressableNumSGPRs = IsaInfo::getAddressableNumSGPRs(Features);

		if (Version.Major >= 8 && !Features.test(FeatureSGPRInitBug) &&
		NumSGPRs > MaxAddressableNumSGPRs)
		return OutOfRangeError(SGPRRange);

		NumSGPRs +=
		IsaInfo::getNumExtraSGPRs(Features, VCCUsed, FlatScrUsed, XNACKUsed);

		if ((Version.Major <= 7 \|\| Features.test(FeatureSGPRInitBug)) &&
		NumSGPRs > MaxAddressableNumSGPRs)
		return OutOfRangeError(SGPRRange);

		if (Features.test(FeatureSGPRInitBug))
		NumSGPRs = IsaInfo::FIXED_NUM_SGPRS_FOR_INIT_BUG;

		VGPRBlocks = IsaInfo::getNumVGPRBlocks(Features, NumVGPRs);
		SGPRBlocks = IsaInfo::getNumSGPRBlocks(Features, NumSGPRs);

		return false;
		}

		bool AMDGPUAsmParser::ParseDirectiveAMDHSAKernel() {
		if (getSTI().getTargetTriple().getArch() != Triple::amdgcn)
		return TokError("directive only supported for amdgcn architecture");

		if (getSTI().getTargetTriple().getOS() != Triple::AMDHSA)
		return TokError("directive only supported for amdhsa OS");

		StringRef KernelName;
		if (getParser().parseIdentifier(KernelName))
		return true;

		kernel_descriptor_t KD = getDefaultAmdhsaKernelDescriptor();

		StringSet<> Seen;

		IsaInfo::IsaVersion IVersion =
		IsaInfo::getIsaVersion(getSTI().getFeatureBits());

		SMRange VGPRRange;
		uint64_t NextFreeVGPR = 0;
		SMRange SGPRRange;
		uint64_t NextFreeSGPR = 0;
		unsigned UserSGPRCount = 0;
		bool ReserveVCC = true;
		bool ReserveFlatScr = true;
		bool ReserveXNACK = hasXNACK();

		while (true) {
		while (getLexer().is(AsmToken::EndOfStatement))
		Lex();

		if (getLexer().isNot(AsmToken::Identifier))
		return TokError("expected .amdhsa_ directive or .end_amdhsa_kernel");

		StringRef ID = getTok().getIdentifier();
		SMRange IDRange = getTok().getLocRange();
		Lex();

		if (ID == ".end_amdhsa_kernel")
		break;

		if (Seen.find(ID) != Seen.end())
		return TokError(".amdhsa_ directives cannot be repeated");
		Seen.insert(ID);

		SMLoc ValStart = getTok().getLoc();
		int64_t IVal;
		if (getParser().parseAbsoluteExpression(IVal))
		return true;
		SMLoc ValEnd = getTok().getLoc();
		SMRange ValRange = SMRange(ValStart, ValEnd);

		if (IVal < 0)
		return OutOfRangeError(ValRange);

		uint64_t Val = IVal;

		#define PARSE_BITS_ENTRY(FIELD, ENTRY, VALUE, RANGE) \
		if (!isUInt<ENTRY##_WIDTH>(VALUE)) \
		return OutOfRangeError(RANGE); \
		AMDHSA_BITS_SET(FIELD, ENTRY, VALUE);

		if (ID == ".amdhsa_group_segment_fixed_size") {
		if (!isUInt<sizeof(KD.group_segment_fixed_size) * CHAR_BIT>(Val))
		return OutOfRangeError(ValRange);
		KD.group_segment_fixed_size = Val;
		} else if (ID == ".amdhsa_private_segment_fixed_size") {
		if (!isUInt<sizeof(KD.private_segment_fixed_size) * CHAR_BIT>(Val))
		return OutOfRangeError(ValRange);
		KD.private_segment_fixed_size = Val;
		} else if (ID == ".amdhsa_user_sgpr_private_segment_buffer") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_PRIVATE_SEGMENT_BUFFER,
		Val, ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_user_sgpr_dispatch_ptr") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_DISPATCH_PTR, Val,
		ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_user_sgpr_queue_ptr") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_QUEUE_PTR, Val,
		ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_user_sgpr_kernarg_segment_ptr") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_KERNARG_SEGMENT_PTR,
		Val, ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_user_sgpr_dispatch_id") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_DISPATCH_ID, Val,
		ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_user_sgpr_flat_scratch_init") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_FLAT_SCRATCH_INIT, Val,
		ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_user_sgpr_private_segment_size") {
		PARSE_BITS_ENTRY(KD.kernel_code_properties,
		KERNEL_CODE_PROPERTY_ENABLE_SGPR_PRIVATE_SEGMENT_SIZE,
		Val, ValRange);
		UserSGPRCount++;
		} else if (ID == ".amdhsa_system_sgpr_private_segment_wavefront_offset") {
		PARSE_BITS_ENTRY(
		KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_SGPR_PRIVATE_SEGMENT_WAVEFRONT_OFFSET, Val,
		ValRange);
		} else if (ID == ".amdhsa_system_sgpr_workgroup_id_x") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_X, Val,
		ValRange);
		} else if (ID == ".amdhsa_system_sgpr_workgroup_id_y") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_Y, Val,
		ValRange);
		} else if (ID == ".amdhsa_system_sgpr_workgroup_id_z") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_Z, Val,
		ValRange);
		} else if (ID == ".amdhsa_system_sgpr_workgroup_info") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_INFO, Val,
		ValRange);
		} else if (ID == ".amdhsa_system_vgpr_workitem_id") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_VGPR_WORKITEM_ID, Val,
		ValRange);
		} else if (ID == ".amdhsa_next_free_vgpr") {
		VGPRRange = ValRange;
		NextFreeVGPR = Val;
		} else if (ID == ".amdhsa_next_free_sgpr") {
		SGPRRange = ValRange;
		NextFreeSGPR = Val;
		} else if (ID == ".amdhsa_reserve_vcc") {
		if (!isUInt<1>(Val))
		return OutOfRangeError(ValRange);
		ReserveVCC = Val;
		} else if (ID == ".amdhsa_reserve_flat_scratch") {
		if (IVersion.Major < 7)
		return getParser().Error(IDRange.Start, "directive requires gfx7+",
		IDRange);
		if (!isUInt<1>(Val))
		return OutOfRangeError(ValRange);
		ReserveFlatScr = Val;
		} else if (ID == ".amdhsa_reserve_xnack_mask") {
		if (IVersion.Major < 8)
		return getParser().Error(IDRange.Start, "directive requires gfx8+",
		IDRange);
		if (!isUInt<1>(Val))
		return OutOfRangeError(ValRange);
		ReserveXNACK = Val;
		} else if (ID == ".amdhsa_float_round_mode_32") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_FLOAT_ROUND_MODE_32, Val, ValRange);
		} else if (ID == ".amdhsa_float_round_mode_16_64") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_FLOAT_ROUND_MODE_16_64, Val, ValRange);
		} else if (ID == ".amdhsa_float_denorm_mode_32") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_FLOAT_DENORM_MODE_32, Val, ValRange);
		} else if (ID == ".amdhsa_float_denorm_mode_16_64") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_FLOAT_DENORM_MODE_16_64, Val,
		ValRange);
		} else if (ID == ".amdhsa_dx10_clamp") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_ENABLE_DX10_CLAMP, Val, ValRange);
		} else if (ID == ".amdhsa_ieee_mode") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1, COMPUTE_PGM_RSRC1_ENABLE_IEEE_MODE,
		Val, ValRange);
		} else if (ID == ".amdhsa_fp16_overflow") {
		if (IVersion.Major < 9)
		return getParser().Error(IDRange.Start, "directive requires gfx9+",
		IDRange);
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc1, COMPUTE_PGM_RSRC1_FP16_OVFL, Val,
		ValRange);
		} else if (ID == ".amdhsa_exception_fp_ieee_invalid_op") {
		PARSE_BITS_ENTRY(
		KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_INVALID_OPERATION, Val,
		ValRange);
		} else if (ID == ".amdhsa_exception_fp_denorm_src") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_FP_DENORMAL_SOURCE,
		Val, ValRange);
		} else if (ID == ".amdhsa_exception_fp_ieee_div_zero") {
		PARSE_BITS_ENTRY(
		KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_DIVISION_BY_ZERO, Val,
		ValRange);
		} else if (ID == ".amdhsa_exception_fp_ieee_overflow") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_OVERFLOW,
		Val, ValRange);
		} else if (ID == ".amdhsa_exception_fp_ieee_underflow") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_UNDERFLOW,
		Val, ValRange);
		} else if (ID == ".amdhsa_exception_fp_ieee_inexact") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_INEXACT,
		Val, ValRange);
		} else if (ID == ".amdhsa_exception_int_div_zero") {
		PARSE_BITS_ENTRY(KD.compute_pgm_rsrc2,
		COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_INT_DIVIDE_BY_ZERO,
		Val, ValRange);
		} else {
		return getParser().Error(IDRange.Start,
		"unknown .amdhsa_kernel directive", IDRange);
		}

		#undef PARSE_BITS_ENTRY
		}

		if (Seen.find(".amdhsa_next_free_vgpr") == Seen.end())
		return TokError(".amdhsa_next_free_vgpr directive is required");

		if (Seen.find(".amdhsa_next_free_sgpr") == Seen.end())
		return TokError(".amdhsa_next_free_sgpr directive is required");

		unsigned VGPRBlocks;
		unsigned SGPRBlocks;
		if (calculateGPRBlocks(getFeatureBits(), ReserveVCC, ReserveFlatScr,
		ReserveXNACK, NextFreeVGPR, VGPRRange, NextFreeSGPR,
		SGPRRange, VGPRBlocks, SGPRBlocks))
		return true;

		if (!isUInt<COMPUTE_PGM_RSRC1_GRANULATED_WORKITEM_VGPR_COUNT_WIDTH>(
		VGPRBlocks))
		return OutOfRangeError(VGPRRange);
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_GRANULATED_WORKITEM_VGPR_COUNT, VGPRBlocks);

		if (!isUInt<COMPUTE_PGM_RSRC1_GRANULATED_WAVEFRONT_SGPR_COUNT_WIDTH>(
		SGPRBlocks))
		return OutOfRangeError(SGPRRange);
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc1,
		COMPUTE_PGM_RSRC1_GRANULATED_WAVEFRONT_SGPR_COUNT,
		SGPRBlocks);

		if (!isUInt<COMPUTE_PGM_RSRC2_USER_SGPR_COUNT_WIDTH>(UserSGPRCount))
		return TokError("too many user SGPRs enabled");
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc2, COMPUTE_PGM_RSRC2_USER_SGPR_COUNT,
		UserSGPRCount);

		getTargetStreamer().EmitAmdhsaKernelDescriptor(
		getSTI(), KernelName, KD, NextFreeVGPR, NextFreeSGPR, ReserveVCC,
		ReserveFlatScr, ReserveXNACK);
		return false;
		}

bool AMDGPUAsmParser::ParseDirectiveHSACodeObjectVersion() {		bool AMDGPUAsmParser::ParseDirectiveHSACodeObjectVersion() {
uint32_t Major;		uint32_t Major;
uint32_t Minor;		uint32_t Minor;

if (ParseDirectiveMajorMinor(Major, Minor))		if (ParseDirectiveMajorMinor(Major, Minor))
return true;		return true;

getTargetStreamer().EmitDirectiveHSACodeObjectVersion(Major, Minor);		getTargetStreamer().EmitDirectiveHSACodeObjectVersion(Major, Minor);
▲ Show 20 Lines • Show All 103 Lines • ▼ Show 20 Lines	bool AMDGPUAsmParser::ParseDirectiveAMDGPUHsaKernel() {
if (getLexer().isNot(AsmToken::Identifier))		if (getLexer().isNot(AsmToken::Identifier))
return TokError("expected symbol name");		return TokError("expected symbol name");

StringRef KernelName = Parser.getTok().getString();		StringRef KernelName = Parser.getTok().getString();

getTargetStreamer().EmitAMDGPUSymbolType(KernelName,		getTargetStreamer().EmitAMDGPUSymbolType(KernelName,
ELF::STT_AMDGPU_HSA_KERNEL);		ELF::STT_AMDGPU_HSA_KERNEL);
Lex();		Lex();
		if (!AMDGPU::IsaInfo::hasCodeObjectV3(&getSTI()))
KernelScope.initialize(getContext());		KernelScope.initialize(getContext());
return false;		return false;
}		}

bool AMDGPUAsmParser::ParseDirectiveISAVersion() {		bool AMDGPUAsmParser::ParseDirectiveISAVersion() {
if (getSTI().getTargetTriple().getArch() != Triple::amdgcn) {		if (getSTI().getTargetTriple().getArch() != Triple::amdgcn) {
return Error(getParser().getTok().getLoc(),		return Error(getParser().getTok().getLoc(),
".amd_amdgpu_isa directive is not available on non-amdgcn "		".amd_amdgpu_isa directive is not available on non-amdgcn "
"architectures");		"architectures");
▲ Show 20 Lines • Show All 87 Lines • ▼ Show 20 Lines	bool AMDGPUAsmParser::ParseDirectivePALMetadata() {
}		}
getTargetStreamer().EmitPALMetadata(PALMetadata);		getTargetStreamer().EmitPALMetadata(PALMetadata);
return false;		return false;
}		}

bool AMDGPUAsmParser::ParseDirective(AsmToken DirectiveID) {		bool AMDGPUAsmParser::ParseDirective(AsmToken DirectiveID) {
StringRef IDVal = DirectiveID.getString();		StringRef IDVal = DirectiveID.getString();

		if (AMDGPU::IsaInfo::hasCodeObjectV3(&getSTI())) {
		if (IDVal == ".amdgcn_target")
		return ParseDirectiveAMDGCNTarget();

		if (IDVal == ".amdhsa_kernel")
		return ParseDirectiveAMDHSAKernel();
		} else {
if (IDVal == ".hsa_code_object_version")		if (IDVal == ".hsa_code_object_version")
return ParseDirectiveHSACodeObjectVersion();		return ParseDirectiveHSACodeObjectVersion();

if (IDVal == ".hsa_code_object_isa")		if (IDVal == ".hsa_code_object_isa")
return ParseDirectiveHSACodeObjectISA();		return ParseDirectiveHSACodeObjectISA();

if (IDVal == ".amd_kernel_code_t")		if (IDVal == ".amd_kernel_code_t")
return ParseDirectiveAMDKernelCodeT();		return ParseDirectiveAMDKernelCodeT();

if (IDVal == ".amdgpu_hsa_kernel")		if (IDVal == ".amdgpu_hsa_kernel")
return ParseDirectiveAMDGPUHsaKernel();		return ParseDirectiveAMDGPUHsaKernel();

if (IDVal == ".amd_amdgpu_isa")		if (IDVal == ".amd_amdgpu_isa")
return ParseDirectiveISAVersion();		return ParseDirectiveISAVersion();
		}

if (IDVal == AMDGPU::HSAMD::AssemblerDirectiveBegin)		if (IDVal == AMDGPU::HSAMD::AssemblerDirectiveBegin)
return ParseDirectiveHSAMetadata();		return ParseDirectiveHSAMetadata();

if (IDVal == PALMD::AssemblerDirective)		if (IDVal == PALMD::AssemblerDirective)
return ParseDirectivePALMetadata();		return ParseDirectivePALMetadata();

return true;		return true;
▲ Show 20 Lines • Show All 2,313 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h

Show All 34 Lines	protected:
unsigned getMACH(StringRef GPU) const;		unsigned getMACH(StringRef GPU) const;

public:		public:
/// \returns Equivalent GPU name for an EF_AMDGPU_MACH_* value.		/// \returns Equivalent GPU name for an EF_AMDGPU_MACH_* value.
static const char *getMachName(unsigned Mach);		static const char *getMachName(unsigned Mach);

AMDGPUTargetStreamer(MCStreamer &S) : MCTargetStreamer(S) {}		AMDGPUTargetStreamer(MCStreamer &S) : MCTargetStreamer(S) {}

		virtual void EmitDirectiveAMDGCNTarget(StringRef Target) = 0;

virtual void EmitDirectiveHSACodeObjectVersion(uint32_t Major,		virtual void EmitDirectiveHSACodeObjectVersion(uint32_t Major,
uint32_t Minor) = 0;		uint32_t Minor) = 0;

virtual void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,		virtual void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,
uint32_t Stepping,		uint32_t Stepping,
StringRef VendorName,		StringRef VendorName,
StringRef ArchName) = 0;		StringRef ArchName) = 0;

Show All 9 Lines	public:

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
virtual bool EmitHSAMetadata(const AMDGPU::HSAMD::Metadata &HSAMetadata) = 0;		virtual bool EmitHSAMetadata(const AMDGPU::HSAMD::Metadata &HSAMetadata) = 0;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
virtual bool EmitPALMetadata(const AMDGPU::PALMD::Metadata &PALMetadata) = 0;		virtual bool EmitPALMetadata(const AMDGPU::PALMD::Metadata &PALMetadata) = 0;

virtual void EmitAmdhsaKernelDescriptor(		virtual void EmitAmdhsaKernelDescriptor(
StringRef KernelName,		const MCSubtargetInfo &STI, StringRef KernelName,
const amdhsa::kernel_descriptor_t &KernelDescriptor) = 0;		const amdhsa::kernel_descriptor_t &KernelDescriptor, uint64_t NextVGPR,
		uint64_t NextSGPR, bool ReserveVCC, bool ReserveFlatScr,
		bool ReserveXNACK) = 0;
};		};

class AMDGPUTargetAsmStreamer final : public AMDGPUTargetStreamer {		class AMDGPUTargetAsmStreamer final : public AMDGPUTargetStreamer {
formatted_raw_ostream &OS;		formatted_raw_ostream &OS;
public:		public:
AMDGPUTargetAsmStreamer(MCStreamer &S, formatted_raw_ostream &OS);		AMDGPUTargetAsmStreamer(MCStreamer &S, formatted_raw_ostream &OS);

		void EmitDirectiveAMDGCNTarget(StringRef Target) override;

void EmitDirectiveHSACodeObjectVersion(uint32_t Major,		void EmitDirectiveHSACodeObjectVersion(uint32_t Major,
uint32_t Minor) override;		uint32_t Minor) override;

void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,		void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,
uint32_t Stepping, StringRef VendorName,		uint32_t Stepping, StringRef VendorName,
StringRef ArchName) override;		StringRef ArchName) override;

void EmitAMDKernelCodeT(const amd_kernel_code_t &Header) override;		void EmitAMDKernelCodeT(const amd_kernel_code_t &Header) override;

void EmitAMDGPUSymbolType(StringRef SymbolName, unsigned Type) override;		void EmitAMDGPUSymbolType(StringRef SymbolName, unsigned Type) override;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
bool EmitISAVersion(StringRef IsaVersionString) override;		bool EmitISAVersion(StringRef IsaVersionString) override;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
bool EmitHSAMetadata(const AMDGPU::HSAMD::Metadata &HSAMetadata) override;		bool EmitHSAMetadata(const AMDGPU::HSAMD::Metadata &HSAMetadata) override;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
bool EmitPALMetadata(const AMDGPU::PALMD::Metadata &PALMetadata) override;		bool EmitPALMetadata(const AMDGPU::PALMD::Metadata &PALMetadata) override;

void EmitAmdhsaKernelDescriptor(		void EmitAmdhsaKernelDescriptor(
StringRef KernelName,		const MCSubtargetInfo &STI, StringRef KernelName,
const amdhsa::kernel_descriptor_t &KernelDescriptor) override;		const amdhsa::kernel_descriptor_t &KernelDescriptor, uint64_t NextVGPR,
		uint64_t NextSGPR, bool ReserveVCC, bool ReserveFlatScr,
		bool ReserveXNACK) override;
};		};

class AMDGPUTargetELFStreamer final : public AMDGPUTargetStreamer {		class AMDGPUTargetELFStreamer final : public AMDGPUTargetStreamer {
MCStreamer &Streamer;		MCStreamer &Streamer;

void EmitAMDGPUNote(const MCExpr *DescSize, unsigned NoteType,		void EmitAMDGPUNote(const MCExpr *DescSize, unsigned NoteType,
function_ref<void(MCELFStreamer &)> EmitDesc);		function_ref<void(MCELFStreamer &)> EmitDesc);

public:		public:
AMDGPUTargetELFStreamer(MCStreamer &S, const MCSubtargetInfo &STI);		AMDGPUTargetELFStreamer(MCStreamer &S, const MCSubtargetInfo &STI);

MCELFStreamer &getStreamer();		MCELFStreamer &getStreamer();

		void EmitDirectiveAMDGCNTarget(StringRef Target) override;

void EmitDirectiveHSACodeObjectVersion(uint32_t Major,		void EmitDirectiveHSACodeObjectVersion(uint32_t Major,
uint32_t Minor) override;		uint32_t Minor) override;

void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,		void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,
uint32_t Stepping, StringRef VendorName,		uint32_t Stepping, StringRef VendorName,
StringRef ArchName) override;		StringRef ArchName) override;

void EmitAMDKernelCodeT(const amd_kernel_code_t &Header) override;		void EmitAMDKernelCodeT(const amd_kernel_code_t &Header) override;

void EmitAMDGPUSymbolType(StringRef SymbolName, unsigned Type) override;		void EmitAMDGPUSymbolType(StringRef SymbolName, unsigned Type) override;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
bool EmitISAVersion(StringRef IsaVersionString) override;		bool EmitISAVersion(StringRef IsaVersionString) override;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
bool EmitHSAMetadata(const AMDGPU::HSAMD::Metadata &HSAMetadata) override;		bool EmitHSAMetadata(const AMDGPU::HSAMD::Metadata &HSAMetadata) override;

/// \returns True on success, false on failure.		/// \returns True on success, false on failure.
bool EmitPALMetadata(const AMDGPU::PALMD::Metadata &PALMetadata) override;		bool EmitPALMetadata(const AMDGPU::PALMD::Metadata &PALMetadata) override;

void EmitAmdhsaKernelDescriptor(		void EmitAmdhsaKernelDescriptor(
StringRef KernelName,		const MCSubtargetInfo &STI, StringRef KernelName,
const amdhsa::kernel_descriptor_t &KernelDescriptor) override;		const amdhsa::kernel_descriptor_t &KernelDescriptor, uint64_t NextVGPR,
		uint64_t NextSGPR, bool ReserveVCC, bool ReserveFlatScr,
		bool ReserveXNACK) override;
};		};

}		}
#endif		#endif

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

Show First 20 Lines • Show All 127 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===//		//===----------------------------------------------------------------------===//
// AMDGPUTargetAsmStreamer		// AMDGPUTargetAsmStreamer
//===----------------------------------------------------------------------===//		//===----------------------------------------------------------------------===//

AMDGPUTargetAsmStreamer::AMDGPUTargetAsmStreamer(MCStreamer &S,		AMDGPUTargetAsmStreamer::AMDGPUTargetAsmStreamer(MCStreamer &S,
formatted_raw_ostream &OS)		formatted_raw_ostream &OS)
: AMDGPUTargetStreamer(S), OS(OS) { }		: AMDGPUTargetStreamer(S), OS(OS) { }

void		void AMDGPUTargetAsmStreamer::EmitDirectiveAMDGCNTarget(StringRef Target) {
AMDGPUTargetAsmStreamer::EmitDirectiveHSACodeObjectVersion(uint32_t Major,		OS << "\t.amdgcn_target \"" << Target << "\"\n";
uint32_t Minor) {		}

		void AMDGPUTargetAsmStreamer::EmitDirectiveHSACodeObjectVersion(
		uint32_t Major, uint32_t Minor) {
OS << "\t.hsa_code_object_version " <<		OS << "\t.hsa_code_object_version " <<
Twine(Major) << "," << Twine(Minor) << '\n';		Twine(Major) << "," << Twine(Minor) << '\n';
}		}

void		void
AMDGPUTargetAsmStreamer::EmitDirectiveHSACodeObjectISA(uint32_t Major,		AMDGPUTargetAsmStreamer::EmitDirectiveHSACodeObjectISA(uint32_t Major,
uint32_t Minor,		uint32_t Minor,
uint32_t Stepping,		uint32_t Stepping,
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines	bool AMDGPUTargetAsmStreamer::EmitPALMetadata(
if (PALMD::toString(PALMetadata, PALMetadataString))		if (PALMD::toString(PALMetadata, PALMetadataString))
return false;		return false;

OS << '\t' << PALMD::AssemblerDirective << PALMetadataString << '\n';		OS << '\t' << PALMD::AssemblerDirective << PALMetadataString << '\n';
return true;		return true;
}		}

void AMDGPUTargetAsmStreamer::EmitAmdhsaKernelDescriptor(		void AMDGPUTargetAsmStreamer::EmitAmdhsaKernelDescriptor(
StringRef KernelName,		const MCSubtargetInfo &STI, StringRef KernelName,
const amdhsa::kernel_descriptor_t &KernelDescriptor) {		const amdhsa::kernel_descriptor_t &KD, uint64_t NextVGPR, uint64_t NextSGPR,
// FIXME: not supported yet.		bool ReserveVCC, bool ReserveFlatScr, bool ReserveXNACK) {
		amdhsa::kernel_descriptor_t DefaultKD = getDefaultAmdhsaKernelDescriptor();

		IsaInfo::IsaVersion IVersion = IsaInfo::getIsaVersion(STI.getFeatureBits());

		OS << "\t.amdhsa_kernel " << KernelName << '\n';

		#define PRINT_IF_NOT_DEFAULT(STREAM, DIRECTIVE, KERNEL_DESC, \
		DEFAULT_KERNEL_DESC, MEMBER_NAME, FIELD_NAME) \
		if (AMDHSA_BITS_GET(KERNEL_DESC.MEMBER_NAME, FIELD_NAME) != \
		AMDHSA_BITS_GET(DEFAULT_KERNEL_DESC.MEMBER_NAME, FIELD_NAME)) \
		STREAM << "\t\t" << DIRECTIVE << " " \
		<< AMDHSA_BITS_GET(KERNEL_DESC.MEMBER_NAME, FIELD_NAME) << '\n';

		if (KD.group_segment_fixed_size != DefaultKD.group_segment_fixed_size)
		OS << "\t\t.amdhsa_group_segment_fixed_size " << KD.group_segment_fixed_size
		<< '\n';
		if (KD.private_segment_fixed_size != DefaultKD.private_segment_fixed_size)
		OS << "\t\t.amdhsa_private_segment_fixed_size "
		<< KD.private_segment_fixed_size << '\n';

		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_user_sgpr_private_segment_buffer", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_PRIVATE_SEGMENT_BUFFER);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_user_sgpr_dispatch_ptr", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_DISPATCH_PTR);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_user_sgpr_queue_ptr", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_QUEUE_PTR);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_user_sgpr_kernarg_segment_ptr", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_KERNARG_SEGMENT_PTR);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_user_sgpr_dispatch_id", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_DISPATCH_ID);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_user_sgpr_flat_scratch_init", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_FLAT_SCRATCH_INIT);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_user_sgpr_private_segment_size", KD, DefaultKD,
		kernel_code_properties,
		amdhsa::KERNEL_CODE_PROPERTY_ENABLE_SGPR_PRIVATE_SEGMENT_SIZE);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_system_sgpr_private_segment_wavefront_offset", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_SGPR_PRIVATE_SEGMENT_WAVEFRONT_OFFSET);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_system_sgpr_workgroup_id_x", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_X);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_system_sgpr_workgroup_id_y", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_Y);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_system_sgpr_workgroup_id_z", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_Z);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_system_sgpr_workgroup_info", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_INFO);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_system_vgpr_workitem_id", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_VGPR_WORKITEM_ID);

		// These directives are required.
		OS << "\t\t.amdhsa_next_free_vgpr " << NextVGPR << '\n';
		OS << "\t\t.amdhsa_next_free_sgpr " << NextSGPR << '\n';

		if (!ReserveVCC)
		OS << "\t\t.amdhsa_reserve_vcc " << ReserveVCC << '\n';
		if (IVersion.Major >= 7 && !ReserveFlatScr)
		OS << "\t\t.amdhsa_reserve_flat_scratch " << ReserveFlatScr << '\n';
		if (IVersion.Major >= 8 && ReserveXNACK != hasXNACK(STI))
		OS << "\t\t.amdhsa_reserve_xnack_mask " << ReserveXNACK << '\n';

		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_float_round_mode_32", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_FLOAT_ROUND_MODE_32);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_float_round_mode_16_64", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_FLOAT_ROUND_MODE_16_64);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_float_denorm_mode_32", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_FLOAT_DENORM_MODE_32);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_float_denorm_mode_16_64", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_FLOAT_DENORM_MODE_16_64);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_dx10_clamp", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_ENABLE_DX10_CLAMP);
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_ieee_mode", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_ENABLE_IEEE_MODE);
		if (IVersion.Major >= 9)
		PRINT_IF_NOT_DEFAULT(OS, ".amdhsa_fp16_overflow", KD, DefaultKD,
		compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_FP16_OVFL);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_fp_ieee_invalid_op", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_INVALID_OPERATION);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_fp_denorm_src", KD, DefaultKD, compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_FP_DENORMAL_SOURCE);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_fp_ieee_div_zero", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_DIVISION_BY_ZERO);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_fp_ieee_overflow", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_OVERFLOW);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_fp_ieee_underflow", KD, DefaultKD,
		compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_UNDERFLOW);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_fp_ieee_inexact", KD, DefaultKD, compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_IEEE_754_FP_INEXACT);
		PRINT_IF_NOT_DEFAULT(
		OS, ".amdhsa_exception_int_div_zero", KD, DefaultKD, compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_EXCEPTION_INT_DIVIDE_BY_ZERO);
		#undef PRINT_IF_NOT_DEFAULT

		OS << "\t.end_amdhsa_kernel\n";
}		}

//===----------------------------------------------------------------------===//		//===----------------------------------------------------------------------===//
// AMDGPUTargetELFStreamer		// AMDGPUTargetELFStreamer
//===----------------------------------------------------------------------===//		//===----------------------------------------------------------------------===//

AMDGPUTargetELFStreamer::AMDGPUTargetELFStreamer(		AMDGPUTargetELFStreamer::AMDGPUTargetELFStreamer(
MCStreamer &S, const MCSubtargetInfo &STI)		MCStreamer &S, const MCSubtargetInfo &STI)
Show All 31 Lines	void AMDGPUTargetELFStreamer::EmitAMDGPUNote(
S.EmitIntValue(NoteType, 4); // type		S.EmitIntValue(NoteType, 4); // type
S.EmitBytes(StringRef(ElfNote::NoteName, NameSZ)); // name		S.EmitBytes(StringRef(ElfNote::NoteName, NameSZ)); // name
S.EmitValueToAlignment(4, 0, 1, 0); // padding 0		S.EmitValueToAlignment(4, 0, 1, 0); // padding 0
EmitDesc(S); // desc		EmitDesc(S); // desc
S.EmitValueToAlignment(4, 0, 1, 0); // padding 0		S.EmitValueToAlignment(4, 0, 1, 0); // padding 0
S.PopSection();		S.PopSection();
}		}

void		void AMDGPUTargetELFStreamer::EmitDirectiveAMDGCNTarget(StringRef Target) {}
AMDGPUTargetELFStreamer::EmitDirectiveHSACodeObjectVersion(uint32_t Major,
uint32_t Minor) {		void AMDGPUTargetELFStreamer::EmitDirectiveHSACodeObjectVersion(
		uint32_t Major, uint32_t Minor) {

EmitAMDGPUNote(		EmitAMDGPUNote(
MCConstantExpr::create(8, getContext()),		MCConstantExpr::create(8, getContext()),
ElfNote::NT_AMDGPU_HSA_CODE_OBJECT_VERSION,		ElfNote::NT_AMDGPU_HSA_CODE_OBJECT_VERSION,
[&](MCELFStreamer &OS){		[&](MCELFStreamer &OS){
OS.EmitIntValue(Major, 4);		OS.EmitIntValue(Major, 4);
OS.EmitIntValue(Minor, 4);		OS.EmitIntValue(Minor, 4);
}		}
▲ Show 20 Lines • Show All 104 Lines • ▼ Show 20 Lines	[&](MCELFStreamer &OS){
for (auto I : PALMetadata)		for (auto I : PALMetadata)
OS.EmitIntValue(I, sizeof(uint32_t));		OS.EmitIntValue(I, sizeof(uint32_t));
}		}
);		);
return true;		return true;
}		}

void AMDGPUTargetELFStreamer::EmitAmdhsaKernelDescriptor(		void AMDGPUTargetELFStreamer::EmitAmdhsaKernelDescriptor(
StringRef KernelName,		const MCSubtargetInfo &STI, StringRef KernelName,
const amdhsa::kernel_descriptor_t &KernelDescriptor) {		const amdhsa::kernel_descriptor_t &KernelDescriptor, uint64_t NextVGPR,
		uint64_t NextSGPR, bool ReserveVCC, bool ReserveFlatScr,
		bool ReserveXNACK) {
auto &Streamer = getStreamer();		auto &Streamer = getStreamer();
auto &Context = Streamer.getContext();		auto &Context = Streamer.getContext();

MCSymbolELF *KernelDescriptorSymbol = cast<MCSymbolELF>(		MCSymbolELF *KernelDescriptorSymbol = cast<MCSymbolELF>(
Context.getOrCreateSymbol(Twine(KernelName) + Twine(".kd")));		Context.getOrCreateSymbol(Twine(KernelName) + Twine(".kd")));
KernelDescriptorSymbol->setBinding(ELF::STB_GLOBAL);		KernelDescriptorSymbol->setBinding(ELF::STB_GLOBAL);
KernelDescriptorSymbol->setType(ELF::STT_OBJECT);		KernelDescriptorSymbol->setType(ELF::STT_OBJECT);
KernelDescriptorSymbol->setSize(		KernelDescriptorSymbol->setSize(
Show All 29 Lines

llvm/trunk/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h

	Show All 10 Lines
	#define LLVM_LIB_TARGET_AMDGPU_UTILS_AMDGPUBASEINFO_H			#define LLVM_LIB_TARGET_AMDGPU_UTILS_AMDGPUBASEINFO_H

	#include "AMDGPU.h"			#include "AMDGPU.h"
	#include "AMDKernelCodeT.h"			#include "AMDKernelCodeT.h"
	#include "SIDefines.h"			#include "SIDefines.h"
	#include "llvm/ADT/StringRef.h"			#include "llvm/ADT/StringRef.h"
	#include "llvm/IR/CallingConv.h"			#include "llvm/IR/CallingConv.h"
	#include "llvm/MC/MCInstrDesc.h"			#include "llvm/MC/MCInstrDesc.h"
				#include "llvm/Support/AMDHSAKernelDescriptor.h"
	#include "llvm/Support/Compiler.h"			#include "llvm/Support/Compiler.h"
	#include "llvm/Support/ErrorHandling.h"			#include "llvm/Support/ErrorHandling.h"
	#include <cstdint>			#include <cstdint>
	#include <string>			#include <string>
	#include <utility>			#include <utility>

	namespace llvm {			namespace llvm {

	class Argument;			class Argument;
	class FeatureBitset;			class FeatureBitset;
	class Function;			class Function;
	class GlobalValue;			class GlobalValue;
	class MachineMemOperand;
	class MCContext;			class MCContext;
	class MCRegisterClass;			class MCRegisterClass;
	class MCRegisterInfo;			class MCRegisterInfo;
	class MCSection;			class MCSection;
	class MCSubtargetInfo;			class MCSubtargetInfo;
				class MachineMemOperand;
	class Triple;			class Triple;

	namespace AMDGPU {			namespace AMDGPU {

	#define GET_MIMGBaseOpcode_DECL			#define GET_MIMGBaseOpcode_DECL
	#define GET_MIMGDim_DECL			#define GET_MIMGDim_DECL
	#define GET_MIMGEncoding_DECL			#define GET_MIMGEncoding_DECL
	#include "AMDGPUGenSearchableTables.inc"			#include "AMDGPUGenSearchableTables.inc"
	▲ Show 20 Lines • Show All 88 Lines • ▼ Show 20 Lines
	/// execution unit requirement for given subtarget \p Features.			/// execution unit requirement for given subtarget \p Features.
	unsigned getMinNumSGPRs(const FeatureBitset &Features, unsigned WavesPerEU);			unsigned getMinNumSGPRs(const FeatureBitset &Features, unsigned WavesPerEU);

	/// \returns Maximum number of SGPRs that meets the given number of waves per			/// \returns Maximum number of SGPRs that meets the given number of waves per
	/// execution unit requirement for given subtarget \p Features.			/// execution unit requirement for given subtarget \p Features.
	unsigned getMaxNumSGPRs(const FeatureBitset &Features, unsigned WavesPerEU,			unsigned getMaxNumSGPRs(const FeatureBitset &Features, unsigned WavesPerEU,
	bool Addressable);			bool Addressable);

				/// \returns Number of extra SGPRs implicitly required by given subtarget \p
				/// Features when the given special registers are used.
				unsigned getNumExtraSGPRs(const FeatureBitset &Features, bool VCCUsed,
				bool FlatScrUsed, bool XNACKUsed);

				/// \returns Number of extra SGPRs implicitly required by given subtarget \p
				/// Features when the given special registers are used. XNACK is inferred from
				/// \p Features.
				unsigned getNumExtraSGPRs(const FeatureBitset &Features, bool VCCUsed,
				bool FlatScrUsed);

				/// \returns Number of SGPR blocks needed for given subtarget \p Features when
				/// \p NumSGPRs are used. \p NumSGPRs should already include any special
				/// register counts.
				unsigned getNumSGPRBlocks(const FeatureBitset &Features, unsigned NumSGPRs);

	/// \returns VGPR allocation granularity for given subtarget \p Features.			/// \returns VGPR allocation granularity for given subtarget \p Features.
	unsigned getVGPRAllocGranule(const FeatureBitset &Features);			unsigned getVGPRAllocGranule(const FeatureBitset &Features);

	/// \returns VGPR encoding granularity for given subtarget \p Features.			/// \returns VGPR encoding granularity for given subtarget \p Features.
	unsigned getVGPREncodingGranule(const FeatureBitset &Features);			unsigned getVGPREncodingGranule(const FeatureBitset &Features);

	/// \returns Total number of VGPRs for given subtarget \p Features.			/// \returns Total number of VGPRs for given subtarget \p Features.
	unsigned getTotalNumVGPRs(const FeatureBitset &Features);			unsigned getTotalNumVGPRs(const FeatureBitset &Features);

	/// \returns Addressable number of VGPRs for given subtarget \p Features.			/// \returns Addressable number of VGPRs for given subtarget \p Features.
	unsigned getAddressableNumVGPRs(const FeatureBitset &Features);			unsigned getAddressableNumVGPRs(const FeatureBitset &Features);

	/// \returns Minimum number of VGPRs that meets given number of waves per			/// \returns Minimum number of VGPRs that meets given number of waves per
	/// execution unit requirement for given subtarget \p Features.			/// execution unit requirement for given subtarget \p Features.
	unsigned getMinNumVGPRs(const FeatureBitset &Features, unsigned WavesPerEU);			unsigned getMinNumVGPRs(const FeatureBitset &Features, unsigned WavesPerEU);

	/// \returns Maximum number of VGPRs that meets given number of waves per			/// \returns Maximum number of VGPRs that meets given number of waves per
	/// execution unit requirement for given subtarget \p Features.			/// execution unit requirement for given subtarget \p Features.
	unsigned getMaxNumVGPRs(const FeatureBitset &Features, unsigned WavesPerEU);			unsigned getMaxNumVGPRs(const FeatureBitset &Features, unsigned WavesPerEU);

				/// \returns Number of VGPR blocks needed for given subtarget \p Features when
				/// \p NumVGPRs are used.
				unsigned getNumVGPRBlocks(const FeatureBitset &Features, unsigned NumSGPRs);

	} // end namespace IsaInfo			} // end namespace IsaInfo

	LLVM_READONLY			LLVM_READONLY
	int16_t getNamedOperandIdx(uint16_t Opcode, uint16_t NamedIdx);			int16_t getNamedOperandIdx(uint16_t Opcode, uint16_t NamedIdx);

	struct MIMGBaseOpcodeInfo {			struct MIMGBaseOpcodeInfo {
	MIMGBaseOpcode BaseOpcode;			MIMGBaseOpcode BaseOpcode;
	bool Store;			bool Store;
	Show All 29 Lines
	int getMaskedMIMGOp(unsigned Opc, unsigned NewChannels);			int getMaskedMIMGOp(unsigned Opc, unsigned NewChannels);

	LLVM_READONLY			LLVM_READONLY
	int getMCOpcode(uint16_t Opcode, unsigned Gen);			int getMCOpcode(uint16_t Opcode, unsigned Gen);

	void initDefaultAMDKernelCodeT(amd_kernel_code_t &Header,			void initDefaultAMDKernelCodeT(amd_kernel_code_t &Header,
	const FeatureBitset &Features);			const FeatureBitset &Features);

				amdhsa::kernel_descriptor_t getDefaultAmdhsaKernelDescriptor();

	bool isGroupSegment(const GlobalValue *GV);			bool isGroupSegment(const GlobalValue *GV);
	bool isGlobalSegment(const GlobalValue *GV);			bool isGlobalSegment(const GlobalValue *GV);
	bool isReadOnlySegment(const GlobalValue *GV);			bool isReadOnlySegment(const GlobalValue *GV);

	/// \returns True if constants should be emitted to .text section for given			/// \returns True if constants should be emitted to .text section for given
	/// target triple \p TT, false otherwise.			/// target triple \p TT, false otherwise.
	bool shouldEmitConstantsToTextSection(const Triple &TT);			bool shouldEmitConstantsToTextSection(const Triple &TT);

	▲ Show 20 Lines • Show All 211 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

Show First 20 Lines • Show All 192 Lines • ▼ Show 20 Lines	void streamIsaVersion(const MCSubtargetInfo *STI, raw_ostream &Stream) {
Stream << TargetTriple.getArchName() << '-'		Stream << TargetTriple.getArchName() << '-'
<< TargetTriple.getVendorName() << '-'		<< TargetTriple.getVendorName() << '-'
<< TargetTriple.getOSName() << '-'		<< TargetTriple.getOSName() << '-'
<< TargetTriple.getEnvironmentName() << '-'		<< TargetTriple.getEnvironmentName() << '-'
<< "gfx"		<< "gfx"
<< ISAVersion.Major		<< ISAVersion.Major
<< ISAVersion.Minor		<< ISAVersion.Minor
<< ISAVersion.Stepping;		<< ISAVersion.Stepping;

		if (hasXNACK(*STI))
		Stream << "+xnack";

Stream.flush();		Stream.flush();
}		}

bool hasCodeObjectV3(const MCSubtargetInfo *STI) {		bool hasCodeObjectV3(const MCSubtargetInfo *STI) {
return STI->getFeatureBits().test(FeatureCodeObjectV3);		return STI->getFeatureBits().test(FeatureCodeObjectV3);
}		}

unsigned getWavefrontSize(const FeatureBitset &Features) {		unsigned getWavefrontSize(const FeatureBitset &Features) {
▲ Show 20 Lines • Show All 120 Lines • ▼ Show 20 Lines	if (Version.Major >= 8 && !Addressable)
AddressableNumSGPRs = 112;		AddressableNumSGPRs = 112;
unsigned MaxNumSGPRs = getTotalNumSGPRs(Features) / WavesPerEU;		unsigned MaxNumSGPRs = getTotalNumSGPRs(Features) / WavesPerEU;
if (Features.test(FeatureTrapHandler))		if (Features.test(FeatureTrapHandler))
MaxNumSGPRs -= std::min(MaxNumSGPRs, (unsigned)TRAP_NUM_SGPRS);		MaxNumSGPRs -= std::min(MaxNumSGPRs, (unsigned)TRAP_NUM_SGPRS);
MaxNumSGPRs = alignDown(MaxNumSGPRs, getSGPRAllocGranule(Features));		MaxNumSGPRs = alignDown(MaxNumSGPRs, getSGPRAllocGranule(Features));
return std::min(MaxNumSGPRs, AddressableNumSGPRs);		return std::min(MaxNumSGPRs, AddressableNumSGPRs);
}		}

		unsigned getNumExtraSGPRs(const FeatureBitset &Features, bool VCCUsed,
		bool FlatScrUsed, bool XNACKUsed) {
		unsigned ExtraSGPRs = 0;
		if (VCCUsed)
		ExtraSGPRs = 2;

		IsaVersion Version = getIsaVersion(Features);
		if (Version.Major < 8) {
		if (FlatScrUsed)
		ExtraSGPRs = 4;
		} else {
		if (XNACKUsed)
		ExtraSGPRs = 4;

		if (FlatScrUsed)
		ExtraSGPRs = 6;
		}

		return ExtraSGPRs;
		}

		unsigned getNumExtraSGPRs(const FeatureBitset &Features, bool VCCUsed,
		bool FlatScrUsed) {
		return getNumExtraSGPRs(Features, VCCUsed, FlatScrUsed,
		Features[AMDGPU::FeatureXNACK]);
		}

		unsigned getNumSGPRBlocks(const FeatureBitset &Features, unsigned NumSGPRs) {
		NumSGPRs = alignTo(std::max(1u, NumSGPRs), getSGPREncodingGranule(Features));
		// SGPRBlocks is actual number of SGPR blocks minus 1.
		return NumSGPRs / getSGPREncodingGranule(Features) - 1;
		}

unsigned getVGPRAllocGranule(const FeatureBitset &Features) {		unsigned getVGPRAllocGranule(const FeatureBitset &Features) {
return 4;		return 4;
}		}

unsigned getVGPREncodingGranule(const FeatureBitset &Features) {		unsigned getVGPREncodingGranule(const FeatureBitset &Features) {
return getVGPRAllocGranule(Features);		return getVGPRAllocGranule(Features);
}		}

Show All 20 Lines	unsigned getMaxNumVGPRs(const FeatureBitset &Features, unsigned WavesPerEU) {
assert(WavesPerEU != 0);		assert(WavesPerEU != 0);

unsigned MaxNumVGPRs = alignDown(getTotalNumVGPRs(Features) / WavesPerEU,		unsigned MaxNumVGPRs = alignDown(getTotalNumVGPRs(Features) / WavesPerEU,
getVGPRAllocGranule(Features));		getVGPRAllocGranule(Features));
unsigned AddressableNumVGPRs = getAddressableNumVGPRs(Features);		unsigned AddressableNumVGPRs = getAddressableNumVGPRs(Features);
return std::min(MaxNumVGPRs, AddressableNumVGPRs);		return std::min(MaxNumVGPRs, AddressableNumVGPRs);
}		}

		unsigned getNumVGPRBlocks(const FeatureBitset &Features, unsigned NumVGPRs) {
		NumVGPRs = alignTo(std::max(1u, NumVGPRs), getVGPREncodingGranule(Features));
		// VGPRBlocks is actual number of VGPR blocks minus 1.
		return NumVGPRs / getVGPREncodingGranule(Features) - 1;
		}

} // end namespace IsaInfo		} // end namespace IsaInfo

void initDefaultAMDKernelCodeT(amd_kernel_code_t &Header,		void initDefaultAMDKernelCodeT(amd_kernel_code_t &Header,
const FeatureBitset &Features) {		const FeatureBitset &Features) {
IsaInfo::IsaVersion ISA = IsaInfo::getIsaVersion(Features);		IsaInfo::IsaVersion ISA = IsaInfo::getIsaVersion(Features);

memset(&Header, 0, sizeof(Header));		memset(&Header, 0, sizeof(Header));

Show All 13 Lines	void initDefaultAMDKernelCodeT(amd_kernel_code_t &Header,

// These alignment values are specified in powers of two, so alignment =		// These alignment values are specified in powers of two, so alignment =
// 2^n. The minimum alignment is 2^4 = 16.		// 2^n. The minimum alignment is 2^4 = 16.
Header.kernarg_segment_alignment = 4;		Header.kernarg_segment_alignment = 4;
Header.group_segment_alignment = 4;		Header.group_segment_alignment = 4;
Header.private_segment_alignment = 4;		Header.private_segment_alignment = 4;
}		}

		amdhsa::kernel_descriptor_t getDefaultAmdhsaKernelDescriptor() {
		amdhsa::kernel_descriptor_t KD;
		memset(&KD, 0, sizeof(KD));
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_FLOAT_DENORM_MODE_16_64,
		amdhsa::FLOAT_DENORM_MODE_FLUSH_NONE);
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_ENABLE_DX10_CLAMP, 1);
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc1,
		amdhsa::COMPUTE_PGM_RSRC1_ENABLE_IEEE_MODE, 1);
		AMDHSA_BITS_SET(KD.compute_pgm_rsrc2,
		amdhsa::COMPUTE_PGM_RSRC2_ENABLE_SGPR_WORKGROUP_ID_X, 1);
		return KD;
		}

bool isGroupSegment(const GlobalValue *GV) {		bool isGroupSegment(const GlobalValue *GV) {
return GV->getType()->getAddressSpace() == AMDGPUAS::LOCAL_ADDRESS;		return GV->getType()->getAddressSpace() == AMDGPUAS::LOCAL_ADDRESS;
}		}

bool isGlobalSegment(const GlobalValue *GV) {		bool isGlobalSegment(const GlobalValue *GV) {
return GV->getType()->getAddressSpace() == AMDGPUAS::GLOBAL_ADDRESS;		return GV->getType()->getAddressSpace() == AMDGPUAS::GLOBAL_ADDRESS;
}		}

▲ Show 20 Lines • Show All 509 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/AMDGPU/code-object-v3.ll

	; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -mattr=+code-object-v3 < %s \| FileCheck --check-prefixes=ALL-ASM,OSABI-AMDHSA-ASM %s			; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -mattr=+code-object-v3 < %s \| FileCheck --check-prefixes=ALL-ASM,OSABI-AMDHSA-ASM %s
	; RUN: llc -filetype=obj -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -mattr=+code-object-v3 < %s \| llvm-readobj -elf-output-style=GNU -notes -relocations -sections -symbols \| FileCheck --check-prefixes=ALL-ELF,OSABI-AMDHSA-ELF %s			; RUN: llc -filetype=obj -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -mattr=+code-object-v3 < %s \| llvm-readobj -elf-output-style=GNU -notes -relocations -sections -symbols \| FileCheck --check-prefixes=ALL-ELF,OSABI-AMDHSA-ELF %s

	; ALL-ASM-LABEL: {{^}}fadd:			; ALL-ASM-LABEL: {{^}}fadd:
	; OSABI-AMDHSA-ASM: s_endpgm			; OSABI-AMDHSA-ASM: s_endpgm
	; OSABI-AMDHSA-ASM: .section .rodata,#alloc			; OSABI-AMDHSA-ASM: .section .rodata,#alloc
	; OSABI-AMDHSA-ASM: .p2align 6			; OSABI-AMDHSA-ASM: .p2align 6
				; OSABI-AMDHSA-ASM: .amdhsa_kernel fadd
				; OSABI-AMDHSA-ASM: .amdhsa_user_sgpr_private_segment_buffer 1
				; OSABI-AMDHSA-ASM: .amdhsa_user_sgpr_kernarg_segment_ptr 1
				; OSABI-AMDHSA-ASM: .amdhsa_next_free_vgpr 3
				; OSABI-AMDHSA-ASM: .amdhsa_next_free_sgpr 6
				; OSABI-AMDHSA-ASM: .amdhsa_reserve_vcc 0
				; OSABI-AMDHSA-ASM: .amdhsa_reserve_flat_scratch 0
				; OSABI-AMDHSA-ASM: .end_amdhsa_kernel
				; OSABI-AMDHSA-ASM: .text

	; ALL-ASM-LABEL: {{^}}fsub:			; ALL-ASM-LABEL: {{^}}fsub:
	; OSABI-AMDHSA-ASM: s_endpgm			; OSABI-AMDHSA-ASM: s_endpgm
	; OSABI-AMDHSA-ASM: .section .rodata,#alloc			; OSABI-AMDHSA-ASM: .section .rodata,#alloc
	; OSABI-AMDHSA-ASM: .p2align 6			; OSABI-AMDHSA-ASM: .p2align 6
				; OSABI-AMDHSA-ASM: .amdhsa_kernel fsub
				; OSABI-AMDHSA-ASM: .amdhsa_user_sgpr_private_segment_buffer 1
				; OSABI-AMDHSA-ASM: .amdhsa_user_sgpr_kernarg_segment_ptr 1
				; OSABI-AMDHSA-ASM: .amdhsa_next_free_vgpr 3
				; OSABI-AMDHSA-ASM: .amdhsa_next_free_sgpr 6
				; OSABI-AMDHSA-ASM: .amdhsa_reserve_vcc 0
				; OSABI-AMDHSA-ASM: .amdhsa_reserve_flat_scratch 0
				; OSABI-AMDHSA-ASM: .end_amdhsa_kernel
				; OSABI-AMDHSA-ASM: .text

	; OSABI-AMDHSA-ASM-NOT: .hsa_code_object_version			; OSABI-AMDHSA-ASM-NOT: .hsa_code_object_version
	; OSABI-AMDHSA-ASM-NOT: .hsa_code_object_isa			; OSABI-AMDHSA-ASM-NOT: .hsa_code_object_isa
	; OSABI-AMDHSA-ASM-NOT: .amd_amdgpu_isa			; OSABI-AMDHSA-ASM-NOT: .amd_amdgpu_isa
	; OSABI-AMDHSA-ASM-NOT: .amd_amdgpu_hsa_metadata			; OSABI-AMDHSA-ASM-NOT: .amd_amdgpu_hsa_metadata
	; OSABI-AMDHSA-ASM-NOT: .amd_amdgpu_pal_metadata			; OSABI-AMDHSA-ASM-NOT: .amd_amdgpu_pal_metadata

	; OSABI-AMDHSA-ELF: Section Headers			; OSABI-AMDHSA-ELF: Section Headers
	Show All 38 Lines

llvm/trunk/test/MC/AMDGPU/hsa-diag-v3.s

				// RUN: not llvm-mc -mattr=+code-object-v3 -triple amdgcn-amd-amdhsa -mcpu=gfx803 -mattr=+xnack -show-encoding %s 2>&1 >/dev/null \| FileCheck %s
				// RUN: not llvm-mc -mattr=+code-object-v3 -triple amdgcn-amd- -mcpu=gfx803 -mattr=+xnack -show-encoding %s 2>&1 >/dev/null \| FileCheck %s --check-prefix=NOT-AMDHSA

				.text

				.amdgcn_target "amdgcn--amdhsa-gfx803+xnack"
				// CHECK: error: target must match options

				.amdhsa_kernel
				// CHECK: error: unknown directive
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				.amdhsa_group_segment_fixed_size -1
				// CHECK: error: value out of range
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				.amdhsa_group_segment_fixed_size 10000000000 + 1
				// CHECK: error: value out of range
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				// NOT-AMDHSA: error: directive only supported for amdhsa OS
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				.amdhsa_group_segment_fixed_size 1
				.amdhsa_group_segment_fixed_size 1
				// CHECK: error: .amdhsa_ directives cannot be repeated
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				// CHECK: error: .amdhsa_next_free_vgpr directive is required
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				.amdhsa_next_free_vgpr 0
				// CHECK: error: .amdhsa_next_free_sgpr directive is required
				.end_amdhsa_kernel

				.amdhsa_kernel foo
				1
				// CHECK: error: expected .amdhsa_ directive or .end_amdhsa_kernel
				.end_amdhsa_kernel

				.set .amdgcn.next_free_vgpr, "foo"
				v_mov_b32_e32 v0, s0
				// CHECK: error: .amdgcn.next_free_{v,s}gpr symbols must be absolute expressions

llvm/trunk/test/MC/AMDGPU/hsa-sgpr-init-bug-v3.s

				// RUN: llvm-mc -mattr=+code-object-v3 -triple amdgcn-amd-amdhsa -mcpu=gfx802 -filetype=obj < %s > %t
				// RUN: llvm-objdump -s -j .rodata %t \| FileCheck --check-prefix=OBJDUMP %s

				// Check that SGPR init bug on gfx803 is corrected by the assembler, setting
				// GRANULATED_WAVEFRONT_SGPR_COUNT to 11.

				// OBJDUMP: Contents of section .rodata
				// OBJDUMP-NEXT: 0000 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0010 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0020 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0030 c002ac00 80000000 00000000 00000000

				.text

				.amdgcn_target "amdgcn-amd-amdhsa--gfx802"

				.p2align 8
				minimal:
				s_endpgm

				.rodata

				.amdhsa_kernel minimal
				.amdhsa_next_free_vgpr 0
				.amdhsa_next_free_sgpr 0
				.end_amdhsa_kernel

llvm/trunk/test/MC/AMDGPU/hsa-v3.s

				// RUN: llvm-mc -mattr=+code-object-v3 -triple amdgcn-amd-amdhsa -mcpu=gfx900 -mattr=+xnack < %s \| FileCheck --check-prefix=ASM %s
				// RUN: llvm-mc -mattr=+code-object-v3 -triple amdgcn-amd-amdhsa -mcpu=gfx900 -mattr=+xnack -filetype=obj < %s > %t
				// RUN: llvm-readobj -elf-output-style=GNU -sections -symbols -relocations %t \| FileCheck --check-prefix=READOBJ %s
				// RUN: llvm-objdump -s -j .rodata %t \| FileCheck --check-prefix=OBJDUMP %s

				// READOBJ: Section Headers
				// READOBJ: .text PROGBITS {{[0-9a-f]+}} {{[0-9a-f]+}} {{[0-9a-f]+}} {{[0-9]+}} AX {{[0-9]+}} {{[0-9]+}} 256
				// READOBJ: .rodata PROGBITS {{[0-9a-f]+}} {{[0-9a-f]+}} 0000c0 {{[0-9]+}} A {{[0-9]+}} {{[0-9]+}} 64

				// READOBJ: Relocation section '.rela.rodata' at offset
				// READOBJ: 0000000000000010 {{[0-9a-f]+}}00000005 R_AMDGPU_REL64 0000000000000000 .text + 10
				// READOBJ: 0000000000000050 {{[0-9a-f]+}}00000005 R_AMDGPU_REL64 0000000000000000 .text + 110
				// READOBJ: 0000000000000090 {{[0-9a-f]+}}00000005 R_AMDGPU_REL64 0000000000000000 .text + 210

				// READOBJ: Symbol table '.symtab' contains {{[0-9]+}} entries:
				// READOBJ: {{[0-9]+}}: 0000000000000100 0 FUNC LOCAL DEFAULT 2 complete
				// READOBJ: {{[0-9]+}}: 0000000000000000 0 FUNC LOCAL DEFAULT 2 minimal
				// READOBJ: {{[0-9]+}}: 0000000000000200 0 FUNC LOCAL DEFAULT 2 special_sgpr
				// READOBJ: {{[0-9]+}}: 0000000000000040 64 OBJECT GLOBAL DEFAULT 3 complete.kd
				// READOBJ: {{[0-9]+}}: 0000000000000000 64 OBJECT GLOBAL DEFAULT 3 minimal.kd
				// READOBJ: {{[0-9]+}}: 0000000000000080 64 OBJECT GLOBAL DEFAULT 3 special_sgpr.kd

				// OBJDUMP: Contents of section .rodata
				// Note, relocation for KERNEL_CODE_ENTRY_BYTE_OFFSET is not resolved here.
				// minimal
				// OBJDUMP-NEXT: 0000 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0010 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0020 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0030 0000ac00 80000000 00000000 00000000
				// complete
				// OBJDUMP-NEXT: 0040 01000000 01000000 00000000 00000000
				// OBJDUMP-NEXT: 0050 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0060 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0070 c2500104 0f0f007f 7f000000 00000000
				// special_sgpr
				// OBJDUMP-NEXT: 0080 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 0090 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 00a0 00000000 00000000 00000000 00000000
				// OBJDUMP-NEXT: 00b0 00010000 80000000 00000000 00000000

				.text
				// ASM: .text

				.amdgcn_target "amdgcn-amd-amdhsa--gfx900+xnack"
				// ASM: .amdgcn_target "amdgcn-amd-amdhsa--gfx900+xnack"

				.p2align 8
				.type minimal,@function
				minimal:
				s_endpgm

				.p2align 8
				.type complete,@function
				complete:
				s_endpgm

				.p2align 8
				.type special_sgpr,@function
				special_sgpr:
				s_endpgm

				.rodata
				// ASM: .rodata

				// Test that only specifying required directives is allowed, and that defaulted
				// values are omitted.
				.p2align 6
				.amdhsa_kernel minimal
				.amdhsa_next_free_vgpr 0
				.amdhsa_next_free_sgpr 0
				.end_amdhsa_kernel

				// ASM: .amdhsa_kernel minimal
				// ASM-NEXT: .amdhsa_next_free_vgpr 0
				// ASM-NEXT: .amdhsa_next_free_sgpr 0
				// ASM-NEXT: .end_amdhsa_kernel

				// Test that we can specify all available directives with non-default values.
				.p2align 6
				.amdhsa_kernel complete
				.amdhsa_group_segment_fixed_size 1
				.amdhsa_private_segment_fixed_size 1
				.amdhsa_user_sgpr_private_segment_buffer 1
				.amdhsa_user_sgpr_dispatch_ptr 1
				.amdhsa_user_sgpr_queue_ptr 1
				.amdhsa_user_sgpr_kernarg_segment_ptr 1
				.amdhsa_user_sgpr_dispatch_id 1
				.amdhsa_user_sgpr_flat_scratch_init 1
				.amdhsa_user_sgpr_private_segment_size 1
				.amdhsa_system_sgpr_private_segment_wavefront_offset 1
				.amdhsa_system_sgpr_workgroup_id_x 0
				.amdhsa_system_sgpr_workgroup_id_y 1
				.amdhsa_system_sgpr_workgroup_id_z 1
				.amdhsa_system_sgpr_workgroup_info 1
				.amdhsa_system_vgpr_workitem_id 1
				.amdhsa_next_free_vgpr 9
				.amdhsa_next_free_sgpr 27
				.amdhsa_reserve_vcc 0
				.amdhsa_reserve_flat_scratch 0
				.amdhsa_reserve_xnack_mask 0
				.amdhsa_float_round_mode_32 1
				.amdhsa_float_round_mode_16_64 1
				.amdhsa_float_denorm_mode_32 1
				.amdhsa_float_denorm_mode_16_64 0
				.amdhsa_dx10_clamp 0
				.amdhsa_ieee_mode 0
				.amdhsa_fp16_overflow 1
				.amdhsa_exception_fp_ieee_invalid_op 1
				.amdhsa_exception_fp_denorm_src 1
				.amdhsa_exception_fp_ieee_div_zero 1
				.amdhsa_exception_fp_ieee_overflow 1
				.amdhsa_exception_fp_ieee_underflow 1
				.amdhsa_exception_fp_ieee_inexact 1
				.amdhsa_exception_int_div_zero 1
				.end_amdhsa_kernel

				// ASM: .amdhsa_kernel complete
				// ASM-NEXT: .amdhsa_group_segment_fixed_size 1
				// ASM-NEXT: .amdhsa_private_segment_fixed_size 1
				// ASM-NEXT: .amdhsa_user_sgpr_private_segment_buffer 1
				// ASM-NEXT: .amdhsa_user_sgpr_dispatch_ptr 1
				// ASM-NEXT: .amdhsa_user_sgpr_queue_ptr 1
				// ASM-NEXT: .amdhsa_user_sgpr_kernarg_segment_ptr 1
				// ASM-NEXT: .amdhsa_user_sgpr_dispatch_id 1
				// ASM-NEXT: .amdhsa_user_sgpr_flat_scratch_init 1
				// ASM-NEXT: .amdhsa_user_sgpr_private_segment_size 1
				// ASM-NEXT: .amdhsa_system_sgpr_private_segment_wavefront_offset 1
				// ASM-NEXT: .amdhsa_system_sgpr_workgroup_id_x 0
				// ASM-NEXT: .amdhsa_system_sgpr_workgroup_id_y 1
				// ASM-NEXT: .amdhsa_system_sgpr_workgroup_id_z 1
				// ASM-NEXT: .amdhsa_system_sgpr_workgroup_info 1
				// ASM-NEXT: .amdhsa_system_vgpr_workitem_id 1
				// ASM-NEXT: .amdhsa_next_free_vgpr 9
				// ASM-NEXT: .amdhsa_next_free_sgpr 27
				// ASM-NEXT: .amdhsa_reserve_vcc 0
				// ASM-NEXT: .amdhsa_reserve_flat_scratch 0
				// ASM-NEXT: .amdhsa_reserve_xnack_mask 0
				// ASM-NEXT: .amdhsa_float_round_mode_32 1
				// ASM-NEXT: .amdhsa_float_round_mode_16_64 1
				// ASM-NEXT: .amdhsa_float_denorm_mode_32 1
				// ASM-NEXT: .amdhsa_float_denorm_mode_16_64 0
				// ASM-NEXT: .amdhsa_dx10_clamp 0
				// ASM-NEXT: .amdhsa_ieee_mode 0
				// ASM-NEXT: .amdhsa_fp16_overflow 1
				// ASM-NEXT: .amdhsa_exception_fp_ieee_invalid_op 1
				// ASM-NEXT: .amdhsa_exception_fp_denorm_src 1
				// ASM-NEXT: .amdhsa_exception_fp_ieee_div_zero 1
				// ASM-NEXT: .amdhsa_exception_fp_ieee_overflow 1
				// ASM-NEXT: .amdhsa_exception_fp_ieee_underflow 1
				// ASM-NEXT: .amdhsa_exception_fp_ieee_inexact 1
				// ASM-NEXT: .amdhsa_exception_int_div_zero 1
				// ASM-NEXT: .end_amdhsa_kernel

				// Test that we are including special SGPR usage in the granulated count.
				.p2align 6
				.amdhsa_kernel special_sgpr
				// Same next_free_sgpr as "complete", but...
				.amdhsa_next_free_sgpr 27
				// ...on GFX9 this should require an additional 6 SGPRs, pushing us from
				// 3 granules to 4
				.amdhsa_reserve_flat_scratch 1

				.amdhsa_reserve_vcc 0
				.amdhsa_reserve_xnack_mask 0

				.amdhsa_float_denorm_mode_16_64 0
				.amdhsa_dx10_clamp 0
				.amdhsa_ieee_mode 0
				.amdhsa_next_free_vgpr 0
				.end_amdhsa_kernel

				// ASM: .amdhsa_kernel special_sgpr
				// ASM-NEXT: .amdhsa_next_free_vgpr 0
				// ASM-NEXT: .amdhsa_next_free_sgpr 27
				// ASM-NEXT: .amdhsa_reserve_vcc 0
				// ASM-NEXT: .amdhsa_reserve_xnack_mask 0
				// ASM-NEXT: .amdhsa_float_denorm_mode_16_64 0
				// ASM-NEXT: .amdhsa_dx10_clamp 0
				// ASM-NEXT: .amdhsa_ieee_mode 0
				// ASM-NEXT: .end_amdhsa_kernel

				.section .foo

				.byte .amdgcn.gfx_generation_number
				// ASM: .byte 9

				.byte .amdgcn.next_free_vgpr
				// ASM: .byte 0
				.byte .amdgcn.next_free_sgpr
				// ASM: .byte 0

				v_mov_b32_e32 v7, s10

				.byte .amdgcn.next_free_vgpr
				// ASM: .byte 8
				.byte .amdgcn.next_free_sgpr
				// ASM: .byte 11

				.set .amdgcn.next_free_vgpr, 0
				.set .amdgcn.next_free_sgpr, 0

				.byte .amdgcn.next_free_vgpr
				// ASM: .byte 0
				.byte .amdgcn.next_free_sgpr
				// ASM: .byte 0

				v_mov_b32_e32 v16, s3

				.byte .amdgcn.next_free_vgpr
				// ASM: .byte 17
				.byte .amdgcn.next_free_sgpr
				// ASM: .byte 4

This is an archive of the discontinued LLVM Phabricator instance.

AMDHSA Code Object v3 assembler syntax update
ClosedPublic

Details

Diff Detail

Event Timeline

Revision Contents

Diff 152360

llvm/trunk/docs/AMDGPUUsage.rst

llvm/trunk/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

llvm/trunk/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

llvm/trunk/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h

llvm/trunk/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

llvm/trunk/test/CodeGen/AMDGPU/code-object-v3.ll

llvm/trunk/test/MC/AMDGPU/hsa-diag-v3.s

llvm/trunk/test/MC/AMDGPU/hsa-sgpr-init-bug-v3.s

llvm/trunk/test/MC/AMDGPU/hsa-v3.s

This is an archive of the discontinued LLVM Phabricator instance.

AMDHSA Code Object v3 assembler syntax updateClosedPublic

Details

Diff Detail

Event Timeline

Revision Contents

Diff 152360

llvm/trunk/docs/AMDGPUUsage.rst

llvm/trunk/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

llvm/trunk/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

llvm/trunk/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h

llvm/trunk/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

llvm/trunk/test/CodeGen/AMDGPU/code-object-v3.ll

llvm/trunk/test/MC/AMDGPU/hsa-diag-v3.s

llvm/trunk/test/MC/AMDGPU/hsa-sgpr-init-bug-v3.s

llvm/trunk/test/MC/AMDGPU/hsa-v3.s

AMDHSA Code Object v3 assembler syntax update
ClosedPublic