Update AMDGOUUsage.rst descriptions

[t-tye]

Summary

• Improve description of XNACK ELF flag.
• Rename all uses of wave to wavefront to be consistent.

[t-tye]

Also rename all uses of wave to wavefront to be consistent.

[kzhuravl]

LGTM.

llvm/trunk/docs/AMDGPUUsage.rst

``EF_AMDGPU_MACH``                0x000000ff AMDGPU processor selection
                                                   mask for
                                                   ``EF_AMDGPU_MACH_xxx`` values
                                                   defined in
                                                   :ref:`amdgpu-ef-amdgpu-mach-table`.
      ``EF_AMDGPU_XNACK``               0x00000100 Indicates if the ``xnack``
                                                   target feature is
                                                   enabled for all code
                                                   contained in the code object.
+                                                  If the processor
+                                                  does not support the
+                                                  ``xnack`` target
+                                                  feature then must
+                                                  be 0.
                                                   See
                                                   :ref:`amdgpu-target-features`.
      ================================= ========== =============================
 
   .. table:: AMDGPU ``EF_AMDGPU_MACH`` Values
      :name: amdgpu-ef-amdgpu-mach-table
 
      ================================= ========== =============================
address to physical address is:
 
   ``wavefront-scratch-base +
   (private-address * wavefront-size * 4) +
   (wavefront-lane-id * 4)``
 
 There are different ways that the wavefront scratch base address is determined
 by a wavefront (see :ref:`amdgpu-amdhsa-initial-kernel-execution-state`). This
 memory can be accessed in an interleaved manner using buffer instruction with
-the scratch buffer descriptor and per wave scratch offset, by the scratch
+the scratch buffer descriptor and per wavefront scratch offset, by the scratch
 instructions, or by flat instructions. If each lane of a wavefront accesses the
 same private address, the interleaving results in adjacent dwords being accessed
 and hence requires fewer cache lines to be fetched. Multi-dword access is not
 supported except by flat and scratch instructions in GFX9.
 
 The generic address space uses the hardware flat address support available in
 GFX7-GFX9. This uses two fixed ranges of virtual addresses (the private and
 local appertures), that are outside the range of addressible global memory, to
..
 
   .. table:: compute_pgm_rsrc2 for GFX6-GFX9
      :name: amdgpu-amdhsa-compute_pgm_rsrc2-gfx6-gfx9-table
 
      ======= ======= =============================== ===========================================================================
      Bits    Size    Field Name                      Description
      ======= ======= =============================== ===========================================================================
      0       1 bit   ENABLE_SGPR_PRIVATE_SEGMENT     Enable the setup of the
-                     _WAVE_OFFSET                    SGPR wave scratch offset
+                     _WAVEFRONT_OFFSET               SGPR wavefront scratch offset
                                                      system register (see
                                                      :ref:`amdgpu-amdhsa-initial-kernel-execution-state`).
 
                                                      Used by CP to set up
                                                      ``COMPUTE_PGM_RSRC2.SCRATCH_EN``.
      5:1     5 bits  USER_SGPR_COUNT                 The total number of SGPR
                                                      user data registers
                                                      requested. This number must
13      1 bit   ENABLE_EXCEPTION_ADDRESS_WATCH  Must be 0.
                                                      runtime requests.
      14      1 bit   ENABLE_EXCEPTION_MEMORY         Must be 0.
 
                                                      Wavefront starts execution
                                                      with memory violation
                                                      exceptions exceptions
                                                      enabled which are generated
                                                      when a memory violation has
-                                                     occurred for this wave from
+                                                     occurred for this wavefront from
                                                      L1 or LDS
                                                      (write-to-read-only-memory,
                                                      mis-aligned atomic, LDS
                                                      address out of range,
                                                      illegal address, etc.).
 
                                                      CP sets the memory
                                                      violation bit in

 The order of the SGPR registers is defined, but the compiler can specify which
 ones are actually setup in the kernel descriptor using the ``enable_sgpr_*`` bit
 fields (see :ref:`amdgpu-amdhsa-kernel-descriptor`). The register numbers used
 for enabled registers are dense starting at SGPR0: the first enabled register is
 SGPR0, the next enabled register is SGPR1 etc.; disabled registers do not have
 an SGPR number.
 
 The initial SGPRs comprise up to 16 User SRGPs that are set by CP and apply to
-all waves of the grid. It is possible to specify more than 16 User SGPRs using
+all wavefronts of the grid. It is possible to specify more than 16 User SGPRs using
 the ``enable_sgpr_*`` bit fields, in which case only the first 16 are actually
 initialized. These are then immediately followed by the System SGPRs that are
-set up by ADC/SPI and can have different values for each wave of the grid
+set up by ADC/SPI and can have different values for each wavefront of the grid
 dispatch.
 
 SGPR register initial state is defined in
 :ref:`amdgpu-amdhsa-sgpr-register-set-up-order-table`.
 
   .. table:: SGPR Register Set Up Order
      :name: amdgpu-amdhsa-sgpr-register-set-up-order-table
 
      ========== ========================== ====== ==============================
      SGPR Order Name                       Number Description
                 (kernel descriptor enable  of
                 field)                     SGPRs
      ========== ========================== ====== ==============================
      First      Private Segment Buffer     4      V# that can be used, together
-                (enable_sgpr_private              with Scratch Wave Offset as an
-                _segment_buffer)                  offset, to access the private
-                                                  memory space using a segment
-                                                  address.
+                (enable_sgpr_private              with Scratch Wavefront Offset
+                _segment_buffer)                  as an offset, to access the
+                                                  private memory space using a
+                                                  segment address.
 
                                                   CP uses the value provided by
                                                   the runtime.
      then       Dispatch Ptr               2      64 bit address of AQL dispatch
                 (enable_sgpr_dispatch_ptr)        packet for kernel dispatch
                                                   actually executing.
      then       Queue Ptr                  2      64 bit address of amd_queue_t
                 (enable_sgpr_queue_ptr)           object for AQL queue on which
then       Flat Scratch Init          2      This is 2 SGPRs:
                                                     for scratch for the queue
                                                     executing the kernel
                                                     dispatch. CP obtains this
                                                     from the runtime. (The
                                                     Scratch Segment Buffer base
                                                     address is
                                                     ``SH_HIDDEN_PRIVATE_BASE_VIMID``
                                                     plus this offset.) The value
-                                                    of Scratch Wave Offset must
+                                                    of Scratch Wavefront Offset must
                                                     be added to this offset by
                                                     the kernel machine code,
                                                     right shifted by 8, and
                                                     moved to the FLAT_SCRATCH_HI
                                                     SGPR register.
                                                     FLAT_SCRATCH_HI corresponds
                                                     to SGPRn-4 on GFX7, and
                                                     SGPRn-6 on GFX8 (where SGPRn
                                                     is the highest numbered SGPR
-                                                    allocated to the wave).
+                                                    allocated to the wavefront).
                                                     FLAT_SCRATCH_HI is
                                                     multiplied by 256 (as it is
                                                     in units of 256 bytes) and
                                                     added to
                                                     ``SH_HIDDEN_PRIVATE_BASE_VIMID``
-                                                    to calculate the per wave
+                                                    to calculate the per wavefront
                                                     FLAT SCRATCH BASE in flat
                                                     memory instructions that
                                                     access the scratch
                                                     apperture.
 
                                                     The second SGPR is 32 bit
                                                     byte size of a single
                                                     work-item's scratch memory
then       Flat Scratch Init          2      This is 2 SGPRs:
                                                     per SPI scratch backing
                                                     memory managed by SPI for
                                                     the queue executing the
                                                     kernel dispatch. CP obtains
                                                     this from the runtime (and
                                                     divides it if there are
                                                     multiple Shader Arrays each
                                                     with its own SPI). The value
-                                                    of Scratch Wave Offset must
+                                                    of Scratch Wavefront Offset must
                                                     be added by the kernel
                                                     machine code and the result
                                                     moved to the FLAT_SCRATCH
                                                     SGPR which is SGPRn-6 and
                                                     SGPRn-5. It is used as the
                                                     FLAT SCRATCH BASE in flat
                                                     memory instructions.
      then       Private Segment Size       1      The 32 bit byte size of a
then       Work-Group Id X            1      32 bit work-group id in X
                 (enable_sgpr_workgroup_id         dimension of grid for
                 _X)                               wavefront.
      then       Work-Group Id Y            1      32 bit work-group id in Y
                 (enable_sgpr_workgroup_id         dimension of grid for
                 _Y)                               wavefront.
      then       Work-Group Id Z            1      32 bit work-group id in Z
                 (enable_sgpr_workgroup_id         dimension of grid for
                 _Z)                               wavefront.
-     then       Work-Group Info            1      {first_wave, 14'b0000,
+     then       Work-Group Info            1      {first_wavefront, 14'b0000,
                 (enable_sgpr_workgroup            ordered_append_term[10:0],
-                _info)                            threadgroup_size_in_waves[5:0]}
-     then       Scratch Wave Offset        1      32 bit byte offset from base
+                _info)                            threadgroup_size_in_wavefronts[5:0]}
+     then       Scratch Wavefront Offset   1      32 bit byte offset from base
                 (enable_sgpr_private              of scratch base of queue
-                _segment_wave_offset)             executing the kernel
+                _segment_wavefront_offset)        executing the kernel
                                                   dispatch. Must be used as an
                                                   offset with Private
                                                   segment address when using
                                                   Scratch Segment Buffer. It
                                                   must be used to set up FLAT
                                                   SCRATCH for flat addressing
                                                   (see
                                                   :ref:`amdgpu-amdhsa-flat-scratch`).
.. table:: VGPR Register Set Up Order
      ========== ========================== ====== ==============================
 
 The setting of registers is is done by GPU CP/ADC/SPI hardware as follows:
 
 1. SGPRs before the Work-Group Ids are set by CP using the 16 User Data
    registers.
 2. Work-group Id registers X, Y, Z are set by ADC which supports any
    combination including none.
-3. Scratch Wave Offset is set by SPI in a per wave basis which is why its value
-   cannot included with the flat scratch init value which is per queue.
+3. Scratch Wavefront Offset is set by SPI in a per wavefront basis which is why
+   its value cannot included with the flat scratch init value which is per queue.
 4. The VGPRs are set by SPI which only supports specifying either (X), (X, Y)
    or (X, Y, Z).
 
 Flat Scratch register pair are adjacent SGRRs so they can be moved as a 64 bit
 value to the hardware required SGPRn-3 and SGPRn-4 respectively.
 
 The global segment can be accessed either using buffer instructions (GFX6 which
 has V# 64 bit address support), flat instructions (GFX7-GFX9), or global

 
 .. _amdgpu-amdhsa-flat-scratch:
 
 Flat Scratch
 ++++++++++++
 
 If the kernel may use flat operations to access scratch memory, the prolog code
 must set up FLAT_SCRATCH register pair (FLAT_SCRATCH_LO/FLAT_SCRATCH_HI which
-are in SGPRn-4/SGPRn-3). Initialization uses Flat Scratch Init and Scratch Wave
+are in SGPRn-4/SGPRn-3). Initialization uses Flat Scratch Init and Scratch Wavefront
 Offset SGPR registers (see :ref:`amdgpu-amdhsa-initial-kernel-execution-state`):
 
 GFX6
   Flat scratch is not supported.
 
 GFX7-GFX8
   1. The low word of Flat Scratch Init is 32 bit byte offset from
      ``SH_HIDDEN_PRIVATE_BASE_VIMID`` to the base of scratch backing memory
      being managed by SPI for the queue executing the kernel dispatch. This is
      the same value used in the Scratch Segment Buffer V# base address. The
-     prolog must add the value of Scratch Wave Offset to get the wave's byte
+     prolog must add the value of Scratch Wavefront Offset to get the wavefront's byte
      scratch backing memory offset from ``SH_HIDDEN_PRIVATE_BASE_VIMID``. Since
      FLAT_SCRATCH_LO is in units of 256 bytes, the offset must be right shifted
      by 8 before moving into FLAT_SCRATCH_LO.
   2. The second word of Flat Scratch Init is 32 bit byte size of a single
      work-items scratch memory usage. This is directly loaded from the kernel
      dispatch packet Private Segment Byte Size and rounded up to a multiple of
      DWORD. Having CP load it once avoids loading it at the beginning of every
      wavefront. The prolog must move it to FLAT_SCRATCH_LO for use as FLAT SCRATCH
      SIZE.
 
 GFX9
   The Flat Scratch Init is the 64 bit address of the base of scratch backing
   memory being managed by SPI for the queue executing the kernel dispatch. The
-  prolog must add the value of Scratch Wave Offset and moved to the FLAT_SCRATCH
+  prolog must add the value of Scratch Wavefront Offset and moved to the FLAT_SCRATCH
   pair for use as the flat scratch base in flat memory instructions.
 
 .. _amdgpu-amdhsa-memory-model:
 
 Memory Model
 ~~~~~~~~~~~~
 
 This section describes the mapping of LLVM memory model onto AMDGPU machine code

 * The wavefronts for a single work-group are executed in the same CU but may be
   executed by different SIMDs.
 * Each CU has a single LDS memory shared by the wavefronts of the work-groups
   executing on it.
 * All LDS operations of a CU are performed as wavefront wide operations in a
   global order and involve no caching. Completion is reported to a wavefront in
   execution order.
 * The LDS memory has multiple request queues shared by the SIMDs of a
-  CU. Therefore, the LDS operations performed by different waves of a work-group
+  CU. Therefore, the LDS operations performed by different wavefronts of a work-group
   can be reordered relative to each other, which can result in reordering the
   visibility of vector memory operations with respect to LDS operations of other
   wavefronts in the same work-group. A ``s_waitcnt lgkmcnt(0)`` is required to
   ensure synchronization between LDS operations and vector memory operations
-  between waves of a work-group, but not between operations performed by the
+  between wavefronts of a work-group, but not between operations performed by the
   same wavefront.
 * The vector memory operations are performed as wavefront wide operations and
   completion is reported to a wavefront in execution order. The exception is
   that for GFX7-GFX9 ``flat_load/store/atomic`` instructions can report out of
   vector memory order if they access LDS memory, and out of LDS operation order
   if they access global memory.
 * The vector memory operations access a single vector L1 cache shared by all
   SIMDs a CU. Therefore, no special action is required for coherence between the
   lanes of a single wavefront, or for coherence between wavefronts in the same
-  work-group. A ``buffer_wbinvl1_vol`` is required for coherence between waves
+  work-group. A ``buffer_wbinvl1_vol`` is required for coherence between wavefronts
   executing in different work-groups as they may be executing on different CUs.
 * The scalar memory operations access a scalar L1 cache shared by all wavefronts
   on a group of CUs. The scalar and vector L1 caches are not coherent. However,
   scalar operations are used in a restricted way so do not impact the memory
   model. See :ref:`amdgpu-amdhsa-memory-spaces`.
 * The vector and scalar memory operations use an L2 cache shared by all CUs on
   the same agent.
 * The L2 cache has independent channels to service disjoint ranges of virtual
   addresses.
 * Each CU has a separate request queue per channel. Therefore, the vector and
-  scalar memory operations performed by waves executing in different work-groups
+  scalar memory operations performed by wavefronts executing in different work-groups
   (which may be executing on different CUs) of an agent can be reordered
   relative to each other. A ``s_waitcnt vmcnt(0)`` is required to ensure
   synchronization between vector memory operations of different CUs. It ensures a
   previous vector memory operation has completed before executing a subsequent
   vector memory or LDS operation and so can be used to meet the requirements of
   acquire and release.
 * The L2 cache can be kept coherent with other agents on some targets, or ranges
   of virtual addresses can be set up to bypass it to ensure system coherence.

 constant address space data may change between kernel dispatch executions. See
 :ref:`amdgpu-amdhsa-memory-spaces`.
 
 The one execption is if scalar writes are used to spill SGPR registers. In this
 case the AMDGPU backend ensures the memory location used to spill is never
 accessed by vector memory operations at the same time. If scalar writes are used
 then a ``s_dcache_wb`` is inserted before the ``s_endpgm`` and before a function
 return since the locations may be used for vector memory instructions by a
-future wave that uses the same scratch area, or a function call that creates a
+future wavefront that uses the same scratch area, or a function call that creates a
 frame at the same address, respectively. There is no need for a ``s_dcache_inv``
 as all scalar writes are write-before-read in the same thread.
 
 Scratch backing memory (which is used for the private address space)
 is accessed with MTYPE NC_NV (non-coherenent non-volatile). Since the private
 address space is only accessed by a single thread, and is always
 write-before-read, there is never a need to invalidate these entries from the L1
 cache. Hence all cache invalidates are done as ``*_vol`` to only invalidate the