[HIP] Start document HIP support by clang

[yaxunl]

Summary

start with example usage, predefined macros, and path setting.

[b-sumner]

clang/docs/HIPSupport.rst

105

Should we include the gfxNNN or GFXN macros here? What about wave size, and CU mode? And what about unsafe FP atomics macro?

[yaxunl]

clang/docs/HIPSupport.rst

105

Those are not only for HIP but for all languages targeting amdgpu. I am thinking we probably need a separate page for amdgpu support by clang.

[scchan]

clang/docs/HIPSupport.rst

34

can we add -xhip to enable HIP for files without the .hip file ext?

57

I'd suggest adding a section to cover compiler options to pass various hip/rocm related paths and that using the compiler options would be a much more preferable alternative.

88

I think this macro is essentially an alias to HIP?

100

It might be helpful to provide more context here as to how a HIP source file gets compiled (once for the host and then once for each device arch). Also I'd suggest moving this up closer to HIP given they are related.

[yaxunl]

clang/docs/HIPSupport.rst

34

will do

57

It may be clearer to put the options and environment vars for setting dependency paths side by side since they are connected

88

Yes. will change the description

100

will do

[yaxunl]

revised by comments

[scchan]

clang/docs/HIPSupport.rst

83

I think it would be easier to understand by explicitly calling out the order of precedence:

• For HIP path:
  • • --hip-path option
    • HIP_PATH env var
    • --rocm-path option
  • ROCM_PATH env var
  • default automatic detection

• For device lib:
  • --hip-device-lib-path option
  • HIP_DEVICE_LIB_PATH env var
  • --rocm-path option
  • ROCM_PATH env var
  • default automatic detection

[yaxunl]

clang/docs/HIPSupport.rst

83

wil do

[yaxunl]

revised by comments

[arsenm]

clang/docs/HIPSupport.rst

50

s/architecture/architectures

100

We should really not have the environment variables I wouldn't go out of the way to advertise them, just mark as deprecated

[yaxunl]

revised by comments

[yaxunl]

clang/docs/HIPSupport.rst

50

done

100

added warnings to discourage use of env vars

[arsenm]

clang/docs/HIPSupport.rst

34

Aren't you supposed to use clang++? Also, could show that .hip is recognized?

[yaxunl]

clang/docs/HIPSupport.rst

34

good point. will fix

[yaxunl]

revised by comments

[keryell]

clang/docs/HIPSupport.rst

25

I thought the idea of HIP was to also target Nvidia GPU? Otherwise I do not understand "portable applications for GPUs" above.

34

I suggest giving also a simpler example without -c to generate directly an executable without split compile and link.
Also avoid using test as an example as it is confusing as it is also a shell builtin on Unix and it might take sometime to understand why the test does not use the GPU. :-)

36–38

142

Should this one be with underscores too?

[keryell]

clang/docs/HIPSupport.rst

25

Thinking to the fact this is used by other projects like https://github.com/CHIP-SPV/chipStar this is even wider but at the same time that project is not up-streamed.

[yaxunl]

clang/docs/HIPSupport.rst

25

will add a brief description of HIP support on other devices.

34

will do

36–38

will fix

142

This is due to design inconsistency. The same with __HIP_NO_IMAGE_SUPPORT.

Since these two macros are new and not widely used, renaming them won't cause significant interruptions. I will let clang emit __HIP_NO_IMAGE_SUPPORT__ and __HIP_API_PER_THREAD_DEFAULT_STREAM__ and make the current ones alias and document them as deprecated.

The macros for HIP memory scope were designed by following the naming convention of clang atomic macros e.g. __ATOMIC_RELAXED, __OPENCL_MEMORY_SCOPE_WORK_ITEM, therefore they will not be changed.

[yaxunl]

revised by comments

[yaxunl]

ping

[scchan]

clang/docs/HIPSupport.rst

66

missing --hip-link

78

missing --hip-link

[arsenm]

clang/docs/HIPSupport.rst

66

What does hip-link do? Why is it needed? I never use it and it works

[yaxunl]

clang/docs/HIPSupport.rst

66

It indicates HIP offloading at the linking stage and creates the HIP toolchain.

For -fno-gpu-rdc mode, it locates the HIP runtime library and links it.

For -fgpu-rdc mode, it unbundles object files and does device linking and embedding.

[yaxunl]

revised by comments

[scchan]

LGTM thanks

[keryell]

clang/docs/HIPSupport.rst

31

Fix the spelling when we talk about the Khronos SPIR-V standard, by opposition to tool names.
Perhaps you can split the line in shorter ones so the diff are more obvious.

66

I am a little lost here.
I have the feeling that the newbie syntax (compile+link in one invocation) should be

clang++ --offload-arch=gfx906 -xhip sample.cpp -o sample

since it should be obvious to the Driver that we want to produce a working program for a GPU.
Or does this work producing a working program with another meaning?

[yaxunl]

clang/docs/HIPSupport.rst

31

Fix the spelling when we talk about the Khronos SPIR-V standard, by opposition to tool names.
Perhaps you can split the line in shorter ones so the diff are more obvious.

will do

66

Currently, the linker will only link HIP runtime library when --hip-link is specified.

The linker does not see -xhip sample.cpp. It only sees the intermediate obj files passed to it, therefore it does not know it is for HIP.

This could be improved by letting the clang driver add an implicit --hip-link option to the linker when the driver sees the linking of HIP input files. However, this needs a separate patch.

clang/docs/HIPSupport.rst

+.. raw:: html
+
+  <style type="text/css">
+    .none { background-color: #FFCCCC }
+    .part { background-color: #FFFF99 }
+    .good { background-color: #CCFF99 }
+  </style>
+
+.. role:: none
+.. role:: part
+.. role:: good
+
+.. contents::
+   :local:
+
+=============
+HIP Support
+=============
+
+HIP (Heterogeneous-Compute Interface for Portability) `<https://github.com/ROCm-Developer-Tools/HIP>`_ is
+a C++ Runtime API and Kernel Language. It enables developers to create portable applications for
+offloading computation to different hardware platforms from a single source code.
+
+AMD GPU Support
+===============
+
+Clang provides HIP support on AMD GPUs via the ROCm platform `<https://rocm.docs.amd.com/en/latest/#>`_.
+The ROCm runtime forms the base for HIP host APIs, while HIP device APIs are realized through HIP header
+files and the ROCm device library. The Clang driver uses the HIPAMD toolchain to compile HIP device code
+to AMDGPU ISA via the AMDGPU backend. The compiled code is then bundled and embedded in the host executables.
+
+Intel GPU Support
+=================
+
+Clang provides partial HIP support on Intel GPUs using the CHIP-Star project `<https://github.com/CHIP-SPV/chipStar>`_.
+CHIP-Star implements the HIP runtime over oneAPI Level Zero or OpenCL runtime. The Clang driver uses the HIPSPV
+toolchain to compile HIP device code into LLVM IR, which is subsequently translated to SPIR-V via the SPIR-V
+backend or the out-of-tree LLVM-SPIRV translator. The SPIR-V is then bundled and embedded into the host executables.
+
+.. note::
+   While Clang does not directly provide HIP support for NVIDIA GPUs and CPUs, these platforms are supported via other means:
+
+   - NVIDIA GPUs: HIP support is offered through the HIP project `<https://github.com/ROCm-Developer-Tools/HIP>`_, which provides a header-only library for translating HIP runtime APIs into CUDA runtime APIs. The code is subsequently compiled using NVIDIA's `nvcc`.
+
+   - CPUs: HIP support is available through the HIP-CPU runtime library `<https://github.com/ROCm-Developer-Tools/HIP-CPU>`_. This header-only library enables CPUs to execute unmodified HIP code.
+
+
+Example Usage
+=============
+
+To compile a HIP program, use the following command:
+
+.. code-block:: shell
+
+   clang++ -c --offload-arch=gfx906 -xhip sample.cpp -o sample.o
+
+The ``-xhip`` option indicates that the source is a HIP program. If the file has a ``.hip`` extension,
+Clang will automatically recognize it as a HIP program:
+
+.. code-block:: shell
+
+   clang++ -c --offload-arch=gfx906 sample.hip -o sample.o
+
+To link a HIP program, use this command:
+
+.. code-block:: shell
+
+   clang++ --hip-link --offload-arch=gfx906 sample.o -o sample
+
+For convenience, Clang also supports compiling and linking in a single step:
+
+.. code-block:: shell
+
+   clang++ --hip-link --offload-arch=gfx906 -xhip sample.cpp -o sample
+
+In the above commands, ``gfx906`` is the GPU architecture that the code is being compiled for. The supported GPU
+architectures can be found in the `AMDGPU Processor Table <https://llvm.org/docs/AMDGPUUsage.html#processors>`_.
+Alternatively, you can use the ``amdgpu-arch`` tool that comes with Clang to list the GPU architecture on your system:
+
+.. code-block:: shell
+
+   amdgpu-arch
+
+You can use ``--offload-arch=native`` to automatically detect the GPU architectures on your system:
+
+.. code-block:: shell
+
+   clang++ --hip-link --offload-arch=native -xhip sample.cpp -o sample
+
+
+Path Setting for Dependencies
+=============================
+
+Compiling a HIP program depends on the HIP runtime and device library. The paths to the HIP runtime and device libraries
+can be specified either using compiler options or environment variables. The paths can also be set through the ROCm path
+if they follow the ROCm installation directory structure.
+
+Order of Precedence for HIP Path
+--------------------------------
+
+1. ``--hip-path`` compiler option
+2. ``HIP_PATH`` environment variable *(use with caution)*
+3. ``--rocm-path`` compiler option
+4. ``ROCM_PATH`` environment variable *(use with caution)*
+5. Default automatic detection (relative to Clang or at the default ROCm installation location)
+
+Order of Precedence for Device Library Path
+-------------------------------------------
+
+1. ``--hip-device-lib-path`` compiler option
+2. ``HIP_DEVICE_LIB_PATH`` environment variable *(use with caution)*
+3. ``--rocm-path`` compiler option
+4. ``ROCM_PATH`` environment variable *(use with caution)*
+5. Default automatic detection (relative to Clang or at the default ROCm installation location)
+
+.. list-table::
+   :header-rows: 1
+
+   * - Compiler Option
+     - Environment Variable
+     - Description
+     - Default Value
+   * - ``--rocm-path=<path>``
+     - ``ROCM_PATH``
+     - Specifies the ROCm installation path.
+     - Automatic detection
+   * - ``--hip-path=<path>``
+     - ``HIP_PATH``
+     - Specifies the HIP runtime installation path.
+     - Determined by ROCm directory structure
+   * - ``--hip-device-lib-path=<path>``
+     - ``HIP_DEVICE_LIB_PATH``
+     - Specifies the HIP device library installation path.
+     - Determined by ROCm directory structure
+
+.. note::
+
+   We recommend using the compiler options as the primary method for specifying these paths. While the environment variables ``ROCM_PATH``, ``HIP_PATH``, and ``HIP_DEVICE_LIB_PATH`` are supported, their use can lead to implicit dependencies that might cause issues in the long run. Use them with caution.
+
+
+Predefined Macros
+=================
+
+.. list-table::
+   :header-rows: 1
+
+   * - Macro
+     - Description
+   * - ``__CLANG_RDC__``
+     - Defined when Clang is compiling code in Relocatable Device Code (RDC) mode. RDC, enabled with the ``-fgpu-rdc`` compiler option, is necessary for linking device codes across translation units.
+   * - ``__HIP__``
+     - Defined when compiling with HIP language support, indicating that the code targets the HIP environment.
+   * - ``__HIPCC__``
+     - Alias to ``__HIP__``.
+   * - ``__HIP_DEVICE_COMPILE__``
+     - Defined during device code compilation in Clang's separate compilation process for the host and each offloading GPU architecture.
+   * - ``__HIP_MEMORY_SCOPE_SINGLETHREAD``
+     - Represents single-thread memory scope in HIP (value is 1).
+   * - ``__HIP_MEMORY_SCOPE_WAVEFRONT``
+     - Represents wavefront memory scope in HIP (value is 2).
+   * - ``__HIP_MEMORY_SCOPE_WORKGROUP``
+     - Represents workgroup memory scope in HIP (value is 3).
+   * - ``__HIP_MEMORY_SCOPE_AGENT``
+     - Represents agent memory scope in HIP (value is 4).
+   * - ``__HIP_MEMORY_SCOPE_SYSTEM``
+     - Represents system-wide memory scope in HIP (value is 5).
+   * - ``__HIP_NO_IMAGE_SUPPORT__``
+     - Defined with a value of 1 when the target device lacks support for HIP image functions.
+   * - ``__HIP_NO_IMAGE_SUPPORT``
+     - Alias to ``__HIP_NO_IMAGE_SUPPORT__``. Deprecated.
+   * - ``__HIP_API_PER_THREAD_DEFAULT_STREAM__``
+     - Defined when the GPU default stream is set to per-thread mode.
+   * - ``HIP_API_PER_THREAD_DEFAULT_STREAM``
+     - Alias to ``__HIP_API_PER_THREAD_DEFAULT_STREAM__``. Deprecated.
+