[HIPSPV] Convert HIP kernels to SPIR-V kernels

[linjamaki]

Summary

This patch translates HIP kernels to SPIR-V kernels when the HIP
compilation mode is targeting SPIR-S. This involves:

• Setting Cuda calling convention to CC_OpenCLKernel (which maps to SPIR_KERNEL in LLVM IR later on).

• Coercing pointer arguments with default address space (AS) qualifier to CrossWorkGroup AS (__global in OpenCL). HIPSPV's device code is ultimately SPIR-V for OpenCL execution environment (as starter/default) where Generic or Function (OpenCL's private) is not supported as storage class for kernel pointer types. This leaves the CrossWorkGroup to be the only reasonable choice for HIP buffers.

[linjamaki]

Rebase.

[linjamaki]

Rebase.

[yaxunl]

LGTM. Thanks!

[Anastasia]

clang/lib/CodeGen/TargetInfo.cpp

10236

It feels like this needs to be in SPIRVABIInfo or something? Or can this be generalized to both - SPIR and SPIR-V?

10239

Can you explain why this mapping is needed? We already have an address space map to perform the mapping of address spaces b/w language and target. It would be good if we don't replicate similar logic in too many places.

10288

Again this feels like we need SPIRVTargetCodeGenInfo unless the logic is generalizable to both.

[linjamaki]

clang/lib/CodeGen/TargetInfo.cpp

10236

A comment was added in D109144 to state that the SPIRABIInfo is an ABI implementation for both the SPIR and SPIR-V. For now, there is not much difference between SPIR and SPIR-V for this class. Would it be satisfactory if the class is renamed to something more general (like CommonSPIRABIInfo)?

10239

HIP does not require address space qualifiers on kernel pointer arguments (e.g. see hipspv-kernel.cpp) nor there are AS qualifiers that can be placed on them. With the default logic, provided by SPIRVTargetInfo’s address space map, the kernel pointer arguments get converted to generic pointers which are not allowed by the OpenCL SPIR-V Environment Specification.

[Anastasia]

clang/lib/CodeGen/TargetInfo.cpp

10236

It might be reasonable to rename indeed or we can just amend its documentation to clarify what it is used for both...

However if you need to specialize different logic I think the conventional way in clang would be to create a subclass...

This is related to a wider discussion that is ongoing about the best way to reuse SPIR for SPIR-V and their evolution... CCing @bader here too.

10239

I feel that it is the same for SYCL... It might be good to check with @bader whether there is already a way to handle this that can be reused for HIP...

[bader]

clang/lib/CodeGen/TargetInfo.cpp

10239

We need to do similar transformation for SYCL, but it's not exactly the same.
For SYCL kernels, which represented as function objects, compiler generates SPIR kernel function and fixes up the address space for pointer arguments in compiler generated declaration. For more details, see the description of https://reviews.llvm.org/D71016 and handlePointerType function code in clang/lib/Sema/SemaSYCL.cpp of this review request (lines 848-876). As address space is fixed in Sema, it works for all targets SYCL currently supports SPIR, NVPTX and AMDGPU.

If I understand it correctly, we are trying to do minimal amount of work for convert HIP kernel function to SPIR kernel function, i.e. fix calling convention and address spaces.
Are these two fixes enough or we need more fixes to enable more sophisticated kernels?

[linjamaki]

clang/lib/CodeGen/TargetInfo.cpp

10239

This patch and D108621 covers only the calling convention and the address space aspects in HIP->SPIR-V conversion. There are still various HIP features [1] which need to be expanded or emulated afterwards. A fully linked device program is needed before the fixes can be applied, so these fixes won’t be implemented at Sema nor CodeGen which operate per translation unit. The full-program fixes are applied by the HIPSPV tool chain by applying LLVM passes provided by a HIPSPV runtime [2]. For the time being, we are not seeing a need to specialize SPIR target infos further.

[1]: “HIP code expansion” section of the “[RFC][HIPSPV] Emitting HIP device code as SPIR-V”
[2]: clang/lib/Driver/ToolChains/HIPSPV.cpp:constructEmitSpirvCommand() @ https://github.com/parmance/llvm-project/tree/hipspv-wip

[linjamaki]

Put HIPSPV logic into new SPIRV{ABI,TargetCodeGen}Info subclassed from
CommonSPIR{ABI,TargetCodeGen}Info.

[linjamaki]

Rebase.

[linjamaki]

Rebase.

[linjamaki]

The patch is ready to land. @Anastasia, @bader, could you commit this patch to the LLVM for us? Thanks.

[bader]

In D109818#3169531, @linjamaki wrote:

The patch is ready to land. @Anastasia, @bader, could you commit this patch to the LLVM for us? Thanks.

Could you rebase on the tip of the main branch, please? I see a couple of conflicts when I cherry-pick the patch.

[linjamaki]

Rebase.

[linjamaki]

Cherry-picking should work now, @bader.

[linjamaki]

Thanks, @bader.