This is an archive of the discontinued LLVM Phabricator instance.

[SYCL] Implement OpenCL kernel function generation
Needs ReviewPublic

Authored by zahiraam on Dec 4 2019, 7:25 AM.

Details

Summary

All SYCL memory objects shared between host and device (buffers/images, these
objects map to OpenCL buffers and images) must be accessed through special
accessor classes. The "device" side implementation of these classes contain
pointers to the device memory. As there is no way in OpenCL to pass
structures with pointers inside as kernel arguments, all memory objects
shared between host and device must be passed to the kernel as raw
pointers. SYCL also has a special mechanism for passing kernel arguments
from host to the device. In OpenCL kernel arguments are set by calling
clSetKernelArg function for each kernel argument, meanwhile in SYCL all the
kernel arguments are fields of "SYCL kernel function" which can be defined
as a lambda function or a named function object and passed as an argument
to SYCL function for invoking kernels (such as parallel_for or single_task).

To facilitate the mapping of SYCL kernel data members to OpenCL kernel
arguments and overcome OpenCL limitations we added the generation of an
OpenCL kernel function inside the compiler. An OpenCL kernel function
contains the body of the SYCL kernel function, receives OpenCL-like
parameters and additionally does some manipulation to initialize SYCL
kernel data members with these parameters. In some pseudo code the OpenCL
kernel function can look like this:

// SYCL kernel is defined in SYCL headers:
template <typename KernelName, typename KernelType/*, ...*/>
__attribute__((sycl_kernel)) void sycl_kernel_function(KernelType KernelFuncObj) {
  // ...
  KernelFuncObj();
}

// Generated OpenCL kernel function
__kernel KernelName(global int* a) {
  KernelType KernelFuncObj; // Actually kernel function object declaration
  // doesn't have a name in AST.
  // Let the kernel function object have one captured field - accessor A.
  // We need to init it with global pointer from arguments:
  KernelFuncObj.A.__init(a);
  // Body of the SYCL kernel from SYCL headers:
  {
    KernelFuncObj();
  }
}

OpenCL kernel function is generated by the compiler inside the Sema
using AST nodes.

Diff Detail

Event Timeline

Fznamznon created this revision.Dec 4 2019, 7:25 AM
Herald added a project: Restricted Project. · View Herald TranscriptDec 4 2019, 7:25 AM
Fznamznon updated this revision to Diff 233013.Dec 10 2019, 1:16 AM

Updated tests using address space attributes added by D71005

keryell added inline comments.Dec 10 2019, 6:04 PM
clang/lib/Sema/SemaSYCL.cpp
44

Feel free to add more comments in this area up to line 103.

417

After more thought, perhaps the solution proposed by Victor Lomüller during the last SYCL upstreaming meeting about marking accessor classes with some attribute instead of detecting concrete type names is a better generic approach.
I am more convinced now by his argument allowing more experimenting with alternative runtimes, since it looks like it is the only place we detect type names. For example the kernels are marked with an attribute in the runtime instead of concretely detecting the parallel_for() functions and so on.

clang/test/CodeGenSYCL/device-functions.cpp
3

Missing description about the purpose of this test

clang/test/SemaSYCL/fake-accessors.cpp
3

Missing description about the purpose of this test.
I am struggling about understanding what this test is for...
OK, after coming back later, I think I got it. I was confused by the fact that in the kernels you are using both true accessors (A, B & C) *and* some objects with names similar to SYCL accessor.
Is it possible to have some tests without true accessors?

clang/test/SemaSYCL/mangle-kernel.cpp
4

Missing description about the purpose of this test

Fznamznon marked an inline comment as done.Dec 13 2019, 6:30 AM
Fznamznon added inline comments.
clang/lib/Sema/SemaSYCL.cpp
417

@Naghasan , what do you think about upstreaming the solution which you proposed?

Fznamznon marked an inline comment as not done.Dec 16 2019, 3:14 AM
bader updated this revision to Diff 319579.Jan 27 2021, 7:49 AM

Depends on D89909

Apply improvements developed in out-of-tree repository, which mostly are refactoring of SemaSYCL.cpp file.
Rebase on ToT + D89909 (which only impacts lit test checks).

bader updated this revision to Diff 319909.Jan 28 2021, 10:04 AM

Addressed comment from Ronan in regression tests and synced them with the latest status.

bader commandeered this revision.Jan 28 2021, 10:05 AM
bader edited reviewers, added: Fznamznon; removed: bader.
bader added inline comments.
clang/test/CodeGenSYCL/device-functions.cpp
3

Done.

clang/test/SemaSYCL/fake-accessors.cpp
3

Added a new test case w/o SYCL accessor and description of the test.

clang/test/SemaSYCL/mangle-kernel.cpp
4

Done.

ABataev added inline comments.Oct 13 2021, 3:33 AM
clang/include/clang/Sema/Sema.h
13129

ArrayRef<Decl *> getSYCLKernels()

clang/lib/Sema/SemaSYCL.cpp
90

final

115

Make functions static?

117

llvm::find_if

186

Type

erichkeane added inline comments.Oct 13 2021, 6:19 AM
clang/lib/Sema/SemaSYCL.cpp
45

Isn't there a big rewrite going on downstream of these with sycl_special_class? Why are we trying to upstream this before that happens?

bader added inline comments.Oct 13 2021, 7:22 AM
clang/lib/Sema/SemaSYCL.cpp
45

Isn't there a big rewrite going on downstream of these with sycl_special_class?

Yes.

Why are we trying to upstream this before that happens?

The downstream work was initiated by this comment: https://reviews.llvm.org/D71016#inline-644645.
This patch was uploaded for review here before refactoring work has started.

FWIW, it seems like precommit CI is failing with problems in x64 windows > Clang.SemaSYCL::accessors-targets.cpp (which is largely hidden by the spam from the CI pipeline, unfortunately). Also, you should address the tidy warnings.

clang/include/clang/Sema/Sema.h
13124

Is 4 accurate? I would assume that 1 is more accurate as most people aren't going to be using SYCL at all.

13129

Also, this needs a const-correct overload (or surface only the const correct version, if possible).

clang/lib/AST/ASTContext.cpp
11069
clang/lib/Sema/SemaSYCL.cpp
45

So should this patch be abandoned until that refactoring work completes, to avoid churn?

100

Please spell out the type as it's not spelled out in the initialization.

It feels like you are doing codegen(OpenCL kernel) in Sema. Are OpenCL kernels the only approach.

bader added a comment.Oct 14 2021, 4:55 AM

It feels like you are doing codegen(OpenCL kernel) in Sema. Are OpenCL kernels the only approach.

We need to update the description of the patch to clarify that it applies to other GPU programming models as well. When the patch was uploaded SYCL targeted OpenCL kernels only and today the downstream implementation can target CUDA, HIP and Intel oneAPI Level Zero kernels as well.
SYCL kernel is defined as C++ callable type, but typical GPU kernel interface is a C-like function. In addition to that there might be additional restrictions on passing data from the host system (e.g. image types can be passed as a member of C++ class, etc.). The solution here is emit a wrapper function, which initializes and invokes SYCL callable object.

Does it answer your question or you would like to see changes in addition to the description update?

It feels like you are doing codegen(OpenCL kernel) in Sema. Are OpenCL kernels the only approach.

We need to update the description of the patch to clarify that it applies to other GPU programming models as well. When the patch was uploaded SYCL targeted OpenCL kernels only and today the downstream implementation can target CUDA, HIP and Intel oneAPI Level Zero kernels as well.
SYCL kernel is defined as C++ callable type, but typical GPU kernel interface is a C-like function. In addition to that there might be additional restrictions on passing data from the host system (e.g. image types can be passed as a member of C++ class, etc.). The solution here is emit a wrapper function, which initializes and invokes SYCL callable object.

Does it answer your question or you would like to see changes in addition to the description update?

Would a codegenSYCL directory help you to separate Sema from code generation?

lebedev.ri added a subscriber: lebedev.ri.

Doesn't this make AST non-representable of the reality,
shouldn't the lowering happen in codegen, not in sema?

bader added a comment.Oct 14 2021, 5:31 AM

Would a codegenSYCL directory help you to separate Sema from code generation?

Moving wrapper kernel function generation to CodeGen library make sense to me.

Doesn't this make AST non-representable of the reality,
shouldn't the lowering happen in codegen, not in sema?

I'm not sure I understand what does "make AST non-representable of the reality" mean, but it seems to be the same suggestion as @tschuett proposed.

zahiraam commandeered this revision.Nov 19 2021, 11:15 AM
zahiraam updated this revision to Diff 388573.
zahiraam added a reviewer: bader.

Added in the sycl_special_class attribute. This is still work in progress as I still have a few LIT tests failing and didn't address the issue of the separating sema from codegen work.