[Docs][OpenCL] Added OpenCL feature description to user manual.

[Anastasia]

Summary

Adding information on OpenCL to Clang documentation. This currently contains the main features only. I am hoping we can improve it with time documenting more features.

Can be build by:

cmake  -DLLVM_ENABLE_SPHINX=ON -DLLVM_BUILD_DOCS=ON ...
make sphinx

[Leporacanthicus]

docs/UsersManual.rst

2083

Should this be opencl_addrsp (two d's)?

2242

Shouldn't that be __attribute__?

2259

Compared to the unrolled code below, this should probably say i < 4 rather than i < 3?

2278

Obviously, change this as well to opencl_addrsp...

[Anastasia]

Updated with comments from Mats!

[Anastasia]

Ping! Any more comments here? It would be nice to commit this before the 4.0 rel branch is taken (on Jan 12).

[yaxunl]

docs/UsersManual.rst

2003

$ clang -target amdgcn-amd-amdhsa-opencl test.cl

2053

Maybe rephrase as "Disables support of OpenCL extensions" instead of "Disable standard target extensions" since "standard target extension" is confusing and also to avoid confusion with the effect of disabling OpenCL extension as in '#pragam OPENCL EXTENSION X: disable'. Disabling support of an extension means that they cannot be enabled by pragmas in the OpenCL kernels since they are deemed as non-supported.

2107

$ clang -target amdgcn-amd-amdhsa-opencl test.cl

2154

To enable modules for OpenCL, add option -fmodules -fimplicit-module-maps -fmodules-cache-path=X where X is the directory for storing the generated modules.

[pekka.jaaskelainen]

Great work.

docs/UsersManual.rst

44

Should this be OpenCL C Language?

2013

I'd like to add a sentence or two for instructing how to use it for non-SPMD targets and of course advertise pocl here too ;)

Something along the lines of:

"For "non-SPMD" targets which cannot spawn multiple work-items on the fly using hardware, which covers practically all non-GPU devices such as CPUs and DSPs, additional processing is needed for the kernels to support multiple WI execution. For this, a 3rd party project such as pocl <http://portablecl.org/>_ can be used."

2078

I'd convert to something like:

"This allows adding explicit address space ids to the bitcode for architectures without separate IDs for each of the OpenCL logical address spaces."

2114

This is an alternative location for my pocl advertisement above with a subtitle:

• For CPU/DSP architectures such as x86, ARM or TCE:

...

2120

Which version of SPIR is generated?

2130

This is a bit confusing paragraph, probably due to my confusing explanations of the problems with pocl. Currently pocl tries not to use FASM for preserving logical AS IDs to LLVM IR due to the bunch of problems it constantly produces with seemingly little benefits for common CPUs. My patch related to this considered only the argument info switch. Now pocl only derives the logical iDS from kernel arguments (and all other IDs within the body of the IR function are lost for flat machines). In my patch, the argument info's address space IDs were made constant and identical to SPIR's as previously they were the target's (which meant losing the AS IDs altogether for flat AS machines).

You seem to document the arg-info md switch later, so this paragraph might be removed or converted to my pocl blurb which mentions the need for further processing for CPUs.

2173

Something like: "The IDs used to encode the OpenCL's logical address spaces in the argument info metadata ..."

2206

add space before /

2310

+conversion

www/index.html

19

OpenCL C. Maybe OpenCL C++ also?

[bader]

Thanks for working on this!
Looks good, except a few pedantic notes.

docs/UsersManual.rst

44

It should and to be perfectly clear we might consider using term "OpenCL C kernel language" to avoid confusion with OpenCL host code language or OpenCL C++ kernel language.

2009

Just for the sake of consistency it would be nice to switch input file name with the options:

clang -c -emit-llvm test.cl

2014

I suggest specifying that Clang support OpenCL C kernel language standards up to v2.0.
For instance OpenCL v2.1 doesn't introduce new OpenCL C kernel language standard - it still uses v2.0. So existing clang is sufficient enough to serve as front-end compiler in OpenCL 2.1 driver.

2120

For -cl-std=CL1.x (where x is 0, 1 or 2), SPIR version is 1.2.
For -cl-std=CL2.0, SPIR version is 2.0.

2142–2143

Not sure it's worth to mention, but even built-in vector types are defined in the opencl-c.h.

[Anastasia]

Addressed comments from Alexey, Pekka, and Sam!

[Anastasia]

docs/UsersManual.rst

2013

Cool! I have added this later in the description of x86 target use for OpenCL. :)

2053

Good point!

2114

Added later in the x86 description.

2120

Is it worth mentioning this in this doc?

2130

Yes, it's perhaps a bit confusing indeed. I find the usage of the x86 backend for OpenCL is generally a bit confusing. Also there are a lot of program paths even in Clang that don't play well for OpenCL purposes or are not handled properly and are therefore a source of bugs.

I was just wondering whether it would be possible to switch to using SPIR as a generic target at some point in the future and "deprecate" the x86 target for OpenCL . Do you think this might work for POCL? At the same time we have upcoming SPIR-V support as a new feature that might reshape things.

Regarding the address space I just know that it's common to use fake address space map with the x86 backend for some out of tree implementations to add missing memory segments to this target. That's why I added this text here.

2142–2143

Good point!

www/index.html

19

We don't have OpenCL C++ officially yet. :(

[bader]

LGTM.
Thanks!

[yaxunl]

Great work. LGTM. Thanks!

[pekka.jaaskelainen]

docs/UsersManual.rst

2120

I think so as SPIR V is now there also.

2130

We have considered using SPIR internally, but it is likely to bring the same problems as with the FASM as we need to convert to the target bitcode to ensure all target specific IR level optimizations (especially vectorization) is applied properly. Why do you think using the real target when generating from OpenCL C kernels is problematic?

2133

ARM (and others) are treated similarly in pocl: OpenCL C is just yet another frontend language in this picture, like C/C++. I would not "deprecate" that path as of yet as forcing to another intermediate representation (which SPIR basically is, albeit derived from LLVM IR) always loses some information on the way.

[Anastasia]

Update based on comments from Pekka.

[Anastasia]

docs/UsersManual.rst

2130

Perhaps, it would be nice to understand better how you use x86 backend in your toolchain. What I have seen in some projects that some invalid IR was produced or Clang hit an asset in the place which has to be done differently in some CodeGen paths just for OpenCL. So the typical fix for this to specialize on the OpenCL is not that easy to get into upstream because the target hasn't been created for what it's being used for and people like to see more modular code in general anyways.

The target is ideally not just a processor architecture but a combination of hardware and runtime software layers (i.e. OpenCL runtime in our case). Btw what triple do you use exactly while compiling OpenCL for x86?

2133

I have removed the deprecation bit from the documentation now. Do you want me to change anything else here?

I think the idea of SPIR target was not to force the developers to yet another IR but rather to create a common generic target that is specifically tailored to OpenCL needs (not just as a language but RT as well). Also the intention of SPIR was to add missing semantics of OpenCL to LLVM IR, so it's supposed not to lose the information ideally even though I am not completely sure the goal was achieved. But since SPIR-V is gaining more popularity I am not sure how much SPIR will continue to be relevant in the community.

One more problem with x86 target is that there are currently not many contributors in the community that are willing to support it for OpenCL. That's why I am a bit reluctant to put it in the document as fully supported target. Do you have any relevant fixes in your internal branches btw?

[Anastasia]

Since everyone agreed on the core part of this documentation I have committed the current revision in 291780 to make it easier for the release.

Feel free to give me more feedback (if any) as small changes can still be merged in.

@yaxunl Sam, I think it would be nice to describe your recent change for adding OpenCL extensions. Would you like to write up some text? Otherwise I can draft something and you can review. :)

[yaxunl]

@yaxunl Sam, I think it would be nice to describe your recent change for adding OpenCL extensions. Would you like to write up some text? Otherwise I can draft something and you can review. :)

Anastasia, I am busy with some other work. If you can add it I will be happy to review it. Thanks!

[Anastasia]

In D28080#644222, @yaxunl wrote:

@yaxunl Sam, I think it would be nice to describe your recent change for adding OpenCL extensions. Would you like to write up some text? Otherwise I can draft something and you can review. :)

Anastasia, I am busy with some other work. If you can add it I will be happy to review it. Thanks!

Sure. No worries!

[pekka.jaaskelainen]

docs/UsersManual.rst

2130

We use just the regular target triplet without any opencl specific parts.This is because of at least two reasons a) we link in builtins as target-specific bitcodes b) we want to ensure all ordinary LLVM IR optimizations (some of which are target specific) are applied to the fully linked kernel bitcode to get all the performance improvements. Apart from the address spaces, I'm not sure which OpenCL-specific code paths you might mean in the code gen. I'd turn it the other way: if there's no need such special casing, it's better to do like with other frontends and have the usual hardware target instead of SPIR (which then needs to be converted to the target IR just for optimizations and bitcode linking).

2133

No additional fixes, the upstream Clang/LLVM works just fine. The main source of grief was the fake address space map to transfer the logical address space data, which I decided to get off of for now.

[pekka.jaaskelainen]

docs/UsersManual.rst

2065

Is this correct? I cannot make it work:

~/local/stow/llvm-4.0-unpatched-Debug/bin/clang -Xclang -finclude-default-header -emit-llvm -cc1 -triple spir64-unknown-unknown kernel/test_halfs.cl -c -S -o -
clang-4.0: error: unknown argument: '-cc1'
clang-4.0: error: unknown argument: '-triple'
clang-4.0: error: no such file or directory: 'spir64-unknown-unknown'

-target works instead. (But reveals another issue, there's no printf() declaration, should I file a bug?)

~/local/stow/llvm-4.0-unpatched-Debug/bin/clang -Xclang -finclude-default-header -emit-llvm -target spir64-unknown-unknown kernel/test_halfs.cl -c -S -o -
kernel/test_halfs.cl:10:9: warning: implicit declaration of function 'printf' is invalid in C99 [-Wimplicit-function-declaration]
        printf("max(a,b)[0] type=uint2 a=0x15b348c9 b=0xf88e7d07 want=0xf88e7d07 got=%x\n", max_[0]);
        ^
kernel/test_halfs.cl:10:9: error: function with no prototype cannot use the spir_function calling convention
1 warning and 1 error generated.

With 3.9 that kernel which calls printf() passes:

/local/stow/llvm-3.9-debug/bin/clang -Xclang -finclude-default-header -emit-llvm -target spir64-unknown-unknown kernel/test_halfs.cl -c -S -o -
; ModuleID = 'kernel/test_halfs.cl'
source_filename = "kernel/test_halfs.cl"
target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024"
target triple = "spir64-unknown-unknown"
....

[Anastasia]

docs/UsersManual.rst

2065

It should be -triple with -cc1 and -target without.

clang -cc1 -triple ...
clang -target ...

We have disabled the C99 builtin functions as per spec v1.2 s6.9.f, but the OpenCL builtins should be available via the header now.

As I can see declaration of printf is in the header under CL1.2 and higher.

Could you try this command instead:

clang -std=CL1.2 test.cl -Xclang -finclude-default-header

[pekka.jaaskelainen]

docs/UsersManual.rst

2065

Yes, std=CL1.2 helped, good point! But -cc1 -triple doesn't work (unknown argument). However -target works. This is with Clang 4.0.

[bader]

docs/UsersManual.rst

2065

I think you pass something before -cc1. -cc1 option must be the first argument to the clang executable, otherwise it won't work.

[Anastasia]

@pekka.jaaskelainen, I just compiled the release 4.0 branch and it all works for me as expected:
clang -cc1 -triple spir64-unknown-unknown test.cl
clang -target spir64-unknown-unknown test.cl

[pekka.jaaskelainen]

In D28080#651732, @Anastasia wrote:

@pekka.jaaskelainen, I just compiled the release 4.0 branch and it all works for me as expected:
clang -cc1 -triple spir64-unknown-unknown test.cl
clang -target spir64-unknown-unknown test.cl

Yes, that seems to work.

The problem was confusion with the frontend flag parameter. With the -cc1 it switches all parameters to
the frontend mode, not only the next one (like -Xclang does).

When I pass -Xclang before -cc1, it fails, if after, it fails also, but differently:

$ clang -cc1 -triple spir64-unknown-unknown -Xclang -finclude-default-header ~/temp/local.cl
error: unknown argument: '-Xclang'
$ clang -Xclang -finclude-default-header -cc1 -triple spir64-unknown-unknown ~/temp/local.cl
clang-4.0: error: unknown argument: '-cc1'
clang-4.0: error: unknown argument: '-triple'
clang-4.0: error: no such file or directory: 'spir64-unknown-unknown'

(I should pass -finclude-default-header after -cc1 without -Xclang then it works, but I think also the
latter version should work, no?)
Also, I cannot add -S (even with -emit-llvm), then it complains about a missing target:

$ clang -cc1 -triple spir64-unknown-unknown ~/temp/local.cl -S
error: unable to create target: 'No available targets are compatible with this triple.'

I think -S used to work with -emit-llvm as a switch to output LLVM text instead of bitcode, but now it seems to
output text by default so it should not be used. Is there are reason to use/document the -cc1 -triple instead of
-target here? Also, for me the command without -emit-llvm doesn't output anything.

[Anastasia]

(I should pass -finclude-default-header after -cc1 without -Xclang then it works, but I think also the
latter version should work, no?)

In general -cc1 and -Xclang have similar effect so I either should work indeed.

Also, I cannot add -S (even with -emit-llvm), then it complains about a missing target:

$ clang -cc1 -triple spir64-unknown-unknown ~/temp/local.cl -S
error: unable to create target: 'No available targets are compatible with this triple.'

I think -S used to work with -emit-llvm as a switch to output LLVM text instead of bitcode, but now it seems to
output text by default so it should not be used.

I think this has to do with spir being a 'generic' target.

Is there are reason to use/document the -cc1 -triple instead of
-target here?

Yes, because in some cases we have added frontend flag only, so we need to either pass -cc1 or -Xclang. But I could change the documentation to try to pass -target before the -cc1 flag.

Also, for me the command without -emit-llvm doesn't output anything.

What targets do you use? I think the problem is that it's not clear what the default output of the standalone compiler should be for the most of the OpenCL targets, since it does't produce a valid binary. If let's say we emit an x86 binary, would it be valid to run it for POCL directly? Perhaps, we could try to emit the bitcode by default then.

Currently if I run:

clang test.cl

I get an error: undefined reference to `main'. But if I add a main (which isn't really valid for OpenCL) I get:
error: function cannot be called 'main'

[pekka.jaaskelainen]

In D28080#653493, @Anastasia wrote:

Also, for me the command without -emit-llvm doesn't output anything.

What targets do you use?

In this case I used only SPIR. I suppose this is fine as the concept of linking is unclear in case of Clang's OpenCL
support.

I think the problem is that it's not clear what the default output of the standalone compiler should be for the most of the OpenCL targets, since it does't produce a valid binary. If let's say we emit an x86 binary, would it be valid to run it for POCL directly? Perhaps, we could try to emit the bitcode by default then.

An x86 OpenCL binary from Clang is not executable as is (especially in case of multi-WI WGs), but needs a transformation to convert the multi-WI
to parallel loops, fibers, straight to vector instructions, or similar. However, I try to keep the Clang's OpenCL output of any target valid for pocl without
a special triplet code for now. This way it should be possible to get OpenCL support for most, if not all, targets supported by LLVM out of the box.

One problem I've found are the host callable kernel functions. We don't want the target-specific CC/ABI used for them as they have clSetKernelArg()
mappable arguments and other interfacing considerations, especially with heterogeneous setups. Recently I discovered that there might be a way to fix this
nicely via the SPIR target. If we use SPIR ABI by default for OpenCL kernel functions, the rest of it might just work with the target's default settings.

Just a heads up, I will propose this patch to upstream after some testing:
https://raw.githubusercontent.com/pocl/pocl/c9ae29f2268d3b92462d6ca97d1d596dae0d15a1/tools/patches/clang-3.9-always-spir-cc-for-kernels.patch

[Anastasia]

@pekka, do you think there is something more to be done here or can I close this revision now?

Apparently, I won't be able to pass` -target` in all cases because -cc has to be always the first parameter.

Also I am preparing additional text and some minor corrections in another review.

[bader]

Closed by https://reviews.llvm.org/rG18e165f50d8c1ab3afe7098dc00557d5f1a43cfa.

docs/UsersManual.rst

    variants depending on base language.
 -  :ref:`C++ Language <cxx>`
 -  :ref:`Objective C++ Language <objcxx>`
+-  :ref:`OpenCL C Language <opencl>`: v1.0, v1.1, v1.2, v2.0.
 
 In addition to these base languages and their dialects, Clang supports a
 broad variety of language extensions, which are documented in the

  is provided or target does not support TLS, code generation for threadprivate
  variables relies on OpenMP runtime library.
 
+.. _opencl:
+
+OpenCL Features
+===============
+
+Clang can be used to compile OpenCL kernels for execution on a device
+(e.g. GPU). It is possible to compile the kernel into a binary (e.g. for AMD or
+Nvidia targets) that can be uploaded to run directly on a device (e.g. using
+`clCreateProgramWithBinary
+<https://www.khronos.org/registry/OpenCL/specs/opencl-1.1.pdf#111>`_) or
+into generic bitcode files loadable into other toolchains.
+
+Compiling to a binary using the default target from the installation can be done
+as follows:
+
+   .. code-block:: console
+
+     $ echo "kernel void k(){}" > test.cl
+     $ clang test.cl
+
+Compiling for a specific target can be done by specifying the triple corresponding
+to the target, for example:
+
+   .. code-block:: console
+
+     $ clang -target nvptx64-unknown-unknown test.cl
+     $ clang -target amdgcn-amd-amdhsa-opencl test.cl
+
+Compiling to bitcode can be done as follows:
+
+   .. code-block:: console
+
+     $ clang -c -emit-llvm test.cl
+
+This will produce a generic test.bc file that can be used in vendor toolchains
+to perform machine code generation.
+
+Clang currently supports OpenCL C language standards up to v2.0.
+
+OpenCL Specific Options
+-----------------------
+
+Most of the OpenCL build options from `the specification v2.0 section 5.8.4
+<https://www.khronos.org/registry/cl/specs/opencl-2.0.pdf#200>`_ are available.
+
+Examples:
+
+   .. code-block:: console
+
+     $ clang -cl-std=CL2.0 -cl-single-precision-constant test.cl
+
+Some extra options are available to support special OpenCL features.
+
+.. option:: -finclude-default-header
+
+Loads standard includes during compilations. By default OpenCL headers are not
+loaded and therefore standard library includes are not available. To load them
+automatically a flag has been added to the frontend (see also :ref:`the section
+on the OpenCL Header <opencl_header>`):
+
+   .. code-block:: console
+
+     $ clang -Xclang -finclude-default-header test.cl
+
+Alternatively ``-include`` or ``-I`` followed by the path to the header location
+can be given manually.
+
+   .. code-block:: console
+
+     $ clang -I<path to clang>/lib/Headers/opencl-c.h test.cl
+
+In this case the kernel code should contain ``#include <opencl-c.h>`` just as a
+regular C include.
+
+.. option:: -cl-ext
+
+Disables support of OpenCL extensions. All OpenCL targets provide a list
+of extensions that they support. Clang allows to amend this using the ``-cl-ext``
+flag with a comma-separated list of extensions prefixed with ``'+'`` or ``'-'``.
+The syntax: ``-cl-ext=<(['-'|'+']<extension>[,])+>``,  where extensions
+can be either one of `the OpenCL specification extensions
+<https://www.khronos.org/registry/cl/sdk/2.0/docs/man/xhtml/EXTENSION.html>`_
+or any known vendor extension. Alternatively, ``'all'`` can be used to enable
+or disable all known extensions.
+Example disabling double support for the 64-bit SPIR target:
+
+   .. code-block:: console
+
+     $ clang -cc1 -triple spir64-unknown-unknown -cl-ext=-cl_khr_fp64 test.cl
+
+Enabling all extensions except double support in R600 AMD GPU can be done using:
+
+   .. code-block:: console
+
+     $ clang -cc1 -triple r600-unknown-unknown -cl-ext=-all,+cl_khr_fp16 test.cl
+
+.. _opencl_fake_address_space_map:
+
+.. option:: -ffake-address-space-map
+
+Overrides the target address space map with a fake map.
+This allows adding explicit address space IDs to the bitcode for non-segmented
+memory architectures that don't have separate IDs for each of the OpenCL
+logical address spaces by default. Passing ``-ffake-address-space-map`` will
+add/override address spaces of the target compiled for with the following values:
+``1-global``, ``2-constant``, ``3-local``, ``4-generic``. The private address
+space is represented by the absence of an address space attribute in the IR (see
+also :ref:`the section on the address space attribute <opencl_addrsp>`).
+
+   .. code-block:: console
+
+     $ clang -ffake-address-space-map test.cl
+
+Some other flags used for the compilation for C can also be passed while
+compiling for OpenCL, examples: ``-c``, ``-O<1-4|s>``, ``-o``, ``-emit-llvm``, etc.
+
+OpenCL Targets
+--------------
+
+OpenCL targets are derived from the regular Clang target classes. The OpenCL
+specific parts of the target representation provide address space mapping as
+well as a set of supported extensions.
+
+Specific Targets
+^^^^^^^^^^^^^^^^
+
+There is a set of concrete HW architectures that OpenCL can be compiled for.
+
+- For AMD target:
+
+   .. code-block:: console
+
+     $ clang -target amdgcn-amd-amdhsa-opencl test.cl
+
+- For Nvidia architectures:
+
+   .. code-block:: console
+
+     $ clang -target nvptx64-unknown-unknown test.cl
+
+
+Generic Targets
+^^^^^^^^^^^^^^^
+
+- SPIR is available as a generic target to allow portable bitcode to be produced
+  that can be used across GPU toolchains. The implementation follows `the SPIR
+  specification <https://www.khronos.org/spir>`_. There are two flavors
+  available for 32 and 64 bits.
+
+   .. code-block:: console
+
+    $ clang -target spir-unknown-unknown test.cl
+    $ clang -target spir64-unknown-unknown test.cl
+
+  All known OpenCL extensions are supported in the SPIR targets. Clang will
+  generate SPIR v1.2 compatible IR for OpenCL versions up to 2.0 and SPIR v2.0
+  for OpenCL v2.0.
+
+- x86 is used by some implementations that are x86 compatible and currently
+  remains for backwards compatibility (with older implementations prior to
+  SPIR target support). For "non-SPMD" targets which cannot spawn multiple
+  work-items on the fly using hardware, which covers practically all non-GPU
+  devices such as CPUs and DSPs, additional processing is needed for the kernels
+  to support multiple work-item execution. For this, a 3rd party toolchain,
+  such as for example `POCL <http://portablecl.org/>`_, can be used.
+
+  This target does not support multiple memory segments and, therefore, the fake
+  address space map can be added using the :ref:`-ffake-address-space-map
+  <opencl_fake_address_space_map>` flag.
+
+.. _opencl_header:
+
+OpenCL Header
+-------------
+
+By default Clang will not include standard headers and therefore OpenCL builtin
+functions and some types (i.e. vectors) are unknown. The default CL header is,
+however, provided in the Clang installation and can be enabled by passing the
+``-finclude-default-header`` flag to the Clang frontend.
+
+   .. code-block:: console
+
+     $ echo "bool is_wg_uniform(int i){return get_enqueued_local_size(i)==get_local_size(i);}" > test.cl
+     $ clang -Xclang -finclude-default-header -cl-std=CL2.0 test.cl
+
+Because the header is very large and long to parse, PCH (:doc:`PCHInternals`)
+and modules (:doc:`Modules`) are used internally to improve the compilation
+speed.
+
+To enable modules for OpenCL:
+
+   .. code-block:: console
+
+     $ clang -target spir-unknown-unknown -c -emit-llvm -Xclang -finclude-default-header -fmodules -fimplicit-module-maps -fmodules-cache-path=<path to the generated module> test.cl
+
+OpenCL Metadata
+---------------
+
+Clang uses metadata to provide additional OpenCL semantics in IR needed for
+backends and OpenCL runtime.
+
+Each kernel will have function metadata attached to it, specifying the arguments.
+Kernel argument metadata is used to provide source level information for querying
+at runtime, for example using the `clGetKernelArgInfo 
+<https://www.khronos.org/registry/OpenCL/specs/opencl-1.2.pdf#167>`_
+call.
+
+Note that ``-cl-kernel-arg-info`` enables more information about the original CL
+code to be added e.g. kernel parameter names will appear in the OpenCL metadata
+along with other information. 
+
+The IDs used to encode the OpenCL's logical address spaces in the argument info
+metadata follows the SPIR address space mapping as defined in the SPIR
+specification `section 2.2
+<https://www.khronos.org/registry/spir/specs/spir_spec-2.0.pdf#18>`_
+
+OpenCL-Specific Attributes
+--------------------------
+
+OpenCL support in Clang contains a set of attribute taken directly from the
+specification as well as additional attributes.
+
+See also :doc:`AttributeReference`.
+
+nosvm
+^^^^^
+
+Clang supports this attribute to comply to OpenCL v2.0 conformance, but it
+does not have any effect on the IR. For more details reffer to the specification
+`section 6.7.2
+<https://www.khronos.org/registry/cl/specs/opencl-2.0-openclc.pdf#49>`_
+
+
+opencl_hint_unroll
+^^^^^^^^^^^^^^^^^^
+
+The implementation of this feature mirrors the unroll hint for C.
+More details on the syntax can be found in the specification
+`section 6.11.5
+<https://www.khronos.org/registry/cl/specs/opencl-2.0-openclc.pdf#61>`_
+
+convergent
+^^^^^^^^^^
+
+To make sure no invalid optimizations occur for single program multiple data
+(SPMD) / single instruction multiple thread (SIMT) Clang provides attributes that
+can be used for special functions that have cross work item semantics.
+An example is the subgroup operations such as `intel_sub_group_shuffle 
+<https://www.khronos.org/registry/cl/extensions/intel/cl_intel_subgroups.txt>`_
+
+   .. code-block:: c
+
+     // Define custom my_sub_group_shuffle(data, c)
+     // that makes use of intel_sub_group_shuffle
+     r1 = … 
+     if (r0) r1 = computeA();
+     // Shuffle data from r1 into r3
+     // of threads id r2.
+     r3 = my_sub_group_shuffle(r1, r2);
+     if (r0) r3 = computeB();
+
+with non-SPMD semantics this is optimized to the following equivalent code:
+
+   .. code-block:: c
+
+     r1 = …
+     if (!r0)
+       // Incorrect functionality! The data in r1
+       // have not been computed by all threads yet.
+       r3 = my_sub_group_shuffle(r1, r2);
+     else {
+       r1 = computeA();
+       r3 = my_sub_group_shuffle(r1, r2);
+       r3 = computeB();
+     }
+
+Declaring the function ``my_sub_group_shuffle`` with the convergent attribute
+would prevent this:
+
+   .. code-block:: c
+
+     my_sub_group_shuffle() __attribute__((convergent));
+
+Using ``convergent`` guarantees correct execution by keeping CFG equivalence
+wrt operations marked as ``convergent``. CFG ``G´`` is equivalent to ``G`` wrt
+node ``Ni`` : ``iff ∀ Nj (i≠j)`` domination and post-domination relations with
+respect to ``Ni`` remain the same in both ``G`` and ``G´``. 
+
+noduplicate
+^^^^^^^^^^^
+
+``noduplicate`` is more restrictive with respect to optimizations than
+``convergent`` because a convergent function only preserves CFG equivalence.
+This allows some optimizations to happen as long as the control flow remains
+unmodified.
+
+   .. code-block:: c
+
+     for (int i=0; i<4; i++)
+       my_sub_group_shuffle()
+
+can be modified to:
+
+   .. code-block:: c
+
+     my_sub_group_shuffle();
+     my_sub_group_shuffle();
+     my_sub_group_shuffle();
+     my_sub_group_shuffle();
+
+while using ``noduplicate`` would disallow this. Also ``noduplicate`` doesn't
+have the same safe semantics of CFG as ``convergent`` and can cause changes in
+CFG that modify semantics of the original program.
+
+``noduplicate`` is kept for backwards compatibility only and it considered to be
+deprecated for future uses.
+
+.. _opencl_addrsp:
+
+address_space
+^^^^^^^^^^^^^
+
+Clang has arbitrary address space support using the ``address_space(N)``
+attribute, where ``N`` is an integer number in the range ``0`` to ``16777215``
+(``0xffffffu``).
+
+An OpenCL implementation provides a list of standard address spaces using
+keywords: ``private``, ``local``, ``global``, and ``generic``. In the AST and
+in the IR local, global, or generic will be represented by the address space
+attribute with the corresponding unique number. Note that private does not have
+any corresponding attribute added and, therefore, is represented by the absence
+of an address space number. The specific IDs for an address space do not have to
+match between the AST and the IR. Typically in the AST address space numbers
+represent logical segments while in the IR they represent physical segments.
+Therefore, machines with flat memory segments can map all AST address space
+numbers to the same physical segment ID or skip address space attribute
+completely while generating the IR. However, if the address space information
+is needed by the IR passes e.g. to improve alias analysis, it is recommended
+to keep it and only lower to reflect physical memory segments in the late
+machine passes.
+
+OpenCL builtins
+---------------
+
+There are some standard OpenCL functions that are implemented as Clang builtins:
+
+- All pipe functions from `section 6.13.16.2/6.13.16.3
+  <https://www.khronos.org/registry/cl/specs/opencl-2.0-openclc.pdf#160>`_ of
+  the OpenCL v2.0 kernel language specification. `
+
+- Address space qualifier conversion functions ``to_global``/``to_local``/``to_private``
+  from `section 6.13.9
+  <https://www.khronos.org/registry/cl/specs/opencl-2.0-openclc.pdf#101>`_.
+
+- All the ``enqueue_kernel`` functions from `section 6.13.17.1
+  <https://www.khronos.org/registry/cl/specs/opencl-2.0-openclc.pdf#164>`_ and
+  enqueue query functions from `section 6.13.17.5
+  <https://www.khronos.org/registry/cl/specs/opencl-2.0-openclc.pdf#171>`_.
+
 .. _target_features:
 
 Target-Specific Features and Limitations

www/index.html

   <h1>clang: a C language family frontend for LLVM</h1>
   <!--*********************************************************************-->
   
-  <p>The goal of the Clang project is to create a new C, C++, Objective C and
-  Objective C++ front-end for the <a href="http://www.llvm.org/">LLVM</a>
-  compiler.  You can <a href="get_started.html">get and build</a> the source
-  today.</p>
+  <p>The goal of the Clang project is to create a new C based language
+  front-end: C, C++, Objective C/C++, OpenCL C and others for the
+  <a href="http://www.llvm.org/">LLVM</a> compiler.  You can
+  <a href="get_started.html">get and build</a> the source  today.</p>
   
   <!--=====================================================================-->
   <h2 id="goals">Features and Goals</h2>