[OpenCL] Mark group functions as convergent in opencl-c.h

[yaxunl]

Summary

Certain OpenCL builtin functions are supposed to be executed by all threads in a work group or sub group. Such functions should not be made divergent during transformation. It makes sense to mark them with convergent attribute.

The adding of convergent attribute is based on Ettore Speziale's work and the original proposal and patch can be found at https://www.mail-archive.com/[e-mail]/msg22271.html.

A test is added to demonstrate that convergent attribute prevents invalid transformations and allows valid loop unrolling whereas noduplicate does not.

[arsenm]

These should be convergent instead

[b-sumner]

clang does not recognize convergent as a valid attribute. There was an attempt to add this, see https://www.mail-archive.com/[e-mail]/msg22271.html but it hasn't had any result. Matt do you see "real uses" for this now?

[yaxunl]

In D25343#564092, @arsenm wrote:

These should be convergent instead

Unfortunately convergent is not supported as Clang attribute. There was patch to add it but the author withdrew it.

Anastasia/Alexey,

Do you think it is a good idea to add attribute((convergent)) to clang and mark these group functions with it?

convergent put more restrictions on what transformations are allowed, whereas noduplicate only forbids duplicate of the function calls.

Thanks.

[Anastasia]

Do you have any code example where Clang/LLVM performs wrong optimizations with respect to the control flow of SPMD execution?

My understanding from the earlier discussion we have had: https://www.mail-archive.com/[e-mail]/msg22643.html that noduplicate is essentially enough for the frontend to prevent erroneous optimizations. Because in general compiler can't do much with unknown function calls.

For LLVM intrinsics it is slightly different as I can deduce from this discussion:http://lists.llvm.org/pipermail/llvm-dev/2015-May/085558.html . It seems like by default it's assumed to be side effect free and can be optimized in various ways.

[• tstellarAMD]

In D25343#565288, @Anastasia wrote:

Do you have any code example where Clang/LLVM performs wrong optimizations with respect to the control flow of SPMD execution?

My understanding from the earlier discussion we have had: https://www.mail-archive.com/[e-mail]/msg22643.html that noduplicate is essentially enough for the frontend to prevent erroneous optimizations. Because in general compiler can't do much with unknown function calls.

noduplicate is enough for correctness, but it prevents legal optimizations, like unrolling loops with barriers. The convergent attribute was added specifically for these kinds of builtins, so we should be using it here instead of noduplicate.

For LLVM intrinsics it is slightly different as I can deduce from this discussion:http://lists.llvm.org/pipermail/llvm-dev/2015-May/085558.html . It seems like by default it's assumed to be side effect free and can be optimized in various ways.

[aaron.ballman]

include/clang/Basic/AttrDocs.td

612 ↗

(On Diff #74394)

on a function declaration

613 ↗

(On Diff #74394)

s/to/into the
s/the call/that the call

621 ↗

(On Diff #74394)

s/since/because

629 ↗

(On Diff #74394)

Since this is a C block, perhaps that should be void convfunc(void) instead?

test/CodeGenOpenCL/convergent.cl

118 ↗

(On Diff #74394)

Missing the Sema tests for the attribute's semantics (applies only to functions, accepts no args, etc).

[yaxunl]

Revised by Aaron's comments. Added a sema test.

[aaron.ballman]

Aside from some minor nits, the attribute functionality looks fine to me. As to whether we think this is a worthy attribute to add or not, I leave that to people who know CUDA and OpenCL better than I do, so please wait for @tstellarAMD or @Anastasia to sign off before committing.

include/clang/Basic/AttrDocs.td

612 ↗

(On Diff #74394)

"declarations" should be singular (declaration).

test/SemaOpenCL/convergent.cl

1 ↗

(On Diff #74514)

Should add -fsyntax-only to the test.

[yaxunl]

Minor revision by Aaron's comments.

[Anastasia]

In D25343#567374, @tstellarAMD wrote:

In D25343#565288, @Anastasia wrote:

Do you have any code example where Clang/LLVM performs wrong optimizations with respect to the control flow of SPMD execution?

My understanding from the earlier discussion we have had: https://www.mail-archive.com/[e-mail]/msg22643.html that noduplicate is essentially enough for the frontend to prevent erroneous optimizations. Because in general compiler can't do much with unknown function calls.

noduplicate is enough for correctness, but it prevents legal optimizations, like unrolling loops with barriers. The convergent attribute was added specifically for these kinds of builtins, so we should be using it here instead of noduplicate.

Should we deprecate noduplicate then as convergent should cover both use cases for OpenCL I believe? As far as I understand noduplicate was added specifically for SPMD use cases...

include/clang/Basic/AttrDocs.td

630 ↗

(On Diff #74541)

Did you mean "in an OpenCL program"?

test/CodeGenOpenCL/convergent.cl

17 ↗

(On Diff #74541)

-> non_convfun();

[yaxunl]

Fix typo in test.

[yaxunl]

Should we deprecate noduplicate then as convergent should cover both use cases for OpenCL I believe? As far as I understand noduplicate was added specifically for SPMD use cases...

noduplicate has different semantics than convergent. Although it is proposed for SPMD originally, it could be used by non-SPMD programs to forbid duplicate of functions. There may be applications using this attribute.

I would suggest to leave this question for future. Probably ask llvm-dev first since the attribute is also in LLVM.

include/clang/Basic/AttrDocs.td

630 ↗

(On Diff #74541)

No.
In C++ program only, it is also valid to set it as C++11 attribute.
OpenCL does not support C++11 attribute.

[Anastasia]

In D25343#567374, @tstellarAMD wrote:

In D25343#565288, @Anastasia wrote:

Do you have any code example where Clang/LLVM performs wrong optimizations with respect to the control flow of SPMD execution?

My understanding from the earlier discussion we have had: https://www.mail-archive.com/[e-mail]/msg22643.html that noduplicate is essentially enough for the frontend to prevent erroneous optimizations. Because in general compiler can't do much with unknown function calls.

noduplicate is enough for correctness, but it prevents legal optimizations, like unrolling loops with barriers. The convergent attribute was added specifically for these kinds of builtins, so we should be using it here instead of noduplicate.

For LLVM intrinsics it is slightly different as I can deduce from this discussion:http://lists.llvm.org/pipermail/llvm-dev/2015-May/085558.html . It seems like by default it's assumed to be side effect free and can be optimized in various ways.

In D25343#567374, @tstellarAMD wrote:

In D25343#565288, @Anastasia wrote:

Do you have any code example where Clang/LLVM performs wrong optimizations with respect to the control flow of SPMD execution?

My understanding from the earlier discussion we have had: https://www.mail-archive.com/[e-mail]/msg22643.html that noduplicate is essentially enough for the frontend to prevent erroneous optimizations. Because in general compiler can't do much with unknown function calls.

noduplicate is enough for correctness, but it prevents legal optimizations, like unrolling loops with barriers. The convergent attribute was added specifically for these kinds of builtins, so we should be using it here instead of noduplicate.

For LLVM intrinsics it is slightly different as I can deduce from this discussion:http://lists.llvm.org/pipermail/llvm-dev/2015-May/085558.html . It seems like by default it's assumed to be side effect free and can be optimized in various ways.

Tom, as far as I understand a valid generalized use case with a barrier is as follows:

a = compute() 
barrier()
use(a)

Regarding unrolling I don't see how the barrier can be unrolled without breaking the correctness of OpenCL programs because all compute() calls of one WG have to complete before use(a) is invoked for the first time. I am not an expert in LLVM transformations but if I read the description of both noduplicate and convergent (http://llvm.org/docs/LangRef.html) none of those seem to make sense to me for the barrier because both allow reordering within the same function or BB respectively and the only valid way seems to mark it as a side-effect intrinsic to prevent any reordering of the barrier itself. I understand within compute() or use() independently there are not limitations on ordering of instructions.

If you have any specific example in mind where the unrolling would work could you please elaborate on it. I think it's essential for the community to undertsnad the use cases for noduplicate/convergent/sideeffect to make better use of them also across similar models i.e. OpenCL, CUDA, etc.

[Anastasia]

LGTM! Thanks! Could you please address the last comment before committing?

test/CodeGenOpenCL/convergent.cl

54 ↗

(On Diff #75203)

Did you mean to check @convfun() here?

[Anastasia]

Additional discussions: http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20161024/175373.html

[yaxunl]

test/CodeGenOpenCL/convergent.cl

54 ↗

(On Diff #75203)

Yes. Will fix it when committing.

cfe/trunk/include/clang/Basic/Attr.td

 }
 
 def NoDuplicate : InheritableAttr {
   let Spellings = [GNU<"noduplicate">, CXX11<"clang", "noduplicate">];
   let Subjects = SubjectList<[Function]>;
   let Documentation = [NoDuplicateDocs];
 }
 
+def Convergent : InheritableAttr {
+  let Spellings = [GNU<"convergent">, CXX11<"clang", "convergent">];
+  let Subjects = SubjectList<[Function]>;
+  let Documentation = [ConvergentDocs];
+}
+
 def NoInline : InheritableAttr {
   let Spellings = [GCC<"noinline">, Declspec<"noinline">];
   let Subjects = SubjectList<[Function]>;
   let Documentation = [Undocumented];
 }
 
 def NoMips16 : InheritableAttr, TargetSpecificAttr<TargetMips> {
   let Spellings = [GCC<"nomips16">];

cfe/trunk/include/clang/Basic/AttrDocs.td

if (a > n) {
     bar();
   }
 
 where the call to "nodupfunc" is duplicated and sunk into the two branches
 of the condition.
   }];
 }
 
+def ConvergentDocs : Documentation {
+  let Category = DocCatFunction;
+  let Content = [{
+The ``convergent`` attribute can be placed on a function declaration. It is
+translated into the LLVM ``convergent`` attribute, which indicates that the call
+instructions of a function with this attribute cannot be made control-dependent
+on any additional values.
+
+In languages designed for SPMD/SIMT programming model, e.g. OpenCL or CUDA,
+the call instructions of a function with this attribute must be executed by
+all work items or threads in a work group or sub group.
+
+This attribute is different from ``noduplicate`` because it allows duplicating
+function calls if it can be proved that the duplicated function calls are
+not made control-dependent on any additional values, e.g., unrolling a loop
+executed by all work items.
+
+Sample usage:
+.. code-block:: c
+
+  void convfunc(void) __attribute__((convergent));
+  // Setting it as a C++11 attribute is also valid in a C++ program.
+  // void convfunc(void) [[clang::convergent]];
+
+  }];
+}
+
 def NoSplitStackDocs : Documentation {
   let Category = DocCatFunction;
   let Content = [{
 The ``no_split_stack`` attribute disables the emission of the split stack
 preamble for a particular function. It has no effect if ``-fsplit-stack``
 is not specified.
   }];
 }

cfe/trunk/lib/CodeGen/CGCall.cpp

if (TargetDecl) {
     if (TargetDecl->hasAttr<ReturnsTwiceAttr>())
       FuncAttrs.addAttribute(llvm::Attribute::ReturnsTwice);
     if (TargetDecl->hasAttr<NoThrowAttr>())
       FuncAttrs.addAttribute(llvm::Attribute::NoUnwind);
     if (TargetDecl->hasAttr<NoReturnAttr>())
       FuncAttrs.addAttribute(llvm::Attribute::NoReturn);
     if (TargetDecl->hasAttr<NoDuplicateAttr>())
       FuncAttrs.addAttribute(llvm::Attribute::NoDuplicate);
+    if (TargetDecl->hasAttr<ConvergentAttr>())
+      FuncAttrs.addAttribute(llvm::Attribute::Convergent);
 
     if (const FunctionDecl *Fn = dyn_cast<FunctionDecl>(TargetDecl)) {
       AddAttributesFromFunctionProtoType(
           getContext(), FuncAttrs, Fn->getType()->getAs<FunctionProtoType>());
       // Don't use [[noreturn]] or _Noreturn for a call to a virtual function.
       // These attributes are not inherited by overloads.
       const CXXMethodDecl *MD = dyn_cast<CXXMethodDecl>(Fn);
       if (Fn->isNoReturn() && !(AttrOnCallSite && MD && MD->isVirtual()))

cfe/trunk/lib/Headers/opencl-c.h

 
 #if __OPENCL_C_VERSION__ >= CL_VERSION_2_0
 #ifndef cl_khr_depth_images
 #define cl_khr_depth_images
 #endif //cl_khr_depth_images
 #endif //__OPENCL_C_VERSION__ >= CL_VERSION_2_0
 
 #define __ovld __attribute__((overloadable))
+#define __conv __attribute__((convergent))
 
 // Optimizations
 #define __purefn __attribute__((pure))
 #define __cnfn __attribute__((const))
 
 // built-in scalar data types:
 
 /**

  * memory operations to local memory.
  * CLK_GLOBAL_MEM_FENCE - The barrier function
  * will queue a memory fence to ensure correct ordering
  * of memory operations to global memory. This can be
  * useful when work-items, for example, write to buffer or
  * image objects and then want to read the updated data.
  */
 
-void __ovld barrier(cl_mem_fence_flags flags);
+void __ovld __conv barrier(cl_mem_fence_flags flags);
 
 #if __OPENCL_C_VERSION__ >= CL_VERSION_2_0
 
 typedef enum memory_scope
 {
   memory_scope_work_item,
   memory_scope_work_group,
   memory_scope_device,
   memory_scope_all_svm_devices,
   memory_scope_sub_group
 } memory_scope;
 
-void __ovld work_group_barrier(cl_mem_fence_flags flags, memory_scope scope);
-void __ovld work_group_barrier(cl_mem_fence_flags flags);
+void __ovld __conv work_group_barrier(cl_mem_fence_flags flags, memory_scope scope);
+void __ovld __conv work_group_barrier(cl_mem_fence_flags flags);
 #endif //__OPENCL_C_VERSION__ >= CL_VERSION_2_0
 
 // OpenCL v1.1 s6.11.9, v1.2 s6.12.9 - Explicit Memory Fence Functions
 
 /**
  * Orders loads and stores of a work-item
  * executing a kernel. This means that loads
  * and stores preceding the mem_fence will

 int __ovld get_image_num_samples(read_write image2d_array_msaa_t image);
 int __ovld get_image_num_samples(read_write image2d_array_msaa_depth_t image);
 #endif //__OPENCL_C_VERSION__ >= CL_VERSION_2_0
 #endif
 
 // OpenCL v2.0 s6.13.15 - Work-group Functions
 
 #if __OPENCL_C_VERSION__ >= CL_VERSION_2_0
-int __ovld work_group_all(int predicate);
-int __ovld work_group_any(int predicate);
+int __ovld __conv work_group_all(int predicate);
+int __ovld __conv work_group_any(int predicate);
 
 #ifdef cl_khr_fp16
-half __ovld work_group_broadcast(half a, size_t local_id);
-half __ovld work_group_broadcast(half a, size_t x, size_t y);
-half __ovld work_group_broadcast(half a, size_t x, size_t y, size_t z);
+half __ovld __conv work_group_broadcast(half a, size_t local_id);
+half __ovld __conv work_group_broadcast(half a, size_t x, size_t y);
+half __ovld __conv work_group_broadcast(half a, size_t x, size_t y, size_t z);
 #endif
-int __ovld work_group_broadcast(int a, size_t local_id);
-int __ovld work_group_broadcast(int a, size_t x, size_t y);
-int __ovld work_group_broadcast(int a, size_t x, size_t y, size_t z);
-uint __ovld work_group_broadcast(uint a, size_t local_id);
-uint __ovld work_group_broadcast(uint a, size_t x, size_t y);
-uint __ovld work_group_broadcast(uint a, size_t x, size_t y, size_t z);
-long __ovld work_group_broadcast(long a, size_t local_id);
-long __ovld work_group_broadcast(long a, size_t x, size_t y);
-long __ovld work_group_broadcast(long a, size_t x, size_t y, size_t z);
-ulong __ovld work_group_broadcast(ulong a, size_t local_id);
-ulong __ovld work_group_broadcast(ulong a, size_t x, size_t y);
-ulong __ovld work_group_broadcast(ulong a, size_t x, size_t y, size_t z);
-float __ovld work_group_broadcast(float a, size_t local_id);
-float __ovld work_group_broadcast(float a, size_t x, size_t y);
-float __ovld work_group_broadcast(float a, size_t x, size_t y, size_t z);
-#ifdef cl_khr_fp64
-double __ovld work_group_broadcast(double a, size_t local_id);
-double __ovld work_group_broadcast(double a, size_t x, size_t y);
-double __ovld work_group_broadcast(double a, size_t x, size_t y, size_t z);
+int __ovld __conv work_group_broadcast(int a, size_t local_id);
+int __ovld __conv work_group_broadcast(int a, size_t x, size_t y);
+int __ovld __conv work_group_broadcast(int a, size_t x, size_t y, size_t z);
+uint __ovld __conv work_group_broadcast(uint a, size_t local_id);
+uint __ovld __conv work_group_broadcast(uint a, size_t x, size_t y);
+uint __ovld __conv work_group_broadcast(uint a, size_t x, size_t y, size_t z);
+long __ovld __conv work_group_broadcast(long a, size_t local_id);
+long __ovld __conv work_group_broadcast(long a, size_t x, size_t y);
+long __ovld __conv work_group_broadcast(long a, size_t x, size_t y, size_t z);
+ulong __ovld __conv work_group_broadcast(ulong a, size_t local_id);
+ulong __ovld __conv work_group_broadcast(ulong a, size_t x, size_t y);
+ulong __ovld __conv work_group_broadcast(ulong a, size_t x, size_t y, size_t z);
+float __ovld __conv work_group_broadcast(float a, size_t local_id);
+float __ovld __conv work_group_broadcast(float a, size_t x, size_t y);
+float __ovld __conv work_group_broadcast(float a, size_t x, size_t y, size_t z);
+#ifdef cl_khr_fp64
+double __ovld __conv work_group_broadcast(double a, size_t local_id);
+double __ovld __conv work_group_broadcast(double a, size_t x, size_t y);
+double __ovld __conv work_group_broadcast(double a, size_t x, size_t y, size_t z);
 #endif //cl_khr_fp64
 
 #ifdef cl_khr_fp16
-half __ovld work_group_reduce_add(half x);
-half __ovld work_group_reduce_min(half x);
-half __ovld work_group_reduce_max(half x);
-half __ovld work_group_scan_exclusive_add(half x);
-half __ovld work_group_scan_exclusive_min(half x);
-half __ovld work_group_scan_exclusive_max(half x);
-half __ovld work_group_scan_inclusive_add(half x);
-half __ovld work_group_scan_inclusive_min(half x);
-half __ovld work_group_scan_inclusive_max(half x);
+half __ovld __conv work_group_reduce_add(half x);
+half __ovld __conv work_group_reduce_min(half x);
+half __ovld __conv work_group_reduce_max(half x);
+half __ovld __conv work_group_scan_exclusive_add(half x);
+half __ovld __conv work_group_scan_exclusive_min(half x);
+half __ovld __conv work_group_scan_exclusive_max(half x);
+half __ovld __conv work_group_scan_inclusive_add(half x);
+half __ovld __conv work_group_scan_inclusive_min(half x);
+half __ovld __conv work_group_scan_inclusive_max(half x);
 #endif
-int __ovld work_group_reduce_add(int x);
-int __ovld work_group_reduce_min(int x);
-int __ovld work_group_reduce_max(int x);
-int __ovld work_group_scan_exclusive_add(int x);
-int __ovld work_group_scan_exclusive_min(int x);
-int __ovld work_group_scan_exclusive_max(int x);
-int __ovld work_group_scan_inclusive_add(int x);
-int __ovld work_group_scan_inclusive_min(int x);
-int __ovld work_group_scan_inclusive_max(int x);
-uint __ovld work_group_reduce_add(uint x);
-uint __ovld work_group_reduce_min(uint x);
-uint __ovld work_group_reduce_max(uint x);
-uint __ovld work_group_scan_exclusive_add(uint x);
-uint __ovld work_group_scan_exclusive_min(uint x);
-uint __ovld work_group_scan_exclusive_max(uint x);
-uint __ovld work_group_scan_inclusive_add(uint x);
-uint __ovld work_group_scan_inclusive_min(uint x);
-uint __ovld work_group_scan_inclusive_max(uint x);
-long __ovld work_group_reduce_add(long x);
-long __ovld work_group_reduce_min(long x);
-long __ovld work_group_reduce_max(long x);
-long __ovld work_group_scan_exclusive_add(long x);
-long __ovld work_group_scan_exclusive_min(long x);
-long __ovld work_group_scan_exclusive_max(long x);
-long __ovld work_group_scan_inclusive_add(long x);
-long __ovld work_group_scan_inclusive_min(long x);
-long __ovld work_group_scan_inclusive_max(long x);
-ulong __ovld work_group_reduce_add(ulong x);
-ulong __ovld work_group_reduce_min(ulong x);
-ulong __ovld work_group_reduce_max(ulong x);
-ulong __ovld work_group_scan_exclusive_add(ulong x);
-ulong __ovld work_group_scan_exclusive_min(ulong x);
-ulong __ovld work_group_scan_exclusive_max(ulong x);
-ulong __ovld work_group_scan_inclusive_add(ulong x);
-ulong __ovld work_group_scan_inclusive_min(ulong x);
-ulong __ovld work_group_scan_inclusive_max(ulong x);
-float __ovld work_group_reduce_add(float x);
-float __ovld work_group_reduce_min(float x);
-float __ovld work_group_reduce_max(float x);
-float __ovld work_group_scan_exclusive_add(float x);
-float __ovld work_group_scan_exclusive_min(float x);
-float __ovld work_group_scan_exclusive_max(float x);
-float __ovld work_group_scan_inclusive_add(float x);
-float __ovld work_group_scan_inclusive_min(float x);
-float __ovld work_group_scan_inclusive_max(float x);
-#ifdef cl_khr_fp64
-double __ovld work_group_reduce_add(double x);
-double __ovld work_group_reduce_min(double x);
-double __ovld work_group_reduce_max(double x);
-double __ovld work_group_scan_exclusive_add(double x);
-double __ovld work_group_scan_exclusive_min(double x);
-double __ovld work_group_scan_exclusive_max(double x);
-double __ovld work_group_scan_inclusive_add(double x);
-double __ovld work_group_scan_inclusive_min(double x);
-double __ovld work_group_scan_inclusive_max(double x);
+int __ovld __conv work_group_reduce_add(int x);
+int __ovld __conv work_group_reduce_min(int x);
+int __ovld __conv work_group_reduce_max(int x);
+int __ovld __conv work_group_scan_exclusive_add(int x);
+int __ovld __conv work_group_scan_exclusive_min(int x);
+int __ovld __conv work_group_scan_exclusive_max(int x);
+int __ovld __conv work_group_scan_inclusive_add(int x);
+int __ovld __conv work_group_scan_inclusive_min(int x);
+int __ovld __conv work_group_scan_inclusive_max(int x);
+uint __ovld __conv work_group_reduce_add(uint x);
+uint __ovld __conv work_group_reduce_min(uint x);
+uint __ovld __conv work_group_reduce_max(uint x);
+uint __ovld __conv work_group_scan_exclusive_add(uint x);
+uint __ovld __conv work_group_scan_exclusive_min(uint x);
+uint __ovld __conv work_group_scan_exclusive_max(uint x);
+uint __ovld __conv work_group_scan_inclusive_add(uint x);
+uint __ovld __conv work_group_scan_inclusive_min(uint x);
+uint __ovld __conv work_group_scan_inclusive_max(uint x);
+long __ovld __conv work_group_reduce_add(long x);
+long __ovld __conv work_group_reduce_min(long x);
+long __ovld __conv work_group_reduce_max(long x);
+long __ovld __conv work_group_scan_exclusive_add(long x);
+long __ovld __conv work_group_scan_exclusive_min(long x);
+long __ovld __conv work_group_scan_exclusive_max(long x);
+long __ovld __conv work_group_scan_inclusive_add(long x);
+long __ovld __conv work_group_scan_inclusive_min(long x);
+long __ovld __conv work_group_scan_inclusive_max(long x);
+ulong __ovld __conv work_group_reduce_add(ulong x);
+ulong __ovld __conv work_group_reduce_min(ulong x);
+ulong __ovld __conv work_group_reduce_max(ulong x);
+ulong __ovld __conv work_group_scan_exclusive_add(ulong x);
+ulong __ovld __conv work_group_scan_exclusive_min(ulong x);
+ulong __ovld __conv work_group_scan_exclusive_max(ulong x);
+ulong __ovld __conv work_group_scan_inclusive_add(ulong x);
+ulong __ovld __conv work_group_scan_inclusive_min(ulong x);
+ulong __ovld __conv work_group_scan_inclusive_max(ulong x);
+float __ovld __conv work_group_reduce_add(float x);
+float __ovld __conv work_group_reduce_min(float x);
+float __ovld __conv work_group_reduce_max(float x);
+float __ovld __conv work_group_scan_exclusive_add(float x);
+float __ovld __conv work_group_scan_exclusive_min(float x);
+float __ovld __conv work_group_scan_exclusive_max(float x);
+float __ovld __conv work_group_scan_inclusive_add(float x);
+float __ovld __conv work_group_scan_inclusive_min(float x);
+float __ovld __conv work_group_scan_inclusive_max(float x);
+#ifdef cl_khr_fp64
+double __ovld __conv work_group_reduce_add(double x);
+double __ovld __conv work_group_reduce_min(double x);
+double __ovld __conv work_group_reduce_max(double x);
+double __ovld __conv work_group_scan_exclusive_add(double x);
+double __ovld __conv work_group_scan_exclusive_min(double x);
+double __ovld __conv work_group_scan_exclusive_max(double x);
+double __ovld __conv work_group_scan_inclusive_add(double x);
+double __ovld __conv work_group_scan_inclusive_min(double x);
+double __ovld __conv work_group_scan_inclusive_max(double x);
 #endif //cl_khr_fp64
 
 #endif //__OPENCL_C_VERSION__ >= CL_VERSION_2_0
 
 // OpenCL v2.0 s6.13.16 - Pipe Functions
 #if __OPENCL_C_VERSION__ >= CL_VERSION_2_0
 #define PIPE_RESERVE_ID_VALID_BIT (1U << 30)
 #define CLK_NULL_RESERVE_ID (__builtin_astype(((void*)(__SIZE_MAX__)), reserve_id_t))

 uint    __ovld get_max_sub_group_size(void);
 uint    __ovld get_num_sub_groups(void);
 #if __OPENCL_C_VERSION__ >= CL_VERSION_2_0
 uint    __ovld get_enqueued_num_sub_groups(void);
 #endif //__OPENCL_C_VERSION__ >= CL_VERSION_2_0
 uint    __ovld get_sub_group_id(void);
 uint    __ovld get_sub_group_local_id(void);
 
-void    __ovld sub_group_barrier(cl_mem_fence_flags flags);
+void    __ovld __conv sub_group_barrier(cl_mem_fence_flags flags);
 #if __OPENCL_C_VERSION__ >= CL_VERSION_2_0
-void    __ovld sub_group_barrier(cl_mem_fence_flags flags, memory_scope scope);
+void    __ovld __conv sub_group_barrier(cl_mem_fence_flags flags, memory_scope scope);
 #endif //__OPENCL_C_VERSION__ >= CL_VERSION_2_0
 
-int     __ovld sub_group_all(int predicate);
-int     __ovld sub_group_any(int predicate);
+int     __ovld __conv sub_group_all(int predicate);
+int     __ovld __conv sub_group_any(int predicate);
 
-int     __ovld sub_group_broadcast(int   x, uint sub_group_local_id);
-uint    __ovld sub_group_broadcast(uint  x, uint sub_group_local_id);
-long    __ovld sub_group_broadcast(long  x, uint sub_group_local_id);
-ulong   __ovld sub_group_broadcast(ulong x, uint sub_group_local_id);
-float   __ovld sub_group_broadcast(float x, uint sub_group_local_id);
-
-int     __ovld sub_group_reduce_add(int   x);
-uint    __ovld sub_group_reduce_add(uint  x);
-long    __ovld sub_group_reduce_add(long  x);
-ulong   __ovld sub_group_reduce_add(ulong x);
-float   __ovld sub_group_reduce_add(float x);
-int     __ovld sub_group_reduce_min(int   x);
-uint    __ovld sub_group_reduce_min(uint  x);
-long    __ovld sub_group_reduce_min(long  x);
-ulong   __ovld sub_group_reduce_min(ulong x);
-float   __ovld sub_group_reduce_min(float x);
-int     __ovld sub_group_reduce_max(int   x);
-uint    __ovld sub_group_reduce_max(uint  x);
-long    __ovld sub_group_reduce_max(long  x);
-ulong   __ovld sub_group_reduce_max(ulong x);
-float   __ovld sub_group_reduce_max(float x);
-
-int     __ovld sub_group_scan_exclusive_add(int   x);
-uint    __ovld sub_group_scan_exclusive_add(uint  x);
-long    __ovld sub_group_scan_exclusive_add(long  x);
-ulong   __ovld sub_group_scan_exclusive_add(ulong x);
-float   __ovld sub_group_scan_exclusive_add(float x);
-int     __ovld sub_group_scan_exclusive_min(int   x);
-uint    __ovld sub_group_scan_exclusive_min(uint  x);
-long    __ovld sub_group_scan_exclusive_min(long  x);
-ulong   __ovld sub_group_scan_exclusive_min(ulong x);
-float   __ovld sub_group_scan_exclusive_min(float x);
-int     __ovld sub_group_scan_exclusive_max(int   x);
-uint    __ovld sub_group_scan_exclusive_max(uint  x);
-long    __ovld sub_group_scan_exclusive_max(long  x);
-ulong   __ovld sub_group_scan_exclusive_max(ulong x);
-float   __ovld sub_group_scan_exclusive_max(float x);
-
-int     __ovld sub_group_scan_inclusive_add(int   x);
-uint    __ovld sub_group_scan_inclusive_add(uint  x);
-long    __ovld sub_group_scan_inclusive_add(long  x);
-ulong   __ovld sub_group_scan_inclusive_add(ulong x);
-float   __ovld sub_group_scan_inclusive_add(float x);
-int     __ovld sub_group_scan_inclusive_min(int   x);
-uint    __ovld sub_group_scan_inclusive_min(uint  x);
-long    __ovld sub_group_scan_inclusive_min(long  x);
-ulong   __ovld sub_group_scan_inclusive_min(ulong x);
-float   __ovld sub_group_scan_inclusive_min(float x);
-int     __ovld sub_group_scan_inclusive_max(int   x);
-uint    __ovld sub_group_scan_inclusive_max(uint  x);
-long    __ovld sub_group_scan_inclusive_max(long  x);
-ulong   __ovld sub_group_scan_inclusive_max(ulong x);
-float   __ovld sub_group_scan_inclusive_max(float x);
-
-#ifdef cl_khr_fp16
-half    __ovld sub_group_broadcast(half x, uint sub_group_local_id);
-half    __ovld sub_group_reduce_add(half x);
-half    __ovld sub_group_reduce_min(half x);
-half    __ovld sub_group_reduce_max(half x);
-half    __ovld sub_group_scan_exclusive_add(half x);
-half    __ovld sub_group_scan_exclusive_min(half x);
-half    __ovld sub_group_scan_exclusive_max(half x);
-half    __ovld sub_group_scan_inclusive_add(half x);
-half    __ovld sub_group_scan_inclusive_min(half x);
-half    __ovld sub_group_scan_inclusive_max(half x);
-#endif //cl_khr_fp16
-
-#ifdef cl_khr_fp64
-double  __ovld sub_group_broadcast(double x, uint sub_group_local_id);
-double  __ovld sub_group_reduce_add(double x);
-double  __ovld sub_group_reduce_min(double x);
-double  __ovld sub_group_reduce_max(double x);
-double  __ovld sub_group_scan_exclusive_add(double x);
-double  __ovld sub_group_scan_exclusive_min(double x);
-double  __ovld sub_group_scan_exclusive_max(double x);
-double  __ovld sub_group_scan_inclusive_add(double x);
-double  __ovld sub_group_scan_inclusive_min(double x);
-double  __ovld sub_group_scan_inclusive_max(double x);
+int     __ovld __conv sub_group_broadcast(int   x, uint sub_group_local_id);
+uint    __ovld __conv sub_group_broadcast(uint  x, uint sub_group_local_id);
+long    __ovld __conv sub_group_broadcast(long  x, uint sub_group_local_id);
+ulong   __ovld __conv sub_group_broadcast(ulong x, uint sub_group_local_id);
+float   __ovld __conv sub_group_broadcast(float x, uint sub_group_local_id);
+
+int     __ovld __conv sub_group_reduce_add(int   x);
+uint    __ovld __conv sub_group_reduce_add(uint  x);
+long    __ovld __conv sub_group_reduce_add(long  x);
+ulong   __ovld __conv sub_group_reduce_add(ulong x);
+float   __ovld __conv sub_group_reduce_add(float x);
+int     __ovld __conv sub_group_reduce_min(int   x);
+uint    __ovld __conv sub_group_reduce_min(uint  x);
+long    __ovld __conv sub_group_reduce_min(long  x);
+ulong   __ovld __conv sub_group_reduce_min(ulong x);
+float   __ovld __conv sub_group_reduce_min(float x);
+int     __ovld __conv sub_group_reduce_max(int   x);
+uint    __ovld __conv sub_group_reduce_max(uint  x);
+long    __ovld __conv sub_group_reduce_max(long  x);
+ulong   __ovld __conv sub_group_reduce_max(ulong x);
+float   __ovld __conv sub_group_reduce_max(float x);
+
+int     __ovld __conv sub_group_scan_exclusive_add(int   x);
+uint    __ovld __conv sub_group_scan_exclusive_add(uint  x);
+long    __ovld __conv sub_group_scan_exclusive_add(long  x);
+ulong   __ovld __conv sub_group_scan_exclusive_add(ulong x);
+float   __ovld __conv sub_group_scan_exclusive_add(float x);
+int     __ovld __conv sub_group_scan_exclusive_min(int   x);
+uint    __ovld __conv sub_group_scan_exclusive_min(uint  x);
+long    __ovld __conv sub_group_scan_exclusive_min(long  x);
+ulong   __ovld __conv sub_group_scan_exclusive_min(ulong x);
+float   __ovld __conv sub_group_scan_exclusive_min(float x);
+int     __ovld __conv sub_group_scan_exclusive_max(int   x);
+uint    __ovld __conv sub_group_scan_exclusive_max(uint  x);
+long    __ovld __conv sub_group_scan_exclusive_max(long  x);
+ulong   __ovld __conv sub_group_scan_exclusive_max(ulong x);
+float   __ovld __conv sub_group_scan_exclusive_max(float x);
+
+int     __ovld __conv sub_group_scan_inclusive_add(int   x);
+uint    __ovld __conv sub_group_scan_inclusive_add(uint  x);
+long    __ovld __conv sub_group_scan_inclusive_add(long  x);
+ulong   __ovld __conv sub_group_scan_inclusive_add(ulong x);
+float   __ovld __conv sub_group_scan_inclusive_add(float x);
+int     __ovld __conv sub_group_scan_inclusive_min(int   x);
+uint    __ovld __conv sub_group_scan_inclusive_min(uint  x);
+long    __ovld __conv sub_group_scan_inclusive_min(long  x);
+ulong   __ovld __conv sub_group_scan_inclusive_min(ulong x);
+float   __ovld __conv sub_group_scan_inclusive_min(float x);
+int     __ovld __conv sub_group_scan_inclusive_max(int   x);
+uint    __ovld __conv sub_group_scan_inclusive_max(uint  x);
+long    __ovld __conv sub_group_scan_inclusive_max(long  x);
+ulong   __ovld __conv sub_group_scan_inclusive_max(ulong x);
+float   __ovld __conv sub_group_scan_inclusive_max(float x);
+
+#ifdef cl_khr_fp16
+half    __ovld __conv sub_group_broadcast(half x, uint sub_group_local_id);
+half    __ovld __conv sub_group_reduce_add(half x);
+half    __ovld __conv sub_group_reduce_min(half x);
+half    __ovld __conv sub_group_reduce_max(half x);
+half    __ovld __conv sub_group_scan_exclusive_add(half x);
+half    __ovld __conv sub_group_scan_exclusive_min(half x);
+half    __ovld __conv sub_group_scan_exclusive_max(half x);
+half    __ovld __conv sub_group_scan_inclusive_add(half x);
+half    __ovld __conv sub_group_scan_inclusive_min(half x);
+half    __ovld __conv sub_group_scan_inclusive_max(half x);
+#endif //cl_khr_fp16
+
+#ifdef cl_khr_fp64
+double  __ovld __conv sub_group_broadcast(double x, uint sub_group_local_id);
+double  __ovld __conv sub_group_reduce_add(double x);
+double  __ovld __conv sub_group_reduce_min(double x);
+double  __ovld __conv sub_group_reduce_max(double x);
+double  __ovld __conv sub_group_scan_exclusive_add(double x);
+double  __ovld __conv sub_group_scan_exclusive_min(double x);
+double  __ovld __conv sub_group_scan_exclusive_max(double x);
+double  __ovld __conv sub_group_scan_inclusive_add(double x);
+double  __ovld __conv sub_group_scan_inclusive_min(double x);
+double  __ovld __conv sub_group_scan_inclusive_max(double x);
 #endif //cl_khr_fp64
 
 #endif //cl_khr_subgroups cl_intel_subgroups
 
 #ifdef cl_amd_media_ops
 uint __ovld amd_bitalign(uint a, uint b, uint c);
 uint2 __ovld amd_bitalign(uint2 a, uint2 b, uint2 c);
 uint3 __ovld amd_bitalign(uint3 a, uint3 b, uint3 c);

cfe/trunk/lib/Sema/SemaDeclAttr.cpp

case AttributeList::AT_Cleanup:
     handleCleanupAttr(S, D, Attr);
     break;
   case AttributeList::AT_NoDebug:
     handleNoDebugAttr(S, D, Attr);
     break;
   case AttributeList::AT_NoDuplicate:
     handleSimpleAttribute<NoDuplicateAttr>(S, D, Attr);
     break;
+  case AttributeList::AT_Convergent:
+    handleSimpleAttribute<ConvergentAttr>(S, D, Attr);
+    break;
   case AttributeList::AT_NoInline:
     handleSimpleAttribute<NoInlineAttr>(S, D, Attr);
     break;
   case AttributeList::AT_NoInstrumentFunction: // Interacts with -pg.
     handleSimpleAttribute<NoInstrumentFunctionAttr>(S, D, Attr);
     break;
   case AttributeList::AT_StdCall:
   case AttributeList::AT_CDecl:

cfe/trunk/test/CodeGenOpenCL/convergent.cl

+// RUN: %clang_cc1 -triple spir-unknown-unknown -emit-llvm %s -o - | FileCheck %s
+
+void convfun(void) __attribute__((convergent));
+void non_convfun(void);
+void nodupfun(void) __attribute__((noduplicate));
+
+void f(void);
+void g(void);
+
+// Test two if's are merged and non_convfun duplicated.
+// The LLVM IR is equivalent to:
+//    if (a) {
+//      f();
+//      non_convfun();
+//      g();
+//    } else {
+//      non_convfun();
+//    }
+//
+// CHECK: define spir_func void @test_merge_if(i32 %[[a:.+]])
+// CHECK: %[[tobool:.+]] = icmp eq i32 %[[a]], 0
+// CHECK: br i1 %[[tobool]], label %[[if_end3_critedge:.+]], label %[[if_then:.+]]
+// CHECK: [[if_then]]:
+// CHECK: tail call spir_func void @f()
+// CHECK: tail call spir_func void @non_convfun()
+// CHECK: tail call spir_func void @g()
+// CHECK: br label %[[if_end3:.+]]
+// CHECK: [[if_end3_critedge]]:
+// CHECK: tail call spir_func void @non_convfun()
+// CHECK: br label %[[if_end3]]
+// CHECK: [[if_end3]]:
+// CHECK-LABEL: ret void
+
+void test_merge_if(int a) {
+  if (a) {
+    f();
+  }
+  non_convfun();
+  if (a) {
+    g();
+  }
+}
+
+// CHECK-DAG: declare spir_func void @f()
+// CHECK-DAG: declare spir_func void @non_convfun()
+// CHECK-DAG: declare spir_func void @g()
+
+// Test two if's are not merged.
+// CHECK: define spir_func void @test_no_merge_if(i32 %[[a:.+]])
+// CHECK:  %[[tobool:.+]] = icmp eq i32 %[[a]], 0
+// CHECK: br i1 %[[tobool]], label %[[if_end:.+]], label %[[if_then:.+]]
+// CHECK: [[if_then]]:
+// CHECK: tail call spir_func void @f()
+// CHECK-NOT: call spir_func void @convfun()
+// CHECK-NOT: call spir_func void @g()
+// CHECK: br label %[[if_end]]
+// CHECK: [[if_end]]:
+// CHECK:  %[[tobool_pr:.+]] = phi i1 [ true, %[[if_then]] ], [ false, %{{.+}} ]
+// CHECK:  tail call spir_func void @convfun() #[[attr5:.+]]
+// CHECK:  br i1 %[[tobool_pr]], label %[[if_then2:.+]], label %[[if_end3:.+]]
+// CHECK: [[if_then2]]:
+// CHECK: tail call spir_func void @g()
+// CHECK:  br label %[[if_end3:.+]]
+// CHECK: [[if_end3]]:
+// CHECK-LABEL:  ret void
+
+void test_no_merge_if(int a) {
+  if (a) {
+    f();
+  }
+  convfun();
+  if(a) {
+    g();
+  }
+}
+
+// CHECK: declare spir_func void @convfun(){{[^#]*}} #[[attr2:[0-9]+]]
+
+// Test loop is unrolled for convergent function.
+// CHECK-LABEL: define spir_func void @test_unroll()
+// CHECK:  tail call spir_func void @convfun() #[[attr5:[0-9]+]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK:  tail call spir_func void @convfun() #[[attr5]]
+// CHECK-LABEL:  ret void
+
+void test_unroll() {
+  for (int i = 0; i < 10; i++)
+    convfun();
+}
+
+// Test loop is not unrolled for noduplicate function.
+// CHECK-LABEL: define spir_func void @test_not_unroll()
+// CHECK:  br label %[[for_body:.+]]
+// CHECK: [[for_cond_cleanup:.+]]:
+// CHECK:  ret void
+// CHECK: [[for_body]]:
+// CHECK:  tail call spir_func void @nodupfun() #[[attr6:[0-9]+]]
+// CHECK-NOT: call spir_func void @nodupfun()
+// CHECK:  br i1 %{{.+}}, label %[[for_body]], label %[[for_cond_cleanup]]
+
+void test_not_unroll() {
+  for (int i = 0; i < 10; i++)
+    nodupfun();
+}
+
+// CHECK: declare spir_func void @nodupfun(){{[^#]*}} #[[attr3:[0-9]+]]
+
+// CHECK-DAG: attributes #[[attr2]] = { {{[^}]*}}convergent{{[^}]*}} }
+// CHECK-DAG: attributes #[[attr3]] = { {{[^}]*}}noduplicate{{[^}]*}} }
+// CHECK-DAG: attributes #[[attr5]] = { {{[^}]*}}convergent{{[^}]*}} }
+// CHECK-DAG: attributes #[[attr6]] = { {{[^}]*}}noduplicate{{[^}]*}} }

cfe/trunk/test/SemaOpenCL/convergent.cl

+// RUN: %clang_cc1 -triple spir-unknown-unknown -fsyntax-only -verify %s
+
+void f1(void) __attribute__((convergent));
+
+void f2(void) __attribute__((convergent(1))); // expected-error {{'convergent' attribute takes no arguments}}
+
+void f3(int a __attribute__((convergent))); // expected-warning {{'convergent' attribute only applies to functions}}
+
+void f4(void) {
+  int var1 __attribute__((convergent)); // expected-warning {{'convergent' attribute only applies to functions}}
+}
+