Add half load and store builtins

[jvesely]

Summary

This enables load/stores of half type, without half being a legal type.

[Anastasia]

include/clang/Basic/Builtins.def

1427 ↗

(On Diff #112965)

I think this should be a language builtin (see above) but perhaps we might need to extend the language version here. Because I believe we only have OpenCL v2.0 currently.

Also this should only be available if cl_khr_fp16 is supported and enabled? I think we are doing similar with some subgroups functions (e.g. get_kernel_sub_group_count_for_ndrange) that are only supported by cl_khr_subgroup but those have custom diagnostic though. May be we could leave this check out since half is not available if cl_khr_fp16 is not enabled anyways.

test/CodeGenOpenCL/no-half.cl

3 ↗

(On Diff #112965)

It seems strange that cl_khr_fp16 is not enabled too.

[jvesely]

include/clang/Basic/Builtins.def

1427 ↗

(On Diff #112965)

This is specifically meant to be used when cl_khr_fp16 is not available.
CLC allows using half as storage format and half pointers without the extension,
vstore_half/vload_half are used to load/store half values. (CL1.2 CH 6.1.1.1)

These builtins are not necessary if cl_khr_fp16 is available (we can use regular loads/stores).

I'll take stab at making these CLC only, but similarly to device specific builtins it looked useful beyond that, since these builtins provide access to half type storage.

[jvesely]

restrict builtins to OCLC langauges

[Anastasia]

include/clang/Basic/Builtins.def

1427 ↗

(On Diff #112965)

Strange. This is not how I would interpret from the extension spec though: https://www.khronos.org/registry/OpenCL/sdk/1.2/docs/man/xhtml/cl_khr_fp16.html

But I think for this change is probably fine indeed because this doesn't affect half type itself.

[Anastasia]

LGTM! Thanks!

[Anastasia]

test/CodeGenOpenCL/no-half.cl

19 ↗

(On Diff #113190)

Would it make sense to add a check for load similarly to store in the test_load_float/test_load_double tests?

27 ↗

(On Diff #113190)

Minor thing: any reason you are not checking the load fully?

[jvesely]

fully check loads in tests

[jvesely]

include/clang/Basic/Builtins.def

1427 ↗

(On Diff #112965)

I'm not sure I see the conflict here. cl_khr_fp16 adds support for half scalar and halfn vector types.
without the extension the specs say (OCL 1.2 Ch. 6.1.1.1):

The half data type can only be used to declare a pointer to a buffer that contains half values.

vload_half and vstore_half used to access those buffers without needing half type (or the cl_khr_fp16 extension).

But I think for this change is probably fine indeed because this doesn't affect half type itself.

exactly. this is needed outside of cl_khr_fp16, or the half type

test/CodeGenOpenCL/no-half.cl

19 ↗

(On Diff #113190)

there is no load. fptrunc double %foo to half uses the function parameter directly

27 ↗

(On Diff #113190)

just my laziness, I've added full check.

[jvesely]

mark load pointers const

[jvesely]

please let me know if your accept still stands for the modified version.

[Anastasia]

test/CodeGenOpenCL/no-half.cl

27 ↗

(On Diff #113190)

Could we do the same for the above examples too?

[jvesely]

test/CodeGenOpenCL/no-half.cl

27 ↗

(On Diff #113190)

I don't understand.
if you mean test_store_*, those functions do not generate any load instructions. the full generated code is:
test_store_double:

entry:
  %0 = fptrunc double %foo to half
  store half %0, half addrspace(1)* %bar, align 2
  ret void

[Anastasia]

Yes, sorry overlooked that. :) LGTM! Thanks!

cfe/trunk/include/clang/Basic/Builtins.h

 class LangOptions;
 
 enum LanguageID {
   GNU_LANG = 0x1,     // builtin requires GNU mode.
   C_LANG = 0x2,       // builtin for c only.
   CXX_LANG = 0x4,     // builtin for cplusplus only.
   OBJC_LANG = 0x8,    // builtin for objective-c and objective-c++
   MS_LANG = 0x10,     // builtin requires MS mode.
-  OCLC20_LANG = 0x20, // builtin for OpenCL C only.
+  OCLC20_LANG = 0x20, // builtin for OpenCL C 2.0 only.
+  OCLC1X_LANG = 0x40, // builtin for OpenCL C 1.x only.
   ALL_LANGUAGES = C_LANG | CXX_LANG | OBJC_LANG, // builtin for all languages.
   ALL_GNU_LANGUAGES = ALL_LANGUAGES | GNU_LANG,  // builtin requires GNU mode.
-  ALL_MS_LANGUAGES = ALL_LANGUAGES | MS_LANG     // builtin requires MS mode.
+  ALL_MS_LANGUAGES = ALL_LANGUAGES | MS_LANG,    // builtin requires MS mode.
+  ALL_OCLC_LANGUAGES = OCLC1X_LANG | OCLC20_LANG // builtin for OCLC languages.
 };
 
 namespace Builtin {
 enum ID {
   NotBuiltin  = 0,      // This is not a builtin function.
 #define BUILTIN(ID, TYPE, ATTRS) BI##ID,
 #include "clang/Basic/Builtins.def"
   FirstTSBuiltin

cfe/trunk/include/clang/Basic/Builtins.def

 LANGBUILTIN(get_kernel_max_sub_group_size_for_ndrange, "Ui.", "tn", OCLC20_LANG)
 LANGBUILTIN(get_kernel_sub_group_count_for_ndrange, "Ui.", "tn", OCLC20_LANG)
 
 // OpenCL v2.0 s6.13.9 - Address space qualifier functions.
 LANGBUILTIN(to_global, "v*v*", "tn", OCLC20_LANG)
 LANGBUILTIN(to_local, "v*v*", "tn", OCLC20_LANG)
 LANGBUILTIN(to_private, "v*v*", "tn", OCLC20_LANG)
 
+// OpenCL half load/store builtin
+LANGBUILTIN(__builtin_store_half, "vdh*", "n", ALL_OCLC_LANGUAGES)
+LANGBUILTIN(__builtin_store_halff, "vfh*", "n", ALL_OCLC_LANGUAGES)
+LANGBUILTIN(__builtin_load_half, "dhC*", "nc", ALL_OCLC_LANGUAGES)
+LANGBUILTIN(__builtin_load_halff, "fhC*", "nc", ALL_OCLC_LANGUAGES)
+
 // Builtins for os_log/os_trace
 BUILTIN(__builtin_os_log_format_buffer_size, "zcC*.", "p:0:nut")
 BUILTIN(__builtin_os_log_format, "v*v*cC*.", "p:0:nt")
 
 // Builtins for XRay
 BUILTIN(__xray_customevent, "vcC*z", "")
 
 // Win64-compatible va_list functions
 BUILTIN(__builtin_ms_va_start, "vc*&.", "nt")
 BUILTIN(__builtin_ms_va_end, "vc*&", "n")
 BUILTIN(__builtin_ms_va_copy, "vc*&c*&", "n")
 
 #undef BUILTIN
 #undef LIBBUILTIN
 #undef LANGBUILTIN

cfe/trunk/lib/Basic/Builtins.cpp

bool BuiltinsUnsupported =
       strchr(BuiltinInfo.Attributes, 'f');
   bool MathBuiltinsUnsupported =
     LangOpts.NoMathBuiltin && BuiltinInfo.HeaderName &&
     llvm::StringRef(BuiltinInfo.HeaderName).equals("math.h");
   bool GnuModeUnsupported = !LangOpts.GNUMode && (BuiltinInfo.Langs & GNU_LANG);
   bool MSModeUnsupported =
       !LangOpts.MicrosoftExt && (BuiltinInfo.Langs & MS_LANG);
   bool ObjCUnsupported = !LangOpts.ObjC1 && BuiltinInfo.Langs == OBJC_LANG;
-  bool OclCUnsupported = LangOpts.OpenCLVersion != 200 &&
-                         BuiltinInfo.Langs == OCLC20_LANG;
+  bool OclC1Unsupported = (LangOpts.OpenCLVersion / 100) != 1 &&
+                          (BuiltinInfo.Langs & ALL_OCLC_LANGUAGES ) ==  OCLC1X_LANG;
+  bool OclC2Unsupported = LangOpts.OpenCLVersion != 200 &&
+                          (BuiltinInfo.Langs & ALL_OCLC_LANGUAGES) == OCLC20_LANG;
+  bool OclCUnsupported = !LangOpts.OpenCL &&
+                         (BuiltinInfo.Langs & ALL_OCLC_LANGUAGES);
   return !BuiltinsUnsupported && !MathBuiltinsUnsupported && !OclCUnsupported &&
+         !OclC1Unsupported && !OclC2Unsupported &&
          !GnuModeUnsupported && !MSModeUnsupported && !ObjCUnsupported;
 }
 
 /// initializeBuiltins - Mark the identifiers for all the builtins with their
 /// appropriate builtin ID # and mark any non-portable builtin identifiers as
 /// such.
 void Builtin::Context::initializeBuiltins(IdentifierTable &Table,
                                           const LangOptions& LangOpts) {

cfe/trunk/lib/CodeGen/CGBuiltin.cpp

const char *Name =
             : "__get_kernel_sub_group_count_for_ndrange_impl";
     return RValue::get(Builder.CreateCall(
         CGM.CreateRuntimeFunction(
             llvm::FunctionType::get(
                 IntTy, {NDRange->getType(), GenericVoidPtrTy}, false),
             Name),
         {NDRange, Block}));
   }
+
+  case Builtin::BI__builtin_store_half:
+  case Builtin::BI__builtin_store_halff: {
+    Value *Val = EmitScalarExpr(E->getArg(0));
+    Address Address = EmitPointerWithAlignment(E->getArg(1));
+    Value *HalfVal = Builder.CreateFPTrunc(Val, Builder.getHalfTy());
+    return RValue::get(Builder.CreateStore(HalfVal, Address));
+  }
+  case Builtin::BI__builtin_load_half: {
+    Address Address = EmitPointerWithAlignment(E->getArg(0));
+    Value *HalfVal = Builder.CreateLoad(Address);
+    return RValue::get(Builder.CreateFPExt(HalfVal, Builder.getDoubleTy()));
+  }
+  case Builtin::BI__builtin_load_halff: {
+    Address Address = EmitPointerWithAlignment(E->getArg(0));
+    Value *HalfVal = Builder.CreateLoad(Address);
+    return RValue::get(Builder.CreateFPExt(HalfVal, Builder.getFloatTy()));
+  }
   case Builtin::BIprintf:
     if (getTarget().getTriple().isNVPTX())
       return EmitNVPTXDevicePrintfCallExpr(E, ReturnValue);
     break;
   case Builtin::BI__builtin_canonicalize:
   case Builtin::BI__builtin_canonicalizef:
   case Builtin::BI__builtin_canonicalizel:
     return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::canonicalize));

cfe/trunk/test/CodeGenOpenCL/no-half.cl

+// RUN: %clang_cc1 %s -cl-std=cl2.0 -emit-llvm -o - -triple spir-unknown-unknown | FileCheck %s
+// RUN: %clang_cc1 %s -cl-std=cl1.2 -emit-llvm -o - -triple spir-unknown-unknown | FileCheck %s
+// RUN: %clang_cc1 %s -cl-std=cl1.1 -emit-llvm -o - -triple spir-unknown-unknown | FileCheck %s
+
+#pragma OPENCL EXTENSION cl_khr_fp64:enable
+
+// CHECK-LABEL: @test_store_float(float %foo, half addrspace({{.}}){{.*}} %bar)
+__kernel void test_store_float(float foo, __global half* bar)
+{
+  __builtin_store_halff(foo, bar);
+// CHECK: [[HALF_VAL:%.*]] = fptrunc float %foo to half
+// CHECK: store half [[HALF_VAL]], half addrspace({{.}})* %bar, align 2
+}
+
+// CHECK-LABEL: @test_store_double(double %foo, half addrspace({{.}}){{.*}} %bar)
+__kernel void test_store_double(double foo, __global half* bar)
+{
+  __builtin_store_half(foo, bar);
+// CHECK: [[HALF_VAL:%.*]] = fptrunc double %foo to half
+// CHECK: store half [[HALF_VAL]], half addrspace({{.}})* %bar, align 2
+}
+
+// CHECK-LABEL: @test_load_float(float addrspace({{.}}){{.*}} %foo, half addrspace({{.}}){{.*}} %bar)
+__kernel void test_load_float(__global float* foo, __global half* bar)
+{
+  foo[0] = __builtin_load_halff(bar);
+// CHECK: [[HALF_VAL:%.*]] = load half, half addrspace({{.}})* %bar
+// CHECK: [[FULL_VAL:%.*]] = fpext half [[HALF_VAL]] to float
+// CHECK: store float [[FULL_VAL]], float addrspace({{.}})* %foo
+}
+
+// CHECK-LABEL: @test_load_double(double addrspace({{.}}){{.*}} %foo, half addrspace({{.}}){{.*}} %bar)
+__kernel void test_load_double(__global double* foo, __global half* bar)
+{
+  foo[0] = __builtin_load_half(bar);
+// CHECK: [[HALF_VAL:%.*]] = load half, half addrspace({{.}})* %bar
+// CHECK: [[FULL_VAL:%.*]] = fpext half [[HALF_VAL]] to double
+// CHECK: store double [[FULL_VAL]], double addrspace({{.}})* %foo
+}