[Polly][GPGPU] Added SPIR Code Generation and Corresponding Runtime Support for Intel

[PhilippSchaad]

Summary

Added SPIR Code Generation to the PPCG Code Generator. This can be invoked using
the polly-gpu-arch flag value 'spir32' or 'spir64' for 32 and 64 bit code respectively.
In addition to that, runtime support has been added to execute said SPIR code on Intel
GPU's, where the system is equipped with Intel's open source driver Beignet (development
version). This requires the cmake flag 'USE_INTEL_OCL' to be turned on, and the polly-gpu-runtime
flag value to be 'libopencl'.
The transformation of LLVM IR to SPIR is currently quite a hack, consisting in part of regex
string transformations.
Has been tested (working) with Polybench 3.2 on an Intel i7-5500U (integrated graphics chip).

[grosser]

Hi Philipp,

this looks indeed already surprisingly good. As you noted, there are still some hacks, but I have good hopes that we can work around most of them. I added some comments. Let me know what you think!

Best,
Tobias

lib/CodeGen/PPCGCodeGeneration.cpp

40

After having removed the hack below, this should hopefully not be needed any more.

1259

Why do you emit NVIDIA intrinsics for SPIR code? Can you not emit an opencl barrier?

If you don't know what intrinsic corresponds to this, create a simple .cl file, compile it with clang to SPIR and see which intrinsic gets emitted.

1782

Instead of spreading the SPIR code all over the place, can we create one function addSPIRMetadata, that takes 'Args' as argument, emits all the metadata needed and also emits the metadata you add a little further below?

1882

Why do you emit NVIDIA intrinsics. As discussed above, this there no way to emit the OpenCL intrinsics instead?

2181

It seems all these regexp hacks are needed because we don't generate the right intrinsics. What is holding us back from generating them directly?

2204

Interesting. So the optimization break stuff. Would be great to understand what exactly we brake!

tools/GPURuntime/GPUJIT.c

158

Why do these declarations need to be optional? I think they should compile easily in any situation, no?

228

I am surprised. Intel's OpenCL uses such a different path. In fact, I would assume that libOpenCL.so as a generic OpenCL library would take care of loading beignet. Is this not the case?

286

Does this break or does this just return nullptr if the function is not available. If we can always try and just detect if it is not around, this would be great!

517

Why do we read the file from "kernel.ll" and not use the file passed to us in BinaryBuffer?

[grosser]

Set this to request changes, that I get notified in case you update this revision.

[PhilippSchaad]

lib/CodeGen/PPCGCodeGeneration.cpp

40

That is correct, yes. This should only be needed as long as we have to manually change the string occurrences for barriers and global/local IDs.

1259

The generated intrinsics are the ones declared in the SPIR reference, so for example @_Z13get_global_idj(i32 0) . I cannot find any corresponding intrinsics in LLVM's TableGen/Intrinsics, did I overlook some? This is essentially the only real point where the hack is needed. If we can insert correct intrinsics, we can ditch the regex. Because fixing this is all the regex does.

1782

Will look into that.

1882

Dito above.

2181

Dito above.

2204

Looking into that. What is clear, is that the optimized version skips a few 'dummy' computations, because the optimized kernels expect a smaller workgroup size in OpenCL. Not sure if and how we could account for that. It seems very tricky to me atm, have played around with it quite a bit.

tools/GPURuntime/GPUJIT.c

158

Oh yeah, my bad. Only the LLVMIntel one has to be conditional.

228

The funny thing is: it does, somewhat. If I only dlopen libOpenCL and then call for example clCreateProgramWithBinary, looking at that in gdb reveals that the beignet routines get loaded by libOpenCL. However as soon as a function is not present in the standard OpenCL API, eg. clCreateProgramWithLLVMIntel, it does not like that.

286

I'm gonna look into that, but iirc it didn't work. Might be wrong, will try it out again!

517

Well the problem is that clCreateProgramWithLLVMIntel wants to read from a file, and requests a filename/filepath as an argument there. BinaryBuffer however holds the complete binary. If you know a simpler solution, that would of course be great, because this is another hacky-part.

[PhilippSchaad]

• Adapted dynamic method loading for intel

[grosser]

CMakeLists.txt

178

Forgot to mention, but would be great if we could get away without a configure test here.

lib/CodeGen/PPCGCodeGeneration.cpp

1259

No, I think they may not exist. However, you can just create declarations for such functions using code as in "GPUNodeBuilder::createCallGetKernel(".

1782

I just see this is not only about the llvm::Type, but also carries an integer which is 0 for local/shared memory and 1 otherwise. So moving this into a function does not seem so easy. If you don't find an easy way, just leave it as it is.

2204

It's OK. Let's get the remainder of the patch updated, then we can look at this issue later on.

tools/GPURuntime/GPUJIT.c

517

I see. Maybe that's fine then. Just add a comment for now.

I would prefer if we could use clCreateProgramWithBinary and just pass an LLVM bitcode file, but this likely only works if the LLVM versions of beignet is very modern. Let's leave it for now like this. It's good to have something working!

[PhilippSchaad]

• Inserting SPIR barriers with custom function.

CMakeLists.txt

178

Yes I agree. I don't know of a method for detecting a Beignet installation with CMake yet, but if that exists, that would of course be great.

tools/GPURuntime/GPUJIT.c

517

Totally agreed, that would be the better option. As you pointed out, it is currently not yet possible.

[PhilippSchaad]

• Added comment clarification for workaround.

[PhilippSchaad]

• Removed Regexp hack

[grosser]

Nice. Can you potentially also add a test case?

CMakeLists.txt

178

I don't think we want to add this to cmake at all. At best, we just compile the beignet support in unconditionally, if this is possibile.

lib/CodeGen/PPCGCodeGeneration.cpp

1948

If you store the group names in an array, you can just index this array with "i".

2080

Maybe call this

insertCUDAKernelIntrinsics

and

insertSPIRKernelIntrincics

for consistency?

tools/GPURuntime/GPUJIT.c

228

OK. Can you then unconditionally try to load beignet/libcl.so into another variable, e.g. HandleOpenCLBeignet? I assume this handle should be nullptr if the dlopen fails, right? Can you then check if this handle is zero and use this to decide if you want to call clCreateProgramWithLLVMIntel? This should allow us to get rid of all the conditional compilation.

[grosser]

tools/GPURuntime/GPUJIT.c

517

Another change. Could you possibly remove the conditional compilation here and just check if clCreateProgramWithLLVMIntelFcnPtr != nullptr?

[PhilippSchaad]

• Simplification
• Removed compile-conditional CMake dependency

[PhilippSchaad]

I am not yet too familiar with how to implement such a case and what exactly to check for, but if that is desired, I can look into it.

This should now address all concerns, minus the file-workaround.

lib/CodeGen/PPCGCodeGeneration.cpp

2080

The reason I kept this separated the way it is now, without specifying "CUDA" in the earlier version, is that SPIR should be the only example. When AMD support follows, or anything where there's a registered LLVM target with corresponding intrinsics (AMDGPU in that case), the original function could be used with just an additional switch case, changing the intrinsics IDs. Do you agree with that?

[PhilippSchaad]

lib/CodeGen/PPCGCodeGeneration.cpp

2080

And with 'example' I mean 'exception', my bad.

[grosser]

Great. Yes, a test case would be great. It is not very complicated. You already updated test cases in your last patch. I would suggest you copy one of the simple double nested loop kernels from the PTX test and just check that the right SPIR intrinsics are generated! That should be enough.

Otherwise, I think this is good to go. Very nice work!

lib/CodeGen/PPCGCodeGeneration.cpp

2080

Sure. Still, the function names could be more similar. What about insertKernelCallsSPIR?

[PhilippSchaad]

• Added testcase for SPIR
• Fixed naming inconsistencies

[grosser]

LGTM from my side.

[grosser]

@singam-sanjay can you have a final look and commit it if you are OK with this patch.

[singam-sanjay]

Sure. Will do in sometime.

[singam-sanjay]

@grosser Thanks for adding me as a reviewer ! It's helped me acquaint myself with the SPIR V codegen and understand GPUJIT better.

@PhilippSchaad I've made suggestions to restructure code, a possibly unintentional debug print statement and added some questions about the SPIR. Do let me know if (and why) some of my suggestions are unnecessary.

General Question: I was following a discussion about integrating the SPIR-V backend as a part of LLVM, i.e. moving it from /tools to lib/Target. If that were to happen, would we be able to generate and feed SPIR IR in the same way as NVPTX ?

lib/CodeGen/PPCGCodeGeneration.cpp

1794

@PhilippSchaad I'm not yet acquainted with the SPIR annotations. Could you please explain why it matters to let the SPIR backend know whether a kernel argument is a read only ?

1796

Why are you setting these metadata to empty strings ? Is this a feature yet to be implemented ?

2097

nitpick: Do we need this ?

2120

nitpick: Do we need this ?

2208–2216

Would it be better to move the SPIR specific code into createKernelASM, as well ? You might not need the switch cases to handle buggy control flow like,

case GPUArch::SPIR64:
case GPUArch::SPIR32:
  llvm_unreachable("Cannot generate ASM for SPIR architecture");

tools/GPURuntime/GPUJIT.c

151

This is new to me. Why are you providing runtime support for Intel to JIT SPIR code ? Is it to run with the integrated GPU on an Intel CPU? Guessing from https://github.com/intel/beignet.

255

void *Handle = ( HandleOpenCLBeignet ? HandleOpenCLBeignet : HandleOpenCL ); maybe ?

Let me know if this isn't recommended.

257

Is it that we prefer to use libraries provided by Intel's SDK and inegrated GPU over other OpenCL drivers and devices ?

I'm assuming that this leads the runtime to use the integrated GPU even when a more powerful Radeon card is present. Please correct me if I'm wrong.

517

@grosser What is your rationale to avoid conditional compilation ?

531

Is this file opened inside <llvm_build>/bin or $PWD ? If it's $PWD, we should to take precautions to not open a file that the user needs. You could instead open a file in /tmp or %TEMP% in Windows, like "/tmp/R@ηdηдm3.ll". Anyways, you could implement this later.

537

Was this supposed to a message letting the user know that the runtime's not using the "beignet" OpenCL implementation ?

[PhilippSchaad]

Will address the points mentioned asap. As for your SPIR-V question: The goal, should that back-end be added, would be to add SPIR-V compilation as an additional GPU target option (In the future). This would allow kernel execution on any SPIR-V supporting target device. This SPIR solution is essentially a 'quickfix' to get it to work on Intel, as long as SPIR-V is not an option.

lib/CodeGen/PPCGCodeGeneration.cpp

1794

The issue here is the Intel Beignet driver. Beignet expects a very specific set of kernel annotations when parsing the Kernel IR. The metadata is simply required to be there, but for the kernels we generate here, it does not matter what is actually contained in there. This is entirely Beignet dependent, maybe Intel will adapt that in the future.

1796

Dito above.

2097

We can change it to an if-clause if that is desired. The compiler just complains when we have a switch and do not check every possible value of Arch. An alternative would be a default clause handling that llvm_unreachable.

2120

Dito above.

2208–2216

Will do, good point. Originally moved it out because we thought we had to do a lot more hacking on the resulting IR.

tools/GPURuntime/GPUJIT.c

151

Your guess is exactly correct. This also corresponds to some of the points mentioned above.

255

Definitely the prettier solution. Will change.

257

This is actually a good point. We don't want to prefer the Intel provided SDK, the choice of selecting Intel's Beignet should be dependent on what -polly-gpu-arch value has been selected. AMD is in the pipeline.

531

I will add this later, but yes it is $PWD. The solution with using /tmp and %TEMP% seem reasonable.

537

Oops, thanks for pointing this out, this was unintentionally left in and should never be reached! Used it for debugging. Removing it.

[singam-sanjay]

lib/CodeGen/PPCGCodeGeneration.cpp

1794

Ohk. Thanks for the info !

1796

Alright. I guess beignet isn't worried about the names of kernel arguments in "kernel_arg_name".

2097

I was talking about the break; after llvm_unreachable. If llvm_unreachable would call abort, the break would be redundant.

2120

Ditto above ;-).

2208–2216

Alright.

tools/GPURuntime/GPUJIT.c

151

Ohk.

255

void *Handle = ( HandleOpenCLBeignet!=NULL ? HandleOpenCLBeignet : HandleOpenCL ); would be better.

257

Alright.

Any ideas on how you'd differentiate between the integrated and dedicated GPU at command line ?

531

Alright.

537

👍

[PhilippSchaad]

Removed left over debug print, moved SPIR creation into createASM, fixed minor issues addressed in comments.

[PhilippSchaad]

tools/GPURuntime/GPUJIT.c

257

The goal is to run on AMD GPUs with the AMDGPU backend, which would mean the differentiation would be done with -polly-gpu-arch=spir32/amdgcn32 for example. I am not yet sure how to transport that choice to here, but I will look into that as I am working on the AMD side of things. Thanks for pointing out this flaw.

[PhilippSchaad]

Ping do the latest changes address your concerns @singam-sanjay ? Can I land this?

[singam-sanjay]

@PhilippSchaad Please go ahead.

[PhilippSchaad]

Rebase for commit

CMakeLists.txt

if (CUDA_FOUND)
   add_definitions(-DHAS_LIBCUDART)
   INCLUDE_DIRECTORIES( ${CUDA_INCLUDE_DIRS} )
 endif(CUDA_FOUND)
 if (OpenCL_FOUND)
   add_definitions(-DHAS_LIBOPENCL)
   INCLUDE_DIRECTORIES( ${OpenCL_INCLUDE_DIR} )
 endif(OpenCL_FOUND)
 
+option(USE_INTEL_OCL "Uses the Intel Beignet driver for GPGPU code" OFF)
+if (USE_INTEL_OCL)
+  add_definitions(-DHAS_INTEL_OCL)
+endif(USE_INTEL_OCL)
+
 option(POLLY_BUNDLED_ISL "Use the bundled version of libisl included in Polly" ON)
 if (NOT POLLY_BUNDLED_ISL)
   find_package(ISL MODULE REQUIRED)
   message(STATUS "Using external libisl ${ISL_VERSION} in: ${ISL_PREFIX}")
   set(ISL_TARGET ISL)
 else()
   set(ISL_INCLUDE_DIRS
     ${CMAKE_CURRENT_BINARY_DIR}/lib/External/isl/include

include/polly/CodeGen/PPCGCodeGeneration.h

 // GPU mapping strategy.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef POLLY_PPCGCODEGENERATION_H
 #define POLLY_PPCGCODEGENERATION_H
 
 /// The GPU Architecture to target.
-enum GPUArch { NVPTX64 };
+enum GPUArch { NVPTX64, SPIR32, SPIR64 };
 
 /// The GPU Runtime implementation to use.
 enum GPURuntime { CUDA, OpenCL };
 
 #endif // POLLY_PPCGCODEGENERATION_H

lib/CodeGen/PPCGCodeGeneration.cpp

 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"
 #include "llvm/Transforms/Utils/BasicBlockUtils.h"
 
+#include <regex>
+
 #include "isl/union_map.h"
 
 extern "C" {
 #include "ppcg/cuda.h"
 #include "ppcg/gpu.h"
 #include "ppcg/gpu_print.h"
 #include "ppcg/ppcg.h"
 #include "ppcg/schedule.h"
private:
   /// Create an in-kernel synchronization call.
   void createKernelSync();
 
   /// Create a PTX assembly string for the current GPU kernel.
   ///
   /// @returns A string containing the corresponding PTX assembly code.
   std::string createKernelASM();
 
+  /// Create a SPIR string for the current GPU kernel.
+  ///
+  /// @returns A string containing the corresponding  SPIR code.
+  std::string createKernelSPIR(std::string IR);
+
   /// Remove references from the dominator tree to the kernel function @p F.
   ///
   /// @param F The function to remove references to.
   void clearDominators(Function *F);
 
   /// Remove references from scalar evolution to the kernel function @p F.
   ///
   /// @param F The function to remove references to.
void GPUNodeBuilder::createScopStmt(isl_ast_expr *Expr,
   if (Stmt->isBlockStmt())
     BlockGen.copyStmt(*Stmt, LTS, Indexes);
   else
     RegionGen.copyStmt(*Stmt, LTS, Indexes);
 }
 
 void GPUNodeBuilder::createKernelSync() {
   Module *M = Builder.GetInsertBlock()->getParent()->getParent();
+  const char *SpirName = "__gen_ocl_barrier_global";
 
   Function *Sync;
 
   switch (Arch) {
+  case GPUArch::SPIR64:
+  case GPUArch::SPIR32:
+    Sync = M->getFunction(SpirName);
+
+    // If Sync is not available, declare it.
+    if (!Sync) {
+      GlobalValue::LinkageTypes Linkage = Function::ExternalLinkage;
+      std::vector<Type *> Args;
+      FunctionType *Ty = FunctionType::get(Builder.getVoidTy(), Args, false);
+      Sync = Function::Create(Ty, Linkage, SpirName, M);
+      Sync->setCallingConv(CallingConv::SPIR_FUNC);
+    }
+    break;
   case GPUArch::NVPTX64:
     Sync = Intrinsic::getDeclaration(M, Intrinsic::nvvm_barrier0);
     break;
   }
 
   Builder.CreateCall(Sync, {});
 }
 
 /// Collect llvm::Values referenced from @p Node
 ///
void GPUNodeBuilder::createKernel(__isl_take isl_ast_node *KernelStmt) {
 
   createKernelFunction(Kernel, SubtreeValues, SubtreeFunctions);
   setupKernelSubtreeFunctions(SubtreeFunctions);
 
   create(isl_ast_node_copy(Kernel->tree));
 
   finalizeKernelArguments(Kernel);
   Function *F = Builder.GetInsertBlock()->getParent();
+  if (Arch == GPUArch::NVPTX64)
     addCUDAAnnotations(F->getParent(), BlockDimX, BlockDimY, BlockDimZ);
   clearDominators(F);
   clearScalarEvolution(F);
   clearLoops(F);
 
   IDToValue = HostIDs;
 
   ValueMap = std::move(HostValueMap);
   ScalarMap = std::move(HostScalarMap);
if (!is64Bit) {
     Ret += "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:"
            "64-f32:32:32-f64:64:64-v16:16:16-v32:32:32-v64:64:"
            "64-v128:128:128-n16:32:64";
   }
 
   return Ret;
 }
 
+/// Compute the DataLayout string for a SPIR kernel.
+///
+/// @param is64Bit Are we looking for a 64 bit architecture?
+static std::string computeSPIRDataLayout(bool is64Bit) {
+  std::string Ret = "";
+
+  if (!is64Bit) {
+    Ret += "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:"
+           "64-f32:32:32-f64:64:64-v16:16:16-v24:32:32-v32:32:"
+           "32-v48:64:64-v64:64:64-v96:128:128-v128:128:128-v192:"
+           "256:256-v256:256:256-v512:512:512-v1024:1024:1024";
+  } else {
+    Ret += "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:"
+           "64-f32:32:32-f64:64:64-v16:16:16-v24:32:32-v32:32:"
+           "32-v48:64:64-v64:64:64-v96:128:128-v128:128:128-v192:"
+           "256:256-v256:256:256-v512:512:512-v1024:1024:1024";
+  }
+
+  return Ret;
+}
+
 Function *
 GPUNodeBuilder::createKernelFunctionDecl(ppcg_kernel *Kernel,
                                          SetVector<Value *> &SubtreeValues) {
   std::vector<Type *> Args;
   std::string Identifier = getKernelFuncName(Kernel->id);
 
+  std::vector<Metadata *> MemoryType;
+
   for (long i = 0; i < Prog->n_array; i++) {
     if (!ppcg_kernel_requires_array_argument(Kernel, i))
       continue;
 
     if (gpu_array_is_read_only_scalar(&Prog->array[i])) {
       isl_id *Id = isl_space_get_tuple_id(Prog->array[i].space, isl_dim_set);
       const ScopArrayInfo *SAI = ScopArrayInfo::getFromId(Id);
       Args.push_back(SAI->getElementType());
+      MemoryType.push_back(
+          ConstantAsMetadata::get(ConstantInt::get(Builder.getInt32Ty(), 0)));
     } else {
       static const int UseGlobalMemory = 1;
       Args.push_back(Builder.getInt8PtrTy(UseGlobalMemory));
+      MemoryType.push_back(
+          ConstantAsMetadata::get(ConstantInt::get(Builder.getInt32Ty(), 1)));
     }
   }
 
   int NumHostIters = isl_space_dim(Kernel->space, isl_dim_set);
 
-  for (long i = 0; i < NumHostIters; i++)
+  for (long i = 0; i < NumHostIters; i++) {
     Args.push_back(Builder.getInt64Ty());
+    MemoryType.push_back(
+        ConstantAsMetadata::get(ConstantInt::get(Builder.getInt32Ty(), 0)));
+  }
 
   int NumVars = isl_space_dim(Kernel->space, isl_dim_param);
 
   for (long i = 0; i < NumVars; i++) {
     isl_id *Id = isl_space_get_dim_id(Kernel->space, isl_dim_param, i);
     Value *Val = IDToValue[Id];
     isl_id_free(Id);
     Args.push_back(Val->getType());
+    MemoryType.push_back(
+        ConstantAsMetadata::get(ConstantInt::get(Builder.getInt32Ty(), 0)));
   }
 
-  for (auto *V : SubtreeValues)
+  for (auto *V : SubtreeValues) {
     Args.push_back(V->getType());
+    MemoryType.push_back(
+        ConstantAsMetadata::get(ConstantInt::get(Builder.getInt32Ty(), 0)));
+  }
 
   auto *FT = FunctionType::get(Builder.getVoidTy(), Args, false);
   auto *FN = Function::Create(FT, Function::ExternalLinkage, Identifier,
                               GPUModule.get());
 
+  std::vector<Metadata *> EmptyStrings;
+
+  for (unsigned int i = 0; i < MemoryType.size(); i++) {
+    EmptyStrings.push_back(MDString::get(FN->getContext(), ""));
+  }
+
+  if (Arch == GPUArch::SPIR32 || Arch == GPUArch::SPIR64) {
+    FN->setMetadata("kernel_arg_addr_space",
+                    MDNode::get(FN->getContext(), MemoryType));
+    FN->setMetadata("kernel_arg_name",
+                    MDNode::get(FN->getContext(), EmptyStrings));
+    FN->setMetadata("kernel_arg_access_qual",
+                    MDNode::get(FN->getContext(), EmptyStrings));
+    FN->setMetadata("kernel_arg_type",
+                    MDNode::get(FN->getContext(), EmptyStrings));
+    FN->setMetadata("kernel_arg_type_qual",
+                    MDNode::get(FN->getContext(), EmptyStrings));
+    FN->setMetadata("kernel_arg_base_type",
+                    MDNode::get(FN->getContext(), EmptyStrings));
+  }
+
   switch (Arch) {
   case GPUArch::NVPTX64:
     FN->setCallingConv(CallingConv::PTX_Kernel);
     break;
+  case GPUArch::SPIR32:
+  case GPUArch::SPIR64:
+    FN->setCallingConv(CallingConv::SPIR_KERNEL);
+    break;
   }
 
   auto Arg = FN->arg_begin();
   for (long i = 0; i < Kernel->n_array; i++) {
     if (!ppcg_kernel_requires_array_argument(Kernel, i))
       continue;
 
     Arg->setName(Kernel->array[i].array->name);
GPUNodeBuilder::createKernelFunctionDecl(ppcg_kernel *Kernel,
   return FN;
 }
 
 void GPUNodeBuilder::insertKernelIntrinsics(ppcg_kernel *Kernel) {
   Intrinsic::ID IntrinsicsBID[2];
   Intrinsic::ID IntrinsicsTID[3];
 
   switch (Arch) {
+  case GPUArch::SPIR64:
+  case GPUArch::SPIR32:
   case GPUArch::NVPTX64:
     IntrinsicsBID[0] = Intrinsic::nvvm_read_ptx_sreg_ctaid_x;
     IntrinsicsBID[1] = Intrinsic::nvvm_read_ptx_sreg_ctaid_y;
 
     IntrinsicsTID[0] = Intrinsic::nvvm_read_ptx_sreg_tid_x;
     IntrinsicsTID[1] = Intrinsic::nvvm_read_ptx_sreg_tid_y;
     IntrinsicsTID[2] = Intrinsic::nvvm_read_ptx_sreg_tid_z;
     break;
for (long i = 0; i < Kernel->n_array; i++) {
     Arg++;
   }
 }
 
 void GPUNodeBuilder::finalizeKernelArguments(ppcg_kernel *Kernel) {
   auto *FN = Builder.GetInsertBlock()->getParent();
   auto Arg = FN->arg_begin();
 
   bool StoredScalar = false;
   for (long i = 0; i < Kernel->n_array; i++) {
     if (!ppcg_kernel_requires_array_argument(Kernel, i))
       continue;
 
     isl_id *Id = isl_space_get_tuple_id(Prog->array[i].space, isl_dim_set);
     const ScopArrayInfo *SAI = ScopArrayInfo::getFromId(isl_id_copy(Id));
     isl_id_free(Id);
 
void GPUNodeBuilder::createKernelFunction(
   switch (Arch) {
   case GPUArch::NVPTX64:
     if (Runtime == GPURuntime::CUDA)
       GPUModule->setTargetTriple(Triple::normalize("nvptx64-nvidia-cuda"));
     else if (Runtime == GPURuntime::OpenCL)
       GPUModule->setTargetTriple(Triple::normalize("nvptx64-nvidia-nvcl"));
     GPUModule->setDataLayout(computeNVPTXDataLayout(true /* is64Bit */));
     break;
+  case GPUArch::SPIR32:
+    GPUModule->setTargetTriple(Triple::normalize("spir-unknown-unknown"));
+    GPUModule->setDataLayout(computeSPIRDataLayout(false /* is64Bit */));
+    break;
+  case GPUArch::SPIR64:
+    GPUModule->setTargetTriple(Triple::normalize("spir64-unknown-unknown"));
+    GPUModule->setDataLayout(computeSPIRDataLayout(true /* is64Bit */));
+    break;
   }
 
   Function *FN = createKernelFunctionDecl(Kernel, SubtreeValues);
 
   BasicBlock *PrevBlock = Builder.GetInsertBlock();
   auto EntryBlock = BasicBlock::Create(Builder.getContext(), "entry", FN);
 
   DT.addNewBlock(EntryBlock, PrevBlock);
 
   Builder.SetInsertPoint(EntryBlock);
   Builder.CreateRetVoid();
   Builder.SetInsertPoint(EntryBlock, EntryBlock->begin());
 
   ScopDetection::markFunctionAsInvalid(FN);
 
   prepareKernelArguments(Kernel, FN);
   createKernelVariables(Kernel, FN);
   insertKernelIntrinsics(Kernel);
 }
 
 std::string GPUNodeBuilder::createKernelASM() {
   llvm::Triple GPUTriple;
 
   switch (Arch) {
   case GPUArch::NVPTX64:
     switch (Runtime) {
     case GPURuntime::CUDA:
       GPUTriple = llvm::Triple(Triple::normalize("nvptx64-nvidia-cuda"));
       break;
     case GPURuntime::OpenCL:
       GPUTriple = llvm::Triple(Triple::normalize("nvptx64-nvidia-nvcl"));
       break;
     }
     break;
+  case GPUArch::SPIR64:
+  case GPUArch::SPIR32:
+    llvm_unreachable("Cannot generate ASM for SPIR architecture");
+    break;
   }
 
   std::string ErrMsg;
   auto GPUTarget = TargetRegistry::lookupTarget(GPUTriple.getTriple(), ErrMsg);
 
   if (!GPUTarget) {
     errs() << ErrMsg << "\n";
     return "";
   }
 
   TargetOptions Options;
   Options.UnsafeFPMath = FastMath;
 
   std::string subtarget;
 
   switch (Arch) {
   case GPUArch::NVPTX64:
     subtarget = CudaVersion;
     break;
+  case GPUArch::SPIR32:
+  case GPUArch::SPIR64:
+    llvm_unreachable("No subtarget for SPIR architecture");
+    break;
   }
 
   std::unique_ptr<TargetMachine> TargetM(GPUTarget->createTargetMachine(
       GPUTriple.getTriple(), subtarget, "", Options, Optional<Reloc::Model>()));
 
   SmallString<0> ASMString;
   raw_svector_ostream ASMStream(ASMString);
   llvm::legacy::PassManager PM;
 
   PM.add(createTargetTransformInfoWrapperPass(TargetM->getTargetIRAnalysis()));
 
   if (TargetM->addPassesToEmitFile(
           PM, ASMStream, TargetMachine::CGFT_AssemblyFile, true /* verify */)) {
     errs() << "The target does not support generation of this file type!\n";
     return "";
   }
 
   PM.run(*GPUModule);
 
   return ASMStream.str();
 }
 
+std::string StringReplace(std::string const &in, std::string const &replace,
+                          std::string const &with) {
+  return std::regex_replace(in, std::regex(replace), with);
+}
+
+std::string GPUNodeBuilder::createKernelSPIR(std::string IR) {
+  IR = StringReplace(IR, "declare i32 @llvm.nvvm.read.ptx.sreg.tid.x\\(\\)",
+                     "declare spir_func i32 @__gen_ocl_get_local_id0()");
+  IR = StringReplace(IR, "declare i32 @llvm.nvvm.read.ptx.sreg.tid.y\\(\\)",
+                     "declare spir_func i32 @__gen_ocl_get_local_id1()");
+  IR = StringReplace(IR, "declare i32 @llvm.nvvm.read.ptx.sreg.tid.z\\(\\)",
+                     "declare spir_func i32 @__gen_ocl_get_local_id2()");
+
+  IR = StringReplace(IR, "declare i32 @llvm.nvvm.read.ptx.sreg.ctaid.x\\(\\)",
+                     "declare spir_func i32 @__gen_ocl_get_group_id0()");
+  IR = StringReplace(IR, "declare i32 @llvm.nvvm.read.ptx.sreg.ctaid.y\\(\\)",
+                     "declare spir_func i32 @__gen_ocl_get_group_id1()");
+  IR = StringReplace(IR, "declare i32 @llvm.nvvm.read.ptx.sreg.ctaid.z\\(\\)",
+                     "declare spir_func i32 @__gen_ocl_get_group_id2()");
+
+  IR = StringReplace(IR, "call i32 @llvm.nvvm.read.ptx.sreg.tid.x\\(\\)",
+                     "call spir_func i32 @__gen_ocl_get_local_id0()");
+  IR = StringReplace(IR, "call i32 @llvm.nvvm.read.ptx.sreg.tid.y\\(\\)",
+                     "call spir_func i32 @__gen_ocl_get_local_id1()");
+  IR = StringReplace(IR, "call i32 @llvm.nvvm.read.ptx.sreg.tid.z\\(\\)",
+                     "call spir_func i32 @__gen_ocl_get_local_id2()");
+
+  IR = StringReplace(IR, "call i32 @llvm.nvvm.read.ptx.sreg.ctaid.x\\(\\)",
+                     "call spir_func i32 @__gen_ocl_get_group_id0()");
+  IR = StringReplace(IR, "call i32 @llvm.nvvm.read.ptx.sreg.ctaid.y\\(\\)",
+                     "call spir_func i32 @__gen_ocl_get_group_id1()");
+  IR = StringReplace(IR, "call i32 @llvm.nvvm.read.ptx.sreg.ctaid.z\\(\\)",
+                     "call spir_func i32 @__gen_ocl_get_group_id2()");
+
+  return IR;
+}
+
 std::string GPUNodeBuilder::finalizeKernelFunction() {
 
   if (verifyModule(*GPUModule)) {
     DEBUG(dbgs() << "verifyModule failed on module:\n";
           GPUModule->print(dbgs(), nullptr); dbgs() << "\n";);
 
     if (FailOnVerifyModuleFailure)
       llvm_unreachable("VerifyModule failed.");
 
     BuildSuccessful = false;
     return "";
   }
 
   if (DumpKernelIR)
     outs() << *GPUModule << "\n";
 
+  if (Arch != GPUArch::SPIR32 && Arch != GPUArch::SPIR64) {
     // Optimize module.
     llvm::legacy::PassManager OptPasses;
     PassManagerBuilder PassBuilder;
     PassBuilder.OptLevel = 3;
     PassBuilder.SizeLevel = 0;
     PassBuilder.populateModulePassManager(OptPasses);
     OptPasses.run(*GPUModule);
+  }
+
+  std::string Assembly;
 
-  std::string Assembly = createKernelASM();
+  if (Arch == GPUArch::SPIR32 || Arch == GPUArch::SPIR64) {
+    std::string IR;
+    raw_string_ostream IROstream(IR);
+    IROstream << *GPUModule;
+    IROstream.flush();
+    Assembly = createKernelSPIR(IR);
+  } else {
+    Assembly = createKernelASM();
+  }
 
   if (DumpKernelASM)
     outs() << Assembly << "\n";
 
   GPUModule.release();
   KernelIDs.clear();
 
   return Assembly;

lib/Support/RegisterPasses.cpp

cl::values(clEnumValN(GPURuntime::CUDA, "libcudart",
                           "use the CUDA Runtime API"),
                clEnumValN(GPURuntime::OpenCL, "libopencl",
                           "use the OpenCL Runtime API")),
     cl::init(GPURuntime::CUDA), cl::ZeroOrMore, cl::cat(PollyCategory));
 
 static cl::opt<GPUArch>
     GPUArchChoice("polly-gpu-arch", cl::desc("The GPU Architecture to target"),
                   cl::values(clEnumValN(GPUArch::NVPTX64, "nvptx64",
-                                        "target NVIDIA 64-bit architecture")),
+                                        "target NVIDIA 64-bit architecture"),
+                             clEnumValN(GPUArch::SPIR32, "spir32",
+                                        "target SPIR 32-bit architecture"),
+                             clEnumValN(GPUArch::SPIR64, "spir64",
+                                        "target SPIR 64-bit architecture")),
                   cl::init(GPUArch::NVPTX64), cl::ZeroOrMore,
                   cl::cat(PollyCategory));
 #endif
 
 VectorizerChoice polly::PollyVectorizerChoice;
 static cl::opt<polly::VectorizerChoice, true> Vectorizer(
     "polly-vectorizer", cl::desc("Select the vectorization strategy"),
     cl::values(

tools/GPURuntime/GPUJIT.c

 #include <cuda.h>
 #include <cuda_runtime.h>
 #endif /* HAS_LIBCUDART */
 
 #ifdef HAS_LIBOPENCL
 #ifdef __APPLE__
 #include <OpenCL/opencl.h>
 #else
+#ifdef HAS_INTEL_OCL
+#include <CL/cl_intel.h>
+#else
 #include <CL/cl.h>
-#endif
+#endif /* HAS_INTEL_OCL */
+#endif /* __APPLE__ */
 #endif /* HAS_LIBOPENCL */
 
 #include <dlfcn.h>
 #include <stdarg.h>
 #include <stdio.h>
 #include <string.h>
+#include <unistd.h>
 
 static int DebugMode;
 static int CacheMode;
 
 static PollyGPURuntime Runtime = RUNTIME_NONE;
 
 static void debug_print(const char *format, ...) {
   if (!DebugMode)

 
 typedef cl_int
 clEnqueueWriteBufferFcnTy(cl_command_queue CommandQueue, cl_mem Buffer,
                           cl_bool BlockingWrite, size_t Offset, size_t Size,
                           const void *Ptr, cl_uint NumEventsInWaitList,
                           const cl_event *EventWaitList, cl_event *Event);
 static clEnqueueWriteBufferFcnTy *clEnqueueWriteBufferFcnPtr;
 
+typedef cl_program
+clCreateProgramWithLLVMIntelFcnTy(cl_context Context, cl_uint NumDevices,
+                                  const cl_device_id *DeviceList,
+                                  const char *Filename, cl_int *ErrcodeRet);
+static clCreateProgramWithLLVMIntelFcnTy *clCreateProgramWithLLVMIntelFcnPtr;
+
 typedef cl_program clCreateProgramWithBinaryFcnTy(
     cl_context Context, cl_uint NumDevices, const cl_device_id *DeviceList,
     const size_t *Lengths, const unsigned char **Binaries, cl_int *BinaryStatus,
     cl_int *ErrcodeRet);
 static clCreateProgramWithBinaryFcnTy *clCreateProgramWithBinaryFcnPtr;
 
 typedef cl_int clBuildProgramFcnTy(
     cl_program Program, cl_uint NumDevices, const cl_device_id *DeviceList,
     const char *Options,
     void(CL_CALLBACK *pfn_notify)(cl_program Program, void *UserData),
     void *UserData);
 static clBuildProgramFcnTy *clBuildProgramFcnPtr;
 
 typedef cl_kernel clCreateKernelFcnTy(cl_program Program,
static void *getAPIHandleCL(void *Handle, const char *FuncName) {
   if ((Err = dlerror()) != 0) {
     fprintf(stderr, "Load OpenCL Runtime API failed: %s. \n", Err);
     return 0;
   }
   return FuncPtr;
 }
 
 static int initialDeviceAPILibrariesCL() {
+#ifdef HAS_INTEL_OCL
+  HandleOpenCL = dlopen("/usr/local/lib/beignet/libcl.so", RTLD_LAZY);
+#else
   HandleOpenCL = dlopen("libOpenCL.so", RTLD_LAZY);
+#endif /* HAS_INTEL_OCL */
   if (!HandleOpenCL) {
     fprintf(stderr, "Cannot open library: %s. \n", dlerror());
     return 0;
   }
   return 1;
 }
 
 /* Get function pointer to OpenCL Runtime API.

  * http://pubs.opengroup.org/onlinepubs/9699919799/functions/dlsym.html
  */
 #pragma GCC diagnostic push
 #pragma GCC diagnostic ignored "-Wpedantic"
 static int initialDeviceAPIsCL() {
   if (initialDeviceAPILibrariesCL() == 0)
     return 0;
 
   clGetPlatformIDsFcnPtr =
       (clGetPlatformIDsFcnTy *)getAPIHandleCL(HandleOpenCL, "clGetPlatformIDs");
 
   clGetDeviceIDsFcnPtr =
       (clGetDeviceIDsFcnTy *)getAPIHandleCL(HandleOpenCL, "clGetDeviceIDs");
 
   clGetDeviceInfoFcnPtr =
       (clGetDeviceInfoFcnTy *)getAPIHandleCL(HandleOpenCL, "clGetDeviceInfo");
 
   clGetKernelInfoFcnPtr =
       (clGetKernelInfoFcnTy *)getAPIHandleCL(HandleOpenCL, "clGetKernelInfo");
 
   clCreateContextFcnPtr =
       (clCreateContextFcnTy *)getAPIHandleCL(HandleOpenCL, "clCreateContext");
 
   clCreateCommandQueueFcnPtr = (clCreateCommandQueueFcnTy *)getAPIHandleCL(
       HandleOpenCL, "clCreateCommandQueue");
 
   clCreateBufferFcnPtr =
       (clCreateBufferFcnTy *)getAPIHandleCL(HandleOpenCL, "clCreateBuffer");
 
   clEnqueueWriteBufferFcnPtr = (clEnqueueWriteBufferFcnTy *)getAPIHandleCL(
       HandleOpenCL, "clEnqueueWriteBuffer");
 
+  clCreateProgramWithLLVMIntelFcnPtr =
+      (clCreateProgramWithLLVMIntelFcnTy *)getAPIHandleCL(
+          HandleOpenCL, "clCreateProgramWithLLVMIntel");
+
   clCreateProgramWithBinaryFcnPtr =
       (clCreateProgramWithBinaryFcnTy *)getAPIHandleCL(
           HandleOpenCL, "clCreateProgramWithBinary");
 
   clBuildProgramFcnPtr =
       (clBuildProgramFcnTy *)getAPIHandleCL(HandleOpenCL, "clBuildProgram");
 
   clCreateKernelFcnPtr =
       (clCreateKernelFcnTy *)getAPIHandleCL(HandleOpenCL, "clCreateKernel");
 
   clSetKernelArgFcnPtr =
       (clSetKernelArgFcnTy *)getAPIHandleCL(HandleOpenCL, "clSetKernelArg");
static PollyGPUFunction *getKernelCL(const char *BinaryBuffer,
   }
 
   if (!GlobalDeviceID) {
     fprintf(stderr, "GPGPU-code generation not initialized correctly.\n");
     exit(-1);
   }
 
   cl_int Ret;
+
+#ifdef HAS_INTEL_OCL
+  // TODO: This is a workaround, since clCreateProgramWithLLVMIntel only
+  // accepts a filename to a valid llvm-ir file as an argument, instead
+  // of accepting the BinaryBuffer directly.
+  FILE *fp = fopen("kernel.ll", "wb");
+  if (fp != NULL) {
+    fputs(BinaryBuffer, fp);
+    fclose(fp);
+  }
+
+  ((OpenCLKernel *)Function->Kernel)->Program =
+      clCreateProgramWithLLVMIntelFcnPtr(
+          ((OpenCLContext *)GlobalContext->Context)->Context, 1,
+          &GlobalDeviceID, "kernel.ll", &Ret);
+  checkOpenCLError(Ret, "Failed to create program from llvm.\n");
+  unlink("kernel.ll");
+#else
   size_t BinarySize = strlen(BinaryBuffer);
   ((OpenCLKernel *)Function->Kernel)->Program = clCreateProgramWithBinaryFcnPtr(
       ((OpenCLContext *)GlobalContext->Context)->Context, 1, &GlobalDeviceID,
       (const size_t *)&BinarySize, (const unsigned char **)&BinaryBuffer, NULL,
       &Ret);
   checkOpenCLError(Ret, "Failed to create program from binary.\n");
+#endif /* HAS_INTEL_OCL */
 
   Ret = clBuildProgramFcnPtr(((OpenCLKernel *)Function->Kernel)->Program, 1,
                              &GlobalDeviceID, NULL, NULL, NULL);
   checkOpenCLError(Ret, "Failed to build program.\n");
 
   ((OpenCLKernel *)Function->Kernel)->Kernel = clCreateKernelFcnPtr(
       ((OpenCLKernel *)Function->Kernel)->Program, KernelName, &Ret);
   checkOpenCLError(Ret, "Failed to create kernel.\n");
 
   ((OpenCLKernel *)Function->Kernel)->BinaryString = BinaryBuffer;
 
   if (CacheMode) {
     if (KernelCache[NextCacheItem])
       freeKernelCL(KernelCache[NextCacheItem]);
 
     KernelCache[NextCacheItem] = Function;
 
     NextCacheItem = (NextCacheItem + 1) % KERNEL_CACHE_SIZE;