This is an archive of the discontinued LLVM Phabricator instance.

[MLIR][mlir-spirv-cpu-runner] A SPIR-V cpu runner prototype
ClosedPublic

Authored by georgemitenkov on Aug 17 2020, 2:16 PM.

Details

Summary

This patch introduces a SPIR-V runner. The aim is to run a gpu
kernel on a CPU via GPU -> SPIRV -> LLVM conversions. This is a first
prototype, so more features will be added in due time.

  • Overview

The runner follows similar flow as the other runners in-tree. However,
having converted the kernel to SPIR-V, we encode the bind attributes of
global variables that represent kernel arguments. Then SPIR-V module is
converted to LLVM. On the host side, we emulate passing the data to device
by creating in main module globals with the same symbolic name as in kernel
module. These global variables are later linked with ones from the nested
module. We copy data from kernel arguments to globals, call the kernel
function from nested module and then copy the data back.

  • Current state

At the moment, the runner is capable of running 2 modules, nested one in
another. The kernel module must contain exactly one kernel function. Also,
the runner supports rank 1 integer memref types as arguments (to be scaled).

  • Enhancement of JitRunner and ExecutionEngine

To translate nested modules to LLVM IR, JitRunner and ExecutionEngine were
altered to take an optional (default to nullptr) function reference that
is a custom LLVM IR module builder. This allows to customize LLVM IR module
creation from MLIR modules.

Diff Detail

Event Timeline

georgemitenkov created this revision.Aug 17 2020, 2:16 PM
georgemitenkov requested review of this revision.Aug 17 2020, 2:17 PM

Can you split this up? Seems like the passes could be split and reviewed separately from the runner code.

Can you split this up? Seems like the passes could be split and reviewed separately from the runner code.

Sure, thanks!

Splitted the patch.

Fixed a typo.

mehdi_amini added inline comments.Aug 17 2020, 5:25 PM
mlir/include/mlir/ExecutionEngine/ExecutionEngine.h
69

Can you document the else case please?

mlir/lib/ExecutionEngine/ExecutionEngine.cpp
231

I'm not convinced this injection is really properly layered: it has some loose contract right now.

mravishankar requested changes to this revision.Aug 17 2020, 11:16 PM

Thanks George!

mlir/lib/ExecutionEngine/ExecutionEngine.cpp
231

How about the translateModuleToLLVMIR be modified to handle multiple modules. So instead of just taking a single ModuleOp takes an ArrayRef<ModuleOp> as arguments and then links them. This way you wouldnt have to thread through the llvmModuleBuilder callback through the entire stack. The spirv-runner can split the nested module out and pass the outer module and the nested modules as two separate modules that are to be compiled and link. Based on a quick survey of the code here, I think that should work.

@mehdi_amini @ftynse, any comments?

mlir/tools/mlir-spirv-runner/runtime-wrappers.cpp
27 ↗(On Diff #286159)

THis seems like a test utility function. If so it might be better to move this into a test/ folder. Something like test/mlir-spirv-runner. See similar examples here : https://github.com/llvm/llvm-project/blob/master/mlir/test/mlir-cpu-runner/mlir_test_cblas.cpp

This revision now requires changes to proceed.Aug 17 2020, 11:16 PM
georgemitenkov marked 2 inline comments as done.

Moved the library to tests. Documented the else case for llvmModuleBuilder in ExecutionEngine.h.

mehdi_amini added inline comments.Aug 18 2020, 10:35 PM
mlir/lib/ExecutionEngine/ExecutionEngine.cpp
231

I am a bit confused by what you're suggesting here, where would the ArrayRef<ModuleOp> come from? ExecutionEngine operates on a single module right now.

mravishankar added inline comments.Aug 18 2020, 11:22 PM
mlir/lib/ExecutionEngine/ExecutionEngine.cpp
231

I am saying that ExecutionEngine::create can take an argument ArrayRef<ModuleOp> instead of just Module. The translateModuleToLLVMIR can be modified to handle multiple modules that get linked together before a single LLVM module is returned

georgemitenkov added inline comments.Aug 19 2020, 1:15 AM
mlir/lib/ExecutionEngine/ExecutionEngine.cpp
231

I am not sure why passing ArrayRef<ModuleOp> will be better? We then dedicate linking to translateModuleToLLVMIR(). If we use LLVM's Linker (https://github.com/llvm/llvm-project/blob/master/llvm/include/llvm/Linker/Linker.h), we may also want to specify some of the optional parameters?

FYI, I made a post on the forum so it is easier to discuss this.

Fixed an error on building SPIR-V runner.

Renamed the runner to mlir-spirv-cpu-runner to resolve the ambiguity of where the runner is actually executed. Also, rebased on updated pass infrastructure.

georgemitenkov retitled this revision from [MLIR][mlir-spirv-runner] A SPIR-V runner prototype to [MLIR][mlir-spirv-cpu-runner] A SPIR-V cpu runner prototype.Aug 20 2020, 3:42 AM

Updated the runner based on the previous patches and added a test for 3D memrefs with floats.

mravishankar requested changes to this revision.Aug 26 2020, 9:50 PM

Looking at the stack here, I think only the change below this one in the stack (which is good to go) and this would probably need me to submit this. It would be good to commit the other two dependent changes and then I could try these two out locally.

This revision now requires changes to proceed.Aug 26 2020, 9:50 PM

Looking at the stack here, I think only the change below this one in the stack (which is good to go) and this would probably need me to submit this. It would be good to commit the other two dependent changes and then I could try these two out locally.

Yes, only the pass converting gpu.launch_func and host code is needed. I am committing all accepted patches now :)

Rebase against master.

Trying this change out, I needed the following changes to get this to build locally. Can you verify this is needed?

index 560dcae8200..41f5c8a9ce5 100644
--- a/mlir/lib/Conversion/GPUCommon/CMakeLists.txt
+++ b/mlir/lib/Conversion/GPUCommon/CMakeLists.txt
@@ -34,6 +34,8 @@ add_mlir_conversion_library(MLIRGPUToGPURuntimeTransforms
   MLIRIR
   MLIRLLVMIR
   MLIRPass
+  MLIRSPIRV
   MLIRSupport
   MLIRStandardToLLVM
+  MLIRSPIRVToLLVM
 )
diff --git a/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt b/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
index 69080ae66dc..8e0c2a6ad67 100644
--- a/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
+++ b/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
@@ -15,6 +15,7 @@ if (MLIR_SPIRV_CPU_RUNNER_ENABLED)
   get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 
   target_link_libraries(mlir-spirv-cpu-runner PRIVATE
+    LLVMLinker
     ${conversion_libs}
     ${dialect_libs}
     MLIRAnalysis

Also there seems to be some failure that is unrelated to this changes, and I am not sure how to run the tests individually. I ran this command locally and I see this error

$ $HOME/llvm/llvm-project/build/bin/mlir-spirv-cpu-runner -e main --entry-point-result=void --shared-libs=$HOME/llvm/llvm-project/build/lib/libmlir_runner_utils.so,$HOME/llvm/llvm-project/build/lib/libmlir_test_spirv_cpu_runner_c_wrappers.so simple_add.mlir 
loc("simple_add.mlir":50:5): error: 'gpu.launch_func' op SPIR-V kernel module '__spv__kernels' is not found
loc("simple_add.mlir":50:5): error: 'gpu.launch_func' op kernel module 'kernels' is undefined

Any ideas about what I might be missing here?

Any ideas about what I might be missing here?

I think this is to do with strings and names in LowerHostToLLVMPass, cause there the build also fails.

Trying this change out, I needed the following changes to get this to build locally. Can you verify this is needed?

I will have a look at this one as well!

Rebased on the correct version of LowerHostToLLVMPass.

Trying this change out, I needed the following changes to get this to build locally. Can you verify this is needed?

index 560dcae8200..41f5c8a9ce5 100644
--- a/mlir/lib/Conversion/GPUCommon/CMakeLists.txt
+++ b/mlir/lib/Conversion/GPUCommon/CMakeLists.txt
@@ -34,6 +34,8 @@ add_mlir_conversion_library(MLIRGPUToGPURuntimeTransforms
   MLIRIR
   MLIRLLVMIR
   MLIRPass
+  MLIRSPIRV
   MLIRSupport
   MLIRStandardToLLVM
+  MLIRSPIRVToLLVM
 )
diff --git a/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt b/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
index 69080ae66dc..8e0c2a6ad67 100644
--- a/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
+++ b/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
@@ -15,6 +15,7 @@ if (MLIR_SPIRV_CPU_RUNNER_ENABLED)
   get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 
   target_link_libraries(mlir-spirv-cpu-runner PRIVATE
+    LLVMLinker
     ${conversion_libs}
     ${dialect_libs}
     MLIRAnalysis

I am building locally without these changes and it is working for me?

Also there seems to be some failure that is unrelated to this changes, and I am not sure how to run the tests individually. I ran this command locally and I see this error

$ $HOME/llvm/llvm-project/build/bin/mlir-spirv-cpu-runner -e main --entry-point-result=void --shared-libs=$HOME/llvm/llvm-project/build/lib/libmlir_runner_utils.so,$HOME/llvm/llvm-project/build/lib/libmlir_test_spirv_cpu_runner_c_wrappers.so simple_add.mlir 
loc("simple_add.mlir":50:5): error: 'gpu.launch_func' op SPIR-V kernel module '__spv__kernels' is not found
loc("simple_add.mlir":50:5): error: 'gpu.launch_func' op kernel module 'kernels' is undefined

Any ideas about what I might be missing here?

This was actually a LowerHostToLLVMPass problem - StringRef objects were destroyed and symbol names were messed up because of that. I fixed that so now everything should be running fine?

Trying this change out, I needed the following changes to get this to build locally. Can you verify this is needed?

index 560dcae8200..41f5c8a9ce5 100644
--- a/mlir/lib/Conversion/GPUCommon/CMakeLists.txt
+++ b/mlir/lib/Conversion/GPUCommon/CMakeLists.txt
@@ -34,6 +34,8 @@ add_mlir_conversion_library(MLIRGPUToGPURuntimeTransforms
   MLIRIR
   MLIRLLVMIR
   MLIRPass
+  MLIRSPIRV
   MLIRSupport
   MLIRStandardToLLVM
+  MLIRSPIRVToLLVM
 )
diff --git a/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt b/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
index 69080ae66dc..8e0c2a6ad67 100644
--- a/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
+++ b/mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt
@@ -15,6 +15,7 @@ if (MLIR_SPIRV_CPU_RUNNER_ENABLED)
   get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 
   target_link_libraries(mlir-spirv-cpu-runner PRIVATE
+    LLVMLinker
     ${conversion_libs}
     ${dialect_libs}
     MLIRAnalysis

I am building locally without these changes and it is working for me?

I was getting missing symbols error. But I dont see the errors in buildbot either. Thats strange.

Also there seems to be some failure that is unrelated to this changes, and I am not sure how to run the tests individually. I ran this command locally and I see this error

$ $HOME/llvm/llvm-project/build/bin/mlir-spirv-cpu-runner -e main --entry-point-result=void --shared-libs=$HOME/llvm/llvm-project/build/lib/libmlir_runner_utils.so,$HOME/llvm/llvm-project/build/lib/libmlir_test_spirv_cpu_runner_c_wrappers.so simple_add.mlir 
loc("simple_add.mlir":50:5): error: 'gpu.launch_func' op SPIR-V kernel module '__spv__kernels' is not found
loc("simple_add.mlir":50:5): error: 'gpu.launch_func' op kernel module 'kernels' is undefined

Any ideas about what I might be missing here?

This was actually a LowerHostToLLVMPass problem - StringRef objects were destroyed and symbol names were messed up because of that. I fixed that so now everything should be running fine?

Ill try it out again. THanks!

Latest patch works, but I still need the fixes to the builds (actually the fixes to GPUTransforms/CMakeLists.txt above is for the patch earlier in the stack).

Thanks George for making this available!

Latest patch works, but I still need the fixes to the builds (actually the fixes to GPUTransforms/CMakeLists.txt above is for the patch earlier in the stack).

Thanks George for making this available!

Great! Let me know if you have any comments.

ftynse accepted this revision.Sep 16 2020, 1:28 AM
ftynse added inline comments.
mlir/lib/ExecutionEngine/ExecutionEngine.cpp
231

I'm fine either way. The injection may be also useful to, e.g., separate the LLVM-only translation from LLVM+OpenMP translation that are now intermixed. It may look strange that a single runner does the module linking, but it removes the need for all other runners to (transitively) depend on the LLVM linker.

On the other hand, the ExecutionEngine already has a linking layer for resolving calls to dynamic libraries, so may be reasonable to have it also link multiple MLIR/LLVM modules together.

The growing complexity of ExecutionEngine API suggests that its users actually want lower-level control over the process, so it may be the right time to reconsider this abstraction. In the longer term, it is preferable to move MLIR-independent parts (better lookup APIs, void ** argument packing, dynamic library loading) into LLVM itself. Ideally, we should be able to produce an LLVM module from MLIR that "just runs" with (augmented) lli instead of having multiple custom runners. For GPU, we may still need some simple shims that split nested modules and link them.

mlir/lib/ExecutionEngine/JitRunner.cpp
299–300

Can we typedef this as TranslatioCallback or something? It's verbose and used too many times.

mlir/test/mlir-spirv-cpu-runner/mlir_test_spirv_cpu_runner_c_wrappers.cpp
16–25

I suspect we already have this declared elsewhere, e.g. https://github.com/llvm/llvm-project/blob/master/mlir/include/mlir/ExecutionEngine/CRunnerUtils.h#L120, could we rather reuse existing things than copy?

mlir/tools/mlir-spirv-cpu-runner/mlir-spirv-cpu-runner.cpp
58

I'd consider having a check that there is only one nested module.

georgemitenkov updated this revision to Diff 293026.EditedSep 20 2020, 12:49 PM
georgemitenkov marked 3 inline comments as done.

Addressed comments:

  • Added a typedef for LLVM module builder function in JitRunner.
  • Added a check for a single nested module and refactored convertMLIRModule function (now getOps<>() is used over the walk function).
  • Removed a duplicate MemRefDescriptor declaration.

Changed includes to match new file locations in lower-host-to-llvm pass.

Thanks George, let me try it. I am trying to add BUILD files for this to make it runnable internally. This refactoring should help. (Will try to land this soon).

This revision was not accepted when it landed; it landed in state Needs Review.Oct 26 2020, 6:10 AM
This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.