This is an archive of the discontinued LLVM Phabricator instance.

[mlir][vulkan-runner] Update mlir-vulkan-runner execution driver.
ClosedPublic

Authored by denis13 on Feb 26 2020, 8:58 AM.

Details

Summary

This patch:

  • Adds GpuLaunchFuncToVulkanLaunchFunc conversion pass.
  • Moves a serialization of the spirv::Module from LaunchFuncToVulkanCalls pass to newly created pass.
  • Updates LaunchFuncToVulkanCalls instrumentation pass, adds initVulkan and deinitVulkan runtime calls.
  • Adds bindResource call to bind specifc resource by the given descriptor set and descriptor binding.
  • Eliminates static construction and desctruction of VulkanRuntimeManager.

Diff Detail

Event Timeline

denis13 created this revision.Feb 26 2020, 8:58 AM
Herald added a project: Restricted Project. · View Herald Transcript

To eliminate static construction and desctruction of VulkanRuntimeManager, we need to instrument runtime calls with first argument as a generic pointer to VulkanRuntimeManager, but the setResource function based on std dialect, which fills the given memref with the given value, could not accept generic pointers, because std dialect does not have generic pointer types. So, to solve it, I added new pass which serializes spirv::Module to binary form appends it as a string attribute to vulkan launch call op. Erases all gpu.module and spirv::Module, so we can lower it to LLVM directly, then we instrument host part with runtime calls as before, but adding initVulkan, deinitVulkan calls and for each lowered memref we create call to bindResource according the current GPUToSPIRV convention (Each resource binds to 0 descriptor set and descriptor binding argument order lined up in argument order.

denis13 edited the summary of this revision. (Show Details)Feb 26 2020, 9:03 AM
denis13 edited the summary of this revision. (Show Details)
denis13 updated this revision to Diff 246788.Feb 26 2020, 10:47 AM

Disable clang-diagnostic for include vulkan header file.

denis13 updated this revision to Diff 247219.Feb 28 2020, 4:03 AM

Rebase on master.

antiagainst requested changes to this revision.Feb 28 2020, 11:19 AM

Nice, thanks Denis!

mlir/lib/Conversion/GPUToVulkan/ConvertGPULaunchFuncToVulkanLaunchFunc.cpp
31

These two are now used as attributes. The convention is to use snake_case for attribute names. So what about naming them as spirv_blob and spirv_entry_point?

33

I think we can just call it vulkanLaunch. The __ prefix can be confusing as for what purpose it serves. (I guess you want to mean this should be an intermediate step? It's not really necessary. If this is influenced by the _ name prefix in SPIR-V dialect, just wanted to let you know that it should also be cleaned up as said in the SPIR-V dialect's doc and I should create a bug for it.)

37

... by creating ... and attaching ....

50

We can drop the mlir:: prefix given we already have using namespace mlir at the beginning. Also for other places.

56

memRefType.hasRank() && memRefType.getRank() == 1

65

Put public section at the beginning.

72

Hmm, I see we want to support lanuching multiple kernels eventually but I think right now it's broken: we walk trough all lanuch_func op here but when serializing the SPIR-V blob we only expect one SPIR-V module. I think we should report error on seeing multiple launch_func ops for now.

88

Hmm, for a vulkan launch we can only control the number of work groups. Workgroup size is written into the kernel. So to properly modelling vulkan launch, we cannot have the local workgroup size configuration here...

Could you put a TODO here to address this later?

90

// Check that all operands have supported types except those for the launch configuration.

131

No need for ConvertGpuLaunchFuncToVulkanLaunchFunc:: given we are in the same class.

163

Convert gpu.launch_func to vulkanLaunch external call

mlir/lib/Conversion/GPUToVulkan/ConvertLaunchFuncToVulkanCalls.cpp
91

Period to end the sentence.

112

Hmm, this is quite complicated and fragile. I think we should consider using the C interface wrappers instead of handling all the details by ourselves. See https://mlir.llvm.org/docs/ConversionToLLVMDialect/#c-compatible-wrapper-emission.

Could you look into that? A TODO for now is fine given this change is quite large already.

119

for the index-th resource.

132–136

Returns the number of resources assuming ...

134

getResourceCount1D?

139

Document these fields.

167

We should be able to do auto operands = SmallVector<Value, 32>{vulkanLaunchCallOp.getOperands()}?

356

Convert vulkanLaunch external call to Vulkan runtime external calls

This revision now requires changes to proceed.Feb 28 2020, 11:19 AM

@antiagainst thanks a lot for detailed review! I'll update soon according to your suggestions, thanks!

denis13 updated this revision to Diff 248594.Mar 5 2020, 1:21 PM

Address comments, sorry for delay.

denis13 retitled this revision from [mlir][spirv] Update mlir-vulkan-runner execution driver. to [mlir][vulkan-runner] Update mlir-vulkan-runner execution driver..Mar 5 2020, 1:24 PM
flaub added a subscriber: flaub.Mar 5 2020, 1:48 PM
antiagainst accepted this revision.Mar 10 2020, 7:28 AM

Awesome, thanks Denis!! I've a few other nits but I'll fix it before landing.

This revision is now accepted and ready to land.Mar 10 2020, 7:28 AM
This revision was automatically updated to reflect the committed changes.
rriddle added inline comments.Mar 10 2020, 1:26 PM
mlir/lib/Conversion/GPUToVulkan/ConvertGPULaunchFuncToVulkanLaunchFunc.cpp
37

Use /// for top level comments.

120

Why not check the range for empty() || hasSingleElement() before doing anything? If there can only be one, we don't need a loop.

mlir/lib/Conversion/GPUToVulkan/ConvertLaunchFuncToVulkanCalls.cpp
130
156

Can we have a comment? Why 6?

mlir/tools/mlir-vulkan-runner/VulkanRuntime.h
25 ↗(On Diff #249483)

Let's remove this.

mlir/tools/mlir-vulkan-runner/vulkan-runtime-wrappers.cpp
66–73

Use /// for top-level comments.

This seems to have broken build on gcc 5:
error: converting to 'std::tuple<mlir::Value, mlir::Value>' from initializer list would use explicit constructor 'constexpr std::tuple<_T1, _T2>::tuple(const _T1&, const _T2&) [with _T1 = mlir::Value; _T2 = mlir::Value]'

operands[numConfigOps + index * loweredMemRefNumOps1D + 3]};
                                                          ^
denis13 marked an inline comment as done.Mar 11 2020, 1:36 AM
denis13 added inline comments.
mlir/lib/Conversion/GPUToVulkan/ConvertLaunchFuncToVulkanCalls.cpp
130

This seems to have broken build on gcc 5:
error: converting to 'std::tuple<mlir::Value, mlir::Value>' from initializer list would use explicit constructor 'constexpr std::tuple<_T1, _T2>::tuple(const _T1&, const _T2&) [with _T1 = mlir::Value; _T2 = mlir::Value]'

operands[numConfigOps + index * loweredMemRefNumOps1D + 3]};
                                                          ^

Sorry about it, should be fixed by https://github.com/llvm/llvm-project/commit/5b0c60c58ea426b215cfd2970c9eaa6eb5b42026

@rriddle thanks for review, will address it in follow up pr.