[MLIR] Provide bare pointer memref lowering option on gpu-to-nvvm pass

[bondhugula]

Summary

Provide the bare pointer memref lowering option on gpu-to-nvvm pass.
This is needed whenever we lower memrefs on the host function side and
the kernel calls on the host-side (gpu-to-llvm) with the bare ptr
convention. The GPU module side of the lowering should also "align" and
use the bare pointer convention.

[bondhugula]

Set the option more naturally.

[krzysz00]

LGTM

[krzysz00]

You might want to add a test though

[bondhugula]

Add integration test cases for bare pointer memref lowering for kernels as well
as host-side IR. Tests the options for both gpu-to-nvvm and gpu-to-llvm.

[bondhugula]

@mehdi_amini Here's the test case for using bare pointers on the host side as well.

[bondhugula]

Adjust comments and dealloc.

[mehdi_amini]

Can you add a unit-test please? Integration test are never replacement for unit-tests.

[bondhugula]

Add unit test case for memref lowering for gpu-to-nvvm.

[bondhugula]

In D152480#4430524, @mehdi_amini wrote:

Can you add a unit-test please? Integration test are never replacement for unit-tests.

Done - added unit test case. (The reason I thought it wasn't necessary was that memref bare ptr lowering already had unit test cases in func-to-llvm. The testing here is repeating/exercising the same lowering. But it's good to have it anyway to show all the users of the feature in the tree and to ensure the option is indeed working.)

mlir/include/mlir/Conversion/Passes.td

let dependentDialects = [
     "NVVM::NVVMDialect",
   ];
   let options = [
     Option<"indexBitwidth", "index-bitwidth", "unsigned",
            /*default=kDeriveIndexBitwidthFromDataLayout*/"0",
            "Bitwidth of the index type, 0 to use size of machine word">,
     Option<"hasRedux", "has-redux", "bool", /*default=*/"false",
            "Target gpu supports redux">,
+    Option<"useBarePtrCallConv", "use-bare-ptr-memref-call-conv", "bool",
+           /*default=*/"false",
+           "Replace memref arguments in GPU functions with bare pointers. "
+           "All memrefs must have static shape.">,
     Option<"useOpaquePointers", "use-opaque-pointers", "bool",
                    /*default=*/"true", "Generate LLVM IR using opaque pointers "
                    "instead of typed pointers">,
   ];
 }
 
 //===----------------------------------------------------------------------===//
 // GPUToROCDL

mlir/lib/Conversion/GPUToNVVM/LowerGpuOpsToNVVMOps.cpp

void runOnOperation() override {
 
     // Customize the bitwidth used for the device side index computations.
     LowerToLLVMOptions options(
         m.getContext(),
         DataLayout(cast<DataLayoutOpInterface>(m.getOperation())));
     if (indexBitwidth != kDeriveIndexBitwidthFromDataLayout)
       options.overrideIndexBitwidth(indexBitwidth);
     options.useOpaquePointers = useOpaquePointers;
+    if (useBarePtrCallConv)
+      options.useBarePtrCallConv = true;
 
     // Apply in-dialect lowering. In-dialect lowering will replace
     // ops which need to be lowered further, which is not supported by a
     // single conversion pass.
     {
       RewritePatternSet patterns(m.getContext());
       populateGpuRewritePatterns(patterns);
       if (failed(applyPatternsAndFoldGreedily(m, std::move(patterns))))