[mlir][openacc] Conversion of data operand to LLVM IR dialect

[clementval]

Summary

Add a conversion pass to convert higher-level type before translation.
This conversion extract meangingful information and pack it into a struct that
the translation (D101504) will be able to understand.

[clementval]

Fix header guard

[ftynse]

Thanks! I think this goes into the right direction. I have a couple of small comments and two concerns:

• the semantics of the OpenACC data descriptor is not documented, so it's hard to tell if the way it is obtained from a memref descriptor is correct or not; most likely it is not because it shouldn't be using the _allocated_ pointer twice;
• the conversion pass shouldn't subsume std-to-llvm conversion, unless you can't make it work without it.

mlir/include/mlir/Dialect/OpenACC/OpenACCOps.td

256

Nit: operand data -> data operand

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

85

91–92

This looks redundant. dataOperands presumably already contains the relevant operands, no need to enumerate it and get them again.

98

Nit: SmallVector has a default for the number of stack elements. Keep it unless there's a strong reason to choose a different value.

104

This should use the aligned pointer rather than allocated pointer for data access. The allocated pointer is there for freeing the data and may point up to (alignment - 1) bytes before the data you want to read.

115

In most similar cases, we just return failure when the pattern doesn't apply. This isn't as nice for error reporting (the pass may need to do a pre- or post-check, or report a general "conversion failure"), but makes patterns compose. Somebody will be able to use this pattern and have a separate pattern handling their custom type.

Also note that emitting an error does not return, so the code below is still executed.

119

I would have used op.getResultTypes instead of always using the empty type range, for future-proofness if anything.

147

Ideally, we shouldn't be running "std-to-llvm" conversion along with other conversions. I cleaned up most of these, but not all. This leads to poor composability and encourages monolithic conversion flows that don't play nicely with each other (again, suppose we have a third dialect that wants to convert to OpenACC, it would have to recreate all duplicate all of this file).

Consider configuring ConversionTarget to provide materialization that inserts llvm.mlir.cast on type mismatch instead, then run std-to-llvm as a separate pass.

153

Nit: I'd rather call this allDataOperandsAreConverted or something similar that indicates universal quantification. I'd expect hasSomething to return true if there is at least one element of something, but this returns true if all elements are something.

190

Please add the newline.

[ftynse]

mlir/lib/Conversion/OpenACCToLLVM/CMakeLists.txt

10

This doesn't look necessary here.

12–13

This neither.

[kiranchandramohan]

It will be good to have the description of the descriptor you are using in the Summary or in the code.

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

114

What about the llvm array type? https://mlir.llvm.org/docs/Dialects/LLVM/#array-types

[clementval]

Address review comments

[clementval]

In D102170#2748010, @ftynse wrote:

Thanks! I think this goes into the right direction. I have a couple of small comments and two concerns:

• the semantics of the OpenACC data descriptor is not documented, so it's hard to tell if the way it is obtained from a memref descriptor is correct or not; most likely it is not because it shouldn't be using the _allocated_ pointer twice;
• the conversion pass shouldn't subsume std-to-llvm conversion, unless you can't make it work without it.

Thanks for the extensive review! I have added information about what kind of information is expected to be hold by the data descriptor. For the composition of passes I guess that if we do not run std-to-llvm together with this pass then we should probably process memref just as any other !llvm.struct since we would not have the relation between memref and the converted !llvm.struct. Or did I miss something?

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

85

Thanks. Don't know what I was trying to say here :-)

91–92

operand and data do not have the same type. In case of memref for example, data is still of type memref while operand has the llvm converted type !llvm.struct<. I thought it was the way to intercept memref conversion and being able to extract the meaningful information while we still know it is coming from a memref. I took this idea from here: mlir/lib/Conversion/SPIRVToLLVM/ConvertLaunchFuncToLLVMCalls.cpp.

114

We might add it later if there is a need. "Unsupported" should be read as "Not supported right now but might get support later".

115

Makes sense. Thanks for the info.

119

The three operations EnterDataOp, ExitDataOp and UpdateOp do not have results so they do not have the getResultTypes() function. At least for now.

147

I fully understand that this lead to poor composability. But if run separately then there is no way to know which !llvm.struct where in fact memref when we run this conversion pass. Or did I miss something?

153

Right, it makes more sense.

[ftynse]

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

91–92

I see. Consider naming them accordingly then, something like originalDataOperand and transformedDataOperand.

119

Nit: you can also use return rewriter.notifyMatchFailure("message") for more detailed information.

147

If you run this pass before -std-to-llvm (most conversions are currently set up so that -std-to-llvm runs last anyway), you can convert

%0 = "memref.producer"() : () -> memref<...>
openacc.enterdata copyin(%0) {
 ...
}

into

%0 = "memref.producer"() : () -> memref<...>
%1 = llvm.mlir.cast %0 : memref<...> to !llvm.struct<normal-descriptor>
%2 = llvm.extractvalue %1[1] : !llvm.struct<normal-descriptor>
...
%42 = llvm.mlir.undef : !llvm.struct<openacc-descriptor>
%43 = llvm.insertvalue %42[0], %2 : !llvm.struct<openacc-descriptor>
...
openacc.enterdata copyin(%43)

Normally, if you use LLVMConversionTarget, you shouldn't even need the to insert the llvm.mlir.cast operation yourself. After std-to-llvm runs, both sides of llvm.mlir.cast will have the same type and it will be folded away.

[clementval]

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

147

Ok I see. Thanks a lot for the hint it makes lots of sense to do it this. I will try to set this up.

[clementval]

@ftynse Thanks for the quick and very useful review. I made the conversion independent from "std-to-llvm".

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

147

Thanks for the useful hint. "std-to-llvm" is not run anymore with this conversions.

[clementval]

Make the conversion independent of std-to-llvm. Conversion can now be run before
std-to-llvm and cast op will be folded.

Address other review comments.

[clementval]

Rebase

[rriddle]

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

75

This class should be in an anonymous namespace.

mlir/include/mlir/Conversion/OpenACCToLLVM/ConvertOpenACCToLLVM.h

+//===- ConvertOpenACCToLLVM.h - OpenACC conversion pass entrypoint --------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#ifndef MLIR_CONVERSION_OPENACCTOLLVM_CONVERTOPENACCTOLLVMPASS_H_
+#define MLIR_CONVERSION_OPENACCTOLLVM_CONVERTOPENACCTOLLVMPASS_H_
+
+#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
+#include <memory>
+
+namespace mlir {
+class LLVMTypeConverter;
+class ModuleOp;
+template <typename T>
+class OperationPass;
+class RewritePatternSet;
+
+static constexpr unsigned kPtrBasePosInDataDescriptor = 0;
+static constexpr unsigned kPtrPosInDataDescriptor = 1;
+static constexpr unsigned kSizePosInDataDescriptor = 2;
+
+class DataDescriptor : public StructBuilder {
+public:
+  /// Construct a helper for the given descriptor value.
+  explicit DataDescriptor(Value descriptor);
+  /// Builds IR creating an `undef` value of the descriptor type.
+  static DataDescriptor undef(OpBuilder &builder, Location loc, Type basePtrTy,
+                              Type ptrTy);
+
+  static bool isValid(Value descriptor);
+
+  void setPointer(OpBuilder &builder, Location loc, Value ptr);
+  void setBasePointer(OpBuilder &builder, Location loc, Value basePtr);
+  void setSize(OpBuilder &builder, Location loc, Value size);
+};
+
+/// Collect the patterns to convert from the OpenACC dialect LLVMIR dialect.
+void populateOpenACCToLLVMConversionPatterns(LLVMTypeConverter &converter,
+                                             RewritePatternSet &patterns);
+
+/// Create a pass to convert the OpenACC dialect into the LLVMIR dialect.
+std::unique_ptr<OperationPass<ModuleOp>> createConvertOpenACCToLLVMPass();
+
+} // namespace mlir
+
+#endif // MLIR_CONVERSION_OPENACCTOLLVM_CONVERTOPENACCTOLLVMPASS_H_

mlir/include/mlir/Conversion/Passes.h

 #include "mlir/Conversion/GPUToNVVM/GPUToNVVMPass.h"
 #include "mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h"
 #include "mlir/Conversion/GPUToSPIRV/GPUToSPIRVPass.h"
 #include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h"
 #include "mlir/Conversion/LinalgToLLVM/LinalgToLLVM.h"
 #include "mlir/Conversion/LinalgToSPIRV/LinalgToSPIRVPass.h"
 #include "mlir/Conversion/LinalgToStandard/LinalgToStandard.h"
 #include "mlir/Conversion/MathToLibm/MathToLibm.h"
+#include "mlir/Conversion/OpenACCToLLVM/ConvertOpenACCToLLVM.h"
 #include "mlir/Conversion/OpenMPToLLVM/ConvertOpenMPToLLVM.h"
 #include "mlir/Conversion/PDLToPDLInterp/PDLToPDLInterp.h"
 #include "mlir/Conversion/SCFToGPU/SCFToGPUPass.h"
 #include "mlir/Conversion/SCFToOpenMP/SCFToOpenMP.h"
 #include "mlir/Conversion/SCFToSPIRV/SCFToSPIRVPass.h"
 #include "mlir/Conversion/SCFToStandard/SCFToStandard.h"
 #include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVMPass.h"
 #include "mlir/Conversion/ShapeToStandard/ShapeToStandard.h"

mlir/include/mlir/Conversion/Passes.td

def ConvertMathToLibm : Pass<"convert-math-to-libm", "ModuleOp"> {
   let description = [{
     This pass converts supported Math ops to libm calls.
   }];
   let constructor = "mlir::createConvertMathToLibmPass()";
   let dependentDialects = ["StandardOpsDialect", "vector::VectorDialect"];
 }
 
 //===----------------------------------------------------------------------===//
+// OpenACCToLLVM
+//===----------------------------------------------------------------------===//
+
+def ConvertOpenACCToLLVM : Pass<"convert-openacc-to-llvm", "ModuleOp"> {
+  let summary = "Convert the OpenACC ops to LLVM dialect";
+  let constructor = "mlir::createConvertOpenACCToLLVMPass()";
+  let dependentDialects = ["LLVM::LLVMDialect"];
+}
+
+//===----------------------------------------------------------------------===//
 // OpenMPToLLVM
 //===----------------------------------------------------------------------===//
 
 def ConvertOpenMPToLLVM : Pass<"convert-openmp-to-llvm", "ModuleOp"> {
   let summary = "Convert the OpenMP ops to OpenMP ops with LLVM dialect";
   let constructor = "mlir::createConvertOpenMPToLLVMPass()";
   let dependentDialects = ["LLVM::LLVMDialect"];
 }

mlir/include/mlir/Dialect/OpenACC/OpenACCOps.td

let arguments = (ins Optional<I1>:$ifCond,
                        Optional<IntOrIndex>:$waitDevnum,
                        Variadic<IntOrIndex>:$waitOperands,
                        UnitAttr:$wait,
                        Variadic<AnyType>:$copyinOperands,
                        Variadic<AnyType>:$createOperands,
                        Variadic<AnyType>:$createZeroOperands,
                        Variadic<AnyType>:$attachOperands);
 
+  let extraClassDeclaration = [{
+    /// The number of data operands.
+    unsigned getNumDataOperands();
+
+    /// The i-th operand data passed.
+    Value getDataOperand(unsigned i);
+  }];
+
   let assemblyFormat = [{
     ( `if` `(` $ifCond^ `)` )?
     ( `async` `(` $asyncOperand^ `:` type($asyncOperand) `)` )?
     ( `wait_devnum` `(` $waitDevnum^ `:` type($waitDevnum) `)` )?
     ( `wait` `(` $waitOperands^ `:` type($waitOperands) `)` )?
     ( `copyin` `(` $copyinOperands^ `:` type($copyinOperands) `)` )?
     ( `create` `(` $createOperands^ `:` type($createOperands) `)` )?
     ( `create_zero` `(` $createZeroOperands^ `:`
let arguments = (ins Optional<I1>:$ifCond,
                        Optional<IntOrIndex>:$waitDevnum,
                        Variadic<IntOrIndex>:$waitOperands,
                        UnitAttr:$wait,
                        Variadic<AnyType>:$copyoutOperands,
                        Variadic<AnyType>:$deleteOperands,
                        Variadic<AnyType>:$detachOperands,
                        UnitAttr:$finalize);
 
+  let extraClassDeclaration = [{
+    /// The number of data operands.
+    unsigned getNumDataOperands();
+
+    /// The i-th operand data passed.
+    Value getDataOperand(unsigned i);
+  }];
+
   let assemblyFormat = [{
     ( `if` `(` $ifCond^ `)` )?
     ( `async` `(` $asyncOperand^ `:` type($asyncOperand) `)` )?
     ( `wait_devnum` `(` $waitDevnum^ `:` type($waitDevnum) `)` )?
     ( `wait` `(` $waitOperands^ `:` type($waitOperands) `)` )?
     ( `copyout` `(` $copyoutOperands^ `:` type($copyoutOperands) `)` )?
     ( `delete` `(` $deleteOperands^ `:` type($deleteOperands) `)` )?
     ( `detach` `(` $detachOperands^ `:` type($detachOperands) `)` )?
let arguments = (ins Optional<IntOrIndex>:$asyncOperand,
                        UnitAttr:$async,
                        UnitAttr:$wait,
                        Variadic<IntOrIndex>:$deviceTypeOperands,
                        Optional<I1>:$ifCond,
                        Variadic<AnyType>:$hostOperands,
                        Variadic<AnyType>:$deviceOperands,
                        UnitAttr:$ifPresent);
 
+  let extraClassDeclaration = [{
+    /// The number of data operands.
+    unsigned getNumDataOperands();
+
+    /// The i-th operand data passed.
+    Value getDataOperand(unsigned i);
+  }];
+
   let assemblyFormat = [{
     ( `if` `(` $ifCond^ `)` )?
     ( `async` `(` $asyncOperand^ `:` type($asyncOperand) `)` )?
     ( `wait_devnum` `(` $waitDevnum^ `:` type($waitDevnum) `)` )?
     ( `device_type` `(` $deviceTypeOperands^ `:`
         type($deviceTypeOperands) `)` )?
     ( `wait` `(` $waitOperands^ `:` type($waitOperands) `)` )?
     ( `host` `(` $hostOperands^ `:` type($hostOperands) `)` )?

mlir/lib/Conversion/CMakeLists.txt

 add_subdirectory(AffineToStandard)
 add_subdirectory(AsyncToLLVM)
 add_subdirectory(ComplexToLLVM)
 add_subdirectory(ComplexToStandard)
 add_subdirectory(GPUCommon)
 add_subdirectory(GPUToNVVM)
 add_subdirectory(GPUToROCDL)
 add_subdirectory(GPUToSPIRV)
 add_subdirectory(GPUToVulkan)
 add_subdirectory(LinalgToLLVM)
 add_subdirectory(LinalgToSPIRV)
 add_subdirectory(LinalgToStandard)
 add_subdirectory(MathToLibm)
+add_subdirectory(OpenACCToLLVM)
 add_subdirectory(OpenMPToLLVM)
 add_subdirectory(PDLToPDLInterp)
 add_subdirectory(SCFToGPU)
 add_subdirectory(SCFToOpenMP)
 add_subdirectory(SCFToSPIRV)
 add_subdirectory(SCFToStandard)
 add_subdirectory(ShapeToStandard)
 add_subdirectory(SPIRVToLLVM)

mlir/lib/Conversion/OpenACCToLLVM/CMakeLists.txt

+add_mlir_conversion_library(MLIROpenACCToLLVM
+  OpenACCToLLVM.cpp
+
+  ADDITIONAL_HEADER_DIRS
+  ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/OpenACCToLLVM
+
+  DEPENDS
+  MLIRConversionPassIncGen
+  intrinsics_gen
+
+  LINK_COMPONENTS
+  Core
+
+  LINK_LIBS PUBLIC
+  MLIRIR
+  MLIRLLVMIR
+  MLIROpenACC
+  MLIRStandardToLLVM
+  MLIRTransforms
+  )

mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp

+//===- OpenACCToLLVM.cpp - Prepare OpenACC data for LLVM translation ------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "../PassDetail.h"
+#include "mlir/Conversion/OpenACCToLLVM/ConvertOpenACCToLLVM.h"
+#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
+#include "mlir/Dialect/OpenACC/OpenACC.h"
+
+using namespace mlir;
+
+//===----------------------------------------------------------------------===//
+// DataDescriptor implementation
+//===----------------------------------------------------------------------===//
+
+constexpr StringRef getStructName() { return "openacc_data"; }
+
+/// Construct a helper for the given descriptor value.
+DataDescriptor::DataDescriptor(Value descriptor) : StructBuilder(descriptor) {
+  assert(value != nullptr && "value cannot be null");
+}
+
+/// Builds IR creating an `undef` value of the data descriptor.
+DataDescriptor DataDescriptor::undef(OpBuilder &builder, Location loc,
+                                     Type basePtrTy, Type ptrTy) {
+  Type descriptorType = LLVM::LLVMStructType::getNewIdentified(
+      builder.getContext(), getStructName(),
+      {basePtrTy, ptrTy, builder.getI64Type()});
+  Value descriptor = builder.create<LLVM::UndefOp>(loc, descriptorType);
+  return DataDescriptor(descriptor);
+}
+
+/// Check whether the type is a valid data descriptor.
+bool DataDescriptor::isValid(Value descriptor) {
+  if (auto type = descriptor.getType().dyn_cast<LLVM::LLVMStructType>()) {
+    if (type.isIdentified() && type.getName().startswith(getStructName()) &&
+        type.getBody().size() == 3 &&
+        type.getBody()[kPtrBasePosInDataDescriptor]
+            .isa<LLVM::LLVMPointerType>() &&
+        type.getBody()[kPtrPosInDataDescriptor].isa<LLVM::LLVMPointerType>() &&
+        type.getBody()[kSizePosInDataDescriptor].isInteger(64))
+      return true;
+  }
+  return false;
+}
+
+/// Builds IR inserting the base pointer value into the descriptor.
+void DataDescriptor::setBasePointer(OpBuilder &builder, Location loc,
+                                    Value basePtr) {
+  setPtr(builder, loc, kPtrBasePosInDataDescriptor, basePtr);
+}
+
+/// Builds IR inserting the pointer value into the descriptor.
+void DataDescriptor::setPointer(OpBuilder &builder, Location loc, Value ptr) {
+  setPtr(builder, loc, kPtrPosInDataDescriptor, ptr);
+}
+
+/// Builds IR inserting the size value into the descriptor.
+void DataDescriptor::setSize(OpBuilder &builder, Location loc, Value size) {
+  setPtr(builder, loc, kSizePosInDataDescriptor, size);
+}
+
+//===----------------------------------------------------------------------===//
+// Conversion patterns
+//===----------------------------------------------------------------------===//
+
+template <typename Op>
+class LegalizeDataOpForLLVMTranslation : public ConvertOpToLLVMPattern<Op> {
+  using ConvertOpToLLVMPattern<Op>::ConvertOpToLLVMPattern;
+
+  LogicalResult
+  matchAndRewrite(Op op, ArrayRef<Value> operands,
+                  ConversionPatternRewriter &builder) const override {
+    Location loc = op.getLoc();
+
+    // Go over the data operand and legalize them for translation.
+    unsigned numDataOperand = op.getNumDataOperands();
+    auto dataOperands = operands.take_back(numDataOperand);
+
+    // Keep the non data operands are kept without modification.
+    auto nonDataOperands =
+        operands.take_front(operands.size() - numDataOperand);
+    SmallVector<Value> convertedOperands;
+    convertedOperands.append(nonDataOperands.begin(), nonDataOperands.end());
+
+    for (auto operand : llvm::enumerate(dataOperands)) {
+      Value data = op.getDataOperand(operand.index());
+
+      // Traverse operands that were converted to MemRefDescriptors.
+      if (auto memRefType = data.getType().dyn_cast<MemRefType>()) {
+        // Calculate the size of the memref and get the pointer to the allocated
+        // buffer.
+        SmallVector<Value, 4> sizes;
+        SmallVector<Value, 4> strides;
+        Value sizeBytes;
+        ConvertToLLVMPattern::getMemRefDescriptorSizes(
+            loc, memRefType, {}, builder, sizes, strides, sizeBytes);
+        MemRefDescriptor descriptor(operand.value());
+        Value dataPtr = descriptor.allocatedPtr(builder, loc);
+        auto ptrType = descriptor.getElementPtrType();
+
+        auto descr = DataDescriptor::undef(builder, loc, ptrType, ptrType);
+        descr.setBasePointer(builder, loc, dataPtr);
+        descr.setPointer(builder, loc, dataPtr);
+        descr.setSize(builder, loc, sizeBytes);
+        convertedOperands.push_back(descr);
+      } else if (data.getType().isa<LLVM::LLVMPointerType>()) {
+        convertedOperands.push_back(data);
+      } else {
+        op.emitError("Unsupported type ") << data.getType();
+      }
+    }
+
+    builder.replaceOpWithNewOp<Op>(op, TypeRange(), convertedOperands,
+                                   op.getOperation()->getAttrs());
+
+    return success();
+  }
+};
+
+void mlir::populateOpenACCToLLVMConversionPatterns(
+    LLVMTypeConverter &converter, OwningRewritePatternList &patterns) {
+  patterns.add<LegalizeDataOpForLLVMTranslation<acc::EnterDataOp>>(converter);
+  patterns.add<LegalizeDataOpForLLVMTranslation<acc::ExitDataOp>>(converter);
+  patterns.add<LegalizeDataOpForLLVMTranslation<acc::UpdateOp>>(converter);
+}
+
+namespace {
+struct ConvertOpenACCToLLVMPass
+    : public ConvertOpenACCToLLVMBase<ConvertOpenACCToLLVMPass> {
+  void runOnOperation() override;
+};
+} // namespace
+
+void ConvertOpenACCToLLVMPass::runOnOperation() {
+  auto op = getOperation();
+  auto *context = op.getContext();
+
+  // Convert to OpenACC operations with LLVM IR dialect
+  RewritePatternSet patterns(context);
+  LLVMTypeConverter converter(context);
+  populateStdToLLVMConversionPatterns(converter, patterns);
+  populateOpenACCToLLVMConversionPatterns(converter, patterns);
+
+  ConversionTarget target(*context);
+  target.addLegalDialect<LLVM::LLVMDialect>();
+
+  auto hasLLVMConvertedDataOperands = [](ValueRange operands) {
+    for (Value operand : operands) {
+      if (!DataDescriptor::isValid(operand) &&
+          !operand.getType().isa<LLVM::LLVMPointerType>())
+        return false;
+    }
+    return true;
+  };
+
+  target.addDynamicallyLegalOp<acc::EnterDataOp>(
+      [hasLLVMConvertedDataOperands](acc::EnterDataOp op) {
+        return hasLLVMConvertedDataOperands(op.copyinOperands()) &&
+               hasLLVMConvertedDataOperands(op.createOperands()) &&
+               hasLLVMConvertedDataOperands(op.createZeroOperands()) &&
+               hasLLVMConvertedDataOperands(op.attachOperands());
+      });
+
+  target.addDynamicallyLegalOp<acc::ExitDataOp>(
+      [hasLLVMConvertedDataOperands](acc::ExitDataOp op) {
+        return hasLLVMConvertedDataOperands(op.copyoutOperands()) &&
+               hasLLVMConvertedDataOperands(op.deleteOperands()) &&
+               hasLLVMConvertedDataOperands(op.detachOperands());
+      });
+
+  target.addDynamicallyLegalOp<acc::UpdateOp>(
+      [hasLLVMConvertedDataOperands](acc::UpdateOp op) {
+        return hasLLVMConvertedDataOperands(op.hostOperands()) &&
+               hasLLVMConvertedDataOperands(op.deviceOperands());
+      });
+
+  if (failed(applyPartialConversion(op, target, std::move(patterns))))
+    signalPassFailure();
+}
+
+std::unique_ptr<OperationPass<ModuleOp>>
+mlir::createConvertOpenACCToLLVMPass() {
+  return std::make_unique<ConvertOpenACCToLLVMPass>();
+}
 No newline at end of file

mlir/lib/Dialect/OpenACC/IR/OpenACC.cpp

if (!op.waitOperands().empty() && op.wait())
     return op.emitError("wait attribute cannot appear with waitOperands");
 
   if (op.waitDevnum() && op.waitOperands().empty())
     return op.emitError("wait_devnum cannot appear without waitOperands");
 
   return success();
 }
 
+unsigned ExitDataOp::getNumDataOperands() {
+  return copyoutOperands().size() + deleteOperands().size() +
+         detachOperands().size();
+}
+
+Value ExitDataOp::getDataOperand(unsigned i) {
+  unsigned numOptional = ifCond() ? 1 : 0;
+  numOptional += asyncOperand() ? 1 : 0;
+  numOptional += waitDevnum() ? 1 : 0;
+  return getOperand(waitOperands().size() + numOptional + i);
+}
+
 //===----------------------------------------------------------------------===//
-// DataEnterOp
+// EnterDataOp
 //===----------------------------------------------------------------------===//
 
 static LogicalResult verify(acc::EnterDataOp op) {
   // 2.6.6. Data Enter Directive restriction
   // At least one copyin, create, or attach clause must appear on an enter data
   // directive.
   if (op.copyinOperands().empty() && op.createOperands().empty() &&
       op.createZeroOperands().empty() && op.attachOperands().empty())
if (!op.waitOperands().empty() && op.wait())
     return op.emitError("wait attribute cannot appear with waitOperands");
 
   if (op.waitDevnum() && op.waitOperands().empty())
     return op.emitError("wait_devnum cannot appear without waitOperands");
 
   return success();
 }
 
+unsigned EnterDataOp::getNumDataOperands() {
+  return copyinOperands().size() + createOperands().size() +
+         createZeroOperands().size() + attachOperands().size();
+}
+
+Value EnterDataOp::getDataOperand(unsigned i) {
+  unsigned numOptional = ifCond() ? 1 : 0;
+  numOptional += asyncOperand() ? 1 : 0;
+  numOptional += waitDevnum() ? 1 : 0;
+  return getOperand(waitOperands().size() + numOptional + i);
+}
+
 //===----------------------------------------------------------------------===//
 // InitOp
 //===----------------------------------------------------------------------===//
 
 static LogicalResult verify(acc::InitOp initOp) {
   Operation *currOp = initOp;
   while ((currOp = currOp->getParentOp())) {
     if (isComputeOperation(currOp))
if (updateOp.waitOperands().size() > 0 && updateOp.wait())
     return updateOp.emitError("wait attribute cannot appear with waitOperands");
 
   if (updateOp.waitDevnum() && updateOp.waitOperands().size() == 0)
     return updateOp.emitError("wait_devnum cannot appear without waitOperands");
 
   return success();
 }
 
+unsigned UpdateOp::getNumDataOperands() {
+  return hostOperands().size() + deviceOperands().size();
+}
+
+Value UpdateOp::getDataOperand(unsigned i) {
+  unsigned numOptional = asyncOperand() ? 1 : 0;
+  numOptional += waitDevnum() ? 1 : 0;
+  numOptional += ifCond() ? 1 : 0;
+  return getOperand(waitOperands().size() + deviceTypeOperands().size() +
+                    numOptional + i);
+}
+
 //===----------------------------------------------------------------------===//
 // WaitOp
 //===----------------------------------------------------------------------===//
 
 static LogicalResult verify(acc::WaitOp waitOp) {
   // The async attribute represent the async clause without value. Therefore the
   // attribute and operand cannot appear at the same time.
   if (waitOp.asyncOperand() && waitOp.async())

mlir/test/Conversion/OpenACCToLLVM/convert-standalone-data-to-llvmir.mlir

+// RUN: mlir-opt -convert-openacc-to-llvm -split-input-file %s | FileCheck %s
+
+func @testenterdataop(%a: memref<10xf32>, %b: memref<10xf32>) -> () {
+  acc.enter_data copyin(%b : memref<10xf32>) create(%a : memref<10xf32>)
+  return
+}
+
+// CHECK: acc.enter_data copyin(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) create(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<f32>, ptr<f32>, i64)>)
+
+// -----
+
+func @testenterdataop(%a: !llvm.ptr<f32>, %b: memref<10xf32>) -> () {
+  acc.enter_data copyin(%b : memref<10xf32>) create(%a : !llvm.ptr<f32>)
+  return
+}
+
+// CHECK: acc.enter_data copyin(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) create(%{{.*}} : !llvm.ptr<f32>)
+
+// -----
+
+func @testenterdataop(%a: memref<10xi64>, %b: memref<10xf32>) -> () {
+  acc.enter_data copyin(%b : memref<10xf32>) create_zero(%a : memref<10xi64>) attributes {async}
+  return
+}
+
+// CHECK: acc.enter_data copyin(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) create_zero(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<i64>, ptr<i64>, i64)>) attributes {async}
+
+// -----
+
+func @testenterdataop(%a: memref<10xf32>, %b: memref<10xf32>) -> () {
+  %ifCond = constant true
+  acc.enter_data if(%ifCond) copyin(%b : memref<10xf32>) create(%a : memref<10xf32>)
+  return
+}
+
+// CHECK: acc.enter_data if(%{{.*}}) copyin(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) create(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<f32>, ptr<f32>, i64)>)
+
+// -----
+
+func @testexitdataop(%a: memref<10xf32>, %b: memref<10xf32>) -> () {
+  acc.exit_data copyout(%b : memref<10xf32>) delete(%a : memref<10xf32>)
+  return
+}
+
+// CHECK: acc.exit_data copyout(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) delete(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<f32>, ptr<f32>, i64)>)
+
+// -----
+
+func @testexitdataop(%a: !llvm.ptr<f32>, %b: memref<10xf32>) -> () {
+  acc.exit_data copyout(%b : memref<10xf32>) delete(%a : !llvm.ptr<f32>)
+  return
+}
+
+// CHECK: acc.exit_data copyout(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) delete(%{{.*}} : !llvm.ptr<f32>)
+
+// -----
+
+func @testexitdataop(%a: memref<10xi64>, %b: memref<10xf32>) -> () {
+  acc.exit_data copyout(%b : memref<10xf32>) delete(%a : memref<10xi64>) attributes {async}
+  return
+}
+
+// CHECK: acc.exit_data copyout(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) delete(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<i64>, ptr<i64>, i64)>) attributes {async}
+
+// -----
+
+func @testexitdataop(%a: memref<10xf32>, %b: memref<10xf32>) -> () {
+  %ifCond = constant true
+  acc.exit_data if(%ifCond) copyout(%b : memref<10xf32>) delete(%a : memref<10xf32>)
+  return
+}
+
+// CHECK: acc.exit_data if(%{{.*}}) copyout(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) delete(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<f32>, ptr<f32>, i64)>)
+
+// -----
+
+func @testupdateop(%a: memref<10xf32>, %b: memref<10xf32>) -> () {
+  acc.update host(%b : memref<10xf32>) device(%a : memref<10xf32>)
+  return
+}
+
+// CHECK: acc.update host(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) device(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<f32>, ptr<f32>, i64)>)
+
+// -----
+
+func @testupdateop(%a: !llvm.ptr<f32>, %b: memref<10xf32>) -> () {
+  acc.update host(%b : memref<10xf32>) device(%a : !llvm.ptr<f32>)
+  return
+}
+
+// CHECK: acc.update host(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) device(%{{.*}} : !llvm.ptr<f32>)
+
+// -----
+
+func @testupdateop(%a: memref<10xi64>, %b: memref<10xf32>) -> () {
+  acc.update host(%b : memref<10xf32>) device(%a : memref<10xi64>) attributes {async}
+  return
+}
+
+// CHECK: acc.update host(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) device(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<i64>, ptr<i64>, i64)>) attributes {async}
+
+// -----
+
+func @testupdateop(%a: memref<10xf32>, %b: memref<10xf32>) -> () {
+  %ifCond = constant true
+  acc.update if(%ifCond) host(%b : memref<10xf32>) device(%a : memref<10xf32>)
+  return
+}
+
+// CHECK: acc.update if(%{{.*}}) host(%{{.*}} : !llvm.struct<"openacc_data", (ptr<f32>, ptr<f32>, i64)>) device(%{{.*}} : !llvm.struct<"openacc_data.1", (ptr<f32>, ptr<f32>, i64)>)