[MLIR, OpenMP] Translation of OpenMP barrier construct to LLVM IR

[kiranchandramohan]

Summary

This patch adds support for translation of the OpenMP barrier construct to LLVM IR. The OpenMP IRBuilder is used for this translation. In this patch the code for translation is added to the existing LLVM dialect translation to LLVM IR.

The patch includes code changes and a testcase.

[merge_guards_bot]

Unit tests: unknown.

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt

[jdoerfert]

mlir/lib/Target/LLVMIR/ConvertToOpenMPLLVMIR.cpp

62 ↗

(On Diff #238903)

Need to be initialized to nullptr if you want to be allowed to call postLLVMTranslation before postLLVMModuleCreation.

[kiranchandramohan]

Made changes suggested by the clang-format bot.

[rriddle]

mlir/include/mlir/Target/OpenMPLLVMIR.h

32 ↗

(On Diff #238919)

I think this revision highlights a problem with the way the LLVM converter(or at least the way it is being used) is structured, mainly that it isn't extensible. I would imagine that lowering support for OpenMP would be an (optional) extension to the LLVM converter, not an entirely different one.

mlir/lib/Target/LLVMIR/ConvertToOpenMPLLVMIR.cpp

61 ↗

(On Diff #238919)

I would imagine this would be a unique_ptr instead of a raw pointer.

[kiranchandramohan]

In this patch the code for translation is added to the existing LLVM dialect translation to LLVM IR. A new flag enable-openmp is used to enable the OpenMP translation.

Had to update makefiles of couple of toy examples and a tool to fix linking issues since the LLVM translator needs the OpenMP libraries.

Also changing the Summary to reflect this patch. Note that the previous version had the translation for OpenMP in a separate class and file.

[kiranchandramohan]

I have taken @rriddle's suggestion and moved the translation into the LLVM translator. Also using unique pointer for the OpenMPIRBuilder. I believe this fixes the null pointer issue pointed out by @jdoerfert.

[jdoerfert]

No need to wait for me on this review but I added two more comments.

mlir/lib/Target/LLVMIR/ConvertToLLVMIR.cpp

25 ↗

(On Diff #239961)

You could just check if OMPBuilder is initialized in order to keep this cmd line option private to a single file. Arguably, you could also create a method getOMPBuilder() which create one if none is present as soon as an OpenMP op is discovered. Maybe I'm missing some intended interaction of OpenMP ops and this flag set to false?

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

345

Nit: If you want it less verbose, OMPBuilder->CreateBarrier(builder.saveIP(), llvm::omp::OMPD_barrier); should work too.

[mehdi_amini]

mlir/include/mlir/Target/LLVMIR/ModuleTranslation.h

29

cl::opt are only intended for testing, I wouldn't expect it in library code.

mlir/lib/Target/LLVMIR/ConvertToLLVMIR.cpp

25 ↗

(On Diff #239961)

Lazily creating the OMPBuilder on demand seems like ideal to me? It there a downside?

What about always creating it eagerly: is it that heavy/costly?

[jdoerfert]

mlir/lib/Target/LLVMIR/ConvertToLLVMIR.cpp

25 ↗

(On Diff #239961)

Lazily creating the OMPBuilder on demand seems like ideal to me? It there a downside?

I can't think of one.

What about always creating it eagerly: is it that heavy/costly?

No it's not. It create a couple of types, incl. function and struct types, but that's it.

[kiranchandramohan]

The enable-openmp flag is now local to file mlir/lib/Target/LLVMIR/ModuleTranslation.cpp. Added a function to lazily create the OpenMPIR builder when an OpenMP dialect op is detected.

Changes to address review comments from @jdoerfert and @mehdi_amini.

[kiranchandramohan]

mlir/include/mlir/Target/LLVMIR/ModuleTranslation.h

29

The cl::opt flag is now only visible in mlir/lib/Target/LLVMIR/ModuleTranslation.cpp. Is that what you suggested?

[ftynse]

Thanks! I have a couple of comments.

mlir/examples/toy/Ch6/CMakeLists.txt

55

Why are these changes necessary? The code doesn't seem to be modifying the tutorial.

mlir/include/mlir/Target/LLVMIR/ModuleTranslation.h

102

Please add a comment for this. "Original and translated module" does not describe it appropriately.

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

342

opInst.getDialect() == context->getRegisteredDialect<OpenMPDialect>() looks cleaner to me, even if it does the same comparison under the hood.

344

isa<omp::BarrierOp>

348

you cannot have non-OpenMP operation here, the outermost if checked that it belongs to the OpenMP dialect.

350

Nit: I'd provide more specific error message in case OpenMP is not enabled. Currenty, we will just fail to translate with the generic "unsupported or non-LLVM operation".

650

This should not be controlled by a command-line flag. It makes this flow unusable as a library (e.g., if the integration does not have a command-line interface). Pass the option to the constructor of ModuleTranslation (or whatever the way it is currently created) instead and default-initialize it from the command-line flag.

mlir/tools/mlir-cpu-runner/CMakeLists.txt

9

Do we really need whole archive link for LLVMFrontendOpenMP, i.e. is there some static registration in that library?

[jdoerfert]

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

650

I don't think we need the switch to begin with. If "omp" ops are supposed to be lowered we need a OMPBuilder. There is no reason not to build one and run into an assertion instead. With the lazy generation of the OMPBuilder we can really just remove the command line flag and get sane behavior w/ and w/o OpenMP ops.

mlir/tools/mlir-cpu-runner/CMakeLists.txt

9

i.e. is there some static registration in that library?

Should not be.

[kiranchandramohan]

Thanks @ftynse for the comments. I will make a patch soon to address comments. The only question is regarding the necessity of the flag. Please see comment inline.

mlir/examples/toy/Ch6/CMakeLists.txt

55

Are you asking specifically for the whole-archive link or for target_link_libraries also?

I was getting some link errors since the OpenMP symbols are needed now.

mlir/include/mlir/Target/LLVMIR/ModuleTranslation.h

102

OK will do.

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

342

Thanks. Will do.

344

Will do.

348

Will change.

350

OK.

650

I think one of the suggestions early on was that if there is no OpenMP support (indicated via the switch/flag) then someone can try to generate code with other parallel libraries (like pthreads) or maybe run sequentially (is that possible? would it be equal to ignoring the openmp op).

For now, is it OK to not have this flag as is being suggested by @jdoerfert ?

mlir/tools/mlir-cpu-runner/CMakeLists.txt

9

I will check and remove if not needed.

[jdoerfert]

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

650

FWIW, running OpenMP programs "sequentially", in the sense that you *ignore* OpenMP, is only sound in *special cases*. Various OpenMP directives change the program semantics (at least potentially) such that ignoring OpenMP directives is not a strategy we should recommend (and we most certainly cannot "support" it in a sane way). You can run them "sequentially" by setting the appropriate upper bounds to 1 though.

[compiler_developer]

MLIROpenMP and LLVMFrontendOpenMP libraries looks like an additional dependency for MLIR users who wants LLVM lowering and doesn't want anything to do with the OpenMP! I think Toy example cmake file changes is one of the early symptoms.

Can't OpenMP dialect have a separate LLVM dialect lowering similar to all other dialects in MLIR? I think it makes sense to lower OpenMP constructs in LLVM dialect rather than having a OpenMPIRBuilder based on LLVM IR.

What happens in case of LLM IR + OpenMP to LLVM dialect conversion? OpenMP dialect will be created too? Or else the LLVM IR to/from LLVM dialect (+ OpenMP dialect) will behave differently!

[kiranchandramohan]

In D72962#1872562, @compiler_developer wrote:

Thanks for your comments.

MLIROpenMP and LLVMFrontendOpenMP libraries looks like an additional dependency for MLIR users who wants LLVM lowering and doesn't want anything to do with the OpenMP! I think Toy example cmake file changes is one of the early symptoms.

One way to handle that would be to go back to the first version of this patch which implements translation by deriving from public LLVM::ModuleTranslation.

Can't OpenMP dialect have a separate LLVM dialect lowering similar to all other dialects in MLIR? I think it makes sense to lower OpenMP constructs in LLVM dialect rather than having a OpenMPIRBuilder based on LLVM IR.

We decided to use the OpenMPIRBuilder since that way we will be able to reuse existing code in the llvm-project (clang) for OpenMP code generation.

What happens in case of LLM IR + OpenMP to LLVM dialect conversion? OpenMP dialect will be created too? Or else the LLVM IR to/from LLVM dialect (+ OpenMP dialect) will behave differently!

I believe we can support this option under a flag. With the flag OpenMP + LLVM dialect is created. Without the flag only LLVM dialect will be created.

[compiler_developer]

In D72962#1872679, @kiranchandramohan wrote:

In D72962#1872562, @compiler_developer wrote:

Thanks for your comments.

MLIROpenMP and LLVMFrontendOpenMP libraries looks like an additional dependency for MLIR users who wants LLVM lowering and doesn't want anything to do with the OpenMP! I think Toy example cmake file changes is one of the early symptoms.

One way to handle that would be to go back to the first version of this patch which implements translation by deriving from public LLVM::ModuleTranslation.

I think conversion patterns would be more appropriate. But, I think, that is dependent on the decision to emit LLVM IR instead of LLVM Dialect.

Can't OpenMP dialect have a separate LLVM dialect lowering similar to all other dialects in MLIR? I think it makes sense to lower OpenMP constructs in LLVM dialect rather than having a OpenMPIRBuilder based on LLVM IR.

We decided to use the OpenMPIRBuilder since that way we will be able to reuse existing code in the llvm-project (clang) for OpenMP code generation.

Given that MLIR has potential to support various parellel IR semantics as different dialects, I think having an LLVM dilaect lowering for OpenMP would be much cleaner solution. Also, shouldn't there be minimum difference between the LLVM Dialect and LLVM IR. If I am not wrong the goal (or atleast it definitely makes sense) is to have a tablegen based LLVM Dialect to LLVM IR conversion and viceversa. Having OpenMP dialect would affect it?

What happens in case of LLM IR + OpenMP to LLVM dialect conversion? OpenMP dialect will be created too? Or else the LLVM IR to/from LLVM dialect (+ OpenMP dialect) will behave differently!

I believe we can support this option under a flag. With the flag OpenMP + LLVM dialect is created. Without the flag only LLVM dialect will be created.

But, would it be possible to extract the OpenMP operations from the LLVM IR? I think that is a difficult problem to solve, especially when the IR is optimized. Also, I dont think it can be under a flag as conversion should be round-trippable?

[kiranchandramohan]

In D72962#1872745, @compiler_developer wrote:

In D72962#1872679, @kiranchandramohan wrote:

In D72962#1872562, @compiler_developer wrote:

Thanks for your comments.

MLIROpenMP and LLVMFrontendOpenMP libraries looks like an additional dependency for MLIR users who wants LLVM lowering and doesn't want anything to do with the OpenMP! I think Toy example cmake file changes is one of the early symptoms.

One way to handle that would be to go back to the first version of this patch which implements translation by deriving from public LLVM::ModuleTranslation.

I think conversion patterns would be more appropriate. But, I think, that is dependent on the decision to emit LLVM IR instead of LLVM Dialect.

OK. I was pointing out that if we do not want these dependencies then we have a way to achieve that by deriving from the LLVM Module translator.

Can't OpenMP dialect have a separate LLVM dialect lowering similar to all other dialects in MLIR? I think it makes sense to lower OpenMP constructs in LLVM dialect rather than having a OpenMPIRBuilder based on LLVM IR.

We decided to use the OpenMPIRBuilder since that way we will be able to reuse existing code in the llvm-project (clang) for OpenMP code generation.

Given that MLIR has potential to support various parellel IR semantics as different dialects, I think having an LLVM dialect lowering for OpenMP would be much cleaner solution. Also, shouldn't there be minimum difference between the LLVM Dialect and LLVM IR. If I am not wrong the goal (or atleast it definitely makes sense) is to have a tablegen based LLVM Dialect to LLVM IR conversion and viceversa. Having OpenMP dialect would affect it?

LLVM dialect and its conversion will not be affected by the OpenMP dialect or the changes in the translation library to lower the dialect. The OpenMP translation will only kick in if there are OpenMP dialect operations during translation.

We did consider lowering to the LLVM dialect as can bee seen in Slide 8 of the presentation (link below) and my first mail to mlir list (link below, see also replies from mlir developers). But given that there is the OpenMPIRBuilder (which provides a good interface to generate LLVM IR for OpenMP constructs), we felt it would be best to use the OpenMPIRBuilder rather than to re-implement that work in MLIR.
https://drive.google.com/file/d/1vU6LsblsUYGA35B_3y9PmBvtKOTXj1Fu/view?usp=sharing
https://groups.google.com/a/tensorflow.org/d/msg/mlir/4Aj_eawdHiw/BSpWvR6zCAAJ

If there was no OpenMP codegen present in LLVM then I would have also have preferred to lower to LLVM dialect. Maybe in future when Clang also uses MLIR, we can modify OpenMPIRBuilder to generate LLVM dialect.

What happens in case of LLM IR + OpenMP to LLVM dialect conversion? OpenMP dialect will be created too? Or else the LLVM IR to/from LLVM dialect (+ OpenMP dialect) will behave differently!

I believe we can support this option under a flag. With the flag OpenMP + LLVM dialect is created. Without the flag only LLVM dialect will be created.

But, would it be possible to extract the OpenMP operations from the LLVM IR? I think that is a difficult problem to solve, especially when the IR is optimized. Also, I dont think it can be under a flag as conversion should be round-trippable?

Yes, this will be a more difficult problem to solve.

[mehdi_amini]

In D72962#1872745, @compiler_developer wrote:

We decided to use the OpenMPIRBuilder since that way we will be able to reuse existing code in the llvm-project (clang) for OpenMP code generation.

Given that MLIR has potential to support various parellel IR semantics as different dialects, I think having an LLVM dilaect lowering for OpenMP would be much cleaner solution.

I agree that having dialect lowering would be cleaner, but I don't get the relationship with "support various parallel IR semantics as different dialects"?

Also, shouldn't there be minimum difference between the LLVM Dialect and LLVM IR. If I am not wrong the goal (or atleast it definitely makes sense) is to have a tablegen based LLVM Dialect to LLVM IR conversion and viceversa. Having OpenMP dialect would affect it?

It wouldn't: as far as I understand every concept in LLVM can be represented in the LLVM dialect and vice-versa.

[mehdi_amini]

(This likely need a rebase)

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

650

For now this is fine with me to skip the flag entirely.

[schweitz]

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

650

There are several design choices one could try here.

Enable/disable OpenMP by:

• some switch in the front-end (throws away the directives, very early)
• creating/not creating the OpenMP dialect ops
• keeping/erasing the OpenMP dialect ops (before the LLVMIR dialect)
• translation modes of OpenMP dialect to LLVMIR dialect
• maybe even closer to OpenMPIRBuilder (late, some sort of target support pushback?)

There is most likely trade offs here in terms of "user experience" and reporting diagnostics as to where and how to balance these approaches. Although supporting the "disabled vs. enabled OpenMP" use cases does seem like one worth providing with some mechanism.

[kiranchandramohan]

This revision includes the following changes,
-> Rebased.
-> Removed the enable-openmp flag.
-> Addressed review comments from @ftynse.

Thanks @schweitz for your input. I have created a project note in https://github.com/orgs/flang-compiler/projects/9 to investigate the various possibilities that you have suggested.

[kiranchandramohan]

mlir/examples/toy/Ch6/CMakeLists.txt

55

As mentioned before, this was required to the dependence of MLIRTranslation on LLVMFrontendOpenMP.

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

650

Have removed the flag for now.

mlir/tools/mlir-cpu-runner/CMakeLists.txt

9

I had to add LLVMFrontendOpenMP to FULL_LIBS used in whole_archive_link in mlir/tools/mlir-translate/CMakeLists.txt since libMLIRTranslation appears after LLVMFrontendOpenMP in the link line.

[mehdi_amini]

mlir/examples/toy/Ch6/CMakeLists.txt

55

This seems like a missing transitive dependency: it does not seem right to add this here. The dependency on LLVMFrontendOpenMP needs to be specified on the library that is depending on it instead of pushed to every client (in this case I believe MLIRLLVMIR)

[kiranchandramohan]

Added LLVMFrontendOpenMP and MLIROpenMP to the dependency of MLIRLLVMIR and also linking these libraries along with MLIRLLVMIR. Removed from other clients.

@mehdi_amini hope this is what you suggested. Thanks.

[kiranchandramohan]

Do @rriddle and @ftynse require further changes?

[kiranchandramohan]

Just wanted to check whether @rriddle has more comments on this patch.

[rriddle]

mlir/include/mlir/Target/LLVMIR/ModuleTranslation.h

105

Variables names should be camelCase.

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

342

When adding more OMP operations(i.e. in a followup) can we split translating these operations into a different function? This one is already getting quite large.

343

We should really pre-compute this. Looking up a registered dialect involves grabbing a lock, and iterating O(N) over all dialects.

[kiranchandramohan]

Thanks for the reviews. Committing with changes suggested.

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

342

Will move to a new function for the next operation.

mlir/examples/toy/Ch6/CMakeLists.txt

 target_link_libraries(toyc-ch6
   PRIVATE
     MLIRAffineOps
     MLIRAffineToStandard
     MLIRAnalysis
     MLIRExecutionEngine
     MLIRIR
     MLIRLLVMIR
+    MLIROpenMP
+    LLVMFrontendOpenMP
     MLIRLoopToStandard
     MLIRParser
     MLIRPass
     MLIRStandardOps
     MLIRStandardToLLVM
     MLIRTargetLLVMIR
     MLIRTransforms
     )
 
 whole_archive_link(toyc-ch6
   MLIRAffineToStandard
   MLIRAffineOps
   MLIRLLVMIR
+  MLIROpenMP
+  LLVMFrontendOpenMP
   MLIRStandardOps
   )

mlir/examples/toy/Ch7/CMakeLists.txt

 target_link_libraries(toyc-ch7
   PRIVATE
     MLIRAffineOps
     MLIRAffineToStandard
     MLIRAnalysis
     MLIRExecutionEngine
     MLIRIR
     MLIRLLVMIR
+    MLIROpenMP
+    LLVMFrontendOpenMP
     MLIRLoopToStandard
     MLIRParser
     MLIRPass
     MLIRStandardOps
     MLIRStandardToLLVM
     MLIRTargetLLVMIR
     MLIRTransforms
     )
 
 whole_archive_link(toyc-ch7
   MLIRAffineToStandard
   MLIRAffineOps
   MLIRLLVMIR
+  MLIROpenMP
+  LLVMFrontendOpenMP
   MLIRStandardOps
   )

mlir/include/mlir/Target/LLVMIR/ModuleTranslation.h

 #ifndef MLIR_TARGET_LLVMIR_MODULETRANSLATION_H
 #define MLIR_TARGET_LLVMIR_MODULETRANSLATION_H
 
 #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
 #include "mlir/IR/Block.h"
 #include "mlir/IR/Module.h"
 #include "mlir/IR/Value.h"
 
+#include "llvm/Frontend/OpenMP/OMPIRBuilder.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/Value.h"
 
 namespace mlir {
 class Attribute;
 class Location;
 class ModuleOp;
 class Operation;
 
 namespace LLVM {
 
 class LLVMFuncOp;
 
protected:
   static std::unique_ptr<llvm::Module> prepareLLVMModule(Operation *m);
 
   /// A helper to look up remapped operands in the value remapping table.
   SmallVector<llvm::Value *, 8> lookupValues(ValueRange values);
 
 private:
   /// Check whether the module contains only supported ops directly in its body.
   static LogicalResult checkSupportedModuleOps(Operation *m);
+  bool existsOrCreateOpenMPIRBuilder();
 
   LogicalResult convertFunctions();
   void convertGlobals();
   LogicalResult convertOneFunction(LLVMFuncOp func);
   void connectPHINodes(LLVMFuncOp func);
   LogicalResult convertBlock(Block &bb, bool ignoreArguments);
 
   llvm::Constant *getLLVMConstant(llvm::Type *llvmType, Attribute attr,
                                   Location loc);
 
   // Original and translated module.
   Operation *mlirModule;
   std::unique_ptr<llvm::Module> llvmModule;
+  std::unique_ptr<llvm::OpenMPIRBuilder> OMPBuilder;
 
   // Mappings between llvm.mlir.global definitions and corresponding globals.
   DenseMap<Operation *, llvm::GlobalValue *> globalsMapping;
 
 protected:
   // Mappings between original and translated values, used for lookups.
   llvm::StringMap<llvm::Function *> functionMapping;
   DenseMap<Value, llvm::Value *> valueMapping;
   DenseMap<Block *, llvm::BasicBlock *> blockMapping;
 };
 
 } // namespace LLVM
 } // namespace mlir
 
 #endif // MLIR_TARGET_LLVMIR_MODULETRANSLATION_H

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

 //===- ModuleTranslation.cpp - MLIR to LLVM conversion --------------------===//
 //
 // Part of the MLIR Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the translation between an MLIR LLVM dialect module and
 // the corresponding LLVMIR module. It only handles core LLVM IR operations.
 //
 //===----------------------------------------------------------------------===//
 
 #include "mlir/Target/LLVMIR/ModuleTranslation.h"
 
 #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
+#include "mlir/Dialect/OpenMP/OpenMPDialect.h"
 #include "mlir/IR/Attributes.h"
 #include "mlir/IR/Module.h"
 #include "mlir/IR/StandardTypes.h"
 #include "mlir/Support/LLVM.h"
 
 #include "llvm/ADT/SetVector.h"
+#include "llvm/Frontend/OpenMP/OMPIRBuilder.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Transforms/Utils/Cloning.h"
 
+static llvm::cl::opt<bool> enableOpenMP{
+    "enable-openmp", llvm::cl::desc("Enable translation of OpenMP dialect Ops"),
+    llvm::cl::init(false)};
+
 using namespace mlir;
 using namespace mlir::LLVM;
 
 /// Builds a constant of a sequential LLVM type `type`, potentially containing
 /// other sequential types recursively, from the individual constant values
 /// provided in `constants`. `shape` contains the number of elements in nested
 /// sequential types. Reports errors at `loc` and returns nullptr on error.
 static llvm::Constant *
if (auto addressOfOp = dyn_cast<LLVM::AddressOfOp>(opInst)) {
     LLVM::GlobalOp global = addressOfOp.getGlobal();
     // The verifier should not have allowed this.
     assert(global && "referencing an undefined global");
 
     valueMapping[addressOfOp.getResult()] = globalsMapping.lookup(global);
     return success();
   }
 
+  if (opInst.getDialect()->getNamespace() == "omp") {
+    if (existsOrCreateOpenMPIRBuilder()) {
+      if (dyn_cast<omp::BarrierOp>(opInst)) {
+        OMPBuilder->CreateBarrier(builder.saveIP(), llvm::omp::OMPD_barrier);
+        return success();
+      }
+      return opInst.emitError("unsupported or non-OpenMP operation: ")
+             << opInst.getName();
+    }
+  }
+
   return opInst.emitError("unsupported or non-LLVM operation: ")
          << opInst.getName();
 }
 
 /// Convert block to LLVM IR.  Unless `ignoreArguments` is set, emit PHI nodes
 /// to define values corresponding to the MLIR block arguments.  These nodes
 /// are not connected to the source basic blocks, which may not exist yet.
 LogicalResult ModuleTranslation::convertBlock(Block &bb, bool ignoreArguments) {
ModuleTranslation::prepareLLVMModule(Operation *m) {
   // memref allocation/deallocation coming from standard ops lowering.
   llvmModule->getOrInsertFunction("malloc", builder.getInt8PtrTy(),
                                   builder.getInt64Ty());
   llvmModule->getOrInsertFunction("free", builder.getVoidTy(),
                                   builder.getInt8PtrTy());
 
   return llvmModule;
 }
+
+bool ModuleTranslation::existsOrCreateOpenMPIRBuilder() {
+  if (!enableOpenMP)
+    return false;
+
+  if (!OMPBuilder) {
+    OMPBuilder =
+        std::move(std::make_unique<llvm::OpenMPIRBuilder>(*llvmModule));
+    OMPBuilder->initialize();
+  }
+
+  return true;
+}

mlir/test/Target/openmp-llvm.mlir

+// RUN: mlir-translate -enable-openmp -mlir-to-llvmir %s | FileCheck %s
+
+// CHECK-LABEL: define void @empty() {
+// CHECK: [[OMP_THREAD:%.*]] = call i32 @__kmpc_global_thread_num(%struct.ident_t* @{{[0-9]+}})
+// CHECK-NEXT:  call void @__kmpc_barrier(%struct.ident_t* @{{[0-9]+}}, i32 [[OMP_THREAD]])
+// CHECK-NEXT:    ret void
+// CHECK-NEXT:  }
+llvm.func @empty() {
+  omp.barrier
+  llvm.return
+}

mlir/tools/mlir-cpu-runner/CMakeLists.txt

 add_llvm_executable(mlir-cpu-runner
   mlir-cpu-runner.cpp
 )
 llvm_update_compile_flags(mlir-cpu-runner)
 whole_archive_link(mlir-cpu-runner
   MLIRAffineOps
   MLIRLLVMIR
+  MLIROpenMP
+  LLVMFrontendOpenMP
   MLIRTargetLLVMIR
   MLIRTranslation
 )
 target_link_libraries(mlir-cpu-runner PRIVATE
   MLIRAnalysis
   MLIREDSC
   MLIRExecutionEngine
   MLIRIR
   MLIRJitRunner
   MLIRLLVMIR
+  MLIROpenMP
+  LLVMFrontendOpenMP
   MLIRParser
   MLIRTargetLLVMIR
   MLIRSupport
   LLVMCore
   LLVMSupport
 )

mlir/tools/mlir-translate/CMakeLists.txt

 set(LIBS
   MLIRParser
   MLIRPass
   MLIRSPIRV
   MLIRSPIRVSerialization
   MLIRTargetLLVMIR
   MLIRTargetNVVMIR
   MLIRTargetROCDLIR
   MLIRTranslation
   MLIRSupport
+  MLIROpenMP
+  LLVMFrontendOpenMP
 )
 add_llvm_executable(mlir-translate
   mlir-translate.cpp
 )
 llvm_update_compile_flags(mlir-translate)
 whole_archive_link(mlir-translate ${LIBS})
 target_link_libraries(mlir-translate PRIVATE MLIRIR MLIRTranslateClParser ${LIBS} LLVMSupport)