[Flang][Driver][MLIR] Add -fopenmp-is-device to Flang and link to an omp.is_device attribute

[agozillon]

Summary

This patch adds the -fopenmp-is-device flag to Flang's FC1 and also
links it up to the application of an attribute to the mlir builtin.module
on lowering. Indiciating whether the module is for host or device.

This omp.is_device module attribute is a light weight adhoc attribute
with noconcrete ODS or class specification, similar to FIR and LLVM's
data_layout, defaultkind and kindmap's. Its intent is simply to carry
information for the moment, it affects transformations (or will in the
future) but does no transformations itself.

This patch currently does not add the fopenmp-targets flag,
e.g. amdgcn-amd-amdhsa=amdgcn-amd-amdhsa which in Clang
populates the cc1 commands with -fopenmp-is-device, so the only
way to utilise the -fopenmp-is-device flag is to directly use -fc1 with flang-new.

[agozillon]

Forgot to add an additional test in initial patch!

[kiranchandramohan]

Could you update the discourse thread (https://discourse.llvm.org/t/rfc-omp-module-and-omp-function-vs-dialect-attributes-to-encode-openmp-properties/67998) with the chosen approach and reasons before proceeding?

[jdoerfert]

Tests?

[agozillon]

In D144864#4156764, @jdoerfert wrote:

Tests?

Happy to add more, I am just a little unsure what the best way to test this is other than application of the attribute to the module when the flag was used and verification that the flag was active in FC1, do you have any suggestions? I don't really think there is a way to test for the attribute in OpenMP/invalid.mlir (as no specified verifier) or in OpenMP/ops.mlir as it is not a predefined operation or attribute, but I can look at how fir.kindmap (which is applied similarly) is tested and see if they do anything particular.

[agozillon]

• [MLIR][OpenMP] Fix attribute helpers to apply to more than builtin.module

[agozillon]

• [Flang][mlir] Adding space at the end of the omp-is-device test file

[agozillon]

After asking @jsjodin and @skatrak for some input on additional tests that could be added and having a look around the existing tests we couldn't really come up with any additional tests to add to the ones that are in the patch at the moment (inside of omp-is-device.f90 and driver-help.f90) unfortunately. Primarily as there isn't any use of this attribute yet, although there will be quite soon.

[awarzynski]

clang/lib/Driver/ToolChains/Flang.cpp

121–125

Can you add a test for this section?

flang/test/Lower/OpenMP/omp-is-device.f90

1 ↗

(On Diff #501645)

What happens if -fopenmp-is-device is used without -fopnemp?

flang/tools/bbc/bbc.cpp

127

This option is not tested

[agozillon]

• [MLIR][OpenMP] Fix attribute helpers to apply to more than builtin.module
• [Flang][mlir] Adding space at the end of the omp-is-device test file
• [Clang][Flang][Driver][Test] Add flang-omp.f90 to test fopenmp/fopenmp-is-device for flang driver-mode
• [Flang][MLIR][OpenMP] Add additional tests to omp-is-device.f90 test

[agozillon]

• [Flang][MLIR][Test] Add newline at end of test file

[agozillon]

The new update attempts to add the suggested new tests! Thank you very much @awarzynski for suggesting them that was a great help. Hopefully the new additions cover what you have requested, if not I am of course happy to change or add more tests as needed.

clang/lib/Driver/ToolChains/Flang.cpp

121–125

I have added a new test file flang-omp.f90 which aims to test that the appropriate flags are generated when openmp related flags are used it currently checks for -fopenmp/fopenmp-is-device

flang/test/Lower/OpenMP/omp-is-device.f90

1 ↗

(On Diff #501645)

I think I now cover this in the test with the *-DEVICE-FLAG-ONLY tests, currently it is silently ignored and nothing is applied to the module.

flang/tools/bbc/bbc.cpp

127

I believe I now test this in the omp-is-device.f90 test!

[awarzynski]

Thank you very much @awarzynski for suggesting them that was a great help.

I'm happy that I could help :) The driver logic LGTM, but please wait for either @jdoerfert and/or @kiranchandramohan to also approve. Thanks for contributing!

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

This test looks correct to me, but please note that:

! Check that flang -fc1 is invoked when in --driver-mode=flang

yet (%clang instead of %flang)

! RUN: %clang --driver-mode=flang -### -fopenmp %s 2>&1 | FileCheck --check-prefixes=CHECK-OPENMP %s

I'm not really sure whether we should be testing Flang-specific logic in Clang. Having said that, Flang does use clangDriver to implement its driver :)

You could consider using https://github.com/llvm/llvm-project/blob/main/flang/test/Driver/frontend-forwarding.f90 instead (or add an OpenMP specific file there).

Not a blocker.

[agozillon]

Thank you very much @awarzynski I will wait for further approval!

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

Yes, I wasn't so sure either as it felt a little weird to test Flang components inside of Clang, but as you said it is a Clang toolchain (that this test is checking) and borrows from the clangDriver!

I borrowed this test from other similar tests in the same folder that test other flang specific driver logic in a similar manner, but I am more than happy to add an additional flang specific driver test as you mention!

[agozillon]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

On further looking into the frontend-forwarding test, I don't know if it is suitable for fopenmp-is-device as it is an fc1 option and won't be forwarded from the flang-new frontend down to fc1 at the moment!

I think this test will be more suitable when additional flags like the fopenmp-targets (or similar flags) that are used in this test are added to the Flang driver. As they spawn/propagate the openmp-is-device flag. However, perhaps I am incorrect.

[awarzynski]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

On further looking into the frontend-forwarding test, I don't know if it is suitable for fopenmp-is-device as it is an fc1 option and won't be forwarded from the flang-new frontend down to fc1 at the moment!

It should be - that's what the logic in Flang.cpp does, no? And if it doesn't, is it a deliberate design decision?

[agozillon]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

I believe the logic in Flang.h/Flang.cpp just invokes for -fc1 not the Frontend driver, at least that's what it looks like from the ConstructJob task that Clang invokes.

It's a deliberate design decision that -fopenmp-is-device is an fc1 option, it mimics the way Clang currently does it, which is a cc1 option, there is no way to directly specify it to clang without -cc1 I think. The hope is to keep the Flang OpenMP flags as similar to Clang's as possible for the time being I believe.

[awarzynski]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

I believe the logic in Flang.h/Flang.cpp just invokes for -fc1 not the Frontend driver.

flang-new -fc1 is the frontend driver ;-) So:

• you've added the logic to forward -fopenmp-is-device that should work for both flang-new and clang --driver-mode=flang, but
• you're only testing one of these.

There should have tests for both. In particular for flang-new given that this patch adds new functionality to LLVM Flang. If that doesn't work, let's try to figure it out together.

The hope is to keep the Flang OpenMP flags as similar to Clang's as possible for the time being I believe.

+1 We should try as much as we can to keep Clang's and Flang's behavior consistent :) (which you do 👍🏻 )

[agozillon]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

Ah I didn't realise that, thank you very much for clarifying that for me! :) I thought it was bypassing it with -fc1, and yjsy flang-new without it was the frontend! I still have a lot to learn, so I apologies for the misunderstanding!

In this case do you mean test with "flang-new -fc1 -fopenmp-is-device" or "flang-new -fopenmp-is-device", as the latter will not be accepted by flang-new in the current patch but I do test the -fc1 variation in the omp-is-device.f90 test which is more about testing the attribute admittedly as opposed to the driver command.

I don't know if testing it in the frontend-forwarding.f90 test is the right location in this case as it doesn't test with -fc1, it seems to test options are forwarded to -fc1 correctly, I could be incorrect though. In future passing an argument like -fopenmp-targets=amdgcn-amd-amdhsa to flang-new would likely expand into -fopenmp-is-device (and other arguments), like it does for Clang at the moment, so this is likely the option that would be tested inside of frontend-forwarding.f90 (and the subsequent check-case would make sure it expanded into -fopenmp-is-device and friends correctly).

However, I am more than happy to add a test for the forwarding behavior, I'm just not necessarily sure what that test would look like unfortunately, so I appologies!

[awarzynski]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

I still have a lot to learn

Don't we all? :)

I apologies for the misunderstanding!

No worries! This could be helpful: https://github.com/llvm/llvm-project/blob/main/flang/docs/FlangDriver.md

I don't know if testing it in the frontend-forwarding.f90

I agree that probably not. In this case, a "compiler driver" flag ( -fopenmp-targets) implies a certain "frontend driver" flag ("-fopenmp-is-device). That's not really forwarding.

In future passing an argument like -fopenmp-targets=amdgcn-amd-amdhsa to flang-new

Ah, that's something I've missed and that's my ignorance kicking in, sorry. Correct me if I am wrong:

1. this patch adds -fopnemp-is-device to Flang's frontend driver (that's the logic in CompilerInvation.cpp),
2. the logic in Flang.cpp means that despite flang-new not supporting -fopenmp-targets= just yet, it is possible to use clang --driver-mode=flang -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa instead of flang-new -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa.

I've completely missed the fact that flang-new is missing -fopenmp-targets (I would appreciate that being mentioned in the summary).

Do we need to have 2. at this point? Ideally, we would add -fopenmp-targets= to flang-new first and then test with:

• flang-new -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa (this could happen in a subsequent patch, the logic for the frontend driver is still fine).

While testing with clang is also correct, this:

• %clang --driver-mode=flang -### -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa %s

feels rather counter-intuitive and indeed very uncommon.

Just from this patch alone I feel that 2. is not yet needed and you could safely:

• remove the changes in Flang.cpp, and
• delete flang-omp.f90).

You could revisit these once -fopenmp-targets is ready. It's totally fine to add "frontend driver" (flang-new -fc1) and "compiler driver" (flang-new) logic separately (just make it clear in the summary). WDYT?

[agozillon]

clang/test/Driver/flang/flang-omp.f90

1 ↗

(On Diff #502139)

Thank you very much for the link! Very useful and comprehensive documentation :)

1. That is correct, and it works like -fopenmp-is-device in Clang
2. That is correct unfortunately, but we do fully intend to support fopenmp-targets in the very near future (another team member is working on this patch), I just thought it would be nice to have all the fopenmp-is-device components in a single patch!

I have added it to the summary, sorry for missing it previously!

It's not needed within this patch no, but it does keep it self contained within one patch and other patches can build off of it a little easier, but I am happy to separate the logic into two different patches if that is preferred!

[agozillon]

• [Flang][ToolChain][OpenMP][Driver] Reduce changes to only add -fc1 support

[agozillon]

The recent update removed the added offload related code from Flang.h/.cpp and removed the related test.

Would it be possible to have an extra sign-off as requested by @awarzynski (or more review points to correct/discuss if we aren't happy with the state/direction of the patch) from either @kiranchandramohan or @jdoerfert my apologies for the pings!

And then provided @awarzynski is happy with the current state of the patch after this recent commit and no further comments I can commit and close the patch :)

Thank you very much for all the excellent comments in this review, and the attention given to it, I appreciate it greatly.

[kiranchandramohan]

LG. See one minor comment in the tests.

I would prefer having an Interface for Target Modules if that could be made to work. I guess this can be taken up separately after https://reviews.llvm.org/D144883.

flang/test/Lower/OpenMP/omp-is-device.f90

8–17 ↗

(On Diff #503008)

Any reason to have separate checks for essentially the same (e.g: DEVICE vs BBC-DEVICE)?

[agozillon]

In D144864#4175257, @kiranchandramohan wrote:

LG. See one minor comment in the tests.

I would prefer having an Interface for Target Modules if that could be made to work. I guess this can be taken up separately after https://reviews.llvm.org/D144883.

Thank you Kiran, by interface for Target Modules do you mean a new omp.module or this type of external interface: https://mlir.llvm.org/docs/Interfaces/#external-models-for-attribute-operation-and-type-interfaces

And quite possibly, @skatrak is currently looking into the review comments in the referenced patch (as he found similar use for the patch), so we shall see where the results lead to. Never opposed to improving on the infrastructure and this is something we will iterate on as we progress with offloading! :)

flang/test/Lower/OpenMP/omp-is-device.f90

8–17 ↗

(On Diff #503008)

In this case to test the addition of the flag to bbc and also to the Flang driver, and testing the flag has the same behavior in both. There is perhaps a way to make the test simpler that I'm unaware of though? e.g. a way to merge DEVICE/BBC-DEVICE into one check

[kiranchandramohan]

In D144864#4175332, @agozillon wrote:

In D144864#4175257, @kiranchandramohan wrote:

LG. See one minor comment in the tests.

I would prefer having an Interface for Target Modules if that could be made to work. I guess this can be taken up separately after https://reviews.llvm.org/D144883.

Thank you Kiran, by interface for Target Modules do you mean a new omp.module or this type of external interface: https://mlir.llvm.org/docs/Interfaces/#external-models-for-attribute-operation-and-type-interfaces

And quite possibly, @skatrak is currently looking into the review comments in the referenced patch (as he found similar use for the patch), so we shall see where the results lead to. Never opposed to improving on the infrastructure and this is something we will iterate on as we progress with offloading! :)

The latter one (external interface).

flang/test/Lower/OpenMP/omp-is-device.f90

8–17 ↗

(On Diff #503008)

If DEVICE and BBC-DEVICE are checking the same things, then you only need to have one of them (say DEVICE) and use it in the check-prefix of both.

[awarzynski]

In D144864#4175001, @agozillon wrote:

And then provided @awarzynski is happy with the current state of the patch

LGTM, thanks for the updates and for working on this!

[agozillon]

• [Flang][Driver] Tidy up omp-is-device.f90 test

[agozillon]

Cleaned up the test, thank you both for teaching me more about the compiler and test infrastructure! If you're happy with the test changes @kiranchandramohan I'll commit the changes upstream to close out the review?

[kiranchandramohan]

LG.

clang/include/clang/Driver/Options.td

 } // let Flags = [CC1Option, NoDriverOption]
 
 //===----------------------------------------------------------------------===//
 // OpenMP Options
 //===----------------------------------------------------------------------===//
 
 def fopenmp_is_device : Flag<["-"], "fopenmp-is-device">,
   HelpText<"Generate code only for an OpenMP target device.">,
-  Flags<[CC1Option, NoDriverOption]>;
+  Flags<[CC1Option, FC1Option, NoDriverOption]>;
 def fopenmp_host_ir_file_path : Separate<["-"], "fopenmp-host-ir-file-path">,
   HelpText<"Path to the IR file produced by the frontend for the host.">,
   Flags<[CC1Option, NoDriverOption]>;
 
 //===----------------------------------------------------------------------===//
 // SYCL Options
 //===----------------------------------------------------------------------===//
 

clang/lib/Driver/ToolChains/Flang.h

private:
   /// Extract target options from the driver arguments and add them to
   /// the command arguments.
   ///
   /// \param [in] Args The list of input driver arguments
   /// \param [out] CmdArgs The list of output command arguments
   void addTargetOptions(const llvm::opt::ArgList &Args,
                         llvm::opt::ArgStringList &CmdArgs) const;
 
+  /// Extract offload options from the driver arguments and add them to
+  /// the command arguments.
+  ///
+  /// \param [in] JA The job action
+  /// \param [in] Args The list of input driver arguments
+  /// \param [out] CmdArgs The list of output command arguments
+  void addOffloadOptions(const JobAction &JA, const llvm::opt::ArgList &Args,
+                         llvm::opt::ArgStringList &CmdArgs) const;
+
   /// Extract other compilation options from the driver arguments and add them
   /// to the command arguments.
   ///
   /// \param [in] Args The list of input driver arguments
   /// \param [out] CmdArgs The list of output command arguments
   void addOtherOptions(const llvm::opt::ArgList &Args,
                        llvm::opt::ArgStringList &CmdArgs) const;
 

clang/lib/Driver/ToolChains/Flang.cpp

void Flang::addTargetOptions(const ArgList &Args,
   case llvm::Triple::x86_64:
     getTargetFeatures(D, Triple, Args, CmdArgs, /*ForAs*/ false);
     break;
   }
 
   // TODO: Add target specific flags, ABI, mtune option etc.
 }
 
+void Flang::addOffloadOptions(const JobAction &JA, const ArgList &Args,
+                              ArgStringList &CmdArgs) const {
+  bool IsOpenMPDevice = JA.isDeviceOffloading(Action::OFK_OpenMP);
+
+  if (IsOpenMPDevice) {
+    // -fopenmp-is-device is passed along to tell the frontend that it is
+    // generating code for a device, so that only the relevant declarations are
+    // emitted.
+    CmdArgs.push_back("-fopenmp-is-device");
+  }
+}
+
 static void addFloatingPointOptions(const Driver &D, const ArgList &Args,
                                     ArgStringList &CmdArgs) {
   StringRef FPContract;
   bool HonorINFs = true;
   bool HonorNaNs = true;
   bool ApproxFunc = false;
   bool SignedZeros = true;
   bool AssociativeMath = false;
void Flang::ConstructJob(Compilation &C, const JobAction &JA,
   addPicOptions(Args, CmdArgs);
 
   // Floating point related options
   addFloatingPointOptions(D, Args, CmdArgs);
 
   // Add target args, features, etc.
   addTargetOptions(Args, CmdArgs);
 
+  // Offloading related options
+  addOffloadOptions(JA, Args, CmdArgs);
+
   // Add other compile options
   addOtherOptions(Args, CmdArgs);
 
   // Forward -Xflang arguments to -fc1
   Args.AddAllArgValues(CmdArgs, options::OPT_Xflang);
 
   // Forward -mllvm options to the LLVM option parser. In practice, this means
   // forwarding to `-fc1` as that's where the LLVM parser is run.

flang/include/flang/Frontend/LangOptions.def

 /// calculation
 LANGOPT(ApproxFunc, 1, false)
 /// Allow optimizations that ignore the sign of floating point zeros
 LANGOPT(NoSignedZeros, 1, false)
 /// Allow reassociation transformations for floating-point instructions
 LANGOPT(AssociativeMath, 1, false)
 /// Allow division operations to be reassociated
 LANGOPT(ReciprocalMath, 1, false)
+/// Generate code only for OpenMP target device
+LANGOPT(OpenMPIsDevice, 1, false)
 
 #undef LANGOPT
 #undef ENUM_LANGOPT

flang/lib/Frontend/CompilerInvocation.cpp

static bool parseDialectArgs(CompilerInvocation &res, llvm::opt::ArgList &args,
   // -fopenmp and -fopenacc
   if (args.hasArg(clang::driver::options::OPT_fopenacc)) {
     res.getFrontendOpts().features.Enable(
         Fortran::common::LanguageFeature::OpenACC);
   }
   if (args.hasArg(clang::driver::options::OPT_fopenmp)) {
     res.getFrontendOpts().features.Enable(
         Fortran::common::LanguageFeature::OpenMP);
+
+    if (args.hasArg(clang::driver::options::OPT_fopenmp_is_device)) {
+      res.getLangOpts().OpenMPIsDevice = 1;
+    }
   }
 
   // -pedantic
   if (args.hasArg(clang::driver::options::OPT_pedantic)) {
     res.setEnableConformanceChecks();
   }
   // -std=f2018 (currently this implies -pedantic)
   // TODO: Set proper options when more fortran standards

flang/lib/Frontend/FrontendActions.cpp

lower::LoweringBridge lb = Fortran::lower::LoweringBridge::create(
       ci.getInvocation().getSemanticsContext().intrinsics(),
       ci.getInvocation().getSemanticsContext().targetCharacteristics(),
       ci.getParsing().allCooked(), ci.getInvocation().getTargetOpts().triple,
       kindMap, ci.getInvocation().getLoweringOpts(),
       ci.getInvocation().getFrontendOpts().envDefaults);
 
   // Fetch module from lb, so we can set
   mlirModule = std::make_unique<mlir::ModuleOp>(lb.getModule());
+
+  if (ci.getInvocation().getFrontendOpts().features.IsEnabled(
+          Fortran::common::LanguageFeature::OpenMP)) {
+    mlir::omp::OpenMPDialect::setIsDevice(
+        *mlirModule, ci.getInvocation().getLangOpts().OpenMPIsDevice);
+  }
+
   setUpTargetMachine();
   const llvm::DataLayout &dl = tm->createDataLayout();
   setMLIRDataLayout(*mlirModule, dl);
 
   // Create a parse tree and lower it to FIR
   Fortran::parser::Program &parseTree{*ci.getParsing().parseTree()};
   lb.lower(parseTree, ci.getInvocation().getSemanticsContext());
 

flang/test/Driver/driver-help.f90

 ! HELP-FC1-NEXT: -fno-analyzed-objects-for-unparse
 ! HELP-FC1-NEXT:                        Do not use the analyzed objects when unparsing
 ! HELP-FC1-NEXT: -fno-automatic         Implies the SAVE attribute for non-automatic local objects in subprograms unless RECURSIVE
 ! HELP-FC1-NEXT: -fno-debug-pass-manager Disables debug printing for the new pass manager
 ! HELP-FC1-NEXT: -fno-reformat          Dump the cooked character stream in -E mode
 ! HELP-FC1-NEXT: -fno-signed-zeros      Allow optimizations that ignore the sign of floating point zeros
 ! HELP-FC1-NEXT: -fno-stack-arrays      Allocate array temporaries on the heap (default)
 ! HELP-FC1-NEXT: -fopenacc              Enable OpenACC
+! HELP-FC1-NEXT: -fopenmp-is-device     Generate code only for an OpenMP target device.
 ! HELP-FC1-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-FC1-NEXT: -fpass-plugin=<dsopath> Load pass plugin from a dynamic shared object file (only with new pass manager).
 ! HELP-FC1-NEXT: -freciprocal-math      Allow division operations to be reassociated
 ! HELP-FC1-NEXT: -fstack-arrays         Attempt to allocate array temporaries on the stack, no matter their size
 ! HELP-FC1-NEXT: -fsyntax-only          Run the preprocessor, parser and semantic analysis stages
 ! HELP-FC1-NEXT: -funderscoring         Appends one trailing underscore to external names
 ! HELP-FC1-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! HELP-FC1-NEXT: -help                  Display available options

flang/tools/bbc/bbc.cpp

 #include "flang/Parser/parsing.h"
 #include "flang/Parser/provenance.h"
 #include "flang/Parser/unparse.h"
 #include "flang/Semantics/expression.h"
 #include "flang/Semantics/runtime-type-info.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Semantics/unparse-with-symbols.h"
 #include "flang/Version.inc"
+#include "mlir/Dialect/OpenMP/OpenMPDialect.h"
 #include "mlir/IR/AsmState.h"
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/MLIRContext.h"
 #include "mlir/Parser/Parser.h"
 #include "mlir/Pass/Pass.h"
 #include "mlir/Pass/PassManager.h"
 #include "mlir/Pass/PassRegistry.h"
 #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
static llvm::cl::opt<bool> pftDumpTest(
     "pft-test",
     llvm::cl::desc("parse the input, create a PFT, dump it, and exit"),
     llvm::cl::init(false));
 
 static llvm::cl::opt<bool> enableOpenMP("fopenmp",
                                         llvm::cl::desc("enable openmp"),
                                         llvm::cl::init(false));
 
+static llvm::cl::opt<bool>
+    enableOpenMPDevice("fopenmp-is-device",
+                       llvm::cl::desc("enable openmp device compilation"),
+                       llvm::cl::init(false));
+
 static llvm::cl::opt<bool> enableOpenACC("fopenacc",
                                          llvm::cl::desc("enable openacc"),
                                          llvm::cl::init(false));
 
 static llvm::cl::opt<bool> enablePolymorphic(
     "polymorphic-type",
     llvm::cl::desc("enable polymorphic type lowering (experimental)"),
     llvm::cl::init(false));
static mlir::LogicalResult convertFortranSourceToMLIR(
   loweringOptions.setPolymorphicTypeImpl(enablePolymorphic);
   loweringOptions.setLowerToHighLevelFIR(useHLFIR);
   auto burnside = Fortran::lower::LoweringBridge::create(
       ctx, semanticsContext, defKinds, semanticsContext.intrinsics(),
       semanticsContext.targetCharacteristics(), parsing.allCooked(), "",
       kindMap, loweringOptions, {});
   burnside.lower(parseTree, semanticsContext);
   mlir::ModuleOp mlirModule = burnside.getModule();
+  if (enableOpenMP)
+    mlir::omp::OpenMPDialect::setIsDevice(mlirModule, enableOpenMPDevice);
   std::error_code ec;
   std::string outputName = outputFilename;
   if (!outputName.size())
     outputName = llvm::sys::path::stem(inputFilename).str().append(".mlir");
   llvm::raw_fd_ostream out(outputName, ec);
   if (ec)
     return mlir::emitError(mlir::UnknownLoc::get(&ctx),
                            "could not open output file ")

mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td

 include "mlir/Dialect/OpenMP/OpenMPOpsInterfaces.td"
 include "mlir/Dialect/OpenMP/OpenMPTypeInterfaces.td"
 
 def OpenMP_Dialect : Dialect {
   let name = "omp";
   let cppNamespace = "::mlir::omp";
   let dependentDialects = ["::mlir::LLVM::LLVMDialect"];
   let useDefaultAttributePrinterParser = 1;
+
+  let extraClassDeclaration = [{
+    // Set the omp.is_device attribute on the module with the specified boolean
+    static void setIsDevice(mlir::ModuleOp module, bool isDevice);
+
+    // Return the value of the omp.is_device attribute stored in the module if it
+    // exists, otherwise return false by default
+    static bool getIsDevice(mlir::ModuleOp module);
+  }];
 }
 
 // OmpCommon requires definition of OpenACC_Dialect.
 include "mlir/Dialect/OpenMP/OmpCommon.td"
 
 class OpenMP_Op<string mnemonic, list<Trait> traits = []> :
       Op<OpenMP_Dialect, mnemonic, traits>;
 

mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp

if ((cct == ClauseCancellationConstructType::Sections) &&
       !(isa<SectionsOp>(parentOp) || isa<SectionOp>(parentOp))) {
     return emitOpError() << "cancellation point sections must appear "
                          << "inside a sections region";
   }
   // TODO : Add more when we support taskgroup.
   return success();
 }
 
+//===----------------------------------------------------------------------===//
+// OpenMPDialect helper functions
+//===----------------------------------------------------------------------===//
+
+// Set the omp.is_device attribute on the module with the specified boolean
+void OpenMPDialect::setIsDevice(mlir::ModuleOp module, bool isDevice) {
+  module->setAttr(
+      mlir::StringAttr::get(module->getContext(), llvm::Twine{"omp.is_device"}),
+      mlir::BoolAttr::get(module->getContext(), isDevice));
+}
+
+// Return the value of the omp.is_device attribute stored in the module if it
+// exists, otherwise return false by default
+bool OpenMPDialect::getIsDevice(mlir::ModuleOp module) {
+  if (Attribute isDevice = module->getAttr("omp.is_device"))
+    if (isDevice.isa<mlir::BoolAttr>())
+      return isDevice.dyn_cast<BoolAttr>().getValue();
+  return false;
+}
+
 #define GET_ATTRDEF_CLASSES
 #include "mlir/Dialect/OpenMP/OpenMPOpsAttributes.cpp.inc"
 
 #define GET_OP_CLASSES
 #include "mlir/Dialect/OpenMP/OpenMPOps.cpp.inc"