[flang][driver] Add support for -mmlir

[awarzynski]

Summary

The semantic of -mmlir are identical to -mllvm. The only notable difference is that -mmlir options should be forwarded to MLIR rather than LLVM.

[rovka]

I don't know the command line library that well, so I have this curiosity: what happens if LLVM and MLIR have 2 different options with the same name? Do we get a compile time error? Or is there a risk that someone might -mllvm -XYZ and it would end up in MLIR's XYZ option instead, because we're processing the MLIR options after the LLVM ones?

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

15

Nit: Should these come after the llvm/ headers? (So it's alphabetical except for flang, which may go first)

flang/test/Driver/mllvm_vs_mmlir.f90

18

Is this the most llvm-ish option we have? I'm concerned that MLIR might also decide to add an option that sounds like this (and for sure -print-ir-before-all is mentioned in the MLIR docs).

[rovka]

flang/test/Driver/mllvm_vs_mmlir.f90

18

Is this the most llvm-ish option we have? I'm concerned that MLIR might also decide to add an option that sounds like this (and for sure -print-ir-before-all is mentioned in the MLIR docs).

Right, I think that might be why the premerge tests are failing...

[awarzynski]

Fix pre-commit CI

[awarzynski]

In D123297#3435816, @rovka wrote:

I don't know the command line library that well, so I have this curiosity: what happens if LLVM and MLIR have 2 different options with the same name? Do we get a compile time error?

The llvm::cl API uses global variables. So, if a variable is defined twice, you will get a compilation error. I will advertise this before merging so that folks from LLVM and MLIR are aware of this change. I'm hoping that they won't mind having to use distinct variable names for CL options in MLIR and LLVM.

Or is there a risk that someone might -mllvm -XYZ and it would end up in MLIR's XYZ option instead, because we're processing the MLIR options after the LLVM ones?

Note that -mllvm options are saved in llvmArgs and -mmlir options are saved in mlirArgs (this is taken care of in "CompilerInvocation.cpp"). This should guarantee that the right option set is used.

flang/test/Driver/mllvm_vs_mmlir.f90

18

Is this the most llvm-ish option we have?

Sadly, most LLVM options have rather generic names that could at some point be added in MLIR. Perhaps you'll spot something more suitable:

USAGE: flang (LLVM option parsing) [options]

OPTIONS:

Color Options:

  --color                                           - Use colors in output (default=autodetect)

General options:

  --aarch64-neon-syntax=<value>                     - Choose style of NEON code to emit from AArch64 backend:
    =generic                                        -   Emit generic NEON assembly
    =apple                                          -   Emit Apple-style NEON assembly
  --aarch64-use-aa                                  - Enable the use of AA during codegen.
  --abort-on-max-devirt-iterations-reached          - Abort when the max iterations for devirtualization CGSCC repeat pass is reached
  --allow-ginsert-as-artifact                       - Allow G_INSERT to be considered an artifact. Hack around AMDGPU test infinite loops.
  --always-execute-loop-body                        - force the body of a loop to execute at least once
  --array-constructor-initial-buffer-size=<uint>    - set the incremental array construction buffer size (default=32)
  --cfg-hide-cold-paths=<number>                    - Hide blocks with relative frequency below the given value
  --cfg-hide-deoptimize-paths                       -
  --cfg-hide-unreachable-paths                      -
  --cost-kind=<value>                               - Target cost kind
    =throughput                                     -   Reciprocal throughput
    =latency                                        -   Instruction latency
    =code-size                                      -   Code size
    =size-latency                                   -   Code size and latency
  --debugify-level=<value>                          - Kind of debug info to add
    =locations                                      -   Locations only
    =location+variables                             -   Locations and Variables
  --debugify-quiet                                  - Suppress verbose debugify output
  --default-kinds=<default-kind-string>             - string to set default kind values
  --disable-i2p-p2i-opt                             - Disables inttoptr/ptrtoint roundtrip optimization
  --dot-cfg-mssa=<file name for generated dot file> - file name for generated dot file
  --enable-cse-in-irtranslator                      - Should enable CSE in irtranslator
  --enable-cse-in-legalizer                         - Should enable CSE in Legalizer
  --enable-gvn-memdep                               -
  --enable-load-in-loop-pre                         -
  --enable-load-pre                                 -
  --enable-loop-simplifycfg-term-folding            -
  --enable-name-compression                         - Enable name/filename string compression
  --enable-split-backedge-in-load-pre               -
  --experimental-debug-variable-locations           - Use experimental new value-tracking variable locations
  --fdebug-dump-pre-fir                             - dump the Pre-FIR tree prior to FIR generation
  --fs-profile-debug-bw-threshold=<uint>            - Only show debug message if the source branch weight is greater  than this value.
  --fs-profile-debug-prob-diff-threshold=<uint>     - Only show debug message if the branch probility is greater than this value (in percentage).
  --gen-array-coor                                  - in lowering create ArrayCoorOp instead of CoordinateOp
  --generate-merged-base-profiles                   - When generating nested context-sensitive profiles, always generate extra base profile for function with all its context profiles merged into it.
  --inline-all                                      - aggressively inline everything
  --instcombine-code-sinking                        - Enable code sinking
  --instcombine-guard-widening-window=<uint>        - How wide an instruction window to bypass looking for another guard
  --instcombine-max-iterations=<uint>               - Limit the maximum number of instruction combining iterations
  --instcombine-max-num-phis=<uint>                 - Maximum number phis to handle in intptr/ptrint folding
  --instcombine-max-sink-users=<uint>               - Maximum number of undroppable users for instruction sinking
  --instcombine-maxarray-size=<uint>                - Maximum array size considered when doing a combine
  --instcombine-negator-enabled                     - Should we attempt to sink negations?
  --instcombine-negator-max-depth=<uint>            - What is the maximal lookup depth when trying to check for viability of negation sinking.
  --kind-mapping=<kind-mapping-string>              - kind mapping string to set kind precision
  --length-to-hash-string-literal=<ulong>           - string literals that exceed this length will use a hash value as their symbol name
  --main-entry-name=<string>                        - override the name of the default PROGRAM entry (may be helpful for using other runtimes)
  --math-runtime=<value>                            - Select math runtime version:
    =fast                                           -   use pgmath fast runtime
    =relaxed                                        -   use pgmath relaxed runtime
    =precise                                        -   use pgmath precise runtime
    =llvm                                           -   only use LLVM intrinsics (may be incomplete)
  --matrix-default-layout=<value>                   - Sets the default matrix layout
    =column-major                                   -   Use column-major layout
    =row-major                                      -   Use row-major layout
  --mir-strip-debugify-only                         - Should mir-strip-debug only strip debug info from debugified modules by default
  --no-discriminators                               - Disable generation of discriminator information.
  --non-recursive-procedures                        - Make procedures non-recursive by default. This was the default for all Fortran standards prior to 2018.
  --opaque-pointers                                 - Use opaque pointers
  --outline-intrinsics                              - Lower all intrinsic procedure implementation in their own functions
  --safepoint-ir-verifier-print-only                -
  --sample-profile-check-record-coverage=<N>        - Emit a warning if less than N% of records in the input profile are matched to the IR.
  --sample-profile-check-sample-coverage=<N>        - Emit a warning if less than N% of samples in the input profile are matched to the IR.
  --sample-profile-max-propagate-iterations=<uint>  - Maximum number of iterations to go through when propagating sample block/edge weights through the CFG.
  --use-alloc-runtime                               - Lower allocations to fortran runtime calls
  --use-desc-for-alloc                              - Always use descriptors for POINTER and ALLOCATABLE
  --verify-legalizer-debug-locs=<value>             - Verify that debug locations are handled
    =none                                           -   No verification
    =legalizations                                  -   Verify legalizations
    =legalizations+artifactcombiners                -   Verify legalizations and artifact combines
  --verify-region-info                              - Verify region info (time consuming)

Generic Options:

  --help                                            - Display available options (--help-hidden for more)
  --help-list                                       - Display list of available options (--help-list-hidden for more)
  --version                                         - Display the version of this program

18

Right, I think that might be why the premerge tests are failing...

Doh, I updated the test before sending this, but forgot to re-run it :( Should be fixed now.

[awarzynski]

Make sure the MLIR CL options are "applied"

[rriddle]

flang/test/Driver/mllvm_vs_mmlir.f90

18

I'd be alright with a patch to MLIR that prefixed the pass manager options with mlir-. The rest of our options (that are provided by libraries) should already be prefixed with that AFAIK.

[awarzynski]

flang/test/Driver/mllvm_vs_mmlir.f90

18

Sounds like a plan, thanks for the suggestion!

[mehdi_amini]

-mmlir is fine with me, but note that MLIR has much less global options than LLVM: you will only get access to context and passmanager options and not individual passes flags. That's not a criticism :)
(that's what makes it not likely to have conflicts by the way: the set of MLIR option is well identified)

[awarzynski]

In D123297#3436867, @mehdi_amini wrote:

-mmlir is fine with me, but note that MLIR has much less global options than LLVM: you will only get access to context and passmanager options and not individual passes flags. That's not a criticism :)
(that's what makes it not likely to have conflicts by the way: the set of MLIR option is well identified)

Thanks for taking a look! All this should be fine - the main rationale for this patch is to enable MLIR options in flang-new so that tests using tco or bbc (and which use MLIR options) can be ported to use flang-new. We were discussing this recently in https://reviews.llvm.org/D121171.

[awarzynski]

flang/test/Driver/mllvm_vs_mmlir.f90

18

See https://reviews.llvm.org/D123495

[rovka]

flang/test/Driver/mllvm_vs_mmlir.f90

18

Should we add this as a parent then?

[awarzynski]

flang/test/Driver/mllvm_vs_mmlir.f90

18

I've just merged it, so can re-base this patch instead.

[awarzynski]

I did a bit of digging and realised that MLIR llvm::cl options are lazily constructed on demand (see the definition of options in PassManagerOptions.cpp). This means that:

• Flang and LLVM global options are always visible and displayed when passing -mllvm -help,
• MLIR global options are only visible when explicitly initialised and displayed when passing -mmlir -help.

Since Flang and LLVM global options are always visible, llvm::cl will display them alongside MLIR options when using -mmlir -help. In other words, -mmlir -help is a superset of --mllvm -help. This is not ideal, but we'd need to refactor all option definitions in Flang and LLVM to improve this. I suggesting leaving this for later.

[awarzynski]

Rebase on top of main, re-fine the test

Make sure that the test file correctly takes into account that -mmlir -help is a superset of -mllvm -help.

[rovka]

LGTM, thanks!

[awarzynski]

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

15

Thanks, I will fix this before merging!

[MaskRay]

clang/include/clang/Driver/Options.td

3272

[awarzynski]

clang/include/clang/Driver/Options.td

3272

Thanks, included in the final version!

clang/include/clang/Driver/Options.td

 def mios_simulator_version_min_EQ : Joined<["-"], "mios-simulator-version-min=">;
 def miphonesimulator_version_min_EQ : Joined<["-"], "miphonesimulator-version-min=">, Alias<mios_simulator_version_min_EQ>;
 def mkernel : Flag<["-"], "mkernel">, Group<m_Group>;
 def mlinker_version_EQ : Joined<["-"], "mlinker-version=">,
   Flags<[NoXarchOption]>;
 def mllvm : Separate<["-"], "mllvm">,Flags<[CC1Option,CC1AsOption,CoreOption,FC1Option,FlangOption]>,
   HelpText<"Additional arguments to forward to LLVM's option processing">,
   MarshallingInfoStringVector<FrontendOpts<"LLVMArgs">>;
+def mmlir : Separate<["-"], "mmlir">,Flags<[CoreOption,FC1Option,FlangOption]>,
+  HelpText<"Additional arguments to forward to MLIR's option processing">;
 def ffuchsia_api_level_EQ : Joined<["-"], "ffuchsia-api-level=">,
   Group<m_Group>, Flags<[CC1Option]>, HelpText<"Set Fuchsia API level">,
   MarshallingInfoInt<LangOpts<"FuchsiaAPILevel">>;
 def mmacosx_version_min_EQ : Joined<["-"], "mmacosx-version-min=">,
   Group<m_Group>, HelpText<"Set Mac OS X deployment target">;
 def mmacos_version_min_EQ : Joined<["-"], "mmacos-version-min=">,
   Group<m_Group>, Alias<mmacosx_version_min_EQ>;
 def mms_bitfields : Flag<["-"], "mms-bitfields">, Group<m_Group>, Flags<[CC1Option]>,

clang/lib/Driver/ToolChains/Flang.cpp

void Flang::ConstructJob(Compilation &C, const JobAction &JA,
 
   // Forward -mllvm options to the LLVM option parser. In practice, this means
   // forwarding to `-fc1` as that's where the LLVM parser is run.
   for (const Arg *A : Args.filtered(options::OPT_mllvm)) {
     A->claim();
     A->render(Args, CmdArgs);
   }
 
+  for (const Arg *A : Args.filtered(options::OPT_mmlir)) {
+    A->claim();
+    A->render(Args, CmdArgs);
+  }
+
   if (Output.isFilename()) {
     CmdArgs.push_back("-o");
     CmdArgs.push_back(Output.getFilename());
   } else {
     assert(Output.isNothing() && "Invalid output.");
   }
 
   assert(Input.isFilename() && "Invalid input.");

flang/include/flang/Frontend/FrontendOptions.h

struct FrontendOptions {
 
   /// The name of the action to run when using a plugin action.
   std::string ActionName;
 
   /// A list of arguments to forward to LLVM's option processing; this
   /// should only be used for debugging and experimental features.
   std::vector<std::string> llvmArgs;
 
+  /// A list of arguments to forward to MLIR's option processing; this
+  /// should only be used for debugging and experimental features.
+  std::vector<std::string> mlirArgs;
+
   // Return the appropriate input kind for a file extension. For example,
   /// "*.f" would return Language::Fortran.
   ///
   /// \return The input kind for the extension, or Language::Unknown if the
   /// extension is not recognized.
   static InputKind GetInputKindForExtension(llvm::StringRef extension);
 };
 } // namespace Fortran::frontend
 
 #endif // LLVM_FLANG_FRONTEND_FRONTENDOPTIONS_H

flang/lib/Frontend/CompilerInvocation.cpp

bool CompilerInvocation::CreateFromArgs(CompilerInvocation &res,
   ParseTargetArgs(res.targetOpts(), args);
   parsePreprocessorArgs(res.preprocessorOpts(), args);
   success &= parseSemaArgs(res, args, diags);
   success &= parseDialectArgs(res, args, diags);
   success &= parseDiagArgs(res, args, diags);
   res.frontendOpts_.llvmArgs =
       args.getAllArgValues(clang::driver::options::OPT_mllvm);
 
+  res.frontendOpts_.mlirArgs =
+      args.getAllArgValues(clang::driver::options::OPT_mmlir);
+
   return success;
 }
 
 void CompilerInvocation::CollectMacroDefinitions() {
   auto &ppOpts = this->preprocessorOpts();
 
   for (unsigned i = 0, n = ppOpts.macros.size(); i != n; ++i) {
     llvm::StringRef macro = ppOpts.macros[i].first;

flang/lib/Frontend/FrontendActions.cpp

void CodeGenAction::GenerateLLVMIR() {
   // Set-up the MLIR pass manager
   mlir::PassManager pm(mlirCtx.get(), mlir::OpPassManager::Nesting::Implicit);
 
   pm.addPass(std::make_unique<Fortran::lower::VerifierPass>());
   pm.enableVerifier(/*verifyPasses=*/true);
 
   // Create the pass pipeline
   fir::createMLIRToLLVMPassPipeline(pm);
+  mlir::applyPassManagerCLOptions(pm);
 
   // Run the pass manager
   if (!mlir::succeeded(pm.run(*mlirModule))) {
     unsigned diagID = ci.diagnostics().getCustomDiagID(
         clang::DiagnosticsEngine::Error, "Lowering to LLVM IR failed");
     ci.diagnostics().Report(diagID);
   }
 

flang/lib/FrontendTool/CMakeLists.txt

 add_flang_library(flangFrontendTool
   ExecuteCompilerInvocation.cpp
 
   DEPENDS
   # This makes sure that the MLIR dependencies of flangFrontend (which are
   # transitively required here) are generated before this target is build.
   flangFrontend
   clangBasic
 
   LINK_LIBS
   flangFrontend
   clangBasic
   clangDriver
+  MLIRPass
 
   LINK_COMPONENTS
   Option
   Support
 )

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

 //===--- ExecuteCompilerInvocation.cpp ------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file holds ExecuteCompilerInvocation(). It is split into its own file to
 // minimize the impact of pulling in essentially everything else in Flang.
 //
 //===----------------------------------------------------------------------===//
 
+#include "mlir/IR/MLIRContext.h"
+#include "mlir/Pass/PassManager.h"
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/FrontendActions.h"
 #include "flang/Frontend/FrontendPluginRegistry.h"
 #include "clang/Driver/Options.h"
 #include "llvm/Option/OptTable.h"
 #include "llvm/Option/Option.h"
 #include "llvm/Support/BuryPointer.h"
 #include "llvm/Support/CommandLine.h"
if (!flang->frontendOpts().llvmArgs.empty()) {
 
     for (unsigned i = 0; i != numArgs; ++i)
       args[i + 1] = flang->frontendOpts().llvmArgs[i].c_str();
 
     args[numArgs + 1] = nullptr;
     llvm::cl::ParseCommandLineOptions(numArgs + 1, args.get());
   }
 
+  // Honor -mmlir. This should happen AFTER plugins have been loaded!
+  if (!flang->frontendOpts().mlirArgs.empty()) {
+    mlir::registerMLIRContextCLOptions();
+    mlir::registerPassManagerCLOptions();
+    unsigned numArgs = flang->frontendOpts().mlirArgs.size();
+    auto args = std::make_unique<const char *[]>(numArgs + 2);
+    args[0] = "flang (MLIR option parsing)";
+
+    for (unsigned i = 0; i != numArgs; ++i)
+      args[i + 1] = flang->frontendOpts().mlirArgs[i].c_str();
+
+    args[numArgs + 1] = nullptr;
+    llvm::cl::ParseCommandLineOptions(numArgs + 1, args.get());
+  }
+
   // If there were errors in processing arguments, don't do anything else.
   if (flang->diagnostics().hasErrorOccurred()) {
     return false;
   }
 
   // Create and execute the frontend action.
   std::unique_ptr<FrontendAction> act(CreateFrontendAction(*flang));
   if (!act)
     return false;
 
   bool success = flang->ExecuteAction(*act);
   return success;
 }
 
 } // namespace Fortran::frontend

flang/test/Driver/driver-help-hidden.f90

 ! CHECK-NEXT: -fno-automatic         Implies the SAVE attribute for non-automatic local objects in subprograms unless RECURSIVE
 ! CHECK-NEXT: -fno-color-diagnostics  Disable colors in diagnostics
 ! CHECK-NEXT: -fopenacc              Enable OpenACC
 ! CHECK-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! CHECK-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! CHECK-NEXT: -help     Display available options
 ! CHECK-NEXT: -I <dir>               Add directory to the end of the list of include search paths
 ! CHECK-NEXT: -mllvm <value>         Additional arguments to forward to LLVM's option processing
+! CHECK-NEXT: -mmlir <value>         Additional arguments to forward to MLIR's option processing
 ! CHECK-NEXT: -module-dir <dir>      Put MODULE files in <dir>
 ! CHECK-NEXT: -nocpp                 Disable predefined and command line preprocessor macros
 ! CHECK-NEXT: -o <file> Write output to <file>
 ! CHECK-NEXT: -pedantic              Warn on language extensions
 ! CHECK-NEXT: -print-effective-triple Print the effective target triple
 ! CHECK-NEXT: -print-target-triple    Print the normalized target triple
 ! CHECK-NEXT: -P                     Disable linemarker output in -E mode
 ! CHECK-NEXT: -std=<value>           Language standard to compile for

flang/test/Driver/driver-help.f90

 ! HELP-NEXT: -fno-automatic         Implies the SAVE attribute for non-automatic local objects in subprograms unless RECURSIVE
 ! HELP-NEXT: -fno-color-diagnostics  Disable colors in diagnostics
 ! HELP-NEXT: -fopenacc              Enable OpenACC
 ! HELP-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! HELP-NEXT: -help                  Display available options
 ! HELP-NEXT: -I <dir>               Add directory to the end of the list of include search paths
 ! HELP-NEXT: -mllvm <value>         Additional arguments to forward to LLVM's option processing
+! HELP-NEXT: -mmlir <value>         Additional arguments to forward to MLIR's option processing
 ! HELP-NEXT: -module-dir <dir>      Put MODULE files in <dir>
 ! HELP-NEXT: -nocpp                 Disable predefined and command line preprocessor macros
 ! HELP-NEXT: -o <file>              Write output to <file>
 ! HELP-NEXT: -pedantic              Warn on language extensions
 ! HELP-NEXT: -print-effective-triple Print the effective target triple
 ! HELP-NEXT: -print-target-triple    Print the normalized target triple
 ! HELP-NEXT: -P                     Disable linemarker output in -E mode
 ! HELP-NEXT: -std=<value>           Language standard to compile for

 ! HELP-FC1-NEXT: -fopenacc              Enable OpenACC
 ! HELP-FC1-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-FC1-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! HELP-FC1-NEXT: -help                  Display available options
 ! HELP-FC1-NEXT: -init-only             Only execute frontend initialization
 ! HELP-FC1-NEXT: -I <dir>               Add directory to the end of the list of include search paths
 ! HELP-FC1-NEXT: -load <dsopath>        Load the named plugin (dynamic shared object)
 ! HELP-FC1-NEXT: -mllvm <value>         Additional arguments to forward to LLVM's option processing
+! HELP-FC1-NEXT: -mmlir <value>         Additional arguments to forward to MLIR's option processing
 ! HELP-FC1-NEXT: -module-dir <dir>      Put MODULE files in <dir>
 ! HELP-FC1-NEXT: -module-suffix <suffix> Use <suffix> as the suffix for module files (the default value is `.mod`)
 ! HELP-FC1-NEXT: -nocpp                 Disable predefined and command line preprocessor macros
 ! HELP-FC1-NEXT: -o <file>              Write output to <file>
 ! HELP-FC1-NEXT: -pedantic              Warn on language extensions
 ! HELP-FC1-NEXT: -plugin <name>         Use the named plugin action instead of the default action (use "help" to list available options)
 ! HELP-FC1-NEXT: -P                     Disable linemarker output in -E mode
 ! HELP-FC1-NEXT: -std=<value>           Language standard to compile for

flang/test/Driver/mllvm_vs_mmlir.f90

+! Verify that `-mllvm` options are forwarded to LLVM and `-mmlir` to MLIR
+
+!------------
+! RUN COMMAND
+!------------
+! RUN: %flang_fc1  -mmlir --help | FileCheck %s --check-prefix=MLIR
+! RUN: %flang_fc1  -mllvm --help | FileCheck %s --check-prefix=MLLVM
+
+!----------------
+! EXPECTED OUTPUT
+!----------------
+! MLIR: flang (MLIR option parsing) [options]
+! MLIR: --mlir-{{.*}}
+
+! MLLVM: flang (LLVM option parsing) [options]
+! MLLVM-NOT: --mlir-{{.*}}