[flang][driver] Extend the `flang` bash script to act as a driver

[awarzynski]

Summary

Until now, f18 would:

1. Use Flang to unparse the input files
2. Call an external Fortran compiler to compile the unparsed source files (generated in step 1)

With this patch, f18 will stop after unparsing the input source files,
i.e. step 1 above. The flang bash script will take care of step 2,
i.e. calling an external Fortran compiler driver to compile them. This
way:

• the functionality of f18 is reduced - it will only drive Flang (as opposed to delegating code-generation to an external tool on top of this)
• we will able to switch between f18 and flang-new for unparsing before an external Fortran compiler is called for code-generation

These changes were discussed in [1] as a requirement for replacing f18
with flang-new.

[1] https://lists.llvm.org/pipermail/flang-dev/2021-April/000677.html

[awarzynski]

I tested the updated flang script with SPEC 2017 and it worked fine for me. However, I don't really use the functionality re-implemented here, so I might have missed something. I would really appreciate if you tested your workflows and let me know whether everything works for you as before.

I still need to:

• remove code for calling external compilers in f18.cpp
• update error messages in flang
• make flang switch between f18 and flang-new based on FLANG_BUILD_NEW_DRIVER

I appreciate that in this respect my patch is a bit incomplete, but I will be away in the coming days and was hoping to get some early feedback in the meantime.

Thank you!

[klausler]

I'm not clear why you plan to remove code from f18.cpp.

[awarzynski]

In D103177#2782459, @klausler wrote:

I'm not clear why you plan to remove code from f18.cpp.

Otherwise flang, the bash script, duplicates functionality available in f18.cpp. The goal is to move that functionality to flang.

[Meinersbur]

A concern that I have is that the bash script will only on platforms that have bash. In addition to Windows, some distribution use different default shells and may not have it installed (MacOS uses zsh and ships only an old version of bash).
However, as long as this is transitional, and to be removed again once the flang toolchain is fully functional, I would be ok with it.

In D103177#2782881, @awarzynski wrote:

Otherwise flang, the bash script, duplicates functionality available in f18.cpp. The goal is to move that functionality to flang.

I think the argument is that f18.cpp will be removed completely, not necessary to remove specific functionality before that.

[awarzynski]

In D103177#2785499, @Meinersbur wrote:

A concern that I have is that the bash script will only on platforms that have bash.

Thanks for bringing this up, I share your concerns. Our goal is to get code generation in llvm-project/flang as soon as possible. Once that's available, we shouldn't need this script and I will be very much of favor of removing it. If it offers a functionality that we'll need long-term, then that functionality should be identified and re-implemented in something platform agnostic (e.g. C++ or Python). For me, this patch is here to facilitate the transition from f18 to flang-new as the default driver.

In D103177#2782881, @awarzynski wrote:

Otherwise flang, the bash script, duplicates functionality available in f18.cpp. The goal is to move that functionality to flang.

I think the argument is that f18.cpp will be removed completely, not necessary to remove specific functionality before that.

Yes, that's the end goal. I guess that I'm focusing more on the intermediate steps.

[awarzynski]

Remove code from f18.cpp

All code that dealt with code-generation in f18.cpp has been removed. The flang script has been extended - it can now compile non-Fortran sources. Also, error messages in the bash script have been updated.

I also had to slightly update the semantics of -funparse-typed-exprs-to-f18-fc and replace it with -fno-unparse-typed-exprs-to-f18-fc. This way I was able to preserve the original behaviour of flang (in general) and f18 (when using -fdebug-unparse).

[klausler]

I still don't understand why you have to mess with f18.cpp. What would break if you left it alone?

[PeteSteinfeld]

In D103177#2782459, @klausler wrote:

I'm not clear why you plan to remove code from f18.cpp.

Even if we decide that it's in our long-term interests to remove this code, it would be better to have a period of transition where we are able to transition to the new functionality while retaining the old as a fallback.

[Leporacanthicus]

In D103177#2791916, @klausler wrote:

I still don't understand why you have to mess with f18.cpp. What would break if you left it alone?

I'm still new here and don't know the whole history, but I think this is meant to be removed eventually, so removing it in pieces seems unnecessary.

Is there any reason we can't remove it immediately? In a separate patch perhaps...
Is there some reason it needs to remain in the source-tree once there's code-generation working with the new driver?

[awarzynski]

Switch between flang-new and f18, depending on the value of FLANG_BUILD_NEW_DRIVER

With this latest update, flang will use flang-new when the new driver is enabled. As previously, I tested this with SPEC 2017 (as well as some trivial examples) and it worked fine for me.

Some extra changes in flang-new were required. I extracted them into seperate patches: D103612, D103613.

[awarzynski]

@klausler , @PeteSteinfeld , thank you for your comments! I'm happy to reduce the changes in f18.cpp to the required minimum. Trimming it "bit-by-bit" seemed like a good idea, but we can move this to a separate patch if that's preferable. I'll send an update shortly!

@Leporacanthicus, yes, our end goal is to replace f18 with flang-new and this means deleting f18.cpp. With this patch, f18 and flang-new become equivalent in terms of functionality and, IIUC, we are ready to delete f18. I'm not aware of any reasons to keep it in-tree long term. Perhaps others could comment?

@PeteSteinfeld, what would be a good transition period? Any suggestions? Is there anything in particular that we could test in the meantime?

[klausler]

In D103177#2796916, @awarzynski wrote:

@klausler , @PeteSteinfeld , thank you for your comments! I'm happy to reduce the changes in f18.cpp to the required minimum. Trimming it "bit-by-bit" seemed like a good idea, but we can move this to a separate patch if that's preferable. I'll send an update shortly!

@Leporacanthicus, yes, our end goal is to replace f18 with flang-new and this means deleting f18.cpp. With this patch, f18 and flang-new become equivalent in terms of functionality and, IIUC, we are ready to delete f18. I'm not aware of any reasons to keep it in-tree long term. Perhaps others could comment?

@PeteSteinfeld, what would be a good transition period? Any suggestions? Is there anything in particular that we could test in the meantime?

What's the timeline for removing the dependency of flang-new on clang?

[PeteSteinfeld]

@awarzynski, I'm not sure what a good transition period would be. Actually, I don't see the benefit of removing any functionality from f18. This might be a good topic to bring up at the Wednesday f18 meeting.

[awarzynski]

In D103177#2796994, @klausler wrote:

What's the timeline for removing the dependency of flang-new on clang?

We don't have a timeline for this, but it will be non-trivial amount of effort.
We discussed the dependency on Clang in our community calls recently. I was under the impression that we agreed that this dependency is a fair compromise for what we are getting in return. At least in the interim.

In D103177#2797954, @PeteSteinfeld wrote:

Actually, I don't see the benefit of removing any functionality from f18.

The new driver is developed to replace f18. In terms of functionality, flang-new and f18 are currently equivalent. However, once we start supporting code-generation, flang-new will be able to leverage clangDriver for support for different tool-chains and operating systems. In f18 we would have to implement this from scratch. Also, the new frontend driver is designed to allow for frontend plugins (akin to plugins in Clang). Again, that's something that would take a lot of effort to implement in f18. Also, flang-new is already more user-friendly than f18:

flang-new -fc1 -random-flag -fdebug-unparse file.f
error: unknown argument: '-random-flag'

This behavior is consistent across all options (no matter the semantics) and it's something that we get for "free" from clangDriver. In comparison, f18 happily ignores unknown flags:

f18 -random-flag -fdebug-unparse file.f

On a different note,

In D103177#2797954, @PeteSteinfeld wrote:

This might be a good topic to bring up at the Wednesday f18 meeting.

Sounds like a good idea! Also, as mentioned previously, we don't need to remove any functionality from f18 in this patch. All changes to f18.cpp have been reverted and we can focus on the bash script instead.

[awarzynski]

Rebase on top of D103612

[klausler]

We don't have a timeline for this, but it will be non-trivial amount of effort. We discussed the dependency on Clang in our community calls recently. I was under the impression that we agreed that this dependency is a fair compromise for what we are getting in return. At least in the interim.

That's great for your community, and I appreciate the update on your activity, but I'd like to avoid the clang dependence for what I'm doing, if I can. I'd prefer that f18.cpp stay in your tree but it's not the end of the world for me if you remove it.

[awarzynski]

In D103177#2799429, @klausler wrote:

I'd like to avoid the clang dependence for what I'm doing, if I can.

Do you require full driver support for what you do? Perhaps we could extract a smaller tool from f18 that would meet your needs and be independent of Clang? For now I'm just thinking out loud, but this might be an option.

We discussed f18 in the call yesterday and agreed to remove it in the near future. I've summarized it here: https://lists.llvm.org/pipermail/flang-dev/2021-June/000742.html. We will have this patch merged first and then wait for at least 2 weeks. Please let me know if you need more time or discover something that this would break for you.

[klausler]

In D103177#2810230, @awarzynski wrote:

In D103177#2799429, @klausler wrote:

I'd like to avoid the clang dependence for what I'm doing, if I can.

Do you require full driver support for what you do? Perhaps we could extract a smaller tool from f18 that would meet your needs and be independent of Clang? For now I'm just thinking out loud, but this might be an option.

We discussed f18 in the call yesterday and agreed to remove it in the near future. I've summarized it here: https://lists.llvm.org/pipermail/flang-dev/2021-June/000742.html. We will have this patch merged first and then wait for at least 2 weeks. Please let me know if you need more time or discover something that this would break for you.

I don't know what "smaller tools" you could extract from the f18 tool that would be useful, but thanks for the offer. The current f18 is still a good fit for my own needs (as one would expect) and I'd keep it in my tree if you delete it from yours.

The source for the f18 and f18-parse-demo tools might be useful to hypothetical future tool-writers as a minimal example of what one needs to do to drive the parser and later phases in a stand-alone environment for purposes other than Fortran compilation (e.g., pretty-printing, source-to-source transformations, program analysis); is your community still interested in such things?

[SouraVX]

Overall I support & I'd like to have one driver i.e flang-new considering all the effort that has gone in it and the long term plan associated with the driver(that will cater to the entire flang project as a primary interface). :)

I can observe/note following benefits of having one driver:

• Keeping things consistent/converge on flang-new will help in synchronizing the effort of the community. This should help anyone who wants to contribute, not to get intimidated by presence of 2 drivers(and implied effort of keeping the feature parity across both).
• flang-new just now gained the feature parity with existing driver f18 (Thanks to @awarzynski and other folks involved :)), from now onwards either one has to bear the cost of maintenance & NOT breaking each other while progressing.
• Speaking of the dependency on clang, I think this will remain and if people don't appreciate it - then this will require entire re-write, non-trivial effort, and needs a solid motivation to move forward along this path. NOT to mention, this entire activity may end up eating most (if not all) efforts that were planned to get the driver functional/usable to a point :)

On the path forward, If people have interest in having f18 (which I can see from above conversation), then at least we can mutually agree upon some clear timelines(2-3 weeks ?) as to when f18 is Okay to remove completely.
This should help in synchronizing folks from both sides :)

[klausler]

In D103177#2813671, @SouraVX wrote:

Overall I support & I'd like to have one driver i.e flang-new considering all the effort that has gone in it and the long term plan associated with the driver(that will cater to the entire flang project as a primary interface). :)

I can observe/note following benefits of having one driver:

• Keeping things consistent/converge on flang-new will help in synchronizing the effort of the community. This should help anyone who wants to contribute, not to get intimidated by presence of 2 drivers(and implied effort of keeping the feature parity across both).
• flang-new just now gained the feature parity with existing driver f18 (Thanks to @awarzynski and other folks involved :)), from now onwards either one has to bear the cost of maintenance & NOT breaking each other while progressing.
• Speaking of the dependency on clang, I think this will remain and if people don't appreciate it - then this will require entire re-write, non-trivial effort, and needs a solid motivation to move forward along this path. NOT to mention, this entire activity may end up eating most (if not all) efforts that were planned to get the driver functional/usable to a point :)

On the path forward, If people have interest in having f18 (which I can see from above conversation), then at least we can mutually agree upon some clear timelines(2-3 weeks ?) as to when f18 is Okay to remove completely.
This should help in synchronizing folks from both sides :)

That's fine with me; as I've said, I don't need to have f18.cpp in your tree to be able to work on my parts of the project. Hypothetical future Fortran tool-writers who just want a parser & maybe semantics can reconstruct it when/if needed, I suppose.

[awarzynski]

Make sure that -fc1 is the first option that flang-new sees

-fc1 is not a regular option and its semantics require that it's the first option that the driver parses. This revision updates flang accordingly.

[Leporacanthicus]

Some minor nit-picks, but nothing wrong - I tested this with some of my fortran code that I've been using for other testing, and it seems to work.

flang/tools/f18/CMakeLists.txt

70

Nit: wouldn't a single blank line be good enough?

flang/tools/f18/flang.in

230

nit: driver

[awarzynski]

Address nits from @Leporacanthicus

Thank you for reviewing, Mats! This patch depends on D103612, so I need to have that one approved before I can merge this.

[awarzynski]

Rebase on top of main + minor update

I updated f18 so that it understands f18 -fdebug-unparse -o <output_file>. The updated flang script relies on this behaviour. Before, f18 would always dump to stdout (with or without -o <output_file>).

[awarzynski]

Make sure that flang exits early when -E is used

SPEC 2017 still builds and runs fine.

[awarzynski]

@sscalpone , thank you for the suggestion on Monday (i.e. to test this change with Fortran Tests in the LLVM Test Suite). While working on it, I discovered that the integration of Flang with CMake is currently not working. I've brought this up on flang-dev for better visibility. As the test-suite requires CMake support, we cannot run it using flang (_before_ or_ after_ this change).

If there are no further comments, I will proceed and merge this change tomorrow. Thank you all for the discussion and for reviewing!

[awarzynski]

Make shellcheck happy

[DavidSpickett]

@awarzynski We have some fortran failures in the test suite after this commit:
https://lab.llvm.org/buildbot/#/builders/184/builds/109

For example:

******************** TEST 'test-suite :: Fortran/UnitTests/fcvs21_f95/FM363.test' FAILED ********************
/home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/tools/timeit-target --limit-core 0 --limit-cpu 7200 --timeout 7200 --limit-file-size 104857600 --limit-rss-size 838860800 --append-exitstatus --redirect-output /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/Output/FM363.test.out --redirect-input /dev/null --chdir /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95 --summary /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/Output/FM363.test.time /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/FM363
cd /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95 ; /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/tools/fpcmp-target /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/Output/FM363.test.out FM363.reference_output

/home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/tools/fpcmp-target: FP Comparison failed, not a numeric difference between 'F' and ' '

Not totally sure this is the cause but it's the only fortran related commit. Some fp related option being dropped or re-ordered?

Oh and that bot is running on SVE hardware but I don't think it's that, it just happens to be the only flang bot that has the test suite enabled.

[rovka]

In D103177#2854933, @DavidSpickett wrote:

@awarzynski We have some fortran failures in the test suite after this commit:
https://lab.llvm.org/buildbot/#/builders/184/builds/109

For example:

******************** TEST 'test-suite :: Fortran/UnitTests/fcvs21_f95/FM363.test' FAILED ********************
/home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/tools/timeit-target --limit-core 0 --limit-cpu 7200 --timeout 7200 --limit-file-size 104857600 --limit-rss-size 838860800 --append-exitstatus --redirect-output /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/Output/FM363.test.out --redirect-input /dev/null --chdir /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95 --summary /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/Output/FM363.test.time /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/FM363
cd /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95 ; /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/tools/fpcmp-target /home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/Fortran/UnitTests/fcvs21_f95/Output/FM363.test.out FM363.reference_output

/home/tcwg-buildslave/worker/clang-aarch64-sve-vls/test/sandbox/build/tools/fpcmp-target: FP Comparison failed, not a numeric difference between 'F' and ' '

Not totally sure this is the cause but it's the only fortran related commit. Some fp related option being dropped or re-ordered?

Oh and that bot is running on SVE hardware but I don't think it's that, it just happens to be the only flang bot that has the test suite enabled.

Hi David, clang-aarch64-full-2stage also has the test-suite enabled, is that one passing?
Sometimes that error message comes up in the test-suite when the executable prints an error message instead of the expected output. Do you think you could dig out the actual output of the test?
Thanks!

[awarzynski]

In D103177#2854933, @DavidSpickett wrote:

@awarzynski We have some fortran failures in the test suite after this commit:
https://lab.llvm.org/buildbot/#/builders/184/builds/109

Thanks for reporting this @DavidSpickett !

Not totally sure this is the cause but it's the only fortran related commit. Some fp related option being dropped or re-ordered?

Yes, it looks like it's related, but it's not immediately obvious how exactly! I've tried testing this patch with LLVM's test-suite, but CMake does not support LLVM Flang yet: https://lists.llvm.org/pipermail/flang-dev/2021-June/000745.html. I've asked around and AFAIK, no one yet has been able to use LLVM Flang with the test-suite. So I'm not sure how people managed to set-up that BuildBot worker to work :)

Oh and that bot is running on SVE hardware but I don't think it's that, it just happens to be the only flang bot that has the test suite enabled.

There is another one here: https://lab.llvm.org/buildbot/#/builders/179. Oddly, it doesn't complain. Also, @rovka kindly tested this patch using that worker's config before it was merged.

So, any clue what the difference between the two workers is?

[DavidSpickett]

So, any clue what the difference between the two workers is?

One tries to enable SVE code, that's it.

Edit: Well, the 1 stage sve just sets cpu to a64fx, the 2 stage tries to vectorise the second stage build.

Hi David, clang-aarch64-full-2stage also has the test-suite enabled, is that one passing?

Yknow, it is, but it was before. I don't see why it turned green the only commit in the next build is unrelated.

Leave this with me, I'll see if the next builds also fail.

I've tried testing this patch with LLVM's test-suite, but CMake does not support LLVM Flang yet

Yeah I don't see any reference to flang bar 1 comment line in the test suite. I see references to nagfortran and f2c in Makefile.FORTRAN. Perhaps we installed one of those in the buildbot image accidentally.

flang/lib/Frontend/FrontendActions.cpp

Unparse(llvm::outs(), *parseTree,
       /*preStatement=*/nullptr,
       invoc.useAnalyzedObjectsForUnparse() ? &invoc.asFortran() : nullptr);
 }
 
 void DebugUnparseAction::ExecuteAction() {
   auto &invoc = this->instance().invocation();
   auto &parseTree{instance().parsing().parseTree()};
 
+  CompilerInstance &ci = this->instance();
+  auto os{ci.CreateDefaultOutputFile(
+      /*Binary=*/false, /*InFile=*/GetCurrentFileOrBufferName())};
+
   // TODO: Options should come from CompilerInvocation
-  Unparse(llvm::outs(), *parseTree,
+  Unparse(*os, *parseTree,
       /*encoding=*/Fortran::parser::Encoding::UTF_8,
       /*capitalizeKeywords=*/true, /*backslashEscapes=*/false,
       /*preStatement=*/nullptr,
       invoc.useAnalyzedObjectsForUnparse() ? &invoc.asFortran() : nullptr);
 
   // Report fatal semantic errors
   reportFatalSemanticErrors(semantics(), this->instance().diagnostics(),
       GetCurrentFileOrBufferName());

flang/tools/f18/CMakeLists.txt

foreach(filename ${MODULES})
   list(APPEND MODULE_FILES ${base}.mod ${base}.f18.mod)
   install(FILES ${base}.mod ${base}.f18.mod DESTINATION include/flang)
 endforeach()
 
 add_custom_target(module_files ALL DEPENDS ${MODULE_FILES})
 
 install(TARGETS f18 DESTINATION bin)
 
+set(FLANG_DEFAULT_DRIVER "flang-new")
+if (NOT FLANG_BUILD_NEW_DRIVER)
+  set(FLANG_DEFAULT_DRIVER "f18")
+endif()
+
 # This flang shell script will only work in a POSIX shell.
 if (NOT WIN32)
-  file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/flang
-    DESTINATION ${CMAKE_BINARY_DIR}/bin
-    FILE_PERMISSIONS OWNER_EXECUTE OWNER_READ OWNER_WRITE)
+  configure_file(${CMAKE_CURRENT_SOURCE_DIR}/flang.in ${CMAKE_BINARY_DIR}/bin/flang @ONLY)
   install(PROGRAMS ${CMAKE_BINARY_DIR}/bin/flang DESTINATION bin)
 endif()

flang/tools/f18/f18.cpp

if (driver.getDefinition) {
       exitStatus = EXIT_FAILURE;
       return {};
     }
   }
   if (driver.dumpParseTree) {
     Fortran::parser::DumpTree(llvm::outs(), parseTree, &asFortran);
   }
   if (driver.dumpUnparse) {
-    Unparse(llvm::outs(), parseTree, driver.encoding, true /*capitalize*/,
+    // Prepare the output stream
+    std::unique_ptr<llvm::raw_fd_ostream> os;
+    std::string outputFile = "-";
+    if (!driver.outputPath.empty()) {
+      outputFile = driver.outputPath;
+    }
+
+    std::error_code EC;
+    os.reset(new llvm::raw_fd_ostream(
+        outputFile, EC, llvm::sys::fs::OF_TextWithCRLF));
+    if (EC) {
+      llvm::errs() << EC.message() << "\n";
+      std::exit(EXIT_FAILURE);
+    }
+
+    Unparse(*os, parseTree, driver.encoding, true /*capitalize*/,
         options.features.IsEnabled(
             Fortran::common::LanguageFeature::BackslashEscapes),
         nullptr /* action before each statement */,
         driver.useAnalyzedObjectsForUnparse ? &asFortran : nullptr);
     return {};
   }
   if (driver.dumpPreFirTree) {
     if (auto ast{Fortran::lower::createPFT(parseTree, semanticsContext)}) {

flang/tools/f18/flang

-#!/bin/bash
-#===-- tools/f18/flang.sh -----------------------------------------*- sh -*-===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
-
-wd=$(cd $(dirname "$0")/.. && pwd)
-opts="-fno-analyzed-objects-for-unparse -module-suffix .f18.mod "
-if ! $wd/bin/f18 $opts "$@"
-then status=$?
-     echo flang: in $PWD, f18 failed with exit status $status: $wd/bin/f18 $opts "$@" >&2
-     exit $status
-fi

flang/tools/f18/flang.in

+#! /usr/bin/env bash
+#===-- tools/f18/flang.sh -----------------------------------------*- sh -*-===#
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+#===------------------------------------------------------------------------===#
+# A wrapper script for Flang's compiler driver that was developed for testing and
+# experimenting. You should be able to use it as a regular compiler driver. It
+# will:
+#   * run Flang's compiler driver to unparse the input source files
+#   * use the external compiler (defined via F18_FC environment variable) to
+#   compile the unparsed source files
+#===------------------------------------------------------------------------===#
+set -euo pipefail
+
+# Global variables to make the parsing of input arguments a bit easier
+INPUT_FILES=()
+OPTIONS=()
+OUTPUT_FILE=""
+MODULE_DIR=""
+INTRINSICS_MOD_DIR=""
+COMPILE_ONLY="False"
+PREPROCESS_ONLY="False"
+TEMP_OUTPUT="flang_temp_out_"
+
+# === parse_args ==============================================================
+#
+# Parse the input arguments passed to this script. Sets the global variables
+# declared at the top.
+#
+# INPUTS:
+#   $1 - all input arguments
+# OUTPUTS:
+#  Saved in the global variables for this script
+# =============================================================================
+parse_args()
+{
+  while [ "${1:-}" != "" ]; do
+      # CASE 1: Compiler option
+      if [[ "${1:0:1}" == "-" ]] ; then
+        # Output file - extract it into a global variable
+        if [[ "$1" == "-o" ]] ; then
+          shift
+          OUTPUT_FILE="$1"
+          shift
+          continue
+        fi
+
+        # Module directory - extract it into a global variable
+        if [[ "$1" == "-module-dir" ]]; then
+          shift
+          MODULE_DIR="$1"
+          shift
+          continue
+        fi
+
+        # Intrinsics module dir - extract it into a global var
+        if [[ "$1" == "-intrinsics-module-directory" ]]; then shift
+          INTRINSICS_MOD_DIR=$1
+          shift
+          continue
+        fi
+
+        # Module suffix cannot be modified - this script defines it before
+        # calling the driver.
+        if [[ "$1" == "-module-suffix" ]]; then
+          echo "ERROR: \'-module-suffix\' is not available when using the \'flang\' script"
+          exit 1
+        fi
+
+        # Special treatment for `J <dir>` and `-I <dir>`. We translate these
+        # into `J<dir>` and `-I<dir>` respectively.
+        if [[ "$1" == "-J" ]] || [[ "$1" == "-I" ]]; then
+          opt=$1
+          shift
+          OPTIONS+=("$opt$1")
+          shift
+          continue
+        fi
+
+        # This is a regular option - just add it to the list.
+        OPTIONS+=($1)
+        if [[ $1 == "-c" ]]; then
+          COMPILE_ONLY="True"
+        fi
+
+        if [[ $1 == "-E" ]]; then
+          PREPROCESS_ONLY="True"
+        fi
+
+        shift
+        continue
+
+      # CASE 2: A regular file (either source or a library file)
+      elif [[ -f "$1" ]]; then
+        INPUT_FILES+=($1)
+        shift
+        continue
+
+      else
+        # CASE 3: Unsupported
+        echo "ERROR: unrecognised option format $1"
+        exit 1
+      fi
+  done
+}
+
+# === categorise_files ========================================================
+#
+# Categorises input files into:
+#   * Fortran source files (to be compiled)
+#   * library files (to be linked into the final executable)
+#
+# INPUTS:
+#   $1 - all input files to be categorised (array, name reference)
+# OUTPUTS:
+#   $2 - Fortran source files extracted from $1 (array, name reference)
+#   $3 - other source files extracted from $1 (array, name reference)
+#   $4 - object files extracted from $1 (array, name reference)
+#   $4 - lib files extracted from $1 (array, name reference)
+# =============================================================================
+categorise_files()
+{
+  local -n -r all_files=$1
+  local -n fortran_sources=$2
+  local -n other_sources=$3
+  local -n libs=$4
+
+  for current_file in "${all_files[@]}"; do
+    file_ext=${current_file##*.}
+    if [[ $file_ext == "f" ]] || [[ $file_ext == "f90" ]] ||
+       [[ $file_ext == "f" ]] || [[ $file_ext == "F" ]] || [[ $file_ext == "ff" ]] ||
+       [[ $file_ext == "f90" ]] || [[ $file_ext == "F90" ]] || [[ $file_ext == "ff90" ]] ||
+       [[ $file_ext == "f95" ]] || [[ $file_ext == "F95" ]] || [[ $file_ext == "ff95" ]] ||
+       [[ $file_ext == "cuf" ]] || [[ $file_ext == "CUF" ]] || [[ $file_ext == "f18" ]] ||
+       [[ $file_ext == "F18" ]] || [[ $file_ext == "ff18" ]]; then
+      fortran_sources+=($current_file)
+    elif [[ $file_ext == "a" ]] || [[ $file_ext == "so" ]]; then
+      libs+=($current_file)
+    elif [[ $file_ext == "o" ]]; then
+      object_files+=($current_file)
+    else
+      other_sources+=($current_file)
+    fi
+  done
+}
+
+# === categorise_opts ==========================================================
+#
+# Categorises compiler options into options for:
+#   * the Flang driver (either new or the "throwaway" driver)
+#   * the external Fortran driver that will generate the code
+# Most options accepted by Flang will be claimed by it. The only exceptions are
+# `-I` and `-J`.
+#
+# INPUTS:
+#   $1 - all compiler options (array, name reference)
+# OUTPUTS:
+#   $2 - compiler options for the Flang driver (array, name reference)
+#   $3 - compiler options for the external driver (array, name reference)
+# =============================================================================
+categorise_opts()
+{
+  local -n all_opts=$1
+  local -n flang_opts=$2
+  local -n fc_opts=$3
+
+  for opt in "${all_opts[@]}"; do
+    # These options are claimed by Flang, but should've been dealt with in parse_args.
+    if  [[ $opt == "-module-dir" ]] ||
+      [[ $opt == "-o" ]] ||
+      [[ $opt == "-fintrinsic-modules-path" ]] ; then
+      echo "ERROR: $opt should've been fully processed by \`parse_args\`"
+      exit 1
+    fi
+
+    if
+      # The options claimed by Flang. This list needs to be compatible with
+      # what's supported by Flang's compiler driver (i.e. `flang-new` and f18).
+      [[ $opt == "-cpp" ]] ||
+      [[ $opt =~ ^-D.* ]] ||
+      [[ $opt == "-E" ]] ||
+      [[ $opt == "-falternative-parameter-statement" ]] ||
+      [[ $opt == "-fbackslash" ]] ||
+      [[ $opt == "-fcolor-diagnostics" ]] ||
+      [[ $opt == "-fdefault-double-8" ]] ||
+      [[ $opt == "-fdefault-integer-8" ]] ||
+      [[ $opt == "-fdefault-real-8" ]] ||
+      [[ $opt == "-ffixed-form" ]] ||
+      [[ $opt =~ ^-ffixed-line-length=.* ]] ||
+      [[ $opt == "-ffree-form" ]] ||
+      [[ $opt == "-fimplicit-none" ]] ||
+      [[ $opt =~ ^-finput-charset=.* ]] ||
+      [[ $opt == "-flarge-sizes" ]] ||
+      [[ $opt == "-flogical-abbreviations" ]] ||
+      [[ $opt == "-fno-color-diagnostics" ]] ||
+      [[ $opt == "-fopenacc" ]] ||
+      [[ $opt == "-fopenmp" ]] ||
+      [[ $opt == "-fxor-operator" ]] ||
+      [[ $opt == "-help" ]] ||
+      [[ $opt == "-nocpp" ]] ||
+      [[ $opt == "-pedantic" ]] ||
+      [[ $opt =~ ^-std=.* ]] ||
+      [[ $opt =~ ^-U.* ]] ||
+      [[ $opt == "--version" ]] ||
+      [[ $opt == "-Werror" ]]; then
+      flang_opts+=($opt)
+    elif [[ $opt =~ -I.* ]] || [[ $opt =~ -J.* ]]; then
+      # Options that are needed for both Flang and the external driver.
+      flang_opts+=($opt)
+      fc_opts+=($opt)
+    else
+      # All other options are claimed for the external driver.
+      fc_opts+=($opt)
+    fi
+  done
+}
+
+# === preprocess ==============================================================
+#
+# Runs the preprocessing. Fortran files are preprocessed using Flang. Other
+# files are preprocessed using the external Fortran compiler.
+#
+# INPUTS:
+#   $1 - Fortran source files (array, name reference)
+#   $2 - other source files (array, name reference)
+#   $3 - compiler flags (array, name reference)
+# =============================================================================
+preprocess() {
+  local -n fortran_srcs=$1
+  local -n other_srcs=$2
+  local -n opts=$3
+
+  local -r ext_fc="${F18_FC:-gfortran}"
+  local -r wd=$(cd "$(dirname "$0")/.." && pwd)
+
+  # Use the provided output file name.
+  if [[ ! -z ${OUTPUT_FILE:+x} ]]; then
+    output_definition="-o $OUTPUT_FILE"
+  fi
+
+  # Preprocess fortran sources using Flang
+  for idx in "${!fortran_srcs[@]}"; do
+    if ! "$wd/bin/@FLANG_DEFAULT_DRIVER@" -E "${opts[@]}" "${fortran_srcs[$idx]}" ${output_definition:+$output_definition}
+    then status=$?
+         echo flang: in "$PWD", @FLANG_DEFAULT_DRIVER@ failed with exit status $status: "$wd/bin/@FLANG_DEFAULT_DRIVER@" "${opts[@]}" "$@" >&2
+         exit $status
+    fi
+  done
+
+  # Preprocess other sources using Flang
+  for idx in "${!other_srcs[@]}"; do
+    if ! $ext_fc -E "${opts[@]}" "${other_srcs[$idx]}" ${output_definition:+$output_definition}
+    then status=$?
+         echo flang: in "$PWD", flang-new failed with exit status $status: "$wd/bin/flang-new" "${flang_options[@]}" "$@" >&2
+         exit $status
+    fi
+  done
+}
+
+# === main ====================================================================
+# Main entry point for this script
+# =============================================================================
+main() {
+  parse_args "$@"
+
+  fortran_source_files=()
+  other_source_files=()
+  object_files=()
+  lib_files=()
+  categorise_files INPUT_FILES fortran_source_files other_source_files object_files lib_files
+
+  if [[ $PREPROCESS_ONLY == "True" ]]; then
+    preprocess fortran_source_files other_source_files OPTIONS
+    exit 0
+  fi
+
+  # Options for the Flang driver.
+  # NOTE: We need `-fc1` to make sure that the frontend driver rather than
+  # compiler driver is used. We also need to make sure that that's the first
+  # flag that the driver will see (otherwise it assumes compiler/toolchain
+  # driver mode).`f18` will just ignore this flag when uparsing, so it's fine
+  # to add it here unconditionally.
+  flang_options=("-fc1")
+  # Options for the external Fortran Compiler
+  ext_fc_options=()
+  categorise_opts OPTIONS flang_options ext_fc_options
+
+  local -r wd=$(cd "$(dirname "$0")/.." && pwd)
+
+  # STEP 1: Unparse
+  local -r unparsed_file="flang_unparsed_source_file"
+  flang_options+=("-module-suffix")
+  flang_options+=(".f18.mod")
+  flang_options+=("-fdebug-unparse")
+  flang_options+=("-fno-analyzed-objects-for-unparse")
+
+  [[ ! -z ${MODULE_DIR} ]] && flang_options+=("-module-dir ${MODULE_DIR}")
+  [[ ! -z ${INTRINSICS_MOD_DIR} ]] && flang_options+=("-intrinsics-module-directory ${INTRINSICS_MOD_DIR}")
+  for idx in "${!fortran_source_files[@]}"; do
+    if ! "$wd/bin/@FLANG_DEFAULT_DRIVER@" "${flang_options[@]}" "${fortran_source_files[$idx]}" -o "${unparsed_file}_${idx}.f90"
+    then status=$?
+         echo flang: in "$PWD", @FLANG_DEFAULT_DRIVER@ failed with exit status $status: "$wd/bin/@FLANG_DEFAULT_DRIVER@" "${flang_options[@]}" "$@" >&2
+         exit $status
+    fi
+  done
+
+  # STEP 2: Compile Fortran Source Files
+  readonly ext_fc="${F18_FC:-gfortran}"
+  for idx in "${!fortran_source_files[@]}"; do
+    # Use the value of $OUTPUT_FILE for the output file iff `-c` was used.
+    if [[ ! -z ${OUTPUT_FILE:+x} ]] && [[ $COMPILE_ONLY == "True" ]]; then
+      output_definition="-o $OUTPUT_FILE"
+    elif [[ $COMPILE_ONLY == "False" ]]; then
+      output_definition="-o ${TEMP_OUTPUT}_${idx}"
+    fi
+
+    if ! $ext_fc -c "${ext_fc_options[@]}" "${unparsed_file}_${idx}.f90" ${output_definition:+$output_definition}
+    then status=$?
+      echo flang: in "$PWD", "$ext_fc" failed with exit status $status: "$ext_fc" "${ext_fc_options[@]}" "$@" >&2
+         exit $status
+    fi
+    object_files+=(${TEMP_OUTPUT}_${idx})
+  done
+
+  # Delete the unparsed files
+  for idx in "${!fortran_source_files[@]}"; do
+    rm "${unparsed_file}_${idx}.f90"
+  done
+
+  # STEP 3: Compile Other Source Files
+  for idx in "${!other_source_files[@]}"; do
+    # Use the value of $OUTPUT_FILE for the output file iff `-c` was used.
+    if [[ ! -z ${OUTPUT_FILE:+x} ]] && [[ $COMPILE_ONLY == "True" ]]; then
+      output_definition="-o $OUTPUT_FILE"
+    elif [[ $COMPILE_ONLY == "False" ]]; then
+      output_definition="-o ${TEMP_OUTPUT}_${idx}"
+    fi
+
+    if ! $ext_fc -c "${ext_fc_options[@]}" "${other_source_files[${idx}]}" ${output_definition:+$output_definition}
+    then status=$?
+      echo flang: in "$PWD", "$ext_fc" failed with exit status $status: "$ext_fc" "${ext_fc_options[@]}" "$@" >&2
+         exit $status
+    fi
+    object_files+=(${TEMP_OUTPUT}_${idx})
+  done
+
+  # STEP 4: Link
+  if [[ $COMPILE_ONLY == "True" ]]; then
+    exit 0;
+  fi
+
+  if [[ ${#object_files[@]} -ge 1 ]]; then
+    # If $OUTPUT_FILE was specified, use it for the output name.
+    if [[ ! -z ${OUTPUT_FILE:+x} ]]; then
+      output_definition="-o $OUTPUT_FILE"
+    else
+      output_definition=""
+    fi
+
+    if ! $ext_fc "${ext_fc_options[@]}" "${object_files[@]}" "${lib_files[@]}" ${output_definition:+$output_definition}
+    then status=$?
+         echo flang: in "$PWD", "$ext_fc" failed with exit status $status: "$ext_fc" "${ext_fc_options[@]}" "$@" >&2
+         exit $status
+    fi
+  fi
+
+  # Delete intermediate object files
+  for idx in "${!fortran_source_files[@]}"; do
+    rm "${TEMP_OUTPUT}_${idx}"
+  done
+}
+
+main "${@}"