[flang][docs] Add documentation for the new Flang driver

[richard.barton.arm]

Great to see some docs added Andrzej - this should help us maintain the new Driver in future.

I have swept through with some comments - mostly typographical nits. More diagrams would help to explain the structure of Actions, and perhaps the relationship between the compiler driver and other jobs/binaries it could call.

flang/docs/FlangDriver.md

23

Is that the right word choice? I think it's for Fortran compilation jobs isn't it?

I wonder if "toolchain driver" == flang and "compiler driver" -> flang -fc1 might work better?

45

I think a diagram would really help explain this part.

61

110

116

123

sp? Should it be FlangOption ?

129

... and be marked as FlangOnlyOption

132

133

I guess you do this by adding FlangFc1Option do you?

157

Not sure what this para is trying to say.

Are you suggesting that all tests for flang (i.e. the Fortran compiler part of it) ought to use -fc1 command lines and only tests for compiler driver should invoke the compiler driver? So tests in flang/test/Driver should use %flang -### and all the others use %flang_fc1? If so, do you think that will work?

Or are you trying to teach a newcomer how to add a tests for the compiler driver and for frontend driver options handling specifically? If so, then I think it needs a bit more prescriptive detail.

159

[awarzynski]

Thank you for taking a look @richard.barton.arm

In D104229#2817089, @richard.barton.arm wrote:

More diagrams would help to explain the structure of Actions, and perhaps the relationship between the compiler driver and other jobs/binaries it could call.

I agree, but I'm finding it difficult to come up with something useful. IMO, the diagrams that I created in the past wouldn't be that helpful here. Also, should we be documenting clangDriver here? I've added a short paragraph on flang, but that's meant to be just a quick intro.

Another reason for not getting into too much detail on the other compilation jobs (e.g. backend, assembler, linker) is that Flang does not support them yet. Also, there's a great diagram and more documentation in Clang: https://clang.llvm.org/docs/DriverInternals.html#design-overview.

flang/docs/FlangDriver.md

23

I find this a bit tricky - I've never seen Clang being referred to as a "toolchain driver". LLDB is tool that belongs to the LLVM toolchain, but it's not driven by Clang. On the other hand, Clang does drive the compiler (i.e. Clang & LLVM) and the linker, so it's more than just a compiler driver.

45

I find the output generated with -ccc-print-phases more helpful than any of the diagrams that I've generated so far :) Let me try that instead.

133

No, these flags are not used in this process. They are not used much beyond defining the output of {flang|clang} -help. Also, the semantics for option flags are not well defined, so we shouldn't rely on them too much.

157

I just want to highlight that there are differences between %flang and %flang_fc1. In some cases the observable behavior will be identical, so IMO it's worthwhile to have this written down. I will rephrase.

[awarzynski]

Updates suggested by @richard.barton.arm

[arnamoy10]

flang/docs/FlangDriver.md

27

[Nit] easy to use --> easy-to-use

57

[Nit] crates -> creates

154

[Nit] Understood by for the frontend -> Understood by the frontend

157

May want to elaborate what you mean by "Plain command line option", or reword

180

[Nit] In some cases --> In some cases,

185

Not sure if we want to document the use of !REQUIRES here

[awarzynski]

Address PR comments from @arnamoy10

Also:

• expanded the section on action options and added some code examples
• moved a couple of paragraphs to improve the flow.

[richard.barton.arm]

flang/docs/FlangDriver.md

45

LGTM

70

I thought this was wrong when I first saw it, but I can reproduce. What's flang-new doing here then? I thought there was no support for these phases in flang yet?

153

[naromero77]

flang/docs/FlangDriver.md

140

Grammar... `There are ..."

[awarzynski]

flang/docs/FlangDriver.md

70

Sorry, I should've clarified.

-ccc-print-phases prints what phases clangDriver _would_ generate. That's not something flang-new is in control of. However, when you try to run it without -ccc-print-phases, it fails during the 2nd phase above:

error: code-generation is not available yet

Making -ccc-print-phases a bit more clever would be nice, but it would have to happen in Clang. In the meantime, I will add a clarification here.

140

Thank you, fixed!

157

I will reword this section a bit, thanks!

185

That's a good suggestion, though I will avoid ! REQUIRES: new-flang-driver, as we want to get rid of it.

[awarzynski]

Fix grammar, add a note on code-generation (not supported)

[naromero77]

LGTM, but please check with Rich to make sure all his comments are addressed.

Also, the pre-merge checks are failing on windows but it seems unrelated to this patch.

[richard.barton.arm]

Happy with the changes made since my comments.
Thanks Andrzej for the great documentation.

flang/docs/FlangDriver.md

23

ok - let's leave it there.

70

That's a nicer error message than I expected too.

flang/docs/FlangDriver.md

+<!--===- docs/FlangDriver.md
+
+   Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+   See https://llvm.org/LICENSE.txt for license information.
+   SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+-->
+
+# Flang drivers
+
+```eval_rst
+.. contents::
+   :local:
+```
+
+There are two main drivers in Flang:
+* the compiler driver, `flang-new`
+* the frontend driver, `flang-new -fc1`
+
+The compiler driver will allow you to control all compilation phases (i.e.
+preprocessing, frontend code-generation, middlend/backend code-optimisation and
+lowering, linking). For frontend specific tasks, the compiler driver creates a
+Fortran compilation job and delegates it to `flang-new -fc1`, the frontend driver.
+
+The frontend driver glues all of the frontend libraries together and provides
+an easy-to-use and intuitive interface to the frontend. It accepts many
+frontend-specific options not available in `flang-new` and as such it provides a
+finer control over the frontend. Similarly to `-Xclang` in `clang`, you can use
+`-Xflang` to forward the frontend specific flags from the compiler directly to
+the frontend driver.
+
+## Compiler Driver
+
+The main entry point for Flang's compiler driver is implemented in
+`flang/tools/flang-driver/driver.cpp`.  Flang's compiler driver is implemented
+in terms of Clang's driver library, `clangDriver`. This approach allows us to:
+* benefit from Clang's support for various targets, platforms and operating systems
+* leverage Clang's ability to drive various backends available in LLVM, as well
+  as linkers and assemblers.
+One implication of this dependency on Clang is that all of Flang's compiler
+options are defined alongside Clang's options in
+`clang/include/clang/Driver/Options.td`. For options that are common for both
+Flang and Clang, the corresponding definitions are shared.
+
+Internally, a `clangDriver` based compiler driver works by creating actions
+that correspond to various compilation phases, e.g. `PreprocessJobClass`,
+`CompileJobClass`, `BackendJobClass` or `LinkJobClass` from the
+`clang::driver::Action::ActionClass` enum. There are also other, more
+specialised actions, e.g. `MigrateJobClass` or `InputClass`, that do not map
+directly to common compilation steps. The actions to run are determined from
+the supplied compiler flags, e.g.
+
+* `-E` for `PreprocessJobClass`,
+* `-c` for `CompileJobClass`.
+
+In most cases, the driver creates a chain of actions/jobs/phases where the
+output from one action is the input for the subsequent one. You can use the
+`-ccc-print-phases` flag to see the sequence of actions that the driver will
+create for your compiler invocation:
+```bash
+flang-new -ccc-print-phases -E file.f
++- 0: input, "file.f", f95-cpp-input
+1: preprocessor, {0}, f95
+```
+As you can see, for `-E` the driver creates only two jobs and stops immediately
+after preprocessing. The first job simply prepares the input. For `-c`, the
+pipeline of the created jobs is more complex:
+```bash
+flang-new -ccc-print-phases -c file.f
+         +- 0: input, "file.f", f95-cpp-input
+      +- 1: preprocessor, {0}, f95
+   +- 2: compiler, {1}, ir
++- 3: backend, {2}, assembler
+4: assembler, {3}, object
+```
+Note that currently Flang does not support code-generation and `flang-new` will
+fail during the second step above with the following error:
+```bash
+error: code-generation is not available yet
+```
+The other phases are printed nonetheless when using `-ccc-print-phases`, as
+that reflects what `clangDriver`, the library, will try to create and run.
+
+For actions specific to the frontend (e.g. preprocessing or code generation), a
+command to call the frontend driver is generated (more specifically, an
+instance of `clang::driver::Command`). Every command is bound to an instance of
+`clang::driver::Tool`. For Flang we introduced a specialisation of this class:
+`clang::driver::Flang`. This class implements the logic to either translate or
+forward compiler options to the frontend driver, `flang-new -fc1`.
+
+You can read more on the design of `clangDriver` in Clang's [Driver Design &
+Internals](https://clang.llvm.org/docs/DriverInternals.html).
+
+## Frontend Driver
+Flang's frontend driver is the main interface between end-users and the Flang
+frontend. The high-level design is similar to Clang's frontend driver, `clang
+-cc1` and consists of the following classes:
+* `CompilerInstance`, which is a helper class that encapsulates and manages
+  various objects that are always required by the frontend (e.g. `AllSources`,
+  `AllCookedSources, `Parsing`, `CompilerInvocation`, etc.). In most cases
+  `CompilerInstance` owns these objects, but it also can share them with its
+  clients when required. It also implements utility methods to construct and
+  manipulate them.
+* `CompilerInvocation` encapsulates the configuration of the current
+  invocation of the compiler as derived from the command-line options and the
+  input files (in particular, file extensions). Among other things, it holds an
+  instance of `FrontendOptions`. Like `CompilerInstance`, it owns the objects
+  that it manages. It can share them with its clients that want to access them
+  even after the corresponding `CompilerInvocation` has been destructed.
+* `FrontendOptions` holds options that control the behaviour of the frontend,
+  as well as e.g. the list of the input files. These options come either
+  directly from the users (through command-line flags) or are derived from
+  e.g. the host system configuration.
+* `FrontendAction` and `FrontendActions` (the former being the base class for
+  the latter) implement the actual actions to perform by the frontend. Usually
+  there is one specialisation of `FrontendActions` for every compiler action flag
+  (e.g. `-E`, `-fdebug-unparse`). These classes also contain various hooks that
+  allow you to e.g. fine-tune the configuration of the frontend based on the
+  input.
+
+This list is not exhaustive and only covers the main classes that implement the
+driver. The main entry point for the frontend driver, `fc1_main`, is
+implemented in `flang/tools/flang-driver/driver.cpp`. It can be accessed by
+invoking the compiler driver, `flang-new`, with the `-fc1` flag.
+
+The frontend driver will only run one action at a time. If you specify multiple
+action flags, only the last one will be taken into account. The default action
+is `ParseSyntaxOnlyAction`, which corresponds to `-fsyntax-only`. In other
+words, `flang-new -fc1 <input-file>` is equivalent to `flang-new -fc1 -fsyntax-only
+<input-file>`.
+
+## Adding new Compiler Options
+Adding a new compiler option in Flang consists of two steps:
+* define the new option in a dedicated TableGen file,
+* parse and implement the option in the relevant drivers that support it.
+
+### Option Definition
+All of Flang's compiler and frontend driver options are defined in
+`clang/include/clang/Driver/Options.td` in Clang. When adding a new option to
+Flang, you will either:
+  * extend the existing definition for an option that is already available
+    in one of Clang's drivers (e.g.  `clang`), but not yet available in Flang, or
+  * add a completely new definition if the option that you are adding has not
+    been defined yet.
+
+There are many predefined TableGen classes and records that you can use to fine
+tune your new option. The list of available configurations can be overwhelming
+at times. Sometimes the easiest approach is to find an existing option that has
+similar semantics to your new option and start by copying that.
+
+For every new option, you will also have to define the visibility of the new
+option. This is controlled through the `Flags` field. You can use the following
+Flang specific option flags to control this:
+  * `FlangOption` - this option will be available in the `flang-new` compiler driver,
+  * `FC1Option` - this option will be available in the `flang-new -fc1` frontend driver,
+  * `FlangOnlyOption` - this option will not be visible in Clang drivers.
+
+Please make sure that options that you add are only visible in drivers that can
+support it. For example, options that only make sense for Fortran input files
+(e.g. `-ffree-form`) should not be visible in Clang and be marked as
+`FlangOnlyOption`.
+
+When deciding what `OptionGroup` to use when defining a new option in the
+`Options.td` file, many new options fall into one of the following two
+categories:
+  * `Action_Group` - options that define an action to run (e.g.
+    `-fsyntax-only`, `-E`)
+  * `f_Group` - target independent compiler flags (e.g. `-ffixed-form`,
+    `-fopenmp`)
+There are also other groups and occasionally you will use them instead of the
+groups listed above.
+
+### Option Implementation
+First, every option needs to be parsed. Flang compiler options are parsed in
+two different places, depending on which driver they belong to:
+
+* frontend driver: `flang/lib/Frontend/CompilerInvocation.cpp`,
+* compiler driver: `clang/lib/Driver/ToolChains/Flang.cpp`.
+
+The parsing will depend on the semantics encoded in the TableGen definition.
+
+When adding a compiler driver option (i.e. an option that contains
+`FlangOption` among its `Flags`) that you also intend to be understood by the
+frontend, make sure that it is either forwarded to `flang-new -fc1` or translated
+into some other option that is accepted by the frontend driver. In the case of
+options that contain both `FlangOption` and `FC1Option` among its flags, we
+usually just forward from `flang-new` to `flang-new -fc1`. This is then tested in
+`flang/test/Driver/frontend-forward.F90`.
+
+What follows is usually very dependant on the meaning of the corresponding
+option. In general, regular compiler flags (e.g. `-ffree-form`) are mapped to
+some state within the driver. A lot of this state is stored within an instance
+of `FrontendOptions`, but there are other more specialised classes too. Action
+flags (e.g. `-fsyntax-only`) are usually more complex overall, but also more
+structured in terms of the implementation.
+
+### Action Options
+For options that correspond to an action (i.e. marked as `Action_Group`), you
+will have to define a dedicated instance of `FrontendActions` in
+`flang/include/flang/Frontend/FrontendOptions.h`. For example, for
+`-fsyntax-only` we defined:
+```cpp
+class ParseSyntaxOnlyAction : public PrescanAndSemaAction {
+  void ExecuteAction() override;
+};
+```
+Command line options are mapped to frontend actions through the
+`Fortran::frontend::ActionKind` enum.  For every new action option that you
+add, you will have to add a dedicated entry in that enum (e.g.
+`ParseSyntaxOnly` for `-fsyntax-only`) and a corresponding `case` in
+`ParseFrontendArgs` function in the `CompilerInvocation.cpp` file, e.g.:
+```cpp
+    case clang::driver::options::OPT_fsyntax_only:
+      opts.programAction_ = ParseSyntaxOnly;
+      break;
+```
+Note that this simply sets the program/frontend action within the frontend
+driver. You still have make sure that the corresponding frontend action class
+is instantiated when your new action option is used. The relevant `switch`
+statement is implemented in `Fortran::frontend::CreatedFrontendBaseAction` in
+the `ExecuteCompilerInvocation.cpp` file. Here's an example for
+`-fsyntax-only`:
+```cpp
+  case ParseSyntaxOnly:
+    return std::make_unique<ParseSyntaxOnlyAction>();
+```
+At this point you should be able to trigger that frontend action that you have
+just added using your new frontend option.
+
+# Testing
+In LIT, we define two variables that you can use to invoke Flang's drivers:
+* `%flang` is expanded as `flang-new` (i.e. the compiler driver)
+* `%flang_fc1` is expanded as `flang-new -fc1` (i.e. the frontend driver)
+
+For most regression tests for the frontend, you will want to use `%flang_fc1`.
+In some cases, the observable behaviour will be identical regardless of whether
+`%flang` or `%flang_fc1` is used. However, when you are using `%flang` instead
+of `%flang_fc1`, the compiler driver will add extra flags to the frontend
+driver invocation (i.e. `flang-new -fc1 -<extra-flags>`). In some cases that might
+be exactly what you want to test.  In fact, you can check these additional
+flags by using the `-###` compiler driver command line option.
+
+Lastly, you can use `! REQUIRES: <feature>` for tests that will only work when
+`<feature>` is available. For example, you can use`! REQUIRES: shell` to mark a
+test as only available on Unix-like systems (i.e. systems that contain a Unix
+shell). In practice this means that the corresponding test is skipped on
+Windows.