This is an archive of the discontinued LLVM Phabricator instance.

[Flang][driver] Update docs
ClosedPublic

Authored by awarzynski on Oct 11 2021, 1:04 PM.

Diff Detail

Event Timeline

awarzynski created this revision.Oct 11 2021, 1:04 PM
Herald added a project: Restricted Project. · View Herald Transcript
awarzynski requested review of this revision.Oct 11 2021, 1:04 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 11 2021, 1:04 PM
NOTE: We cannot merge this just yet. First, we need add support for code-generation in LLVM Flang. This is currently being developed on fir-dev.

I've started drafting this to clarify and to justify the design. I'm sharing it here so that people have access to it while we are still implementing the basic functionality.

tschuett added a subscriber: tschuett.EditedOct 11 2021, 1:14 PM

I always liked the idea that the driver (clang, flang,swift) orchestrates the build. It uses various tools like e.g. compilers, linkers, assemblers, ... to build libraries, executables, ... (artefacts)

For historical reasons, there is only one clang binary. For swift I have swift, swiftc, swift-frontend, and swift-driver. Actually swiftc is a symlink to swift. Maybe give the flang frontend and the flang driver different names and use symlinks.

richard.barton.arm added inline comments.
flang/docs/FlangDriver.md
47

external ^tools like assemblers.

tschuett added inline comments.Oct 12 2021, 1:24 AM
flang/docs/FlangDriver.md
54

It is easier to understand, if talk about the driver and the frontend. frontend-driver is misleading.

Leporacanthicus added inline comments.
flang/docs/FlangDriver.md
48

"Nor does it need"?

awarzynski added inline comments.Oct 12 2021, 5:14 AM
flang/docs/FlangDriver.md
54

"frontend" can mean different things, though. For somebody from e.g. llvm-project/llvm (i.e. people working on the backend), probably both "llvm-project/clang" and "llvm-project/flang" are frontends (i.e. the entire sub-projects, with all the libraries and tools). For an "llvm-project/{flang|clang}" developer, it's probably just the core libraries (e.g. clangSema, clangParse, FortranParser or FortranSemantics).

I think that both perspectives are correct and simply depend on the point of reference. But it means that the term is ambiguous and I try to be careful about the context in which it is used. Also, "frontend" or "frontend tool"? The latter is more accurate (indeed, it's just a tool), but then which "frontend tool" does it refer to? There's already flang-new, flang-new -fc1, tco, bbc.

I really appreciate the suggestion! This is tricky. To me, "frontend driver" makes a lot of sense (i.e. it drives the frontend libraries), but I'm biased. Let's wait and see what others think!

Address comments from @richard.barton.arm and @Leporacanthicus

Thank you all for taking a look!

Thanks for working on this, Andrzej.

In addition to the inline comments, there are some things that I don't understand about the driver.

When invoking the compiler driver, I'll want to specify options to the frontend. How do I do that?

The document implies that the frontend options are specifically designed to be of interest to compiler developers and are potentially unstable. Neither one of these seems true to me.

As I hinted in some of my inline comments, my overall understanding of the drivers would be enhanced if I better understood the overall compilation process that they control. I think it would be good to include a brief overview of the compilation processes controlled by the compiler driver and the frontend driver.

flang/docs/FlangDriver.md
16–18

It would help me to understand the difference between the frontend driver and the compiler driver if I knew what the inputs and outputs of the compiler and frontend were. I assume that the inputs of the compiler are source code and module files and that its outputs are executables, libraries, and module files. I assume that the inputs of the frontend are the same as those of the compiler. I'm not sure about its outputs. Where do the tools bbc and tco fit into this picture? Are there other tools in the LLVM compilation tool chain that the compiler driver might invoke to do optimization or analysis?

25

Rather than "e.g.", I would say "other tools such as"

30

Should read "viewd as the driver".

45

There is no scanner in the compiler, so I don't think we should mention scanning.

46

I think that this should read "have to be".

54

In the rest of this document, "frontend driver" is spelled without a hyphen in the middle.

58

This should read "A similar model".

awarzynski marked 6 inline comments as done.

Address some of the comments from @PeteSteinfeld

Thank you for the review, Pete.

When invoking the compiler driver, I'll want to specify options to the frontend. How do I do that?

You can do this with -Xflang:

flang-new -Xflang -fdebug-dump-parse-tree input.f95

In the above ^^^ invocation, -fdebug-dump-parse-tree is forwarded to flang-new -fc1. Here are a few small experiments that illustrate this:

# Print the input file fist
$ cat input.f95
end program

# 1. Compiler driver: `-fdebug-dump-parse-tree` *is not* supported
$ bin/flang-new -fdebug-dump-parse-tree file.f95
flang-new: warning: argument unused during compilation: '-fdebug-dump-parse-tree'

# 2. Frontend driver: `-fdebug-dump-parse-tree` *is* supported
$ bin/flang-new -fc1 -fdebug-dump-parse-tree file.f95
Program -> ProgramUnit -> MainProgram
| SpecificationPart
| | ImplicitPart ->
| ExecutionPart -> Block
| EndProgramStmt ->

# 3. Compiler driver: `-fdebug-dump-parse-tree` is forwarded to `flang-new -fc1` with `-Xflang`
$ bin/flang-new -Xflang -fdebug-dump-parse-tree file.f95
Program -> ProgramUnit -> MainProgram
| SpecificationPart
| | ImplicitPart ->
| ExecutionPart -> Block
| EndProgramStmt ->

Note that Xflang is similar to the following option flags in Clang:

Other, somewhat similar and very popular option: mllvm. GCC has similar options too, see e.g. Xassembler.

The document implies that the frontend options are specifically designed to be of interest to compiler developers and are potentially unstable. Neither one of these seems true to me.

The frontend driver has options like -fdebug-dump-symbols, -fdebug-dump-parse-tree or -fdebug-dump-provenance. These expose the internals of the compiler. Would end-users be interested in these? The very interested ones can use flang-new -fc1. By limiting certain options to flang-new -fc1, we give ourselves a framework for e.g.:

  • experimental flags (i.e. features that we are not yet sure about)
  • flags to control code-paths that might be unstable or under active development

For example, what should we do with flags that control pass pipelines for generating MLIR or LLVM IR? In the frontend driver we can introduce many experimental flags to fine-tune every pass separately. But should these flags be available in our main user facing interface (i.e. flang-new)? In most cases, we would like users to rely on -O{0|1|2|3|s|z} instead. We've also discussed this briefly with folks from MLIR (https://llvm.discourse.group/t/passpipelineclparser-and-flang-driver/4291/2):

I’d be very cautious about exposing this being a flag that make it very clear that these aren’t stable and meant for development / debugging purpose only: they can change from one Flang release to another.

As for your final point,

I think it would be good to include a brief overview of the compilation processes controlled by the compiler driver and the frontend driver.

We probably need a diagram for this. I will try to produce something.

flang/docs/FlangDriver.md
30

There are many tools that "drive" LLVM (e.g. opt, clang etc). So we probably want "viewed as a driver", right?

45

Would replacing scanning with prescanning & preprocessing make sense?

Thanks for the updates and explanations, Andrzej.

Actually, I do think that end users would find it useful to dump out the symbol table and the parse tree. This is because Fortran's language definition has aspects that make it possible to create programs that are confusing to look at and interpret -- spaces are not syntactically significant, there are no reserved words, and names can be implicitly declared. I personally have spent many hours puzzling over Fortran source code, and I routinely dump out the symbol table and parse tree to help me understand what the compiler thinks the program represents.

I'm not so sure about -fdebug-dump-provenance. I rarely use it.

I agree with you that having experimental flags might be useful. But I don't see why this utility is specific to the frontend. Might it not also be useful to have experimental flags for the compiler driver?

flang/docs/FlangDriver.md
45

I'm not sure what prescanning is. Replacing with preprocessing makes sense.

awarzynski added inline comments.Oct 13 2021, 7:59 AM
flang/docs/FlangDriver.md
45

I've extracted that from the Flang documentation:

Prescan and preprocess

https://github.com/llvm/llvm-project/blob/main/flang/docs/Overview.md#prescan-and-preprocess

PeteSteinfeld added inline comments.Oct 13 2021, 8:09 AM
flang/docs/FlangDriver.md
45

I stand corrected! Bizarrely, we prescan, but we don't scan. Go figure.

Actually, I do think that end users would find it useful to dump out the symbol table and the parse tree.

We can (and probably should) make them available in flang-new then. As for -fdebug-dump-provenance, it sounds like it should remain in flang-new -fc1.

I agree with you that having experimental flags might be useful. But I don't see why this utility is specific to the frontend. Might it not also be useful to have experimental flags for the compiler driver?

We can have experimental flags in both drivers. I'm not suggesting any strict rules here. But, for every option that we add, we need to make a decision whether it's:

  1. only available in flang-new
  2. only available in flang-new -fc1
  3. available in both flang-new and flang-new -fc1

Added a diagram for the frontend and the compiler driver, and also a bit more explanation to better highlight the difference between the two.

PeteSteinfeld accepted this revision.Oct 28 2021, 6:56 AM

Ship it!

This revision is now accepted and ready to land.Oct 28 2021, 6:56 AM
awarzynski updated this revision to Diff 384688.Nov 4 2021, 2:56 AM

Add a note that the document is based on a design that we are working towards
rather than the current state of affairs. In particular, at assumes that
code-generation is already available, which is not the case (it's under active
development on fir-dev).

This revision was automatically updated to reflect the committed changes.