[flang][driver]Add experimental flang driver and frontend with help screen

[awarzynski]

Summary

This is the first patch implementing the new Flang driver as outlined in [1],
[2] & [3]. It created Flang driver (flang-new and Flang frontend driver
(flang-new -fc1) with help screen(-help) and version(--version) options.
These will be renamed as flang and flang -fc1 once the current Flang
throwaway driver, flang, can be replaced

flang-new is implemented in terms of libclangDriver, defaulting the driver
mode to flang (added to libclangDriver in [2]). This ensures that the driver
runs in flang mode regardless of the name of the binary inferred from argv[0].

The design of the new flang compiler and frontend drivers is inspired by it
counterparts in Clang [3]. Currently, the new flang compiler and frontend
drivers re-use Clang libraries: clangBasic, clangDriver and clangFrontend.
To identify Flang options, this patch adds FlangOption enum. printHelp is
updated so that flang-new prints only Flang options.

The new Flang driver is disabled by default. To build it, set
-DBUILD_FLANG_NEW_DRIVER =ON when configuring Flang and LLVM_ENABLE_PROJECTS
with clang.

[1] RFC: new Flang driver - next steps
http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
[2] "RFC: Adding a fortran mode to the clang driver for flang"
http://lists.llvm.org/pipermail/cfe-dev/2019-June/062669.html
[3]RFC: refactoring libclangDriver/libclangFrontend to share with Flan
http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html

[tskeith]

flang/include/flang/Frontend/CompilerInstance.h

12

Why is this called "Frontend" rather than "Driver"?

17

Nothing else is in namespace flang. Fortran::frontend would be more consistent.

22

Data members, local variables, etc. should start with lower case.
Non-public data members should end with underscore.

Please follow the style in flang/documentation/C++style.md.

flang/include/flang/Frontend/FrontendOptions.h

32

Why isn't the type Format?

33

Why isn't the type bool?

66

Why aren't the types bool?

flang/include/flang/FrontendTool/Utils.h

19

Is this needed?

flang/lib/Frontend/CompilerInstance.cpp

40

The "then" and "else" statements should have braces around them.

flang/lib/Frontend/CompilerInvocation.cpp

47

Why not return InputKind{Language::Unknown};?

92

What is the purpose of DashX here?

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

21

What is this for?

flang/tools/flang-driver/driver.cpp

31

Why isn't this declared in a header?

flang/tools/flang-driver/fc1_main.cpp

33

MainAddr isn't used.

55

Why is this statement in a block? Is the result of CreateFromArgs supposed to affect the return value of this function?

[richard.barton.arm]

A few specific comments to address here as well as the pre-commit linting ones.

clang/lib/Driver/Driver.cpp

1584

Instead of this early exit, could we instead of calling getClangFullVersion below call getClangToolFullVersion with a different string here?

if (IsFlangMode())
  OS >> getClangToolFullVersion("flang-new") << '\n';
else
  OS >> getClangFullVersion();

That way, we get the full clang version screen already implemented 'for free' and the code is nicer too (IMO)

flang/CMakeLists.txt

20

Generally we should make sure all Flang-specific CMake build configuration variables start with FLANG. Suggest this is FLANG_BUILD_NEW_DRIVER or similar.

86

Remove

flang/test/lit.cfg.py

40

I think it would be cleaner to define config.excludes unconditionally then append the Flang-Driver dir if our condition passes.

68

Rather than always trying to add flang-new and ignoring failure, I think it would be better to conditionally add this to tools iff we are building the new driver and so have config.include_flang_new_driver_test = "ON". This way if you are building the new driver and for some reason lit fails to resolve the tool then you get a nice error before trying to run tests on a binary that is not there.

flang/test/lit.site.cfg.py.in

14

typo "controld"

flang/unittests/CMakeLists.txt

27

indentation?

llvm/lib/Option/OptTable.cpp

621 ↗

(On Diff #286047)

This is not the sort of change I expect to see in an llvm support library given that it seems very clang and flang specific.

I think this needs to be re-written to be more generic, perhaps tweaking the interface to be able to express the logic you want to use for flang and clang.

Why is it not sufficient to call this function unchanged from both flang and clang but with a different set of FlagsToInclude/FlagsToExclude passed in using this logic to set that on the clang/flang side?

[sameeranjoshi]

Thanks for the work.

A couple of comments on clang/ related changes:
An out-of-tree build with this patch fails for me:
Here's what I did:
I initially used ENABLE_PROJECTS="clang;mlir" to build llvm-project, I didn't build flang during this run.

Then I passed following to the cmake when building flang as out of tree.

cmake -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DFLANG_ENABLE_WERROR=On \
  -DCMAKE_CXX_STANDARD=17 \
  -DLLVM_TARGETS_TO_BUILD=host \
  -DLLVM_LIT_ARGS=-v \
  -D LLVM_DIR=${LLVM_MLIR_CLANG_BUILD}/lib/cmake/llvm \
  -D MLIR_DIR=${LLVM_MLIR_CLANG_BUILD}/lib/cmake/mlir \
  -DBUILD_FLANG_NEW_DRIVER=ON \
  ../
ninja -j16

where LLVM_MLIR_CLANG_BUILD points to initially built llvm-project .
I see below error message:

../lib/Frontend/CompilerInvocation.cpp: In static member function ‘static bool flang::CompilerInvocation::CreateFromArgs(flang::CompilerInvocation&, llvm::ArrayRef<const char*>, clang::DiagnosticsEngine&)’:
../lib/Frontend/CompilerInvocation.cpp:85:67: error: ‘FlangOption’ is not a member of ‘clang::driver::options’
       clang::driver::options::CC1Option | clang::driver::options::FlangOption;
                                                                   ^~~~~~~~~~~
../lib/Frontend/CompilerInvocation.cpp:85:67: note: suggested alternative: ‘LastOption’
       clang::driver::options::CC1Option | clang::driver::options::FlangOption;
                                                                   ^~~~~~~~~~~
                                                                   LastOption

Do you need to add -DCLANG_DIR flag, as there seem to be a dependency for this patch on clang as libraries?

clang/include/clang/Driver/Options.td

60

nit: a missing period.
flang mode -> flang mode.

2110

nit: - A space before FlangOption
CC1AsOption,FlangOption -> CC1AsOption, FlangOption

3059–3060

Same as above.
nit: - A space before FlangOption

clang/lib/Driver/Driver.cpp

1574

Can IsFlangMode() be used instead?

1578

In enum ClangFlags
inside File clang/include/clang/Driver/Options.h
there are various other options do they need to be considered?
If not, how are they handled?

[richard.barton.arm]

flang/test/lit.cfg.py

40

I _think_ the pattern to follow to exclude tests for something you haven't built is to use lit features.

So you would add a feature like:
config.available_features.add("new-driver")

then each test that only works on the new driver would be prefixed with a statement:

REQUIRES: new-driver

This means that the tests can go in the test/Driver directory and you don't need to create a new directory for these tests or this hack. The additional benefit would be that all the existing tests for the throwaway driver can be re-used on the new Driver to test it. There are not many of those though and we are using a different driver name so they can't be shared either. Still think it would be a worthwhile thing to do because when looking at the test itself it is clear why it is not being run whereas with this hack it is hidden away.

Sorry for not thinking this first time around.

[sameeranjoshi]

Thanks for working on it.
A few review comments/questions on changes in flang part from the patch.

flang/include/flang/Frontend/CompilerInstance.h

94

The block of comments above make sense for this function and not the currently mentioned one.
Please interchange/replace the comments to later declared function.
Wrong comments above could reflect in doxygen APIs, misleading the reader of code.

flang/include/flang/FrontendTool/Utils.h

2

nit:: blank line.

[sameeranjoshi]

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

36

With

clang --version
clang version 11.0.0 (https://github.com/llvm/llvm-project.git 1acf129bcf9a1b51e301a9fef151254ec4c7ec43)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

whereas with

./bin/flang-new --vesion
Flang experimental driver (flang-new)

I see both clang & flang call
llvm::cl::PrintVersionMessage() internally.

Is more information need to be registered in llvm(llvm::cl) for flang to give more detailed output or will that come later once we start adding more patch for driver?

40

The comment in header for ExecuteCompilerInvocation mentions,

	/// \return - True on success.
	bool ExecuteCompilerInvocation(CompilerInstance *Flang);

Do we need to have a false somewhere here?

I see 2 scenarios when ExecuteCompilerInvocation might fail (there could definitely be more) and there we need an indication of failure by returning false,

1. When there is no actual execution of compiler.
2. The compilation in not successful.

[CarolineConcatto]

Address review comments

Replace namespace flang for Fortran::frontend
and fix style

[CarolineConcatto]

Hey Folks,
Fist of all, thank you very much for your review.
I have answered some questions and update some of the requests.
There are some that I did not had time to go inside yet.
I have the week of 24th off. IN my return I will tackle all the reviews as:
out of build tree
and some question about the driver

flang/include/flang/Frontend/CompilerInstance.h

12

The way we think is that the driver is split in:
the driver libDriver -> that implements the driver. ATM it is inside clang/lib/Driver. And it can be clang( called inside clang/tools/driver/driver.cpp) or flang-new(called inside flang/tools/driver/driver.cpp) according to the mode the driver is called.
the frontend driver -> it can be:

clang frontend driver: clang -cc1  that calls libclangfrontend 
flang frontend driver: flang-new -fc1 that calls libflangfrontend.

the -xc1 has its functions/methods implemented inside the driver Frontend

So this folder is called Frontend because it belongs to the part that implements the driver frontend of the compiler and we would like to leave this way so it has the same design as clang.

22

I believe I have changed the style of the patch to be flang style.

flang/include/flang/Frontend/FrontendOptions.h

32

This is to not mistake with: enum Format { Source, ModuleMap, Precompiled };

33

Most of these things are going to be set by option::driver. That is the reason it is as it is.

66

Most of these things are going to be set by option::driver. That is the reason it is as it is.

flang/lib/Frontend/CompilerInvocation.cpp

47

Just because the next lines also return DashX.

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

21

Because of CompilerInstance *Flang.

flang/test/lit.cfg.py

40

I like to have the test at a different folder for now so it is clear that the tests inside this folder all belongs to the new driver. So I don't need to open the test to know.
I can implement the requires, but in the future will not need the REQUIRES for the driver test.

[SouraVX]

flang/tools/flang-driver/fc1_main.cpp

2

This file-name seems odd considering LLVM style. How about just Flang.cpp or FlangMain.cpp ? or other

[awarzynski]

@CarolineConcatto - thank you for working on this! @all, thank you for your reviews, this is much appreciated!

While Carol is away, I'll try my best to address all the outstanding comments. I will also update the patch accordingly. I've identified a few small issues myself - that will also be updated in the next patch.

clang/lib/Driver/Driver.cpp

1578

do they need to be considered?

Not at this stage. This is sufficient to make sure that flang-new -help only prints options relevant to Flang.

We may want to fine-tune this in the future, but currently it would be a bit tricky with only 2 options being supported ;-)

flang/include/flang/Frontend/CompilerInstance.h

12

Just to further clarify, I think that it's important to distinguish between:

• the compiler driver, flang, implemented in terms of libclandDriver (it creates and executes jobs/commands, e.g. linker or frontend jobs)
• the frontend driver, flang -fc1, implemented in terms of libflangFrontend (it creates frontend actions, e.g. preprocessor actions)

"Frontend" in this case means "Frontend Driver". "Driver" is reserved for the compiler driver, i.e. "libclangDriver".

As Carol mentioned, our terminology is inspired by Clang. Keeping the names consistent felt like a good idea. @tskeith, would you prefer "FrontendDriver" instead of "Frontend"?

(libclangDriver is the Clang compiler driver library that we want to re-use, libflangFrontend is the Flang frontend driver library, inspired by libclangFrontend)

flang/include/flang/Frontend/FrontendOptions.h

32–33

Why isn't the type bool?

@tskeith Bitfields are used here for optimization. An instance of InputKind is created per input file, so this could be a substantial saving (if the bitfields happen to be laid out nicely).

Btw, we won't need fmt_ or preprocessed_ (and the corresponding member methods) just now. I will delete them.

66

Why aren't the types bool?

@tskeith See above.

flang/lib/Frontend/CompilerInvocation.cpp

47

I agree with @tskeith , return InputKind{Language::Unknown}; would be cleaner. I will update this.

92

Not needed for now, I will delete it. Thanks!

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

21

It's not needed, good catch, cheers!

40

@sameeranjoshi These are good points, thank you! However, currently ExecuteCompilerInvocation doesn't do much. When this patch is approved and commited, we'll start adding actual _actions_ (to be executed by this method) that _could_ fail. At that point we will introduce code paths that could lead to return false.

flang/test/lit.cfg.py

40

Let me clarify a bit. I assume that FLANG_BUILD_NEW_DRIVER is Off.

SCENARIO 1: In order to make sure that bin/llvm-lit tools/flang/test/ does not _attempt to run_ the new driver tests, add:
config.excludes.append('Flang-Driver')

With this, the new driver tests will neither be run nor summarized (e.g. as UNSUPPORTED) in the final output.

SCENARIO 2: config.excludes.append('Flang-Driver') will not affect bin/llvm-lit tools/flang/test/Flang-Driver (this time I explicitly specify the Flang-Driver directory). We need:

REQUIERES: new-flang-driver

(or similar) for the new Flang driver tests to be reported as UNSUPPORTED.

I agree with @richard.barton.arm that REQUIRES would be the more canonical approach. We could still leave config.excludes.append('Flang-Driver') there to reduce the noise when running bin/llvm-lit tools/flang/test/. Unless people object, I recommend that we have both.

flang/test/lit.site.cfg.py.in

15

FLANG_BUILD_NEW_DRIVER should be canonicalized in CMake first: https://github.com/llvm/llvm-project/blob/c79a366ec0f87eafca123138b550b039618bf880/llvm/cmake/modules/AddLLVM.cmake#L1424-L1435

flang/tools/flang-driver/driver.cpp

31

We'd be creating a dedicated header file for no real benefit (this method should not be re-used anywhere else).

flang/tools/flang-driver/fc1_main.cpp

2

This is for consistency with clang: https://github.com/llvm/llvm-project/tree/master/clang/tools/driver. Perhaps Fc1Main.cpp instead?

55

Copy and paste error - cheers for pointing out, fixed in the next revision!

llvm/lib/Option/OptTable.cpp

621 ↗

(On Diff #286047)

I agree and have updated this. Thanks for pointing it out!

This part is particularly tricky for Flang. Flang has options with flags that fall both into FlagsToExclude and into FlagsToInclude. For example:

def help : Flag<["-", "--"], "help">, Flags<[CC1Option,CC1AsOption,FlangOption]>,
  HelpText<"Display available options">;

For Flang, before this method is called,

• CC1Option is added to FlagsToExclude, and
• FlangOption is added to FlagsToInclude

AFAIK, this is unique for Flang. Other tools never set both FlagsToExclude and FlagsToInclude. As a result, the currently vague relation between FlagsToExclude and FlagsToInclude needs to be clarified. In my opinion, FlagsToInclude should take precedence over FlagsToExclude. That's what I've implemented (that's clarified in the next revision).

[andwar]

• Canonicalised FLANG_BUILD_NEW_DRIVER in flang/test/CMakeLists.txt and updated the LIT config scripts accordingly
• Implemented proper handling of -emit-obj (via diagnostics) in CompilerInvocation.cpp
• Removed members from InputKind which are not needed (and the related methods)
• Removed/updated comments, added new comments to clarify the design and to highlight the future steps
• Simplified the original change in OptTable::PrintHelp (made it more generic)
• Removed unused #includes
• Fine-tuned the tests (I wanted to make sure that they clearly communicate what is currently supported and what is not)
• Addressed all the outstanding comments

[awarzynski]

@andwar is my Phabricator alter-ego. Apologies for the confusion - I had my Arcanist misconfigured.

[awarzynski]

Added support for out-of-tree builds. It requires -DCLANG_DIR=<llvm-buuld-dir>/lib/cmake/clang when configuring CMake. That's on top of LLVM_DIR and MLIR_DIR.

[awarzynski]

@sameeranjoshi Thanks for testing out-of-tree builds. We were missing some minor changes in CMake. I have now uploaded these. Out-of-tree will require -DCLANG_DIR when configuring build while the new driver depends on Clang.

As for your original issue - I suspect that you forgot to update your in-tree directory. This patch needs to be applied both in-tree (in llvm-project) and out-of-tree (flang).

[sameeranjoshi]

Thank you for changes.
I was able to build successfully out-of-tree.
Please update the README.md with the necessary changes.

LGTM!

[awarzynski]

Update README.md with instructions for building flang-new

[richard.barton.arm]

Requesting changes mostly because of the exit status issue on the Driver tests.

A few general questions as well:

1. Why not implement -### as part of this patch to enable testing more easily?
2. How come there are no regression tests for -fc1 in flang/test/Frontend? I suppose these come when the first real FrontendAction is implemented?

flang/test/CMakeLists.txt

12

So do the other bools like LINK_WITH_FIR also need this treatment and this is a latent bug? Seems amazing we've not hit that before now.

From an offline conversation ISTR you saying this was to do with how the variable is translated into the lit config? If that is the case then I think there is a function you can use called lit.util.pythonize_bool which converts various strings that mean "True/False" into a real bool for python. That would also let you clean up the lit.cfg.py later on (separate comments).

flang/test/Flang-Driver/driver-error-cc1.c

8

I am surprised that you don't need to prefix this run line with 'not' indicating that flang-new returns 0 exit code in this instance, which seems wrong to me.

flang/test/Flang-Driver/driver-error-cc1.cpp

2

Same comment as above on exit code.

flang/test/Flang-Driver/emit-obj.f90

3

Seems like it would be helpful to have also implemented -### in this patch so that you don't need to write tests like this. You could simply run flang-new -### then check the -fc1 line that would have been emitted for the presence of -emit-obj.

Same comment above regarding exit code.

flang/test/lit.cfg.py

39

This comment should be on line 41

40

Happy with this approach :-)

flang/test/lit.site.cfg.py.in

15

I think it might be better to use lit.util.pythonize_bool to do the canonicalisation. It would let you use config.include_flang_new_driver_test as a real bool later in the file rather than comparing to 1 or 0.

llvm/lib/Option/OptTable.cpp

621 ↗

(On Diff #286047)

This change is much better as it is more general and I think it could be acceptable.

I still don't get why you need to have anything set for FlagsToExclude in Flang's case? We only want to display flags that are specifically set as Flang options I think. So we set:

FlagsToInclude=FlangOption
FlagsToExclude=0

And all the options marked as both CC1Option, FlangOption are included because FlangOption is in the include list and all options without FlangOption are skipped.

This will only display all the flags with FlangOption set.

[richard.barton.arm]

Another random thought that just came to me: what does the new driver do when you invoke it with no input files or options? I could imagine a few sensible outcomes (error: no input (clang and gcc/gfortran behaviour), print version and exit, print help and exit, etc.) and I don't have a strong preference over which we chose here but I think there should be a test for it in test/Driver

[awarzynski]

• Adddressed comments from @richard.barton.arm
• Added FC1Option to ClangFlags (and made other related changes)
• Added missing code to check return codes from subcommands (flang-new)

[awarzynski]

In D86089#2256139, @richard.barton.arm wrote:

Another random thought that just came to me: what does the new driver do when you invoke it with no input files or options? I could imagine a few sensible outcomes (error: no input (clang and gcc/gfortran behaviour), print version and exit, print help and exit, etc.) and I don't have a strong preference over which we chose here but I think there should be a test for it in test/Driver

flang-new inherits generic things like this from libclangDriver. In this case:

error: no input files

This is consistent with clang. I've added a test for this. As for flang-new -fc1 - it in general does very little right now :) clang -cc1 just sits there waiting

flang/test/CMakeLists.txt

12

AFAIK, LINK_WITH_FIR is not used in LIT tests.

I switched to lit.util.pythonize_bool , thanks for the suggestion!

flang/test/Flang-Driver/driver-error-cc1.c

8

Good catch, thanks! Fixed in the latest revision.

flang/test/Flang-Driver/driver-error-cc1.cpp

2

Good catch, thanks! Fixed in the latest revision.

flang/test/Flang-Driver/emit-obj.f90

3

Seems like it would be helpful to have also implemented -### in this patch

As`flang-new` is based on libclangDriver, we get -### for free (i.e. it's already supported).

However, there's a catch: https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Driver/Options.td#L369-L370. Currently flang-new --help will not display -### because the the corresponding option definition hasn't been updated (i.e. it is not flagged as a Flang option):

def _HASH_HASH_HASH : Flag<["-"], "###">, Flags<[DriverOption, CoreOption]>,
    HelpText<"Print (but do not run) the commands to run for this compilation">;

We have three options:

1. Make flang-new --help display options that are flagged as DriverOption or CoreOption. Note this will include many other options (currently unsupported) and extra filtering would be required.
2. Add FlangOption to the definition of _HASH_HASH_HASH, but IMO that's a bit controversial. Is that what we want to do long-term? Or shall we create a separate category for generic driver options that apply to both Clang and Flang?
3. Delay the decision until there is more code to base our design on.

I'm in favor of Option 3.

flang/test/lit.cfg.py

39

Updated, ta!

flang/test/lit.site.cfg.py.in

15

I did not know that one - thanks! Updated.

llvm/lib/Option/OptTable.cpp

621 ↗

(On Diff #286047)

IIUC, you're assuming that FlagsToInclude != 0 implies that FlagsToExclude == 0. However, once we have an option that's only relevant to Flang we'll start setting both FlagsToInclude and FlagsToExclude (i.e. that assumption will no longer hold).

But this change is needed _just yet_ and hence the confusion. Thanks for pointing out, I will keep it for later!

[awarzynski]

Add missing update in lit.cfg.py

[richard.barton.arm]

This LGTM. I think all the previous comments from other reviewers and me have been dealt with so I am happy to accept this revision based on the reviews so far.

I have a few small inline comments to resolve in PrintHelp now we have reverted the functional changes there. Happy to approve on the assumption that they are dealt with and I don't need to see another patch, you can accept it yourself.

I think the non-flang changes to clang and llvm are in-line with what we said in our RFC. I think the summary of these changes are:

• Don't hard-code the name of the driver in the object constructor, pass it in as an argument + fix up all the clang callsites.
• Tweak the --help and --version logic to be conditional on driver mode
• Add some new FlangOption flags to Options.td
• Change the flang driver binary name to flang-new (in the already approved flang mods)

I am happy to approve these in good faith.

flang/test/Flang-Driver/emit-obj.f90

3

Fair enough. Happy to cross this bridge when we come to it later on. I certainly think that flang-new should support -### one day.

llvm/include/llvm/Option/OptTable.h

246

I don't think this change is correct now that you have reverted the code change to the function.

If I have the same bit set in FlagsToInclude and FlagsToExclude, the FlagsToExclude check fires second and would mean the option is not printed. So FlagsToExclude takes precedence.

Suggest to drop the edit or to correct it.

llvm/lib/Option/OptTable.cpp

613 ↗

(On Diff #290435)

With the diff to this logic gone, we should also remove the new newlines so as not to make textual changes unrelated to this patch.

621 ↗

(On Diff #286047)

Thanks for the explanation. I agree that it is best to wait for us to have a concrete motivating example before making this tweak to the PrintHelp logic. It will make it easier to test for one thing and will also make the patches more logically consistent.

[awarzynski]

Address final comments + rebase

I also took the liberty and made the following changes in CMake scripts:

• replaced add_library wiht add_flang_library
• replaced add_executable with add_flang_tool

These are all NFCs.

[richard.barton.arm]

LGTM

[awarzynski]

We forgot to update the signature and definition of OptTable::findByPrefix: https://github.com/llvm/llvm-project/blob/257b29715bb27b7d9f6c3c40c481b6a4af0b37e5/llvm/include/llvm/Option/OptTable.h#L154-L155. That's required after updating the definition of OptTable::Flags. Not sure none of the compilers complained.

Fixed here: https://github.com/llvm/llvm-project/commit/4eed800b18abaeba3082bf950fbe5c3020c4b592

clang/include/clang/Driver/Driver.h

 public:
 
   /// Takes the path to a binary that's either in bin/ or lib/ and returns
   /// the path to clang's resource directory.
   static std::string GetResourcesPath(StringRef BinaryPath,
                                       StringRef CustomResourceDir = "");
 
   Driver(StringRef ClangExecutable, StringRef TargetTriple,
-         DiagnosticsEngine &Diags,
+         DiagnosticsEngine &Diags, std::string Title = "clang LLVM compiler",
          IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS = nullptr);
 
   /// @name Accessors
   /// @{
 
   /// Name to use when invoking gcc/g++.
   const std::string &getCCCGenericGCCName() const { return CCCGenericGCCName; }
 

clang/include/clang/Driver/Options.h

enum ClangFlags {
   NoArgumentUnused = (1 << 6),
   Unsupported = (1 << 7),
   CoreOption = (1 << 8),
   CLOption = (1 << 9),
   CC1Option = (1 << 10),
   CC1AsOption = (1 << 11),
   NoDriverOption = (1 << 12),
   LinkOption = (1 << 13),
-  Ignored = (1 << 14),
+  FlangOption = (1 << 14),
+  FC1Option = (1 << 15),
+  Ignored = (1 << 16),
 };
 
 enum ID {
     OPT_INVALID = 0, // This is not an option ID.
 #define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
                HELPTEXT, METAVAR, VALUES)                                      \
   OPT_##ID,
 #include "clang/Driver/Options.inc"

clang/include/clang/Driver/Options.td

 
 // NoDriverOption - This option should not be accepted by the driver.
 def NoDriverOption : OptionFlag;
 
 // If an option affects linking, but has a primary group (so Link_Group cannot
 // be used), add this flag.
 def LinkOption : OptionFlag;
 
+// FlangOption - This is considered a "core" Flang option, available in
+// flang mode.
+def FlangOption : OptionFlag;
+
+// FC1Option - This option should be accepted by flang -fc1.
+def FC1Option : OptionFlag;
+
 // A short name to show in documentation. The name will be interpreted as rST.
 class DocName<string name> { string DocName = name; }
 
 // A brief description to show in documentation, interpreted as rST.
 class DocBrief<code descr> { code DocBrief = descr; }
 
 // Indicates that this group should be flattened into its parent when generating
 // documentation.

 def gz_EQ : Joined<["-"], "gz=">, Group<g_flags_Group>,
     HelpText<"DWARF debug sections compression type">;
 def gembed_source : Flag<["-"], "gembed-source">, Group<g_flags_Group>, Flags<[CC1Option]>,
     HelpText<"Embed source text in DWARF debug sections">;
 def gno_embed_source : Flag<["-"], "gno-embed-source">, Group<g_flags_Group>,
     Flags<[DriverOption]>,
     HelpText<"Restore the default behavior of not embedding source text in DWARF debug sections">;
 def headerpad__max__install__names : Joined<["-"], "headerpad_max_install_names">;
-def help : Flag<["-", "--"], "help">, Flags<[CC1Option,CC1AsOption]>,
+def help : Flag<["-", "--"], "help">, Flags<[CC1Option,CC1AsOption, FC1Option, FlangOption]>,
   HelpText<"Display available options">;
 def ibuiltininc : Flag<["-"], "ibuiltininc">,
   HelpText<"Enable builtin #include directories even when -nostdinc is used "
            "before or after -ibuiltininc. "
            "Using -nobuiltininc after the option disables it">;
 def index_header_map : Flag<["-"], "index-header-map">, Flags<[CC1Option]>,
   HelpText<"Make the next included directory (-I or -F) an indexer header map">;
 def idirafter : JoinedOrSeparate<["-"], "idirafter">, Group<clang_i_Group>, Flags<[CC1Option]>,

 def _profile_blocks : Flag<["--"], "profile-blocks">, Alias<a>;
 def _profile : Flag<["--"], "profile">, Alias<p>;
 def _resource_EQ : Joined<["--"], "resource=">, Alias<fcompile_resource_EQ>;
 def _resource : Separate<["--"], "resource">, Alias<fcompile_resource_EQ>;
 def _rtlib : Separate<["--"], "rtlib">, Alias<rtlib_EQ>;
 def _serialize_diags : Separate<["-", "--"], "serialize-diagnostics">, Flags<[DriverOption]>,
   HelpText<"Serialize compiler diagnostics to a file">;
 // We give --version different semantics from -version.
-def _version : Flag<["--"], "version">, Flags<[CoreOption, CC1Option]>,
+def _version : Flag<["--"], "version">,
+  Flags<[CoreOption, CC1Option, FC1Option, FlangOption]>,
   HelpText<"Print version information">;
 def _signed_char : Flag<["--"], "signed-char">, Alias<fsigned_char>;
 def _std : Separate<["--"], "std">, Alias<std_EQ>;
 def _stdlib : Separate<["--"], "stdlib">, Alias<stdlib_EQ>;
 def _sysroot_EQ : Joined<["--"], "sysroot=">;
 def _sysroot : Separate<["--"], "sysroot">, Alias<_sysroot_EQ>;
 def _target_help : Flag<["--"], "target-help">;
 def _trace_includes : Flag<["--"], "trace-includes">, Alias<H>;

clang/lib/Driver/Driver.cpp

if (CustomResourceDir != "") {
     llvm::sys::path::append(P, Twine("lib") + CLANG_LIBDIR_SUFFIX, "clang",
                             CLANG_VERSION_STRING);
   }
 
   return std::string(P.str());
 }
 
 Driver::Driver(StringRef ClangExecutable, StringRef TargetTriple,
-               DiagnosticsEngine &Diags,
+               DiagnosticsEngine &Diags, std::string Title,
                IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS)
     : Diags(Diags), VFS(std::move(VFS)), Mode(GCCMode),
       SaveTemps(SaveTempsNone), BitcodeEmbed(EmbedNone), LTOMode(LTOK_None),
       ClangExecutable(ClangExecutable), SysRoot(DEFAULT_SYSROOT),
-      DriverTitle("clang LLVM compiler"), CCPrintOptionsFilename(nullptr),
+      DriverTitle(Title), CCPrintOptionsFilename(nullptr),
       CCPrintHeadersFilename(nullptr), CCLogDiagnosticsFilename(nullptr),
       CCCPrintBindings(false), CCPrintOptions(false), CCPrintHeaders(false),
       CCLogDiagnostics(false), CCGenDiagnostics(false),
       TargetTriple(TargetTriple), CCCGenericGCCName(""), Saver(Alloc),
       CheckInputsExist(true), GenReproducer(false),
       SuppressMissingInputWarning(false) {
   // Provide a sane fallback if no VFS is specified.
   if (!this->VFS)
void Driver::PrintHelp(bool ShowHidden) const {
   unsigned ExcludedFlagsBitmask;
   std::tie(IncludedFlagsBitmask, ExcludedFlagsBitmask) =
       getIncludeExcludeOptionFlagMasks(IsCLMode());
 
   ExcludedFlagsBitmask |= options::NoDriverOption;
   if (!ShowHidden)
     ExcludedFlagsBitmask |= HelpHidden;
 
+  if (IsFlangMode())
+    IncludedFlagsBitmask |= options::FlangOption;
+
   std::string Usage = llvm::formatv("{0} [options] file...", Name).str();
   getOpts().PrintHelp(llvm::outs(), Usage.c_str(), DriverTitle.c_str(),
                       IncludedFlagsBitmask, ExcludedFlagsBitmask,
                       /*ShowAllAliases=*/false);
 }
 
 void Driver::PrintVersion(const Compilation &C, raw_ostream &OS) const {
+  if (IsFlangMode()) {
+    OS << getClangToolFullVersion("flang-new") << '\n';
+  } else {
     // FIXME: The following handlers should use a callback mechanism, we don't
     // know what the client would like to do.
     OS << getClangFullVersion() << '\n';
+  }
   const ToolChain &TC = C.getDefaultToolChain();
   OS << "Target: " << TC.getTripleString() << '\n';
 
   // Print the threading model.
   if (Arg *A = C.getArgs().getLastArg(options::OPT_mthread_model)) {
     // Don't print if the ToolChain would have barfed on it already
     if (TC.isThreadModelSupported(A->getValue()))
       OS << "Thread model: " << A->getValue();

 void Driver::HandleAutocompletions(StringRef PassedFlags) const {
   if (PassedFlags == "")
     return;
   // Print out all options that start with a given argument. This is used for
   // shell autocompletion.
   std::vector<std::string> SuggestedCompletions;
   std::vector<std::string> Flags;
 
-  unsigned short DisableFlags =
+  unsigned int DisableFlags =
       options::NoDriverOption | options::Unsupported | options::Ignored;
 
   // Distinguish "--autocomplete=-someflag" and "--autocomplete=-someflag,"
   // because the latter indicates that the user put space before pushing tab
   // which should end up in a file completion.
   const bool HasSpace = PassedFlags.endswith(",");
 
   // Parse PassedFlags by "," as all the command-line flags are passed to this

clang/lib/Driver/ToolChains/Flang.cpp

if (Output.isFilename()) {
     assert(Output.isNothing() && "Invalid output.");
   }
 
   const InputInfo &Input = Inputs[0];
   assert(Input.isFilename() && "Invalid input.");
   CmdArgs.push_back(Input.getFilename());
 
   const auto& D = C.getDriver();
-  const char* Exec = Args.MakeArgString(D.GetProgramPath("flang", TC));
+  // TODO: Replace flang-new with flang once the new driver replaces the
+  // throwaway driver
+  const char *Exec = Args.MakeArgString(D.GetProgramPath("flang-new", TC));
   C.addCommand(std::make_unique<Command>(
       JA, *this, ResponseFileSupport::AtFileUTF8(), Exec, CmdArgs, Inputs));
 }
 
-Flang::Flang(const ToolChain &TC) : Tool("flang", "flang frontend", TC) {}
+Flang::Flang(const ToolChain &TC) : Tool("flang-new", "flang frontend", TC) {}
 
 Flang::~Flang() {}

clang/lib/Frontend/CreateInvocationFromCommandLine.cpp

std::unique_ptr<CompilerInvocation> clang::createInvocationFromCommandLine(
   }
 
   SmallVector<const char *, 16> Args(ArgList.begin(), ArgList.end());
 
   // FIXME: Find a cleaner way to force the driver into restricted modes.
   Args.push_back("-fsyntax-only");
 
   // FIXME: We shouldn't have to pass in the path info.
-  driver::Driver TheDriver(Args[0], llvm::sys::getDefaultTargetTriple(),
-                           *Diags, VFS);
+  driver::Driver TheDriver(Args[0], llvm::sys::getDefaultTargetTriple(), *Diags,
+                           "clang LLVM compiler", VFS);
 
   // Don't check that inputs exist, they may have been remapped.
   TheDriver.setCheckInputsExist(false);
 
   std::unique_ptr<driver::Compilation> C(TheDriver.BuildCompilation(Args));
   if (!C)
     return nullptr;
 

clang/lib/Tooling/Tooling.cpp

 // it to be based on the same framework.
 
 /// Builds a clang driver initialized for running clang tools.
 static driver::Driver *
 newDriver(DiagnosticsEngine *Diagnostics, const char *BinaryName,
           IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS) {
   driver::Driver *CompilerDriver =
       new driver::Driver(BinaryName, llvm::sys::getDefaultTargetTriple(),
-                         *Diagnostics, std::move(VFS));
+                         *Diagnostics, "clang LLVM compiler", std::move(VFS));
   CompilerDriver->setTitle("clang_based_tool");
   return CompilerDriver;
 }
 
 /// Retrieves the clang CC1 specific flags out of the compilation's jobs.
 ///
 /// Returns nullptr on error.
 static const llvm::opt::ArgStringList *getCC1Arguments(

clang/test/Driver/flang/flang.f90

 ! Check that flang -fc1 is invoked when in --driver-mode=flang.
 
 ! This is a copy of flang_ucase.F90 because the driver has logic in it which
 ! differentiates between F90 and f90 files. Flang will not treat these files
 ! differently.
 
 ! Test various output types:
 ! * -E
 ! * -fsyntax-only
 ! * -emit-llvm -S
 ! * -emit-llvm
 ! * -S
 ! * (no type specified, resulting in an object file)
 
 ! All invocations should begin with flang -fc1, consume up to here.
-! ALL-LABEL: "{{[^"]*}}flang" "-fc1"
+! ALL-LABEL: "{{[^"]*}}flang-new" "-fc1"
 
 ! Check that f90 files are not treated as "previously preprocessed"
 ! ... in --driver-mode=flang.
 ! RUN: %clang --driver-mode=flang -### -E                  %s 2>&1 | FileCheck --check-prefixes=ALL,CHECK-E %s
 ! CHECK-E-NOT: previously preprocessed input
 ! CHECK-E-DAG: "-E"
 ! CHECK-E-DAG: "-o" "-"
 

clang/test/Driver/flang/flang_ucase.F90

 ! Check that flang -fc1 is invoked when in --driver-mode=flang.
 
 ! This is a copy of flang.f90 because the driver has logic in it which
 ! differentiates between F90 and f90 files. Flang will not treat these files
 ! differently.
 
 ! Test various output types:
 ! * -E
 ! * -fsyntax-only
 ! * -emit-llvm -S
 ! * -emit-llvm
 ! * -S
 ! * (no type specified, resulting in an object file)
 
 ! All invocations should begin with flang -fc1, consume up to here.
-! ALL-LABEL: "{{[^"]*}}flang" "-fc1"
+! ALL-LABEL: "{{[^"]*}}flang-new" "-fc1"
 
 ! Check that f90 files are not treated as "previously preprocessed"
 ! ... in --driver-mode=flang.
 ! RUN: %clang --driver-mode=flang -### -E                  %s 2>&1 | FileCheck --check-prefixes=ALL,CHECK-E %s
 ! CHECK-E-NOT: previously preprocessed input
 ! CHECK-E-DAG: "-E"
 ! CHECK-E-DAG: "-o" "-"
 

clang/test/Driver/flang/multiple-inputs-mixed.f90

 ! Check that flang can handle mixed C and fortran inputs.
 
 ! RUN: %clang --driver-mode=flang -### -fsyntax-only %S/Inputs/one.f90 %S/Inputs/other.c 2>&1 | FileCheck --check-prefixes=CHECK-SYNTAX-ONLY %s
-! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}flang{{[^"/]*}}" "-fc1"
+! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}flang-new{{[^"/]*}}" "-fc1"
 ! CHECK-SYNTAX-ONLY: "{{[^"]*}}/Inputs/one.f90"
 ! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}clang{{[^"/]*}}" "-cc1"
 ! CHECK-SYNTAX-ONLY: "{{[^"]*}}/Inputs/other.c"

clang/test/Driver/flang/multiple-inputs.f90

 ! Check that flang driver can handle multiple inputs at once.
 
 ! RUN: %clang --driver-mode=flang -### -fsyntax-only %S/Inputs/one.f90 %S/Inputs/two.f90 2>&1 | FileCheck --check-prefixes=CHECK-SYNTAX-ONLY %s
-! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}flang" "-fc1"
+! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}flang-new" "-fc1"
 ! CHECK-SYNTAX-ONLY: "{{[^"]*}}/Inputs/one.f90"
-! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}flang" "-fc1"
+! CHECK-SYNTAX-ONLY-LABEL: "{{[^"]*}}flang-new" "-fc1"
 ! CHECK-SYNTAX-ONLY: "{{[^"]*}}/Inputs/two.f90"

clang/unittests/Driver/SanitizerArgsTest.cpp

const Command &emulateSingleCompilation(std::vector<std::string> ExtraArgs,
                                           std::vector<std::string> ExtraFiles) {
     assert(!DriverInstance && "Running twice is not allowed");
 
     llvm::IntrusiveRefCntPtr<DiagnosticOptions> Opts = new DiagnosticOptions;
     DiagnosticsEngine Diags(
         new DiagnosticIDs, Opts,
         new TextDiagnosticPrinter(llvm::errs(), Opts.get()));
     DriverInstance.emplace(ClangBinary, "x86_64-unknown-linux-gnu", Diags,
-                           prepareFS(ExtraFiles));
+                           "clang LLVM compiler", prepareFS(ExtraFiles));
 
     std::vector<const char *> Args = {ClangBinary};
     for (const auto &A : ExtraArgs)
       Args.push_back(A.c_str());
     Args.push_back("-c");
     Args.push_back(InputFile);
 
     CompilationJob.reset(DriverInstance->BuildCompilation(Args));

clang/unittests/Driver/ToolChainTest.cpp

TEST(ToolChainTest, VFSGCCInstallation) {
   IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
 
   IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
   struct TestDiagnosticConsumer : public DiagnosticConsumer {};
   DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
   IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> InMemoryFileSystem(
       new llvm::vfs::InMemoryFileSystem);
   Driver TheDriver("/bin/clang", "arm-linux-gnueabihf", Diags,
-                   InMemoryFileSystem);
+                   "clang LLVM compiler", InMemoryFileSystem);
 
   const char *EmptyFiles[] = {
       "foo.cpp",
       "/bin/clang",
       "/usr/lib/gcc/arm-linux-gnueabi/4.6.1/crtbegin.o",
       "/usr/lib/gcc/arm-linux-gnueabi/4.6.1/crtend.o",
       "/usr/lib/gcc/arm-linux-gnueabihf/4.6.3/crtbegin.o",
       "/usr/lib/gcc/arm-linux-gnueabihf/4.6.3/crtend.o",
TEST(ToolChainTest, VFSGCCInstallationRelativeDir) {
   IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
 
   IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
   struct TestDiagnosticConsumer : public DiagnosticConsumer {};
   DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
   IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> InMemoryFileSystem(
       new llvm::vfs::InMemoryFileSystem);
   Driver TheDriver("/home/test/bin/clang", "arm-linux-gnueabi", Diags,
-                   InMemoryFileSystem);
+                   "clang LLVM compiler", InMemoryFileSystem);
 
   const char *EmptyFiles[] = {
       "foo.cpp", "/home/test/lib/gcc/arm-linux-gnueabi/4.6.1/crtbegin.o",
       "/home/test/include/arm-linux-gnueabi/.keep"};
 
   for (const char *Path : EmptyFiles)
     InMemoryFileSystem->addFile(Path, 0,
                                 llvm::MemoryBuffer::getMemBuffer("\n"));
TEST(ToolChainTest, DefaultDriverMode) {
 
   IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
   struct TestDiagnosticConsumer : public DiagnosticConsumer {};
   DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
   IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> InMemoryFileSystem(
       new llvm::vfs::InMemoryFileSystem);
 
   Driver CCDriver("/home/test/bin/clang", "arm-linux-gnueabi", Diags,
-                  InMemoryFileSystem);
+                  "clang LLVM compiler", InMemoryFileSystem);
   CCDriver.setCheckInputsExist(false);
   Driver CXXDriver("/home/test/bin/clang++", "arm-linux-gnueabi", Diags,
-                   InMemoryFileSystem);
+                   "clang LLVM compiler", InMemoryFileSystem);
   CXXDriver.setCheckInputsExist(false);
   Driver CLDriver("/home/test/bin/clang-cl", "arm-linux-gnueabi", Diags,
-                  InMemoryFileSystem);
+                  "clang LLVM compiler", InMemoryFileSystem);
   CLDriver.setCheckInputsExist(false);
 
   std::unique_ptr<Compilation> CC(CCDriver.BuildCompilation(
       { "/home/test/bin/clang", "foo.cpp"}));
   std::unique_ptr<Compilation> CXX(CXXDriver.BuildCompilation(
       { "/home/test/bin/clang++", "foo.cpp"}));
   std::unique_ptr<Compilation> CL(CLDriver.BuildCompilation(
       { "/home/test/bin/clang-cl", "foo.cpp"}));

flang/CMakeLists.txt

 endif()
 
 # option() honors normal variables.
 if (POLICY CMP0077)
   cmake_policy(SET CMP0077 NEW)
 endif()
 
 option(LINK_WITH_FIR "Link driver with FIR and LLVM" ON)
+option(FLANG_BUILD_NEW_DRIVER "Build the flang compiler driver" OFF)
 
 # Flang requires C++17.
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED TRUE)
 set(CMAKE_CXX_EXTENSIONS OFF)
 
 set(FLANG_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 
if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
 
   # We need a pre-built/installed version of LLVM.
   find_package(LLVM REQUIRED HINTS "${LLVM_CMAKE_PATH}")
   # If the user specifies a relative path to LLVM_DIR, the calls to include
   # LLVM modules fail. Append the absolute path to LLVM_DIR instead.
   get_filename_component(LLVM_DIR_ABSOLUTE ${LLVM_DIR} REALPATH)
   list(APPEND CMAKE_MODULE_PATH ${LLVM_DIR_ABSOLUTE})
 
+  if(FLANG_BUILD_NEW_DRIVER)
+    # TODO: Remove when libclangDriver is lifted out of Clang
+    list(APPEND CMAKE_MODULE_PATH ${CLANG_DIR})
+    find_package(Clang REQUIRED HINTS "${CLANG_DIR}")
+  endif()
+
   # If LLVM links to zlib we need the imported targets so we can too.
   if(LLVM_ENABLE_ZLIB)
     find_package(ZLIB REQUIRED)
   endif()
   option(LLVM_ENABLE_WARNINGS "Enable compiler warnings." ON)
   option(LLVM_ENABLE_PEDANTIC "Compile with pedantic enabled." ON)
   if(CMAKE_COMPILER_IS_GNUCXX)
     set(USE_NO_MAYBE_UNINITIALIZED 1)
   endif()
 
   include(CMakeParseArguments)
   include(AddLLVM)
   include(HandleLLVMOptions)
   include(VersionFromVCS)
 
   if(LINK_WITH_FIR)
     include(TableGen)
     find_package(MLIR REQUIRED CONFIG)
     # Use SYSTEM for the same reasons as for LLVM includes
     include_directories(SYSTEM ${MLIR_INCLUDE_DIRS})
     # If the user specifies a relative path to MLIR_DIR, the calls to include
     # MLIR modules fail. Append the absolute path to MLIR_DIR instead.
     get_filename_component(MLIR_DIR_ABSOLUTE ${MLIR_DIR} REALPATH)
     list(APPEND CMAKE_MODULE_PATH ${MLIR_DIR_ABSOLUTE})
if (LINK_WITH_FIR)
     set(MLIR_INCLUDE_DIR ${LLVM_MAIN_SRC_DIR}/../mlir/include ) # --includedir
     set(MLIR_TABLEGEN_OUTPUT_DIR ${CMAKE_BINARY_DIR}/tools/mlir/include)
     set(MLIR_TABLEGEN_EXE $<TARGET_FILE:mlir-tblgen>)
     include_directories(SYSTEM ${MLIR_INCLUDE_DIR})
     include_directories(SYSTEM ${MLIR_TABLEGEN_OUTPUT_DIR})
   endif()
 endif()
 
+if(FLANG_BUILD_NEW_DRIVER)
+    # TODO: Remove when libclangDriver is lifted out of Clang
+    if(FLANG_STANDALONE_BUILD)
+      set(CLANG_INCLUDE_DIR ${CLANG_INCLUDE_DIRS} )
+      # No need to specify TableGen output dir as that's embedded in CLANG_DIR
+    else()
+      set(CLANG_INCLUDE_DIR ${LLVM_MAIN_SRC_DIR}/../clang/include )
+      # Specify TableGen output dir for things like DiagnosticCommonKinds.inc,
+      # DiagnosticDriverKinds.inc (required for reporting diagnostics)
+      set(CLANG_TABLEGEN_OUTPUT_DIR ${CMAKE_BINARY_DIR}/tools/clang/include)
+      include_directories(SYSTEM ${CLANG_TABLEGEN_OUTPUT_DIR})
+    endif()
+    include_directories(SYSTEM ${CLANG_INCLUDE_DIR})
+endif()
+
 if(LINK_WITH_FIR)
   # tco tool and FIR lib output directories
   if(FLANG_STANDALONE_BUILD)
     set(LLVM_RUNTIME_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/bin)
     set(LLVM_LIBRARY_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/lib)
   endif()
   # Always build tco tool
   set(LLVM_BUILD_TOOLS ON)

flang/README.md

 Release builds execute quickly.
 
 ### Build Flang out of tree
 ```
 cd ~/flang/build
 cmake -DLLVM_DIR=$LLVM -DMLIR_DIR=$MLIR ~/flang/src
 make
 ```
+
+### Build The New Flang Driver
+The new Flang driver, `flang-new`, is currently under active development and
+should be considered as an experimental feature. For this reason it is disabled
+by default. This will change once the new driver replaces the _throwaway_
+driver, `flang`.
+
+In order to build the new driver, add `-DBUILD_FLANG_NEW_DRIVER=ON` to your
+CMake invocation line. Additionally, when building out-of-tree, use `CLANG_DIR`
+(similarly to `LLVM_DIR` and `MLIR_DIR`) to find the installed Clang
+components.
+
+**Note:** `CLANG_DIR` is only required when building the new Flang driver,
+which currently depends on Clang.
+
 # How to Run Tests
 
 Flang supports 2 different categories of tests
 1. Regression tests (https://www.llvm.org/docs/TestingGuide.html#regression-tests)
 2. Unit tests (https://www.llvm.org/docs/TestingGuide.html#unit-tests)
 
 ## For out of tree builds
 To run all tests:

flang/include/flang/Frontend/CompilerInstance.h

+//===-- CompilerInstance.h - Flang Compiler Instance ------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#ifndef LLVM_FLANG_FRONTEND_COMPILERINSTANCE_H
+#define LLVM_FLANG_FRONTEND_COMPILERINSTANCE_H
+
+#include "flang/Frontend/CompilerInvocation.h"
+
+#include <cassert>
+#include <memory>
+
+namespace Fortran::frontend {
+
+class CompilerInstance {
+
+  /// The options used in this compiler instance.
+  std::shared_ptr<CompilerInvocation> invocation_;
+
+  /// The diagnostics engine instance.
+  llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine> diagnostics_;
+
+public:
+  explicit CompilerInstance();
+
+  ~CompilerInstance();
+  CompilerInvocation &GetInvocation() {
+    assert(invocation_ && "Compiler instance has no invocation!");
+    return *invocation_;
+  };
+
+  /// }
+  /// @name Forwarding Methods
+  /// {
+
+  clang::DiagnosticOptions &GetDiagnosticOpts() {
+    return invocation_->GetDiagnosticOpts();
+  }
+  const clang::DiagnosticOptions &GetDiagnosticOpts() const {
+    return invocation_->GetDiagnosticOpts();
+  }
+
+  FrontendOptions &GetFrontendOpts() { return invocation_->GetFrontendOpts(); }
+  const FrontendOptions &GetFrontendOpts() const {
+    return invocation_->GetFrontendOpts();
+  }
+
+  /// }
+  /// @name Diagnostics Engine
+  /// {
+
+  bool HasDiagnostics() const { return diagnostics_ != nullptr; }
+
+  /// Get the current diagnostics engine.
+  clang::DiagnosticsEngine &GetDiagnostics() const {
+    assert(diagnostics_ && "Compiler instance has no diagnostics!");
+    return *diagnostics_;
+  }
+
+  /// SetDiagnostics - Replace the current diagnostics engine.
+  void SetDiagnostics(clang::DiagnosticsEngine *value);
+
+  clang::DiagnosticConsumer &GetDiagnosticClient() const {
+    assert(diagnostics_ && diagnostics_->getClient() &&
+        "Compiler instance has no diagnostic client!");
+    return *diagnostics_->getClient();
+  }
+
+  /// Get the current diagnostics engine.
+  clang::DiagnosticsEngine &getDiagnostics() const {
+    assert(diagnostics_ && "Compiler instance has no diagnostics!");
+    return *diagnostics_;
+  }
+
+  /// }
+  /// @name Construction Utility Methods
+  /// {
+
+  /// Create a DiagnosticsEngine object with a the TextDiagnosticPrinter.
+  ///
+  /// If no diagnostic client is provided, this creates a
+  /// DiagnosticConsumer that is owned by the returned diagnostic
+  /// object, if using directly the caller is responsible for
+  /// releasing the returned DiagnosticsEngine's client eventually.
+  ///
+  /// \param opts - The diagnostic options; note that the created text
+  /// diagnostic object contains a reference to these options.
+  ///
+  /// \param client If non-NULL, a diagnostic client that will be
+  /// attached to (and, then, owned by) the returned DiagnosticsEngine
+  /// object.
+  ///
+  /// \return The new object on success, or null on failure.
+  static clang::IntrusiveRefCntPtr<clang::DiagnosticsEngine> CreateDiagnostics(
+      clang::DiagnosticOptions *opts,
+      clang::DiagnosticConsumer *client = nullptr, bool shouldOwnClient = true);
+  void CreateDiagnostics(
+      clang::DiagnosticConsumer *client = nullptr, bool shouldOwnClient = true);
+};
+
+} // end namespace Fortran::frontend
+#endif // LLVM_FLANG_FRONTEND_COMPILERINSTANCE_H

flang/include/flang/Frontend/CompilerInvocation.h

+//===- CompilerInvocation.h - Compiler Invocation Helper Data ---*- C -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#ifndef LLVM_FLANG_FRONTEND_COMPILERINVOCATION_H
+#define LLVM_FLANG_FRONTEND_COMPILERINVOCATION_H
+
+#include "flang/Frontend/FrontendOptions.h"
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticOptions.h"
+
+namespace Fortran::frontend {
+class CompilerInvocationBase {
+public:
+  /// Options controlling the diagnostic engine.$
+  llvm::IntrusiveRefCntPtr<clang::DiagnosticOptions> diagnosticOpts_;
+
+  CompilerInvocationBase();
+  CompilerInvocationBase(const CompilerInvocationBase &x);
+  ~CompilerInvocationBase();
+
+  clang::DiagnosticOptions &GetDiagnosticOpts() {
+    return *diagnosticOpts_.get();
+  }
+  const clang::DiagnosticOptions &GetDiagnosticOpts() const {
+    return *diagnosticOpts_.get();
+  }
+};
+
+class CompilerInvocation : public CompilerInvocationBase {
+  /// Options controlling the frontend itself.
+  FrontendOptions frontendOpts_;
+
+public:
+  CompilerInvocation() = default;
+
+  FrontendOptions &GetFrontendOpts() { return frontendOpts_; }
+  const FrontendOptions &GetFrontendOpts() const { return frontendOpts_; }
+
+  /// Create a compiler invocation from a list of input options.
+  /// \returns true on success.
+  /// \returns false if an error was encountered while parsing the arguments
+  /// \param [out] res - The resulting invocation.
+  static bool CreateFromArgs(CompilerInvocation &res,
+      llvm::ArrayRef<const char *> commandLineArgs,
+      clang::DiagnosticsEngine &diags);
+};
+
+} // end namespace Fortran::frontend
+#endif // LLVM_FLANG_FRONTEND_COMPILERINVOCATION_H

flang/include/flang/Frontend/FrontendOptions.h

+//===- FrontendOptions.h ----------------------------------------*- C -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#ifndef LLVM_FLANG_FRONTEND_FRONTENDOPTIONS_H
+#define LLVM_FLANG_FRONTEND_FRONTENDOPTIONS_H
+
+#include <cstdint>
+#include <string>
+namespace Fortran::frontend {
+
+enum class Language : uint8_t {
+  Unknown,
+
+  /// LLVM IR: we accept this so that we can run the optimizer on it,
+  /// and compile it to assembly or object code.
+  LLVM_IR,
+
+  ///@{ Languages that the frontend can parse and compile.
+  Fortran,
+  ///@}
+};
+
+/// The kind of a file that we've been handed as an input.
+class InputKind {
+private:
+  Language lang_;
+
+public:
+  /// The input file format.
+  enum Format { Source, ModuleMap, Precompiled };
+
+  constexpr InputKind(Language l = Language::Unknown) : lang_(l) {}
+
+  Language GetLanguage() const { return static_cast<Language>(lang_); }
+
+  /// Is the input kind fully-unknown?
+  bool IsUnknown() const { return lang_ == Language::Unknown; }
+};
+
+/// FrontendOptions - Options for controlling the behavior of the frontend.
+class FrontendOptions {
+public:
+  /// Show the -help text.
+  unsigned showHelp_ : 1;
+
+  /// Show the -version text.
+  unsigned showVersion_ : 1;
+
+public:
+  FrontendOptions() : showHelp_(false), showVersion_(false) {}
+};
+} // namespace Fortran::frontend
+
+#endif // LLVM_FLANG_FRONTEND_FRONTENDOPTIONS_H

flang/include/flang/FrontendTool/Utils.h

+//===--- Utils.h - Misc utilities for the flang front-end --------*- C++-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+//  This header contains miscellaneous utilities for various front-end actions
+//  which were split from Frontend to minimise Frontend's dependencies.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_FLANG_FRONTENDTOOL_UTILS_H
+#define LLVM_FLANG_FRONTENDTOOL_UTILS_H
+
+namespace Fortran::frontend {
+
+class CompilerInstance;
+
+/// ExecuteCompilerInvocation - Execute the given actions described by the
+/// compiler invocation object in the given compiler instance.
+///
+/// \return - True on success.
+bool ExecuteCompilerInvocation(CompilerInstance *flang);
+
+} // end namespace Fortran::frontend
+
+#endif // LLVM_FLANG_FRONTENDTOOL_UTILS_H

flang/lib/CMakeLists.txt

 add_subdirectory(Common)
 add_subdirectory(Evaluate)
 add_subdirectory(Decimal)
 add_subdirectory(Lower)
 add_subdirectory(Parser)
 add_subdirectory(Semantics)
 
+if(FLANG_BUILD_NEW_DRIVER)
+  add_subdirectory(Frontend)
+  add_subdirectory(FrontendTool)
+endif()
+
 if(LINK_WITH_FIR)
   add_subdirectory(Optimizer)
 endif()

flang/lib/Frontend/CMakeLists.txt

+add_flang_library(flangFrontend
+  CompilerInstance.cpp
+  CompilerInvocation.cpp
+  FrontendOptions.cpp
+
+  LINK_LIBS
+  clangBasic
+  clangDriver
+  # TODO: Added to re-use clang's TextDiagnosticBuffer & TextDiagnosticPrinter.
+  # Add a custom implementation for Flang and remove this dependency.
+  clangFrontend
+
+  LINK_COMPONENTS
+  Option
+  Support
+)

flang/lib/Frontend/CompilerInstance.cpp

+//===--- CompilerInstance.cpp ---------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/CompilerInstance.h"
+#include "flang/Frontend/CompilerInvocation.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace Fortran::frontend;
+
+CompilerInstance::CompilerInstance() : invocation_(new CompilerInvocation()) {}
+
+CompilerInstance::~CompilerInstance() = default;
+
+void CompilerInstance::CreateDiagnostics(
+    clang::DiagnosticConsumer *client, bool shouldOwnClient) {
+  diagnostics_ =
+      CreateDiagnostics(&GetDiagnosticOpts(), client, shouldOwnClient);
+}
+
+clang::IntrusiveRefCntPtr<clang::DiagnosticsEngine>
+CompilerInstance::CreateDiagnostics(clang::DiagnosticOptions *opts,
+    clang::DiagnosticConsumer *client, bool shouldOwnClient) {
+  clang::IntrusiveRefCntPtr<clang::DiagnosticIDs> diagID(
+      new clang::DiagnosticIDs());
+  clang::IntrusiveRefCntPtr<clang::DiagnosticsEngine> diags(
+      new clang::DiagnosticsEngine(diagID, opts));
+
+  // Create the diagnostic client for reporting errors or for
+  // implementing -verify.
+  if (client) {
+    diags->setClient(client, shouldOwnClient);
+  } else {
+    diags->setClient(new clang::TextDiagnosticPrinter(llvm::errs(), opts));
+  }
+  return diags;
+}

flang/lib/Frontend/CompilerInvocation.cpp

+//===- CompilerInvocation.cpp ---------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/CompilerInvocation.h"
+#include "clang/Basic/AllDiagnostics.h"
+#include "clang/Basic/DiagnosticDriver.h"
+#include "clang/Basic/DiagnosticOptions.h"
+#include "clang/Driver/DriverDiagnostic.h"
+#include "clang/Driver/Options.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/Option/Arg.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/OptTable.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace Fortran::frontend;
+
+//===----------------------------------------------------------------------===//
+// Initialization.
+//===----------------------------------------------------------------------===//
+CompilerInvocationBase::CompilerInvocationBase()
+    : diagnosticOpts_(new clang::DiagnosticOptions()) {}
+
+CompilerInvocationBase::CompilerInvocationBase(const CompilerInvocationBase &x)
+    : diagnosticOpts_(new clang::DiagnosticOptions(x.GetDiagnosticOpts())) {}
+
+CompilerInvocationBase::~CompilerInvocationBase() = default;
+
+//===----------------------------------------------------------------------===//
+// Deserialization (from args)
+//===----------------------------------------------------------------------===//
+static InputKind ParseFrontendArgs(FrontendOptions &opts,
+    llvm::opt::ArgList &args, clang::DiagnosticsEngine &diags) {
+  // Identify the action (i.e. opts.ProgramAction)
+  if (const llvm::opt::Arg *a =
+          args.getLastArg(clang::driver::options::OPT_Action_Group)) {
+    switch (a->getOption().getID()) {
+    default: {
+      llvm_unreachable("Invalid option in group!");
+    }
+      // TODO:
+      // case clang::driver::options::OPT_E:
+      // case clang::driver::options::OPT_emit_obj:
+      // case calng::driver::options::OPT_emit_llvm:
+      // case clang::driver::options::OPT_emit_llvm_only:
+      // case clang::driver::options::OPT_emit_codegen_only:
+      // case clang::driver::options::OPT_emit_module:
+      // (...)
+    }
+  }
+
+  opts.showHelp_ = args.hasArg(clang::driver::options::OPT_help);
+  opts.showVersion_ = args.hasArg(clang::driver::options::OPT_version);
+
+  // Get the input kind (from the value passed via `-x`)
+  InputKind dashX(Language::Unknown);
+  if (const llvm::opt::Arg *a =
+          args.getLastArg(clang::driver::options::OPT_x)) {
+    llvm::StringRef XValue = a->getValue();
+    // Principal languages.
+    dashX = llvm::StringSwitch<InputKind>(XValue)
+                .Case("f90", Language::Fortran)
+                .Default(Language::Unknown);
+
+    // Some special cases cannot be combined with suffixes.
+    if (dashX.IsUnknown())
+      dashX = llvm::StringSwitch<InputKind>(XValue)
+                  .Case("ir", Language::LLVM_IR)
+                  .Default(Language::Unknown);
+
+    if (dashX.IsUnknown())
+      diags.Report(clang::diag::err_drv_invalid_value)
+          << a->getAsString(args) << a->getValue();
+  }
+
+  return dashX;
+}
+
+bool CompilerInvocation::CreateFromArgs(CompilerInvocation &res,
+    llvm::ArrayRef<const char *> commandLineArgs,
+    clang::DiagnosticsEngine &diags) {
+
+  bool success = true;
+
+  // Parse the arguments
+  const llvm::opt::OptTable &opts = clang::driver::getDriverOptTable();
+  const unsigned includedFlagsBitmask =
+      clang::driver::options::FC1Option;
+  unsigned missingArgIndex, missingArgCount;
+  llvm::opt::InputArgList args = opts.ParseArgs(
+      commandLineArgs, missingArgIndex, missingArgCount, includedFlagsBitmask);
+
+  // Issue errors on unknown arguments
+  for (const auto *a : args.filtered(clang::driver::options::OPT_UNKNOWN)) {
+    auto argString = a->getAsString(args);
+    std::string nearest;
+    if (opts.findNearest(argString, nearest, includedFlagsBitmask) > 1)
+      diags.Report(clang::diag::err_drv_unknown_argument) << argString;
+    else
+      diags.Report(clang::diag::err_drv_unknown_argument_with_suggestion)
+          << argString << nearest;
+    success = false;
+  }
+
+  // Parse the frontend args
+  ParseFrontendArgs(res.GetFrontendOpts(), args, diags);
+
+  return success;
+}

flang/lib/Frontend/FrontendOptions.cpp

+//===- FrontendOptions.cpp ------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/FrontendOptions.h"

flang/lib/FrontendTool/CMakeLists.txt

+add_flang_library(flangFrontendTool
+  ExecuteCompilerInvocation.cpp
+
+  LINK_LIBS
+  clangBasic
+  clangDriver
+
+  LINK_COMPONENTS
+  Option
+  Support
+)

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

+//===--- ExecuteCompilerInvocation.cpp ------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file holds ExecuteCompilerInvocation(). It is split into its own file to
+// minimize the impact of pulling in essentially everything else in Flang.
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/CompilerInstance.h"
+#include "clang/Driver/Options.h"
+#include "llvm/Option/OptTable.h"
+#include "llvm/Support/CommandLine.h"
+
+namespace Fortran::frontend {
+bool ExecuteCompilerInvocation(CompilerInstance *flang) {
+  // Honor -help.
+  if (flang->GetFrontendOpts().showHelp_) {
+    clang::driver::getDriverOptTable().PrintHelp(llvm::outs(),
+        "flang-new -fc1 [options] file...", "LLVM 'Flang' Compiler",
+        /*Include=*/clang::driver::options::FlangOption,
+        /*Exclude=*/0, /*ShowAllAliases=*/false);
+    return true;
+  }
+
+  // Honor -version.
+  if (flang->GetFrontendOpts().showVersion_) {
+    llvm::cl::PrintVersionMessage();
+    return true;
+  }
+
+  return true;
+}
+
+} // namespace Fortran::frontend

flang/test/CMakeLists.txt

 # Test runner infrastructure for Flang. This configures the Flang test trees
 # for use by Lit, and delegates to LLVM's lit test handlers.
 
 llvm_canonicalize_cmake_booleans(
   FLANG_STANDALONE_BUILD
 )
 
 set(FLANG_INTRINSIC_MODULES_DIR ${FLANG_BINARY_DIR}/include/flang)
 
 set(FLANG_TOOLS_DIR ${FLANG_BINARY_DIR}/bin)
 
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
   ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
   MAIN_CONFIG
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
 )
 
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in

 set(FLANG_TEST_DEPENDS
   f18 FileCheck count not module_files
 )
 
 if (LINK_WITH_FIR)
   list(APPEND FLANG_TEST_DEPENDS tco)
 endif()
 
+if (FLANG_BUILD_NEW_DRIVER)
+  list(APPEND FLANG_TEST_DEPENDS flang-new)
+endif()
+
 if (FLANG_INCLUDE_TESTS)
   if (FLANG_GTEST_AVAIL)
     list(APPEND FLANG_TEST_DEPENDS FlangUnitTests)
   endif()
 endif()
 
 add_custom_target(flang-test-depends DEPENDS ${FLANG_TEST_DEPENDS})
 

flang/test/Flang-Driver/driver-error-cc1.c

+// RUN: not %flang-new %s 2>&1 | FileCheck %s
+
+// REQUIRES: new-flang-driver
+
+// C files are currently not supported (i.e. `flang -cc1`)
+
+// CHECK:error: unknown integrated tool '-cc1'. Valid tools include '-fc1'.

flang/test/Flang-Driver/driver-error-cc1.cpp

+// RUN: not %flang-new %s 2>&1 | FileCheck %s
+
+// REQUIRES: new-flang-driver
+
+// C++ files are currently not supported (i.e. `flang -cc1`)
+
+// CHECK:error: unknown integrated tool '-cc1'. Valid tools include '-fc1'.

flang/test/Flang-Driver/driver-help.f90

+! RUN: %flang-new -help 2>&1 | FileCheck %s
+! RUN: %flang-new -fc1 -help 2>&1 | FileCheck %s
+! RUN: not %flang-new -helps 2>&1 | FileCheck %s --check-prefix=ERROR
+
+! REQUIRES: new-flang-driver
+
+! CHECK:USAGE: flang-new
+! CHECK-EMPTY:
+! CHECK-NEXT:OPTIONS:
+! CHECK-NEXT: -help     Display available options
+! CHECK-NEXT: --version Print version information
+
+! ERROR: error: unknown argument '-helps'; did you mean '-help'

flang/test/Flang-Driver/driver-version.f90

+! RUN: %flang-new --version 2>&1 | FileCheck %s
+! RUN: not %flang-new --versions 2>&1 | FileCheck %s --check-prefix=ERROR
+
+! REQUIRES: new-flang-driver
+
+! CHECK:flang-new version 
+! CHECK-NEXT:Target:
+! CHECK-NEXT:Thread model:
+! CHECK-NEXT:InstalledDir:
+
+! ERROR: error: unsupported option '--versions'; did you mean '--version'?

flang/test/Flang-Driver/emit-obj.f90

+! RUN: not %flang-new  %s 2>&1 | FileCheck %s --check-prefix=ERROR-IMPLICIT
+! RUN: not %flang-new  -emit-obj %s 2>&1 | FileCheck %s --check-prefix=ERROR-EXPLICIT
+! RUN: not %flang-new  -fc1 -emit-obj %s 2>&1 | FileCheck %s --check-prefix=ERROR-FC1
+
+! REQUIRES: new-flang-driver
+
+! By default (e.g. when no options like `-E` are passed) flang-new
+! creates a job that corresponds to `-emit-obj`. This option/action is
+! not yet supported. Verify that this is correctly reported as error.
+
+! ERROR-IMPLICIT: error: unknown argument: '-triple'
+! ERROR-IMPLICIT: error: unknown argument: '-emit-obj'
+! ERROR-IMPLICIT: error: unknown argument: '-o'
+
+! ERROR-EXPLICIT: error: unknown argument: '-o'
+
+! ERROR-FC1: error: unknown argument: '-emit-obj'

flang/test/Flang-Driver/missing-input.f90

+! RUN: not %flang-new  2>&1 | FileCheck %s
+
+! REQUIRES: new-flang-driver
+
+! CHECK: error: no input files

flang/test/lit.cfg.py

 
 # testFormat: The test format to use to interpret tests.
 #
 # For now we require '&&' between commands, until they get globally killed and
 # the test runner updated.
 config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
 
 # suffixes: A list of file extensions to treat as test files.
-config.suffixes = ['.f', '.F', '.ff', '.FOR', '.for', '.f77', '.f90', '.F90',
+config.suffixes = ['.c', '.cpp', '.f', '.F', '.ff', '.FOR', '.for', '.f77', '.f90', '.F90',
                    '.ff90', '.f95', '.F95', '.ff95', '.fpp', '.FPP', '.cuf',
                    '.CUF', '.f18', '.F18', '.fir']
 
 config.substitutions.append(('%PATH%', config.environment['PATH']))
 
 llvm_config.use_default_substitutions()
 
 # excludes: A list of directories to exclude from the testsuite. The 'Inputs'
 # subdirectories contain auxiliary inputs for various tests in their parent
 # directories.
 config.excludes = ['Inputs', 'CMakeLists.txt', 'README.txt', 'LICENSE.txt']
 
+# If the new Flang driver is enabled, add the corresponding feature to
+# config. Otherwise, exclude the corresponding test directory.
+if config.include_flang_new_driver_test:
+  config.available_features.add('new-flang-driver')
+else:
+  config.excludes.append('Flang-Driver')
+
 # test_source_root: The root path where tests are located.
 config.test_source_root = os.path.dirname(__file__)
 
 # test_exec_root: The root path where tests should be run.
 config.test_exec_root = os.path.join(config.flang_obj_root, 'test')
 
 # Tweak the PATH to include the tools dir.
 llvm_config.with_environment('PATH', config.flang_tools_dir, append_path=True)
 llvm_config.with_environment('PATH', config.llvm_tools_dir, append_path=True)
 
 if config.flang_standalone_build:
     # For builds with FIR, set path for tco and enable related tests
     if config.flang_llvm_tools_dir != "":
         config.available_features.add('fir')
         if config.llvm_tools_dir != config.flang_llvm_tools_dir:
             llvm_config.with_environment('PATH', config.flang_llvm_tools_dir, append_path=True)
 
 # For each occurrence of a flang tool name, replace it with the full path to
 # the build directory holding that tool.
 tools = [
   ToolSubst('%f18', command=FindTool('f18'),
     extra_args=["-intrinsic-module-directory "+config.flang_intrinsic_modules_dir],
     unresolved='fatal')
 ]
 
+if config.include_flang_new_driver_test:
+   tools.append(ToolSubst('%flang-new', command=FindTool('flang-new'), unresolved='fatal'))
+
 if config.flang_standalone_build:
     llvm_config.add_tool_substitutions(tools, [config.flang_llvm_tools_dir])
 else:
     llvm_config.add_tool_substitutions(tools, config.llvm_tools_dir)
 
 # Enable libpgmath testing
 result = lit_config.params.get("LIBPGMATH")
 if result:
     config.environment["LIBPGMATH"] = True

flang/test/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.flang_obj_root = "@FLANG_BINARY_DIR@"
 config.flang_src_dir = "@FLANG_SOURCE_DIR@"
 config.flang_tools_dir = "@FLANG_TOOLS_DIR@"
 config.flang_intrinsic_modules_dir = "@FLANG_INTRINSIC_MODULES_DIR@"
 config.flang_llvm_tools_dir = "@CMAKE_BINARY_DIR@/bin"
 config.python_executable = "@PYTHON_EXECUTABLE@"
 config.flang_standalone_build = @FLANG_STANDALONE_BUILD@
 
+# Control the regression test for flang-new driver
+import lit.util
+config.include_flang_new_driver_test = \
+    lit.util.pythonize_bool("@FLANG_BUILD_NEW_DRIVER@")
+
 # Support substitution of the tools_dir with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.flang_tools_dir = config.flang_tools_dir % lit_config.params
 except KeyError:
     e = sys.exc_info()[1]
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@FLANG_SOURCE_DIR@/test/lit.cfg.py")

flang/tools/CMakeLists.txt

 #===-- tools/CMakeLists.txt ------------------------------------------------===#
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 #===------------------------------------------------------------------------===#
 
 add_subdirectory(f18)
+if(FLANG_BUILD_NEW_DRIVER)
+  add_subdirectory(flang-driver)
+endif()
 if(LINK_WITH_FIR)
   add_subdirectory(tco)
 endif()
 add_subdirectory(f18-parse-demo)

flang/tools/flang-driver/CMakeLists.txt

+# Infrastructure to build flang driver entry point. Flang driver depends on
+# LLVM libraries.
+
+# Set your project compile flags.
+link_directories(${LLVM_LIBRARY_DIR})
+
+add_flang_tool(flang-new
+  driver.cpp
+  fc1_main.cpp
+)
+
+# Link against LLVM and Clang libraries
+target_link_libraries(flang-new
+  PRIVATE
+  ${LLVM_COMMON_LIBS}
+  flangFrontend
+  flangFrontendTool
+  clangDriver
+  clangBasic
+  LLVMSupport
+  LLVMTarget
+  LLVMOption
+)
+
+install(TARGETS flang-new DESTINATION bin)

flang/tools/flang-driver/driver.cpp

+//===-- driver.cpp - Flang Driver -----------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This is the entry point to the flang driver; it is a thin wrapper
+// for functionality in the Driver flang library.
+//
+//===----------------------------------------------------------------------===//
+#include "clang/Driver/Driver.h"
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticIDs.h"
+#include "clang/Basic/DiagnosticOptions.h"
+#include "clang/Driver/Compilation.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/IntrusiveRefCntPtr.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Support/Host.h"
+#include "llvm/Support/InitLLVM.h"
+#include "llvm/Support/VirtualFileSystem.h"
+
+// main frontend method. Lives inside fc1_main.cpp
+extern int fc1_main(llvm::ArrayRef<const char *> argv, const char *argv0);
+
+std::string GetExecutablePath(const char *argv0) {
+  // This just needs to be some symbol in the binary
+  void *p = (void *)(intptr_t)GetExecutablePath;
+  return llvm::sys::fs::getMainExecutable(argv0, p);
+}
+
+// This lets us create the DiagnosticsEngine with a properly-filled-out
+// DiagnosticOptions instance
+static clang::DiagnosticOptions *CreateAndPopulateDiagOpts(
+    llvm::ArrayRef<const char *> argv) {
+  auto *diagOpts = new clang::DiagnosticOptions;
+  return diagOpts;
+}
+
+static int ExecuteFC1Tool(llvm::SmallVectorImpl<const char *> &argV) {
+  llvm::StringRef tool = argV[1];
+  if (tool == "-fc1")
+    return fc1_main(makeArrayRef(argV).slice(2), argV[0]);
+
+  // Reject unknown tools.
+  // ATM it only supports fc1. Any fc1[*] is rejected.
+  llvm::errs() << "error: unknown integrated tool '" << tool << "'. "
+               << "Valid tools include '-fc1'.\n";
+  return 1;
+}
+
+int main(int argc_, const char **argv_) {
+
+  // Initialize variables to call the driver
+  llvm::InitLLVM x(argc_, argv_);
+  llvm::SmallVector<const char *, 256> argv(argv_, argv_ + argc_);
+
+  clang::driver::ParsedClangName targetandMode("flang", "--driver-mode=flang");
+  std::string driverPath = GetExecutablePath(argv[0]);
+
+  // Check if flang-new is in the frontend mode
+  auto firstArg = std::find_if(
+      argv.begin() + 1, argv.end(), [](const char *a) { return a != nullptr; });
+  if (firstArg != argv.end()) {
+    if (llvm::StringRef(argv[1]).startswith("-cc1")) {
+      llvm::errs() << "error: unknown integrated tool '" << argv[1] << "'. "
+                   << "Valid tools include '-fc1'.\n";
+      return 1;
+    }
+    // Call flang-new frontend
+    if (llvm::StringRef(argv[1]).startswith("-fc1")) {
+      return ExecuteFC1Tool(argv);
+    }
+  }
+
+  // Not in the frontend mode - continue in the compiler driver mode.
+
+  // Create DiagnosticsEngine for the compiler driver
+  llvm::IntrusiveRefCntPtr<clang::DiagnosticOptions> diagOpts =
+      CreateAndPopulateDiagOpts(argv);
+  llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> diagID(
+      new clang::DiagnosticIDs());
+  clang::TextDiagnosticPrinter *diagClient =
+      new clang::TextDiagnosticPrinter(llvm::errs(), &*diagOpts);
+  clang::DiagnosticsEngine diags(diagID, &*diagOpts, diagClient);
+
+  // Prepare the driver
+  clang::driver::Driver theDriver(driverPath,
+      llvm::sys::getDefaultTargetTriple(), diags, "flang LLVM compiler");
+  theDriver.setTargetAndMode(targetandMode);
+  std::unique_ptr<clang::driver::Compilation> c(
+      theDriver.BuildCompilation(argv));
+  llvm::SmallVector<std::pair<int, const clang::driver::Command *>, 4>
+      failingCommands;
+
+  // Run the driver
+  int res = 1;
+  bool isCrash = false;
+  res = theDriver.ExecuteCompilation(*c, failingCommands);
+
+  for (const auto &p : failingCommands) {
+    int CommandRes = p.first;
+    const clang::driver::Command *failingCommand = p.second;
+    if (!res)
+      res = CommandRes;
+
+    // If result status is < 0 (e.g. when sys::ExecuteAndWait returns -1),
+    // then the driver command signalled an error. On Windows, abort will
+    // return an exit code of 3. In these cases, generate additional diagnostic
+    // information if possible.
+    isCrash = CommandRes < 0;
+#ifdef _WIN32
+    IsCrash |= CommandRes == 3;
+#endif
+    if (isCrash) {
+      theDriver.generateCompilationDiagnostics(*c, *failingCommand);
+      break;
+    }
+  }
+
+  diags.getClient()->finish();
+
+  // If we have multiple failing commands, we return the result of the first
+  // failing command.
+  return res;
+}

flang/tools/flang-driver/fc1_main.cpp

+//===-- fc1_main.cpp - Flang FC1 Compiler Frontend ------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This is the entry point to the flang -fc1 functionality, which implements the
+// core compiler functionality along with a number of additional tools for
+// demonstration and testing purposes.
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/CompilerInstance.h"
+#include "flang/Frontend/CompilerInvocation.h"
+#include "flang/FrontendTool/Utils.h"
+#include "clang/Driver/DriverDiagnostic.h"
+#include "clang/Frontend/TextDiagnosticBuffer.h"
+#include "llvm/Option/Arg.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/OptTable.h"
+
+#include <cstdio>
+
+using namespace Fortran::frontend;
+
+int fc1_main(llvm::ArrayRef<const char *> argv, const char *argv0) {
+  // Create CompilerInstance
+  std::unique_ptr<CompilerInstance> flang(new CompilerInstance());
+
+  // Create DiagnosticsEngine for the frontend driver
+  flang->CreateDiagnostics();
+  if (!flang->HasDiagnostics())
+    return 1;
+
+  // Create CompilerInvocation - use a dedicated instance of DiagnosticsEngine
+  // for parsing the arguments
+  llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> diagID(
+      new clang::DiagnosticIDs());
+  llvm::IntrusiveRefCntPtr<clang::DiagnosticOptions> diagOpts =
+      new clang::DiagnosticOptions();
+  clang::TextDiagnosticBuffer *diagsBuffer = new clang::TextDiagnosticBuffer;
+  clang::DiagnosticsEngine diags(diagID, &*diagOpts, diagsBuffer);
+  bool success =
+      CompilerInvocation::CreateFromArgs(flang->GetInvocation(), argv, diags);
+
+  diagsBuffer->FlushDiagnostics(flang->getDiagnostics());
+  if (!success)
+    return 1;
+
+  // Execute the frontend actions.
+  success = ExecuteCompilerInvocation(flang.get());
+
+  return !success;
+}

flang/unittests/CMakeLists.txt

function(add_flang_nongtest_unittest test_name)
   add_dependencies(FlangUnitTests ${test_name}.test)
 endfunction()
 
 add_subdirectory(Optimizer)
 add_subdirectory(Decimal)
 add_subdirectory(Evaluate)
 add_subdirectory(Runtime)
 add_subdirectory(Lower)
+
+if (FLANG_BUILD_NEW_DRIVER)
+  add_subdirectory(Frontend)
+endif()

flang/unittests/Frontend/CMakeLists.txt

+add_flang_unittest(FlangFrontendTests
+  CompilerInstanceTest.cpp
+)
+
+target_link_libraries(FlangFrontendTests
+  PRIVATE
+  LLVMSupport
+  clangBasic
+  flangFrontend
+  flangFrontendTool)

flang/unittests/Frontend/CompilerInstanceTest.cpp

+//===- unittests/Frontend/CompilerInstanceTest.cpp - CI tests -------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/CompilerInstance.h"
+#include "gtest/gtest.h"
+#include "flang/Frontend/CompilerInvocation.h"
+#include "clang/Basic/DiagnosticOptions.h"
+#include "clang/Driver/Options.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <filesystem>
+using namespace llvm;
+using namespace Fortran::frontend;
+
+namespace {
+
+TEST(CompilerInstance, AllowDiagnosticLogWithUnownedDiagnosticConsumer) {
+  // 1. Set-up a basic DiagnosticConsumer
+  std::string diagnosticOutput;
+  llvm::raw_string_ostream diagnosticsOS(diagnosticOutput);
+  auto diagPrinter = std::make_unique<clang::TextDiagnosticPrinter>(
+      diagnosticsOS, new clang::DiagnosticOptions());
+
+  // 2. Create a CompilerInstance (to manage a DiagnosticEngine)
+  CompilerInstance compInst;
+
+  // 3. Set-up DiagnosticOptions
+  auto diagOpts = new clang::DiagnosticOptions();
+  // Tell the diagnostics engine to emit the diagnostic log to STDERR. This
+  // ensures that a chained diagnostic consumer is created so that the test can
+  // exercise the unowned diagnostic consumer in a chained consumer.
+  diagOpts->DiagnosticLogFile = "-";
+
+  // 4. Create a DiagnosticEngine with an unowned consumer
+  IntrusiveRefCntPtr<clang::DiagnosticsEngine> diags =
+      compInst.CreateDiagnostics(diagOpts, diagPrinter.get(),
+          /*ShouldOwnClient=*/false);
+
+  // 5. Report a diagnostic
+  diags->Report(clang::diag::err_expected) << "no crash";
+
+  // 6. Verify that the reported diagnostic wasn't lost and did end up in the
+  // output stream
+  ASSERT_EQ(diagnosticsOS.str(), "error: expected no crash\n");
+}
+} // namespace

llvm/include/llvm/Option/OptTable.h

struct Info {
     /// matching.
     const char *const *Prefixes;
     const char *Name;
     const char *HelpText;
     const char *MetaVar;
     unsigned ID;
     unsigned char Kind;
     unsigned char Param;
-    unsigned short Flags;
+    unsigned int Flags;
     unsigned short GroupID;
     unsigned short AliasID;
     const char *AliasArgs;
     const char *Values;
   };
 
 private:
   /// The option information table.
InputArgList parseArgs(int Argc, char *const *Argv, OptSpecifier Unknown,
                          function_ref<void(StringRef)> ErrorFn) const;
 
   /// Render the help text for an option table.
   ///
   /// \param OS - The stream to write the help text to.
   /// \param Usage - USAGE: Usage
   /// \param Title - OVERVIEW: Title
   /// \param FlagsToInclude - If non-zero, only include options with any
   ///                         of these flags set.
   /// \param FlagsToExclude - Exclude options with any of these flags set.
   /// \param ShowAllAliases - If true, display all options including aliases
   ///                         that don't have help texts. By default, we display
   ///                         only options that are not hidden and have help
   ///                         texts.
   void PrintHelp(raw_ostream &OS, const char *Usage, const char *Title,
                  unsigned FlagsToInclude, unsigned FlagsToExclude,
                  bool ShowAllAliases) const;