[Flang][Driver] Add PrintPreprocessed FrontendAction

[awarzynski]

Summary

[Flang][Driver] Add PrintPreprocessed FrontendAction

This patch implements the first frontend action for the Flang parser
(i.e. Fortran::parser). This action runs the preprocessor on the input
Fortran file and prints the generated output either to stdout or the
output file (specified with - or -o <output-file>).

Note that currently there is no mechanism to map options for the
frontend driver (i.e. Fortran::frontend::FrontendOptions) to options for
the parser (i.e. Fortran::parser::Options). Instead,
Frotran::parser::options are hard-coded to:

std::vector<std::string> searchDirectories{"."s};
searchDirectories = searchDirectories;
isFixedForm = false;
_encoding(Fortran::parser::Encoding::UTF_8);

These default settings are compatible with the current driver. Further
work is required in order for CompilerInvocation to read and map
clang::driver::options to Fortran::parser::options.

Co-authored-by: Andrzej Warzynski <[e-mail]>

Depends on D87989

[CarolineConcatto]

Fix typos

[CarolineConcatto]

Remove -E from FlangOptions

[awarzynski]

Rebase + refector the unit test

This is re-based on top of the latest version of D87989. The unit test is
update for consistency with D87989.

[awarzynski]

Since @CarolineConcatto has recently moved to a different project, I am assigning this to myself and will be responding to the future review comments. Thank you for all the effort @CarolineConcatto !

[sameeranjoshi]

Thank you, this patch looks easy to understand as it's clearly separated from(D87989) the infrastructure changes needed for frontend actions.
A few comments inline.

flang/include/flang/Frontend/CompilerInstance.h

85

If I am correct this seems to be an accessor member function and it should follow point 3 from flang style guide mentioned at https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#naming

I am not aware if driver related patches follow llvm-project style.

flang/include/flang/Frontend/CompilerInvocation.h

12

Nit: I believe clang-format is missing.

flang/lib/Frontend/CMakeLists.txt

17 ↗

(On Diff #298635)

I believe this patch is a parsing and preprocessing related patch, is this used somewhere or should this be removed for now?

flang/lib/Frontend/CompilerInstance.cpp

136

May be you meant as it is used?

flang/lib/Frontend/FrontendAction.cpp

8

undo

58

Nit: Converts to phases in english according to my browser. :)

flang/lib/Frontend/FrontendActions.cpp

62

https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#c-language
point 4
Can you use braced initializers ?

A more better version I think would be to simplify this to

if (auto os{ci.CreateDefaultOutputFile(/*Binary=*/true, /*InFile=*/GetCurrentFileOrBufferName())}){
    (*os) << out.str();
} else {
  llvm::errs() << "Unable to create the output file\n";
  return;
}

flang/test/Frontend/Inputs/hello-world.c

2

a - ?

[awarzynski]

Address PR comments, make clearer separation between option types, simplify/fix tests

Based on the feedback, I've made a clearer separation between the parser and
the frontend driver options. I've also emphasised that mapping between the two
is a TODO. Initialisation to the defaults has been extracted out of
ExecuteAction (these should be set before an action is run).

I've also fixed a bug in the unit test and simplified the regression tests.
There's no need to differentiation between:

• -E -, -E -o <output-file> and -E

That is done in https://reviews.llvm.org/D87989. In this patch it is sufficient
to focus on testing -E.

Thank you for reviewing @sameeranjoshi !

[awarzynski]

flang/include/flang/Frontend/CompilerInstance.h

85

Thanks, good catch!

Flang driver is a subproject within Flang and we've assumed that it should follow Flang's coding style. I'm much more used to LLVM's coding style and also spend a lot of time in Clang - hence mistakes like this.

Personally I think that it's unfortunate that there are multiple coding styles within llvm-project. But this is neither time nor place to challenge that :)

Btw, sadly we've made similar mistake in other places: https://github.com/llvm/llvm-project/blob/master/flang/include/flang/Frontend/CompilerInstance.h#L30. I will refactor that in a separate patch.

flang/include/flang/Frontend/CompilerInvocation.h

12

I did apply it and this line looks OK to me - what's missing?

49–52

This comment should clearly communicate that these are _frontend_ options. frontendOpts_ are _frontend_ driver options. The naming is perhaps a bit unfortunate :/

flang/lib/Frontend/CMakeLists.txt

17 ↗

(On Diff #298635)

Not sure how that crept in - thank you!

flang/lib/Frontend/CompilerInstance.cpp

136

Good point :)

I have even more drastic suggestion - IMO the defaults should be set in a dedicated method. And the encoding for files could be set in the constructor of CompilerInstance.

I will implement that change. I will also add comments to highlight that we are using these defaults during the development. However, the end goal is to provide APIs for the user the set-up the frontend options.

flang/lib/Frontend/FrontendAction.cpp

58

Fixed :)

flang/lib/Frontend/FrontendActions.cpp

62

Neat suggestion, though: https://llvm.org/docs/CodingStandards.html#do-not-use-braced-initializer-lists-to-call-a-constructor. For consistency sake, I will follow the convention from Flang. I will also simplify this function a bit.

flang/test/Frontend/print-preprocess-C-file.f90

9

Probably copied from a different file - can be removed.

11–12

These 2 tests are no different from the previous one. The key thing to test here is that flang-new -E triggers an error for C files. I recommend removing that.

flang/test/Frontend/print-preprocessed-file.f90

10–11

Similar comment as for print-preprocess-C-file.f90 (i.e. only one version is needed).

flang/unittests/Frontend/PrintPreprocessedTest.cpp

10

Not sure how this crept in here. Need to delete.

73

This should be startswith instead of equals. The output buffer is not initialized and hence there will be garbage past "program b".`

[awarzynski]

Apply fix to unit test that got lost in the previous patch, simplify C input file

[sameeranjoshi]

A few nits: and mostly style comments inline:

flang/include/flang/Frontend/CompilerInvocation.h

12

See point 2[1] below the code block.
Ideally clang comes alphabetically first.

[1] https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#files

flang/include/flang/Frontend/FrontendActions.h

9

Why is this not following the style guide mentioned at [1] point 2.

[1] https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md#files

[awarzynski]

flang/include/flang/Frontend/CompilerInvocation.h

12

Thanks for pointing this out! I appreciate that Flang has its own coding style, but I think that in terms of the order of include files, the rules from LLVM's coding standard [1] and Flang's are actually similar. From [1]:

We prefer these #includes to be listed in this order:

Main Module Header
Local/Private Headers
LLVM project/subproject headers (clang/..., lldb/..., llvm/..., etc)
System #includes
and each category should be sorted lexicographically by the full path.

This is then further clarified as:

LLVM project and subproject headers should be grouped from most specific to least specific, for the same reasons described above. For example, LLDB depends on both clang and LLVM, and clang depends on LLVM. So an LLDB source file should include lldb headers first, followed by clang headers, followed by llvm headers, to reduce the possibility (for example) of an LLDB header accidentally picking up a missing include due to the previous inclusion of that header in the main source file or some earlier header file. clang should similarly include its own headers before including llvm headers. This rule applies to all LLVM subprojects.

That's the rule that I applied here. This is consistent with the current .clang-format configuration for Flang: https://github.com/llvm/llvm-project/blob/d26dd743084a886382204ede5eeed146cd29fcd6/flang/.clang-format#L10-L18. This is also why clang-format believes this is correct.

In other words, there are two rules:

• project specific header files first (i.e. header files from Flang first)
• then, use alphabetic order.

I do feel that both [1] and [2] leave room for interpretation. If we are still in disagreement, we can follow-up on flang-dev or in one of the weekly calls. I'm keen to get this right!

[1] https://llvm.org/docs/CodingStandards.html#include-style
[2] https://github.com/llvm/llvm-project/blob/master/flang/docs/C++style.md

flang/include/flang/Frontend/FrontendActions.h

9

I'm guessing that you are referring to this rule specifically:

Header files should be idempotent. Use the usual technique:

#ifndef FORTRAN_header_H_
#define FORTRAN_header_H_
// code
#endif  // FORTRAN_header_H_

To me this rule reads: "use hash guards". The rule doesn't mention what the format of the hash guard should be. However, LLVM's style guide does: https://llvm.org/docs/CodingStandards.html#header-guard. We've adopted it based on the following rule from Flang's style guide:

Otherwise, where LLVM's C++ style guide is clear on usage, follow it.

The format of the hash guards hasn't been contested in any of the previous reviews, so we assume that the format is fine.

And if this is not about the format of the hash guard - please clarify :)

[sameeranjoshi]

Looks good, I would wait for a couple of more days for someone to review from community may be from Nvidia's side if someone would verify the initial design.
Thank you.

flang/include/flang/Frontend/CompilerInvocation.h

12

Thanks for details.
I was unaware of the fact that local project headers should come first, I was assuming everything is lexicographically sorted.

flang/include/flang/Frontend/FrontendActions.h

9

I reverted back seeing the clang-tidy warnings and no FORTRAN prefix but a LLVM_FLANG prefix.
If it was approved in previous reviews, I think that OK to keep it.

[awarzynski]

Rebase on top of master + rename accessor method

[awarzynski]

In D88381#2357123, @sameeranjoshi wrote:

I would wait for a couple of more days for someone to review from community may be from Nvidia's side if someone would verify the initial design.

As there were no new reviews, I assumed that this good to land and have just merged it. I encourage people to submit post-commit reviews if they identify any issues with this patch that we've missed.

Thank you all for reviewing!

clang/include/clang/Driver/Options.td

def B : JoinedOrSeparate<["-"], "B">, MetaVarName<"<dir>">,
     HelpText<"Add <dir> to search path for binaries and object files used implicitly">;
 def CC : Flag<["-"], "CC">, Flags<[CC1Option]>, Group<Preprocessor_Group>,
     HelpText<"Include comments from within macros in preprocessed output">;
 def C : Flag<["-"], "C">, Flags<[CC1Option]>, Group<Preprocessor_Group>,
     HelpText<"Include comments in preprocessed output">;
 def D : JoinedOrSeparate<["-"], "D">, Group<Preprocessor_Group>,
     Flags<[CC1Option]>, MetaVarName<"<macro>=<value>">,
     HelpText<"Define <macro> to <value> (or 1 if <value> omitted)">;
-def E : Flag<["-"], "E">, Flags<[NoXarchOption,CC1Option]>, Group<Action_Group>,
+def E : Flag<["-"], "E">, Flags<[NoXarchOption,CC1Option, FlangOption, FC1Option]>, Group<Action_Group>,
     HelpText<"Only run the preprocessor">;
 def F : JoinedOrSeparate<["-"], "F">, Flags<[RenderJoined,CC1Option]>,
     HelpText<"Add directory to framework include search path">;
 def G : JoinedOrSeparate<["-"], "G">, Flags<[NoXarchOption]>, Group<m_Group>,
     MetaVarName<"<size>">, HelpText<"Put objects of at most <size> bytes "
     "into small data section (MIPS / Hexagon)">;
 def G_EQ : Joined<["-"], "G=">, Flags<[NoXarchOption]>, Group<m_Group>, Alias<G>;
 def H : Flag<["-"], "H">, Flags<[CC1Option]>, Group<Preprocessor_Group>,

flang/include/flang/Frontend/CompilerInstance.h

 //===-- CompilerInstance.h - Flang Compiler Instance ------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 #ifndef LLVM_FLANG_FRONTEND_COMPILERINSTANCE_H
 #define LLVM_FLANG_FRONTEND_COMPILERINSTANCE_H
 
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Frontend/FrontendAction.h"
+#include "flang/Parser/parsing.h"
 #include "flang/Parser/provenance.h"
 #include "llvm/Support/raw_ostream.h"
 
-#include <cassert>
-#include <memory>
-
 namespace Fortran::frontend {
 
 class CompilerInstance {
 
   /// The options used in this compiler instance.
   std::shared_ptr<CompilerInvocation> invocation_;
 
   /// Flang file  manager.
   std::shared_ptr<Fortran::parser::AllSources> allSources_;
 
+  std::shared_ptr<Fortran::parser::AllCookedSources> allCookedSources_;
+
+  std::shared_ptr<Fortran::parser::Parsing> parsing_;
+
   /// The diagnostics engine instance.
   llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine> diagnostics_;
 
   /// Holds information about the output file.
   struct OutputFile {
     std::string filename_;
     OutputFile(std::string inputFilename)
         : filename_(std::move(inputFilename)) {}
public:
   /// {
 
   /// Return the current allSources.
   Fortran::parser::AllSources &allSources() const { return *allSources_; }
 
   bool HasAllSources() const { return allSources_ != nullptr; }
 
   /// }
+  /// @name Parser Operations
+  /// {
+
+  /// Return parsing to be used by Actions.
+  Fortran::parser::Parsing &parsing() const { return *parsing_; }
+
+  /// }
   /// @name High-Level Operations
   /// {
 
   /// Execute the provided action against the compiler's
   /// CompilerInvocation object.
   /// \param act - The action to execute.
   /// \return - True on success.
   bool ExecuteAction(FrontendAction &act);

flang/include/flang/Frontend/CompilerInvocation.h

 //===- CompilerInvocation.h - Compiler Invocation Helper Data ---*- C -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 #ifndef LLVM_FLANG_FRONTEND_COMPILERINVOCATION_H
 #define LLVM_FLANG_FRONTEND_COMPILERINVOCATION_H
 
 #include "flang/Frontend/FrontendOptions.h"
+#include "flang/Parser/parsing.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "llvm/Option/ArgList.h"
 
 namespace Fortran::frontend {
 
 /// Fill out Opts based on the options given in Args.
 ///
clang::DiagnosticOptions &GetDiagnosticOpts() {
     return *diagnosticOpts_.get();
   }
   const clang::DiagnosticOptions &GetDiagnosticOpts() const {
     return *diagnosticOpts_.get();
   }
 };
 
 class CompilerInvocation : public CompilerInvocationBase {
-  /// Options controlling the frontend itself.
+  /// Options for the frontend driver
+  // TODO: Merge with or translate to parserOpts_. We shouldn't need two sets of
+  // options.
   FrontendOptions frontendOpts_;
 
+  /// Options for Flang parser
+  // TODO: Merge with or translate to frontendOpts_. We shouldn't need two sets
+  // of options.
+  Fortran::parser::Options parserOpts_;
+
 public:
   CompilerInvocation() = default;
 
   FrontendOptions &frontendOpts() { return frontendOpts_; }
   const FrontendOptions &frontendOpts() const { return frontendOpts_; }
 
+  Fortran::parser::Options &fortranOpts() { return parserOpts_; }
+  const Fortran::parser::Options &fortranOpts() const { return parserOpts_; }
+
   /// Create a compiler invocation from a list of input options.
   /// \returns true on success.
   /// \returns false if an error was encountered while parsing the arguments
   /// \param [out] res - The resulting invocation.
   static bool CreateFromArgs(CompilerInvocation &res,
       llvm::ArrayRef<const char *> commandLineArgs,
       clang::DiagnosticsEngine &diags);
+
+  /// Set the Fortran options to predifined defaults. These defaults are
+  /// consistend with f18/f18.cpp.
+  // TODO: We should map frontendOpts_ to parserOpts_ instead. For that, we
+  // need to extend frontendOpts_ first. Next, we need to add the corresponding
+  // compiler driver options in libclangDriver.
+  void SetDefaultFortranOpts();
 };
 
 } // end namespace Fortran::frontend
 #endif // LLVM_FLANG_FRONTEND_COMPILERINVOCATION_H

flang/include/flang/Frontend/FrontendActions.h

 //===- FrontendActions.h -----------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H
 #define LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H
 
 #include "flang/Frontend/FrontendAction.h"
 
 namespace Fortran::frontend {
 
 //===----------------------------------------------------------------------===//
 // Custom Consumer Actions
 //===----------------------------------------------------------------------===//
 
 class InputOutputTestAction : public FrontendAction {
   void ExecuteAction() override;
 };
 
+class PrintPreprocessedAction : public FrontendAction {
+  void ExecuteAction() override;
+};
+
 } // namespace Fortran::frontend
 
 #endif // LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H

flang/include/flang/Frontend/FrontendOptions.h

 namespace Fortran::frontend {
 
 enum ActionKind {
   InvalidAction = 0,
 
   /// -test-io mode
   InputOutputTest,
 
-  // TODO: ADD flags as the Actions are implemented, e.g.
-  // RunPreprocessor, ParserSyntaxOnly, EmitLLVM, EmitLLVMOnly,
-  // EmitCodeGenOnly, EmitAssembly, (...)
+  /// -E mode.
+  PrintPreprocessedInput,
+  /// TODO: RunPreprocessor, ParserSyntaxOnly, EmitLLVM, EmitLLVMOnly,
+  /// EmitCodeGenOnly, EmitAssembly, (...)
 };
 
 inline const char *GetActionKindName(const ActionKind ak) {
   switch (ak) {
   case InputOutputTest:
     return "InputOutputTest";
+  case PrintPreprocessedInput:
+    return "PrintPreprocessedInput";
   default:
     return "<unknown ActionKind>";
     // TODO:
     // case RunPreprocessor:
     // case ParserSyntaxOnly:
     // case EmitLLVM:
     // case EmitLLVMOnly:
     // case EmitCodeGenOnly:

flang/lib/Frontend/CompilerInstance.cpp

 //===--- CompilerInstance.cpp ---------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Frontend/TextDiagnosticPrinter.h"
+#include "flang/Parser/parsing.h"
 #include "flang/Parser/provenance.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace Fortran::frontend;
 
 CompilerInstance::CompilerInstance()
     : invocation_(new CompilerInvocation()),
-      allSources_(new Fortran::parser::AllSources()) {}
+      allSources_(new Fortran::parser::AllSources()),
+      allCookedSources_(new Fortran::parser::AllCookedSources(*allSources_)),
+      parsing_(new Fortran::parser::Parsing(*allCookedSources_)) {
+
+  // TODO: This is a good default during development, but ultimately we should
+  // give the user the opportunity to specify this.
+  allSources_->set_encoding(Fortran::parser::Encoding::UTF_8);
+}
 
 CompilerInstance::~CompilerInstance() {
   assert(outputFiles_.empty() && "Still output files in flight?");
 }
 
 void CompilerInstance::set_invocation(
     std::shared_ptr<CompilerInvocation> value) {
   invocation_ = std::move(value);
for (OutputFile &of : outputFiles_)
     if (!of.filename_.empty() && eraseFiles)
       llvm::sys::fs::remove(of.filename_);
 
   outputFiles_.clear();
   nonSeekStream_.reset();
 }
 
 bool CompilerInstance::ExecuteAction(FrontendAction &act) {
+  // Set some sane defaults for the frontend.
+  // TODO: Instead of defaults we should be setting these options based on the
+  // user input.
+  this->invocation().SetDefaultFortranOpts();
 
   // Connect Input to a CompileInstance
   for (const FrontendInputFile &fif : frontendOpts().inputs_) {
     if (act.BeginSourceFile(*this, fif)) {
       if (llvm::Error err = act.Execute()) {
         consumeError(std::move(err));
       }
       act.EndSourceFile();
     }
   }
   return true;
 }

flang/lib/Frontend/CompilerInvocation.cpp

if (const llvm::opt::Arg *a =
           args.getLastArg(clang::driver::options::OPT_Action_Group)) {
     switch (a->getOption().getID()) {
     default: {
       llvm_unreachable("Invalid option in group!");
     }
     case clang::driver::options::OPT_test_io:
       opts.programAction_ = InputOutputTest;
       break;
+    case clang::driver::options::OPT_E:
+      opts.programAction_ = PrintPreprocessedInput;
+      break;
+
       // TODO:
-      // case clang::driver::options::OPT_E:
       // case clang::driver::options::OPT_emit_obj:
       // case calng::driver::options::OPT_emit_llvm:
       // case clang::driver::options::OPT_emit_llvm_only:
       // case clang::driver::options::OPT_emit_codegen_only:
       // case clang::driver::options::OPT_emit_module:
       // (...)
     }
   }
for (const auto *a : args.filtered(clang::driver::options::OPT_UNKNOWN)) {
     success = false;
   }
 
   // Parse the frontend args
   ParseFrontendArgs(res.frontendOpts(), args, diags);
 
   return success;
 }
+
+void CompilerInvocation::SetDefaultFortranOpts() {
+  auto fortranOptions = fortranOpts();
+
+  // These defaults are based on the defaults in f18/f18.cpp.
+  std::vector<std::string> searchDirectories{"."s};
+  fortranOptions.searchDirectories = searchDirectories;
+  fortranOptions.isFixedForm = false;
+}

flang/lib/Frontend/FrontendAction.cpp

 //===--- FrontendAction.cpp -----------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "flang/Frontend/FrontendAction.h"
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/FrontendActions.h"
 #include "llvm/Support/Errc.h"
 
 using namespace Fortran::frontend;
 
 void FrontendAction::set_currentInput(const FrontendInputFile &currentInput) {
bool FrontendAction::BeginSourceFile(
   return true;
 }
 
 bool FrontendAction::ShouldEraseOutputFiles() {
   return instance().diagnostics().hasErrorOccurred();
 }
 
 llvm::Error FrontendAction::Execute() {
+  std::string currentInputPath{GetCurrentFileOrBufferName()};
+
+  Fortran::parser::Options parserOptions =
+      this->instance().invocation().fortranOpts();
+
+  this->instance().parsing().Prescan(currentInputPath, parserOptions);
+
   ExecuteAction();
+
   return llvm::Error::success();
 }
 
 void FrontendAction::EndSourceFile() {
   CompilerInstance &ci = instance();
 
   // Cleanup the output streams, and erase the output files if instructed by the
   // FrontendAction.
   ci.ClearOutputFiles(/*EraseFiles=*/ShouldEraseOutputFiles());
 
   set_instance(nullptr);
   set_currentInput(FrontendInputFile());
 }

flang/lib/Frontend/FrontendActions.cpp

 //===--- FrontendActions.cpp ----------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
+
 #include "flang/Frontend/FrontendActions.h"
-#include "flang/Common/Fortran-features.h"
-#include "flang/Common/default-kinds.h"
 #include "flang/Frontend/CompilerInstance.h"
+#include "flang/Parser/parsing.h"
+#include "flang/Parser/provenance.h"
 #include "flang/Parser/source.h"
-#include "clang/Serialization/PCHContainerOperations.h"
 
 using namespace Fortran::frontend;
 
 void InputOutputTestAction::ExecuteAction() {
 
   // Get the name of the file from FrontendInputFile current.
   std::string path{GetCurrentFileOrBufferName()};
   std::string buf;
os = ci.CreateDefaultOutputFile(
         binaryMode, GetCurrentFileOrBufferName(), "txt");
     if (!os)
       return;
     (*os) << fileContent.data();
   } else {
     ci.WriteOutputStream(fileContent.data());
   }
 }
+
+void PrintPreprocessedAction::ExecuteAction() {
+  std::string buf;
+  llvm::raw_string_ostream outForPP{buf};
+
+  // Run the preprocessor
+  CompilerInstance &ci = this->instance();
+  ci.parsing().DumpCookedChars(outForPP);
+
+  // If a pre-defined output stream exists, dump the preprocessed content there
+  if (!ci.IsOutputStreamNull()) {
+    // Send the output to the pre-defined output buffer.
+    ci.WriteOutputStream(outForPP.str());
+    return;
+  }
+
+  // Create a file and save the preprocessed output there
+  if (auto os{ci.CreateDefaultOutputFile(
+          /*Binary=*/true, /*InFile=*/GetCurrentFileOrBufferName())}) {
+    (*os) << outForPP.str();
+  } else {
+    llvm::errs() << "Unable to create the output file\n";
+    return;
+  }
+}

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

 static std::unique_ptr<FrontendAction> CreateFrontendBaseAction(
     CompilerInstance &ci) {
 
   ActionKind ak = ci.frontendOpts().programAction_;
   switch (ak) {
   case InputOutputTest:
     return std::make_unique<InputOutputTestAction>();
     break;
+  case PrintPreprocessedInput:
+    return std::make_unique<PrintPreprocessedAction>();
+    break;
   default:
     break;
     // TODO:
     // case RunPreprocessor:
     // case ParserSyntaxOnly:
     // case EmitLLVM:
     // case EmitLLVMOnly:
     // case EmitCodeGenOnly:

flang/test/Flang-Driver/driver-help-hidden.f90

 ! RUN: not %flang-new -fc1  -help-hidden 2>&1 | FileCheck %s --check-prefix=ERROR-FLANG-FC1
 
 !----------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG DRIVER (flang-new)
 !----------------------------------------------------
 ! CHECK:USAGE: flang-new
 ! CHECK-EMPTY:
 ! CHECK-NEXT:OPTIONS:
+! CHECK-NEXT: -E        Only run the preprocessor
 ! CHECK-NEXT: -fcolor-diagnostics    Enable colors in diagnostics
 ! CHECK-NEXT: -fno-color-diagnostics Disable colors in diagnostics
 ! CHECK-NEXT: -help     Display available options
 ! CHECK-NEXT: -o <file> Write output to <file>
 ! CHECK-NEXT: -test-io  Run the InputOuputTest action. Use for development and testing only.
 ! CHECK-NEXT: --version Print version information
 
 !-------------------------------------------------------------

flang/test/Flang-Driver/driver-help.f90

 ! REQUIRES: new-flang-driver
 
 !--------------------------
 ! FLANG DRIVER (flang-new)
 !--------------------------
 ! RUN: %flang-new -help 2>&1 | FileCheck %s --check-prefix=HELP
 ! RUN: not %flang-new -helps 2>&1 | FileCheck %s --check-prefix=ERROR
 
 !----------------------------------------
 ! FLANG FRONTEND DRIVER (flang-new -fc1)
 !----------------------------------------
 ! RUN: %flang-new -fc1 -help 2>&1 | FileCheck %s --check-prefix=HELP-FC1
 ! RUN: not %flang-new -fc1 -helps 2>&1 | FileCheck %s --check-prefix=ERROR
 
-!-----------------------------
-! EXPECTED OUTPUT (flang-new)
-!-----------------------------
+!----------------------------------------------------
+! EXPECTED OUTPUT FOR FLANG DRIVER (flang-new)
+!----------------------------------------------------
 ! HELP:USAGE: flang-new
 ! HELP-EMPTY:
 ! HELP-NEXT:OPTIONS:
+! HELP-NEXT: -E                     Only run the preprocessor
 ! HELP-NEXT: -fcolor-diagnostics    Enable colors in diagnostics
 ! HELP-NEXT: -fno-color-diagnostics Disable colors in diagnostics
 ! HELP-NEXT: -help                  Display available options
 ! HELP-NEXT: -o <file>              Write output to <file>
 ! HELP-NEXT: --version              Print version information
 
-!----------------------------------
-! EXPECTED OUTPUT (flang-new -fc1)
-!----------------------------------
+!-------------------------------------------------------------
+! EXPECTED OUTPUT FOR FLANG FRONTEND DRIVER (flang-new -fc1)
+!-------------------------------------------------------------
 ! HELP-FC1:USAGE: flang-new
 ! HELP-FC1-EMPTY:
 ! HELP-FC1-NEXT:OPTIONS:
+! HELP-FC1-NEXT: -E        Only run the preprocessor
 ! HELP-FC1-NEXT: -help     Display available options
 ! HELP-FC1-NEXT: -o <file> Write output to <file>
 ! HELP-FC1-NEXT: --version Print version information
 
 !---------------
 ! EXPECTED ERROR
 !---------------
 ! ERROR: error: unknown argument '-helps'; did you mean '-help'

flang/test/Frontend/Inputs/hello-world.c

+int main() { return 0; }

flang/test/Frontend/print-preprocess-C-file.f90

+! Test preprocessing for C files using Flang driver
+
+! REQUIRES: new-flang-driver
+
+!--------------------------
+! FLANG DRIVER (flang-new)
+!--------------------------
+! RUN: not %flang-new -E %S/Inputs/hello-world.c  2>&1 | FileCheck %s
+
+!-----------------------
+! EXPECTED OUTPUT
+!-----------------------
+! CHECK: error: unknown integrated tool '-cc1'. Valid tools include '-fc1'.

flang/test/Frontend/print-preprocessed-file.f90

+! Test printpreprocessed action
+
+! REQUIRES: new-flang-driver
+
+!--------------------------
+! FLANG DRIVER (flang-new)
+!--------------------------
+! RUN: %flang-new -E %s  2>&1 | FileCheck %s
+
+!-----------------------------------------
+!   FRONTEND FLANG DRIVER (flang-new -fc1)
+!-----------------------------------------
+! RUN: %flang-new -fc1 -E %s  2>&1 | FileCheck %s
+
+
+!-----------------------
+! EXPECTED OUTPUT
+!-----------------------
+! flang-new -E  %s
+! CHECK:program a
+! CHECK-NOT:program b
+! CHECK-NEXT:x = 1
+! CHECK-NEXT:write(*,*) x
+! CHECK-NEXT:end
+
+! Preprocessed-file.F:
+#define NEW
+#ifdef NEW
+  program A
+#else
+  program B
+#endif
+    x = 1
+    write(*,*) x
+  end

flang/unittests/Frontend/CMakeLists.txt

 add_flang_unittest(FlangFrontendTests
   CompilerInstanceTest.cpp
   InputOutputTest.cpp
+  PrintPreprocessedTest.cpp
 )
 
 target_link_libraries(FlangFrontendTests
   PRIVATE
   clangBasic
   clangFrontend
   flangFrontend
   flangFrontendTool
   FortranParser
-  )

flang/unittests/Frontend/PrintPreprocessedTest.cpp

+//===- unittests/Frontend/PrintPreprocessedTest.cpp  FrontendAction tests--===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "gtest/gtest.h"
+#include "flang/Frontend/CompilerInstance.h"
+#include "flang/Frontend/FrontendOptions.h"
+#include "flang/FrontendTool/Utils.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace Fortran::frontend;
+
+namespace {
+
+TEST(FrontendAction, PrintPreprocessedInput) {
+  std::string inputFile = "test-file.f";
+  std::error_code ec;
+
+  // 1. Create the input file for the file manager
+  // AllSources (which is used to manage files inside every compiler instance),
+  // works with paths. This means that it requires a physical file. Create one.
+  std::unique_ptr<llvm::raw_fd_ostream> os{
+      new llvm::raw_fd_ostream(inputFile, ec, llvm::sys::fs::OF_None)};
+  if (ec)
+    FAIL() << "Fail to create the file need by the test";
+
+  // Populate the input file with the pre-defined input and flush it.
+  *(os) << "! test-file.F:\n"
+        << "#ifdef NEW\n"
+        << "  Program A \n"
+        << "#else\n"
+        << "  Program B\n"
+        << "#endif";
+  os.reset();
+
+  // Get the path of the input file
+  llvm::SmallString<64> cwd;
+  if (std::error_code ec = llvm::sys::fs::current_path(cwd))
+    FAIL() << "Failed to obtain the current working directory";
+  std::string testFilePath(cwd.c_str());
+  testFilePath += "/" + inputFile;
+
+  // 2. Prepare the compiler (CompilerInvocation + CompilerInstance)
+  CompilerInstance compInst;
+  compInst.CreateDiagnostics();
+  auto invocation = std::make_shared<CompilerInvocation>();
+  invocation->frontendOpts().programAction_ = PrintPreprocessedInput;
+
+  compInst.set_invocation(std::move(invocation));
+  compInst.frontendOpts().inputs_.push_back(
+      FrontendInputFile(testFilePath, Language::Fortran));
+
+  // 3. Set-up the output stream. Using output buffer wrapped as an output
+  // stream, as opposed to an actual file (or a file descriptor).
+  llvm::SmallVector<char, 256> outputFileBuffer;
+  std::unique_ptr<llvm::raw_pwrite_stream> outputFileStream(
+      new llvm::raw_svector_ostream(outputFileBuffer));
+  compInst.set_outputStream(std::move(outputFileStream));
+
+  // 4. Run the earlier defined FrontendAction
+  bool success = ExecuteCompilerInvocation(&compInst);
+
+  // 5. Validate the expected output
+  EXPECT_TRUE(success);
+  EXPECT_TRUE(!outputFileBuffer.empty());
+  EXPECT_TRUE(
+      llvm::StringRef(outputFileBuffer.data()).startswith("program b\n"));
+
+  // 6. Clear the input and the output files. Since we used an output buffer,
+  // there are no physical output files to delete.
+  llvm::sys::fs::remove(inputFile);
+  compInst.ClearOutputFiles(/*EraseFiles=*/true);
+}
+} // namespace