[flang] Produce proper "preprocessor output" for -E option

[klausler]

Summary

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.

The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.

Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).

An effort has been made to preserve original alphabetic character case
and source indentation.

Add -P and -fno-reformat to the new drivers.

Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.

[klausler]

Update code to match commit comments.

[awarzynski]

Thank you for submitting this. It will unblock https://gitlab.kitware.com/cmake/cmake/-/issues/22387, which is quite an important milestone for Flang.

With this change, tests in https://github.com/llvm/llvm-project/tree/main/flang/test/Preprocessing require re-formatting or they will start failing.

Also, could a regression test be added that fails without your change? For example:

! Verify that the output from `-E` is valid fixed-form source. See
! https://bugs.llvm.org/show_bug.cgi?id=50993.

! RUN: %flang_fc1 -E %s 2>&1 | %flang_fc1 -fsyntax-only -ffixed-form 2>&1 | FileCheck %s --allow-empty

! CHECK-NOT: error
! CHECK-NOT: warning

! https://bugs.llvm.org/show_bug.cgi?id=51219
! CHECK-NOT: Character in fixed-form label field must be a digit

      PROGRAM HELLO
        write(*, *), "hello, world!"
      END PROGRAM

Feel free to edit and re-use this one ^^^. I also created https://bugs.llvm.org/show_bug.cgi?id=5121.9. Otherwise that final CHECK-NOT above is confusing.

[klausler]

I was hoping to get a code review. Tests will be added before submission.

The link you gave to bugzilla does not work.

[klausler]

In D106727#2904410, @awarzynski wrote:

With this change, tests in https://github.com/llvm/llvm-project/tree/main/flang/test/Preprocessing require re-formatting or they will start failing.

They all pass without change.

[awarzynski]

In D106727#2904725, @klausler wrote:

I was hoping to get a code review. Tests will be added before submission.

I read it and it made a lot of sense to me. It's fairly straightforward, like you suggested it would. The commit message makes it clear _what_ and _why_ is happening and that's consistent with what the code does. Glad to see -fdebug-dump-cooked-chars and -P being introduced. The latter is consistent with gfortran, which is important for us. This change should be sufficient for us to make progress with CMake.

I downloaded and tested your patch - hence my remark about the tests (they failed for me). Also, extra tests would really help to identify potential corner cases. Anyway, I scanned the implementation again and left a few inline comments.

The link you gave to bugzilla does not work.

Sorry, my bad: https://bugs.llvm.org/show_bug.cgi?id=51219.

flang/include/flang/Parser/parsing.h

40

[nit] Why not preprocess only? This is triggered by -E and does a bit more than just prescanning. But I don't mind.

flang/lib/Parser/parsing.cpp

104

This should be Fortran::parser::Options::fixedFormColumns rather than hard-coded 72.

136

Could you add an assert here assert(position->line > sourceLine && "sourceLine exceeded the actual source line")? There are not too many comments here and asserts can be very informative.

138

Why 10 and why make this case special?

149

This will only make sense for fixed-form, right?

[awarzynski]

In D106727#2904864, @klausler wrote:

They all pass without change.

Yes, but with this patch they fail. For example, https://github.com/llvm/llvm-project/blob/main/flang/test/Preprocessing/pp001.F:

bin/f18 -E /llvm-project/flang/test/Preprocessing/pp001.F
!dir$ fixed
#line "/llvm-project/flang/test/Preprocessing/pp001.F" 4
      integer, parameter :: KWM = 666

      if (777 .eq. 777) then
        print *, 'pp001.F yes'
      else
        print *, 'pp001.F no: ',777
      end if
      end

i.e., spaces in if (777 .eq. 777) then are preserved and hence ! CHECK: if(777.eq.777)then is no longer valid. This is the output that I get without your change:

integer,parameter::kwm=666
if(777.eq.777)then
print*,'pp001.F yes'
else
print*,'pp001.F no: ',777
endif
end

[klausler]

flang/include/flang/Parser/parsing.h

40

From the perspective of the Parsing module, we're just going to run the prescanner. F18 has no preprocessing phase; there is no way to just run its preprocessor. If there's a better accurate name that you would prefer, perhaps "preprocessedOutputOnly", let me know.

flang/lib/Parser/parsing.cpp

104

There's no guarantee that a later Fortran compilation of the -E output will also be using a nonstandard fixed form statement length. 72 is safer.

136

That case could arise in some future where f18 honors #line directives in its input.

138

Emitting a few newlines can be more readable and occupy fewer bytes than the equivalent #line directive would have required. The 10 is the number of bytes in a #line 123 directive (with newline).

149

The output is always valid fixed form Fortran.

[awarzynski]

flang/include/flang/Parser/parsing.h

40

prescanAndReformat?

flang/lib/Parser/parsing.cpp

104

Makes sense. Could you add a comment then? Otherwise it's just a magic number. A comment will prevent folks (including my future self) from changing this to Fortran::parser::Options::fixedFormColumns.

138

[nit] A non-magic number would be great (e.g. unsigned newLineCharLitmit = 10). Or a comment.

155

Non-digit in the first 6 columns (fixed form) is an error, right? Why not assert or issue a diagnostic here? This could hide a potential bug.

[klausler]

flang/lib/Parser/parsing.cpp

155

It would be an error if we were examining the original source file, yes, but this code runs later in compilation when we are reformatting the cooked character stream, from which excess spaces have been removed by prescanning.

[awarzynski]

flang/lib/Parser/parsing.cpp

155

Ah, so ch is assumed to be a valid character here. That wasn't clear.

A comment for this case would be appreciated (similar to e.g. // Wrap long lines in a portable fashion, perhaps // Add whitespaces in the first 6 columns to turn this into a valid fixed-form file?).

[klausler]

flang/include/flang/Parser/parsing.h

40

Whatever you like; will do.

[klausler]

Add new/newly supported options to new driver.

Update options in tests.

Address review comments.

[awarzynski]

Thank you for updating the new driver and fixing the tests. Everything builds fine and tests pass. Tested with the new driver. Since you are adding this in the new driver too, I suggest that for consistency sake we rename -fdebug-dump-cooked-chars . Let me elaborate.

All options in the form of -fdebug-dump-<something> correspond to an action and have a dedicated instance of FrontendAction (in FrontendActions.h). -fdebug-dump-cooked-chars is a feature option that configures -E (i.e. preprocessing) rather than an action in itself. Indeed, you tweaked PrintPreprocessedAction rather than created a new action, which makes a lot of sense. As -fdebug-dump-cooked-chars is not an action, there are a few changes here that are not required (see inline comments). As for the spelling itself, -fdebug-dump-cooked-chars will not dump anything on its own. It needs to be paired with -E. So the name is a bit misleading. Perhaps we could have something like -fcooked-only or -fno-reformat? Naming is hard, I have no particular preferences.

Apologies for not bringing this up earlier. In f18, we don't differentiate between "action" options and "feature" options, so it doesn't matter that much. But in flang-new it becomes more relevant. Let me know if you'd like me to go into more detail with respect to action/non-action flags.

clang/include/clang/Driver/Options.td

4535–4536

This is the decision point (action vs non-action option):

• Group<Preprocessor_Group> --> -fdebug-dump-cooked-chars is a preprocessor flag (i.e. tweaks the behavior of -E)
• Group<Action_Group> --> -fdebug-dump-cooked-chars is an action flag

Just highlighting. I think that Preprocessor_Group is exactly what we want here.

flang/include/flang/Frontend/FrontendOptions.h

31–33

Not needed, -fdebug-dump-cooked-chars is not an action.

255–276

Do you think that we need accessors and mutators here? We've kept FrontendOptions rather basic - it's just a plain aggregate. There's no intention to make it any more "clever". Keeping everything "public" allows us to have a rather basic and concise interface.

If we do add member methods for noLineDirectives_ and noLineDirectives_, then perhaps we should do it for all member variables? But then we'll bloat this class.

Also, we've been using bitfields rather than bools for this class to keep it slim. Could we stick to bitfields for consistency?

flang/lib/Frontend/CompilerInvocation.cpp

122–124

In this switch statement we only decide _which_ action to run. In this case, it's PrintPreprocessedInput.

Preprocessor options are set in parsePreprocessorArgs. Could you move this code there?

126–127

This case is not needed, -fdebug-dump-cooked-chars is not an action.

flang/lib/Parser/parsing.cpp

57–58

This is nly required for fixed-form, but not for free-form. That's fine, but it's worth adding a comment that this is intentional. For example:

// Make sure that the first 6 columns are formatted correctly. We focus on the standard Fortran fixed form, which is also valid free form.

My future self will be grateful.

flang/test/Preprocessing/dash-E.F90

2

How about statement labels? That's still not tested.

[klausler]

flang/include/flang/Frontend/FrontendOptions.h

31–33

It used to be so in my first patch. Removed.

255–276

I put them there because you're using names in that class with terminal underscores, and that's a convention in f18 (and google3) meant only for private data members; I didn't want to perpetuate that error. If everything here is meant to be public, this should be a struct whose members are all data and all public.

Bitfields can't have default initializers, which is the best way to initialize these things; otherwise the default value is separated from the declaration and possibly in another file.

[klausler]

As for the spelling itself, -fdebug-dump-cooked-chars will not dump anything on its own. It needs to be paired with -E. So the name is a bit misleading. Perhaps we could have something like -fcooked-only or -fno-reformat? Naming is hard, I have no particular preferences.

Tell me what you want, and I'll change the patch to use it. Otherwise I'll have to iterate.

[klausler]

Address review comments (except for renaming -fdebug-dump-cooked-chars, need to know new name for that)

[klausler]

Fix new test's RUN line; add --strict-whitespace to FileCheck.

[awarzynski]

In D106727#2910910, @klausler wrote:

As for the spelling itself, -fdebug-dump-cooked-chars will not dump anything on its own. It needs to be paired with -E. So the name is a bit misleading. Perhaps we could have something like -fcooked-only or -fno-reformat? Naming is hard, I have no particular preferences.

Tell me what you want, and I'll change the patch to use it. Otherwise I'll have to iterate.

-fno-reformat, please.

flang/include/flang/Frontend/FrontendOptions.h

255–276

I put them there because you're using names in that class with terminal underscores, and that's a convention in f18 (and google3) meant only for private data members; I didn't want to perpetuate that error. If everything here is meant to be public, this should be a struct whose members are all data and all public.

Ack. Fixed in https://reviews.llvm.org/D107062

Bitfields can't have default initializers, which is the best way to initialize these things; otherwise the default value is separated from the declaration and possibly in another file.

Sounds like each approach has its pros and cons. I would still prefer consistency and to rely on the constructor to initialise everything (which is nearby, in the same file). If you are strongly in favour bools, then let's update all fields at the same time. Preferably in a dedicated patch.

[awarzynski]

clang/include/clang/Driver/Options.td

4540–4541

In this group, all options have the following flags set:

let Flags = [FC1Option, FlangOnlyOption] in {

This means that we only support them in the frontend driver (FC1Option). Frontend-only options should be skipped in clang/lib/Driver/ToolChains/Flang.cpp (the bridge between the compiler driver and the frontend driver). This way, flang-new will ignore it:

bin/flang-new  -E -P  -fdebug-dump-cooked-chars  file.f
flang-new: warning: argument unused during compilation: '-fdebug-dump-cooked-chars'

Users can still use it with -Xflang, e.g.

flang-new -E -Xflang -fdebug-dump-cooked-chars file.f

This way, we avoid exposing all of the frontend options in the compiler driver mode.

clang/lib/Driver/ToolChains/Flang.cpp

43

You can delete options::OPT_fdebug_dump_cooked_chars from here. It won't be supported by the compiler driver, but that's fine. It's mostly for compiler developers, so it's fine to restrict it to the frontend driver. You can still pass it to flang-new -fc1 with -Xflang.

As for -P, could you add it to https://github.com/llvm/llvm-project/blob/main/flang/test/Driver/frontend-forwarding.f90?

[awarzynski]

flang/lib/Frontend/CompilerInvocation.cpp

378–379

I would replace this comment with a diagnostic, e.g. :

const unsigned diagID = diags.getCustomDiagID(clang::DiagnosticsEngine::Warning, "'-P' will be ignored ('-P' and '-fno-reformat' are incompatible)");
diags.Report(diagID);
}

This way, the driver becomes a bit more friendly towards end-users. You will have to update this method a tiny bit. Here's a patch with a test (it's a small change):

diff --git a/flang/lib/Frontend/CompilerInvocation.cpp b/flang/lib/Frontend/CompilerInvocation.cpp
index 04905f103daf..3d38d6dba3ff 100644
--- a/flang/lib/Frontend/CompilerInvocation.cpp
+++ b/flang/lib/Frontend/CompilerInvocation.cpp
@@ -342,8 +342,10 @@ static std::string getIntrinsicDir() {
 ///
 /// \param [in] opts The preprocessor options instance
 /// \param [out] args The list of input arguments
-static void parsePreprocessorArgs(
-    Fortran::frontend::PreprocessorOptions &opts, llvm::opt::ArgList &args) {
+static bool parsePreprocessorArgs(Fortran::frontend::PreprocessorOptions &opts,
+    llvm::opt::ArgList &args, clang::DiagnosticsEngine &diags) {
+  unsigned numErrorsBefore = diags.getNumErrors();
+
   // Add macros from the command line.
   for (const auto *currentArg : args.filtered(
            clang::driver::options::OPT_D, clang::driver::options::OPT_U)) {
@@ -372,13 +374,24 @@ static void parsePreprocessorArgs(
         ? PPMacrosFlag::Include
         : PPMacrosFlag::Exclude;

+
   if (args.hasArg(clang::driver::options::OPT_fdebug_dump_cooked_chars)) {
     opts.set_dumpCookedChars(true);
-  } else if (args.hasArg(clang::driver::options::OPT_P)) {
-    // If both -P and -fdebug-dump-cooked-chars appear, -P is
-    // unclaimed and will elicit a warning
+  }
+
+  if (args.hasArg(clang::driver::options::OPT_P)) {
     opts.set_noLineDirectives(true);
   }
+
+  if (opts.noLineDirectives() && opts.dumpCookedChars()) {
+    const unsigned diagID =
+        diags.getCustomDiagID(clang::DiagnosticsEngine::Warning,
+            "'-P' will be ignored ('-P' and '-fno-reformat' are incompatible)");
+    diags.Report(diagID);
+    opts.set_noLineDirectives(false);
+  }
+
+  return diags.getNumErrors() == numErrorsBefore;
 }

 /// Parses all semantic related arguments and populates the variables
@@ -541,7 +554,7 @@ bool CompilerInvocation::CreateFromArgs(CompilerInvocation &res,
   }

   success &= ParseFrontendArgs(res.frontendOpts(), args, diags);
-  parsePreprocessorArgs(res.preprocessorOpts(), args);
+  success &= parsePreprocessorArgs(res.preprocessorOpts(), args, diags);
   success &= parseSemaArgs(res, args, diags);
   success &= parseDialectArgs(res, args, diags);
   success &= parseDiagArgs(res, args, diags);
diff --git a/flang/test/Driver/pp-flags-warning.f90 b/flang/test/Driver/pp-flags-warning.f90
new file mode 100644
index 000000000000..5fbf3a1574a5
--- /dev/null
+++ b/flang/test/Driver/pp-flags-warning.f90
@@ -0,0 +1,8 @@
+! REQUIRES: new-flang-driver
+
+! RUN: %flang_fc1 -E -P -fdebug-dump-cooked-chars %s 2>&1 | FileCheck %s
+
+! CHECK: warning: '-P' will be ignored ('-P' and '-fno-reformat' are incompatible)
+
+program hello
+end

Feel free to use this (with or without edits).

[klausler]

flang/include/flang/Frontend/FrontendOptions.h

255–276

They've been moved to PreprocessorOptions.

flang/lib/Frontend/CompilerInvocation.cpp

378–379

Now that -fno-reformat is a developer-only flag, this is less important. Changing to accept both. The output of -fno-reformat has no #line directives so the -P is at worst redundant when both appear.

[klausler]

Rename original -fdebug-dump-cooked-chars to -fno-reformat, available only in the frontend driver.

[awarzynski]

Thank you for all the comments in EmitPreprocessedSource! I've left a couple of final suggestions, but nothing major.

flang/include/flang/Frontend/PreprocessorOptions.h

47–68

Could you make noReformat and noLineDirective public (for consistency with other member variables here)? If you have a strong preference for private members, then all of them should be private.

Btw, I've updated PreprocessorOptions in https://reviews.llvm.org/D107062 so that it is consistent with FrontendOptions.

flang/lib/Parser/parsing.cpp

113

If you insert continue after ++sourceLine, you can avoid the following else block. That's also one level of indentation less, which makes things easier to read.

[klausler]

flang/lib/Parser/parsing.cpp

113

Adding gotos, however spelled, tends to make code harder to follow than simple nested structures. No.

[klausler]

Address final(?) comments

[awarzynski]

LGTM. This patch is much appreciated and thank you for addressing my comments.

flang/lib/Parser/parsing.cpp

113

continue is not goto. Also: https://llvm.org/docs/CodingStandards.html#use-early-exits-and-continue-to-simplify-code.

[klausler]

Revert some test changes; instead of adding "-fno-reformat" to most tests using -E, apply a mix of updated test results with some use of -P where necessary.

clang/include/clang/Driver/Options.td

 def O : Joined<["-"], "O">, Group<O_Group>, Flags<[CC1Option]>;
 def O_flag : Flag<["-"], "O">, Flags<[CC1Option]>, Alias<O>, AliasArgs<["1"]>;
 def Ofast : Joined<["-"], "Ofast">, Group<O_Group>, Flags<[CC1Option]>;
-def P : Flag<["-"], "P">, Flags<[CC1Option]>, Group<Preprocessor_Group>,
+def P : Flag<["-"], "P">, Flags<[CC1Option,FlangOption,FC1Option]>, Group<Preprocessor_Group>,
   HelpText<"Disable linemarker output in -E mode">,
   MarshallingInfoNegativeFlag<PreprocessorOutputOpts<"ShowLineMarkers">>;
 def Qy : Flag<["-"], "Qy">, Flags<[CC1Option]>,

   HelpText<"Measure the parse tree">;
 def fdebug_pre_fir_tree : Flag<["-"], "fdebug-pre-fir-tree">, Group<Action_Group>,
   HelpText<"Dump the pre-FIR tree">;
-def fdebug_module_writer : Flag<["-"],"fdebug-module-writer">, 
+def fdebug_module_writer : Flag<["-"],"fdebug-module-writer">,
   HelpText<"Enable debug messages while writing module files">;
 def fget_symbols_sources : Flag<["-"], "fget-symbols-sources">, Group<Action_Group>,
   HelpText<"Dump symbols and their source code locations">;

 def fno_analyzed_objects_for_unparse : Flag<["-"],
   "fno-analyzed-objects-for-unparse">,  Group<f_Group>,
   HelpText<"Do not use the analyzed objects when unparsing">;
-
+def fno_reformat : Flag<["-"], "fno-reformat">, Group<Preprocessor_Group>,
+  HelpText<"Dump the cooked character stream in -E mode">;
 }
 
 //===----------------------------------------------------------------------===//

clang/lib/Driver/ToolChains/Flang.cpp

 
 void Flang::AddPreprocessingOptions(const ArgList &Args,
                                     ArgStringList &CmdArgs) const {
-  Args.AddAllArgs(CmdArgs, {options::OPT_D, options::OPT_U, options::OPT_I,
-                            options::OPT_cpp, options::OPT_nocpp});
+  Args.AddAllArgs(CmdArgs,
+                  {options::OPT_P, options::OPT_D, options::OPT_U,
+                   options::OPT_I, options::OPT_cpp, options::OPT_nocpp});
 }
 
 void Flang::AddOtherOptions(const ArgList &Args, ArgStringList &CmdArgs) const {

flang/include/flang/Frontend/FrontendOptions.h

   /// -test-io mode
   InputOutputTest,
 
-  /// -E mode.
+  /// -E mode
   PrintPreprocessedInput,
 
   /// -fsyntax-only

flang/include/flang/Frontend/PreprocessorOptions.h

 
   PPMacrosFlag macrosFlag_ = PPMacrosFlag::Unknown;
 
+  // -P: Suppress #line directives in -E output
+  bool noLineDirectives{false};
+
+  // -fno-reformat: Emit cooked character stream as -E output
+  bool noReformat{false};
+
 public:
   PreprocessorOptions() {}
 

flang/include/flang/Parser/parsing.h

   bool isModuleFile{false};
   bool needProvenanceRangeToCharBlockMappings{false};
   Fortran::parser::Encoding encoding{Fortran::parser::Encoding::UTF_8};
+  bool prescanAndReformat{false}; // -E
 };
 
 class Parsing {

   bool consumedWholeFile() const { return consumedWholeFile_; }
   const char *finalRestingPlace() const { return finalRestingPlace_; }
   AllCookedSources &allCooked() { return allCooked_; }
+  const AllCookedSources &allCooked() const { return allCooked_; }
   Messages &messages() { return messages_; }
   std::optional<Program> &parseTree() { return parseTree_; }
 
   const CookedSource &cooked() const { return DEREF(currentCooked_); }
 
   const SourceFile *Prescan(const std::string &path, Options);
+  void EmitPreprocessedSource(
+      llvm::raw_ostream &, bool lineDirectives = true) const;
   void DumpCookedChars(llvm::raw_ostream &) const;
   void DumpProvenance(llvm::raw_ostream &) const;
   void DumpParsingLog(llvm::raw_ostream &) const;

flang/lib/Frontend/CompilerInvocation.cpp

         (currentArg->getOption().matches(clang::driver::options::OPT_cpp))
         ? PPMacrosFlag::Include
         : PPMacrosFlag::Exclude;
+
+  opts.noReformat = args.hasArg(clang::driver::options::OPT_fno_reformat);
+  opts.noLineDirectives = args.hasArg(clang::driver::options::OPT_P);
 }
 
 /// Parses all semantic related arguments and populates the variables

flang/lib/Frontend/FrontendActions.cpp

   std::string buf;
   llvm::raw_string_ostream outForPP{buf};
 
-  // Run the preprocessor
+  // Format or dump the prescanner's output
   CompilerInstance &ci = this->instance();
-  ci.parsing().DumpCookedChars(outForPP);
+  if (ci.invocation().preprocessorOpts().noReformat) {
+    ci.parsing().DumpCookedChars(outForPP);
+  } else {
+    ci.parsing().EmitPreprocessedSource(
+        outForPP, !ci.invocation().preprocessorOpts().noLineDirectives);
+  }
 
   // If a pre-defined output stream exists, dump the preprocessed content there
   if (!ci.IsOutputStreamNull()) {

     return;
   }
 
-  // Print diagnostics from the preprocessor
+  // Print diagnostics from the prescanner
   ci.parsing().messages().Emit(llvm::errs(), ci.allCookedSources());
 
   // Create a file and save the preprocessed output there

     (*os) << outForPP.str();
   } else {
     llvm::errs() << "Unable to create the output file\n";
-    return;
   }
 }
 

flang/lib/Frontend/FrontendOptions.cpp

 //===----------------------------------------------------------------------===//
 
 #include "flang/Frontend/FrontendOptions.h"
-#include "flang/Evaluate/expression.h"
 
 using namespace Fortran::frontend;
 

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

   switch (ak) {
   case InputOutputTest:
     return std::make_unique<InputOutputTestAction>();
-    break;
   case PrintPreprocessedInput:
     return std::make_unique<PrintPreprocessedAction>();
-    break;
   case ParseSyntaxOnly:
     return std::make_unique<ParseSyntaxOnlyAction>();
   case EmitObj:
     return std::make_unique<EmitObjAction>();
-    break;
   case DebugUnparse:
     return std::make_unique<DebugUnparseAction>();
-    break;
   case DebugUnparseNoSema:
     return std::make_unique<DebugUnparseNoSemaAction>();
-    break;
   case DebugUnparseWithSymbols:
     return std::make_unique<DebugUnparseWithSymbolsAction>();
-    break;
   case DebugDumpSymbols:
     return std::make_unique<DebugDumpSymbolsAction>();
-    break;
   case DebugDumpParseTree:
     return std::make_unique<DebugDumpParseTreeAction>();
-    break;
   case DebugDumpParseTreeNoSema:
     return std::make_unique<DebugDumpParseTreeNoSemaAction>();
-    break;
   case DebugDumpAll:
     return std::make_unique<DebugDumpAllAction>();
-    break;
   case DebugDumpProvenance:
     return std::make_unique<DebugDumpProvenanceAction>();
-    break;
   case DebugDumpParsingLog:
     return std::make_unique<DebugDumpParsingLogAction>();
-    break;
   case DebugMeasureParseTree:
     return std::make_unique<DebugMeasureParseTreeAction>();
-    break;
   case DebugPreFIRTree:
     return std::make_unique<DebugPreFIRTreeAction>();
-    break;
   case GetDefinition:
     return std::make_unique<GetDefinitionAction>();
-    break;
   case GetSymbolsSources:
     return std::make_unique<GetSymbolsSourcesAction>();
-    break;
   case InitOnly:
     return std::make_unique<InitOnlyAction>();
-    break;
   default:
     break;
     // TODO:
-    // case RunPreprocessor:
     // case ParserSyntaxOnly:
     // case EmitLLVM:
     // case EmitLLVMOnly:

flang/lib/Parser/parsing.cpp

   return sourceFile;
 }
 
+void Parsing::EmitPreprocessedSource(
+    llvm::raw_ostream &out, bool lineDirectives) const {
+  const SourceFile *sourceFile{nullptr};
+  int sourceLine{0};
+  int column{1};
+  bool inDirective{false};
+  bool inContinuation{false};
+  const AllSources &allSources{allCooked().allSources()};
+  for (const char &atChar : cooked().AsCharBlock()) {
+    char ch{atChar};
+    if (ch == '\n') {
+      out << '\n'; // TODO: DOS CR-LF line ending if necessary
+      column = 1;
+      inDirective = false;
+      inContinuation = false;
+      ++sourceLine;
+    } else {
+      if (ch == '!') {
+        // Other comment markers (C, *, D) in original fixed form source
+        // input card column 1 will have been deleted or normalized to !,
+        // which signifies a comment (directive) in both source forms.
+        inDirective = true;
+      }
+      auto provenance{cooked().GetProvenanceRange(CharBlock{&atChar, 1})};
+      std::optional<SourcePosition> position{provenance
+              ? allSources.GetSourcePosition(provenance->start())
+              : std::nullopt};
+      if (lineDirectives && column == 1 && position) {
+        if (&position->file != sourceFile) {
+          out << "#line \"" << position->file.path() << "\" " << position->line
+              << '\n';
+        } else if (position->line != sourceLine) {
+          if (sourceLine < position->line &&
+              sourceLine + 10 >= position->line) {
+            // Emit a few newlines to catch up when they'll likely
+            // require fewer bytes than a #line directive would have
+            // occupied.
+            while (sourceLine++ < position->line) {
+              out << '\n';
+            }
+          } else {
+            out << "#line " << position->line << '\n';
+          }
+        }
+        sourceFile = &position->file;
+        sourceLine = position->line;
+      }
+      if (column > 72) {
+        // Wrap long lines in a portable fashion that works in both
+        // of the Fortran source forms.  The first free-form continuation
+        // marker ("&") lands in column 73, which begins the card commentary
+        // field of fixed form, and the second one is put in column 6,
+        // where it signifies fixed form line continuation.
+        // The standard Fortran fixed form column limit (72) is used
+        // for output, even if the input was parsed with a nonstandard
+        // column limit override option.
+        out << "&\n     &";
+        column = 7; // start of fixed form source field
+        ++sourceLine;
+        inContinuation = true;
+      } else if (!inDirective && ch != ' ' && (ch < '0' || ch > '9')) {
+        // Put anything other than a label or directive into the
+        // Fortran fixed form source field (columns [7:72]).
+        for (; column < 7; ++column) {
+          out << ' ';
+        }
+      }
+      if (!inContinuation && position && position->column <= 72 && ch != ' ') {
+        // Preserve original indentation
+        for (; column < position->column; ++column) {
+          out << ' ';
+        }
+      }
+      if (ch >= 'a' && ch <= 'z' && provenance && provenance->size() == 1) {
+        // Preserve original case
+        if (const char *orig{allSources.GetSource(*provenance)}) {
+          auto upper{static_cast<char>(ch + 'A' - 'a')};
+          if (*orig == upper) {
+            ch = upper;
+          }
+        }
+      }
+      out << ch;
+      ++column;
+    }
+  }
+}
+
 void Parsing::DumpCookedChars(llvm::raw_ostream &out) const {
   UserState userState{allCooked_, common::LanguageFeatureControl{}};
   ParseState parseState{cooked()};

flang/lib/Parser/provenance.cpp

 std::optional<SourcePosition> AllSources::GetSourcePosition(
     Provenance prov) const {
   const Origin &origin{MapToOrigin(prov)};
-  if (const auto *inc{std::get_if<Inclusion>(&origin.u)}) {
-    std::size_t offset{origin.covers.MemberOffset(prov)};
-    return inc->source.FindOffsetLineAndColumn(offset);
-  } else {
-    return std::nullopt;
-  }
+  return std::visit(
+      common::visitors{
+          [&](const Inclusion &inc) -> std::optional<SourcePosition> {
+            std::size_t offset{origin.covers.MemberOffset(prov)};
+            return inc.source.FindOffsetLineAndColumn(offset);
+          },
+          [&](const Macro &) {
+            return GetSourcePosition(origin.replaces.start());
+          },
+          [](const CompilerInsertion &) -> std::optional<SourcePosition> {
+            return std::nullopt;
+          },
+      },
+      origin.u);
 }
 
 std::optional<ProvenanceRange> AllSources::GetFirstFileProvenance() const {

       return result;
     }
   }
-  return nullptr;
+  return std::nullopt;
 }
 
 void AllCookedSources::Dump(llvm::raw_ostream &o) const {

flang/lib/Parser/token-sequence.h

   TokenSequence &ClipComment(bool skipFirst = false);
   const TokenSequence &CheckBadFortranCharacters(Messages &) const;
   void Emit(CookedSource &) const;
-  void Dump(llvm::raw_ostream &) const;
+  llvm::raw_ostream &Dump(llvm::raw_ostream &) const;
 
 private:
   std::size_t TokenBytes(std::size_t token) const {

flang/lib/Parser/token-sequence.cpp

   cooked.PutProvenanceMappings(provenances_);
 }
 
-void TokenSequence::Dump(llvm::raw_ostream &o) const {
+llvm::raw_ostream &TokenSequence::Dump(llvm::raw_ostream &o) const {
   o << "TokenSequence has " << char_.size() << " chars; nextStart_ "
     << nextStart_ << '\n';
   for (std::size_t j{0}; j < start_.size(); ++j) {
     o << '[' << j << "] @ " << start_[j] << " '" << TokenAt(j).ToString()
       << "'\n";
   }
+  return o;
 }
 
 Provenance TokenSequence::GetTokenProvenance(

flang/test/Driver/cpp-nocpp-command-line-macro.f90

 !-----------------
 ! EXPECTED OUTPUT
 !-----------------
-! UNDEFINED:program b
-! UNDEFINED-NOT:program a
+! UNDEFINED:program B
+! UNDEFINED-NOT:program A
 
-! DEFINED:program a
-! DEFINED-NOT:program b
+! DEFINED:program A
+! DEFINED-NOT:program B
 
 #ifdef X
 program X

flang/test/Driver/driver-help-hidden.f90

 ! CHECK-NEXT: -nocpp                 Disable predefined and command line preprocessor macros
 ! CHECK-NEXT: -o <file> Write output to <file>
 ! CHECK-NEXT: -pedantic              Warn on language extensions
+! CHECK-NEXT: -P                     Disable linemarker output in -E mode
 ! CHECK-NEXT: -std=<value>           Language standard to compile for
 ! CHECK-NEXT: -U <macro>             Undefine macro <macro>
 ! CHECK-NEXT: --version Print version information

flang/test/Driver/driver-help.f90

 ! HELP-NEXT: -nocpp                 Disable predefined and command line preprocessor macros
 ! HELP-NEXT: -o <file>              Write output to <file>
 ! HELP-NEXT: -pedantic              Warn on language extensions
+! HELP-NEXT: -P                     Disable linemarker output in -E mode
 ! HELP-NEXT: -std=<value>           Language standard to compile for
 ! HELP-NEXT: -U <macro>             Undefine macro <macro>
 ! HELP-NEXT: --version              Print version information

 ! HELP-FC1-NEXT: -flogical-abbreviations Enable logical abbreviations
 ! HELP-FC1-NEXT: -fno-analyzed-objects-for-unparse
 ! HELP-FC1-NEXT:                        Do not use the analyzed objects when unparsing
+! HELP-FC1-NEXT: -fno-reformat          Dump the cooked character stream in -E mode
 ! HELP-FC1-NEXT: -fopenacc              Enable OpenACC
 ! HELP-FC1-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-FC1-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.

 ! HELP-FC1-NEXT: -nocpp                 Disable predefined and command line preprocessor macros
 ! HELP-FC1-NEXT: -o <file>              Write output to <file>
 ! HELP-FC1-NEXT: -pedantic              Warn on language extensions
+! HELP-FC1-NEXT: -P                     Disable linemarker output in -E mode
 ! HELP-FC1-NEXT: -std=<value>           Language standard to compile for
 ! HELP-FC1-NEXT: -test-io               Run the InputOuputTest action. Use for development and testing only.
 ! HELP-FC1-NEXT: -U <macro>             Undefine macro <macro>

flang/test/Driver/escaped-backslash.f90

 !-----------------------------------------
 ! EXPECTED OUTPUT FOR ESCAPED BACKSLASHES
 !-----------------------------------------
-! ESCAPED:program backslash
+! ESCAPED:program Backslash
 ! ESCAPED-NEXT:New\\nline
 ! ESCAPED-NOT:New\nline
 
 !-------------------------------------------
 ! EXPECTED OUTPUT FOR UNESCAPED BACKSLASHES
 !-------------------------------------------
-! UNESCAPED:program backslash
+! UNESCAPED:program Backslash
 ! UNESCAPED-NEXT:New\nline
 ! UNESCAPED-NOT:New\\nline
 

flang/test/Driver/fixed-free-detection.f90

 ! Ensure the driver correctly switches between fixed and free form based on the file extension.
-! This test exploits the fact that the preprocessor treats white-spaces differently for free
+! This test exploits the fact that the prescanner treats whitespace differently for free
 ! and fixed form input files.
 
 !--------------------------
 ! FLANG DRIVER (flang)
 !--------------------------
-! RUN: %flang -E %S/Inputs/free-form-test.f90  2>&1 | FileCheck %s --check-prefix=FREEFORM
-! RUN: %flang -E %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=FIXEDFORM
-! RUN: %flang -E %S/Inputs/free-form-test.f90 %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=MULTIPLEFORMS
+! RUN: %flang -E -Xflang -fno-reformat %S/Inputs/free-form-test.f90  2>&1 | FileCheck %s --check-prefix=FREEFORM
+! RUN: %flang -E -Xflang -fno-reformat %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=FIXEDFORM
+! RUN: %flang -E -Xflang -fno-reformat %S/Inputs/free-form-test.f90 %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=MULTIPLEFORMS
 
 !-----------------------------------------
 ! FRONTEND FLANG DRIVER (flang_fc1)
 !-----------------------------------------
-! RUN: %flang_fc1 -E %S/Inputs/free-form-test.f90  2>&1 | FileCheck %s --check-prefix=FREEFORM
-! RUN: %flang_fc1 -E %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=FIXEDFORM
-! RUN: %flang_fc1 -E %S/Inputs/free-form-test.f90 %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=MULTIPLEFORMS
+! RUN: %flang_fc1 -E -fno-reformat %S/Inputs/free-form-test.f90  2>&1 | FileCheck %s --check-prefix=FREEFORM
+! RUN: %flang_fc1 -E -fno-reformat %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=FIXEDFORM
+! RUN: %flang_fc1 -E -fno-reformat %S/Inputs/free-form-test.f90 %S/Inputs/fixed-form-test.f  2>&1 | FileCheck %s --check-prefix=MULTIPLEFORMS
 
 !-------------------------------------
 ! EXPECTED OUTPUT FOR A FREE FORM FILE

flang/test/Driver/fixed-line-length.f90

 !--------------------------
 ! FLANG DRIVER (flang)
 !--------------------------
-! RUN: %flang -E %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=DEFAULTLENGTH
-! RUN: not %flang -E -ffixed-line-length=-2 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=NEGATIVELENGTH
-! RUN: not %flang -E -ffixed-line-length=3 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=INVALIDLENGTH
-! RUN: %flang -E -ffixed-line-length=none %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
-! RUN: %flang -E -ffixed-line-length=0 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
-! RUN: %flang -E -ffixed-line-length=13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
+! RUN: %flang -E -Xflang -fno-reformat %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=DEFAULTLENGTH
+! RUN: not %flang -E -Xflang -fno-reformat -ffixed-line-length=-2 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=NEGATIVELENGTH
+! RUN: not %flang -E -Xflang -fno-reformat -ffixed-line-length=3 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=INVALIDLENGTH
+! RUN: %flang -E -Xflang -fno-reformat -ffixed-line-length=none %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
+! RUN: %flang -E -Xflang -fno-reformat -ffixed-line-length=0 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
+! RUN: %flang -E -Xflang -fno-reformat -ffixed-line-length=13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
 
 !----------------------------------------
 ! FRONTEND FLANG DRIVER (flang -fc1)
 !----------------------------------------
-! RUN: %flang_fc1 -E %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=DEFAULTLENGTH
-! RUN: not %flang_fc1 -E -ffixed-line-length=-2 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=NEGATIVELENGTH
-! RUN: not %flang_fc1 -E -ffixed-line-length=3 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=INVALIDLENGTH
-! RUN: %flang_fc1 -E -ffixed-line-length=none %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
-! RUN: %flang_fc1 -E -ffixed-line-length=0 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
-! RUN: %flang_fc1 -E -ffixed-line-length=13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
+! RUN: %flang_fc1 -E -fno-reformat %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=DEFAULTLENGTH
+! RUN: not %flang_fc1 -E -fno-reformat -ffixed-line-length=-2 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=NEGATIVELENGTH
+! RUN: not %flang_fc1 -E -fno-reformat -ffixed-line-length=3 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=INVALIDLENGTH
+! RUN: %flang_fc1 -E -fno-reformat -ffixed-line-length=none %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
+! RUN: %flang_fc1 -E -fno-reformat -ffixed-line-length=0 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=UNLIMITEDLENGTH
+! RUN: %flang_fc1 -E -fno-reformat -ffixed-line-length=13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
 
 !-------------------------------------
 ! COMMAND ALIAS -ffixed-line-length-n
 !-------------------------------------
-! RUN: %flang -E -ffixed-line-length-13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
-! RUN: %flang_fc1 -E -ffixed-line-length-13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
+! RUN: %flang -E -Xflang -fno-reformat -ffixed-line-length-13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
+! RUN: %flang_fc1 -E -fno-reformat -ffixed-line-length-13 %S/Inputs/fixed-line-length-test.f  2>&1 | FileCheck %s --check-prefix=LENGTH13
 
 !-------------------------------------
 ! EXPECTED OUTPUT WITH DEFAULT LENGTH

flang/test/Driver/frontend-forwarding.f90

 ! RUN:     -fdefault-integer-8 \
 ! RUN:     -fdefault-real-8 \
 ! RUN:     -flarge-sizes \
+! RUN:     -P \
 ! RUN:   | FileCheck %s
 
+! CHECK: "-P"
 ! CHECK: "-finput-charset=utf-8"
 ! CHECK: "-fdefault-double-8"
 ! CHECK: "-fdefault-integer-8"

flang/test/Driver/include-header.f90

 !---------------------------------------------
 ! EXPECTED OUTPUT FOR A SINGLE INCLUDED FOLDER
 !--------------------------------------------
-! SINGLEINCLUDE:program maindirectoryone
-! SINGLEINCLUDE-NOT:program x
-! SINGLEINCLUDE-NOT:program b
-! SINGLEINCLUDE-NEXT:end
-! SINGLEINCLUDE-NEXT:program maindirectorytwo
-! SINGLEINCLUDE-NOT:program y
-! SINGLEINCLUDE-NOT:program c
+! SINGLEINCLUDE:program MainDirectoryOne
+! SINGLEINCLUDE-NOT:program X
+! SINGLEINCLUDE-NOT:program B
+! SINGLEINCLUDE:program MainDirectoryTwo
+! SINGLEINCLUDE-NOT:program Y
+! SINGLEINCLUDE-NOT:program C
 
 !-------------------------------------------------------
 ! EXPECTED OUTPUT FOR Inputs/ DIRECTORY SPECIFIED FIRST
 !-------------------------------------------------------
-! MAINDIRECTORY:program maindirectoryone
-! MAINDIRECTORY-NOT:program subdirectoryone
-! MAINDIRECTORY-NOT:program b
-! MAINDIRECTORY-NEXT:end
-! MAINDIRECTORY-NEXT:program maindirectorytwo
-! MAINDIRECTORY-NOT:program subdirectorytwo
-! MAINDIRECTORY-NOT:program c
+! MAINDIRECTORY:program MainDirectoryOne
+! MAINDIRECTORY-NOT:program SubDirectoryOne
+! MAINDIRECTORY-NOT:program B
+! MAINDIRECTORY:program MainDirectoryTwo
+! MAINDIRECTORY-NOT:program SubDirectoryTwo
+! MAINDIRECTORY-NOT:program C
 
 !------------------------------------------------------------------
 ! EXPECTED OUTPUT FOR Inputs/header-dir/ DIRECTORY SPECIFIED FIRST
 !------------------------------------------------------------------
-! SUBDIRECTORY:program subdirectoryone
-! SUBDIRECTORY-NOT:program maindirectoryone
-! SUBDIRECTORY-NOT:program b
-! SUBDIRECTORY-NEXT:end
-! SUBDIRECTORY-NEXT:program subdirectorytwo
-! SUBDIRECTORY-NOT:program maindirectorytwo
-! SUBDIRECTORY-NOT:program c
+! SUBDIRECTORY:program SubDirectoryOne
+! SUBDIRECTORY-NOT:program MainDirectoryOne
+! SUBDIRECTORY-NOT:program B
+! SUBDIRECTORY:program SubDirectoryTwo
+! SUBDIRECTORY-NOT:program MainDirectoryTwo
+! SUBDIRECTORY-NOT:program C
 
 ! include-test-one.f90
 #include <basic-header-one.h>

flang/test/Driver/input-from-stdin.f90

 !-------------------------
 ! EXPECTED OUTPUT for `-E`
 !-------------------------
-! PP-NOT-DEFINED: program b
-! PP-DEFINED: program a
+! PP-NOT-DEFINED: Program B
+! PP-DEFINED: Program A
 
 !-------------------------------
 ! EXPECTED OUTPUT for `-test-io`

flang/test/Driver/macro-def-undef.F90

 !--------------------------
 ! FLANG DRIVER (flang-new)
 !--------------------------
-! RUN: %flang -E %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
-! RUN: %flang -E -DX=A %s  2>&1 | FileCheck %s --check-prefix=DEFINED
-! RUN: %flang -E -DX=A -UX %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
+! RUN: %flang -E -P %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
+! RUN: %flang -E -P -DX=A %s  2>&1 | FileCheck %s --check-prefix=DEFINED
+! RUN: %flang -E -P -DX=A -UX %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
 
 !-----------------------------------------
 ! FRONTEND FLANG DRIVER (flang-new -fc1)
 !-----------------------------------------
-! RUN: %flang_fc1 -E %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
-! RUN: %flang_fc1 -E -DX=A %s  2>&1 | FileCheck %s --check-prefix=DEFINED
-! RUN: %flang_fc1 -E -DX -UX %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
+! RUN: %flang_fc1 -E -P %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
+! RUN: %flang_fc1 -E -P -DX=A %s  2>&1 | FileCheck %s --check-prefix=DEFINED
+! RUN: %flang_fc1 -E -P -DX -UX %s  2>&1 | FileCheck %s --check-prefix=UNDEFINED
 
 !--------------------------------------------
 ! EXPECTED OUTPUT FOR AN UNDEFINED MACRO
 !--------------------------------------------
-! UNDEFINED:program b
-! UNDEFINED-NOT:program x
-! UNDEFINED-NEXT:end
+! UNDEFINED:program B
+! UNDEFINED-NOT:program X
 
 !--------------------------------------------
 ! EXPECTED OUTPUT FOR MACRO 'X' DEFINED AS A
 !--------------------------------------------
-! DEFINED:program a
-! DEFINED-NOT:program b
-! DEFINED-NEXT:end
+! DEFINED:program A
+! DEFINED-NOT:program B
 
 #ifdef X
 program X

flang/test/Driver/macro-multiline.F90

 !--------------------------
 ! FLANG DRIVER (flang)
 !--------------------------
-! RUN: printf -- "-DX=A\\\\\nTHIS_SHOULD_NOT_EXIST_IN_THE_OUTPUT\n" | xargs %flang -E %s  2>&1 | FileCheck --strict-whitespace --match-full-lines %s
+! RUN: printf -- "-DX=A\\\\\nTHIS_SHOULD_NOT_EXIST_IN_THE_OUTPUT\n" | xargs %flang -E -P %s  2>&1 | FileCheck --strict-whitespace --match-full-lines %s
 
 !-----------------------------------------
 ! FRONTEND FLANG DRIVER (flang_fc1)
 !-----------------------------------------
-! RUN: printf -- "-DX=A\\\\\nTHIS_SHOULD_NOT_EXIST_IN_THE_OUTPUT\n" | xargs %flang_fc1 -E %s  2>&1 | FileCheck --strict-whitespace --match-full-lines %s
+! RUN: printf -- "-DX=A\\\\\nTHIS_SHOULD_NOT_EXIST_IN_THE_OUTPUT\n" | xargs %flang_fc1 -E -P %s  2>&1 | FileCheck --strict-whitespace --match-full-lines %s
 
 !-------------------------------
 ! EXPECTED OUTPUT FOR MACRO 'X'
 !-------------------------------
-! CHECK:start a end
+! CHECK:       START A END
 ! CHECK-NOT:THIS_SHOULD_NOT_EXIST_IN_THE_OUTPUT
 ! CHECK-NOT:this_should_not_exist_in_the_output
 
        START X END

flang/test/Parser/badlabel.f

-! RUN: %flang_fc1 -E %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -E -fno-reformat %s 2>&1 | FileCheck %s
 ! CHECK: Label digit is not in fixed-form label field
       1 continue
 ! CHECK: Label digit is not in fixed-form label field

flang/test/Preprocessing/assert.F90

-!RUN: %flang -E %s 2>&1 | FileCheck %s
+!RUN: %flang -E -Xflang -fno-reformat %s 2>&1 | FileCheck %s
 !CHECK: if(.not.(.true.)) error stop "assert(" // ".TRUE." // ") failed " // "at ""
 !CHECK-SAME: assert.F90"": " // "7"
 #define STR(x) #x

flang/test/Preprocessing/dash-E.F90

+! RUN: %flang -E %s 2>&1 | FileCheck --strict-whitespace %s
+!CHECK:       program Main
+program Main
+#define ADD(x,y) (x)+(y)
+!CHECK:       integer :: j = (1)+( 2)
+  integer :: j = ADD(1,&
+                     2)
+!CHECK:1     format('This is a very long output literal edit descriptor for a F&
+!CHECK:     &ORMAT statement, and it will require statement continuation.')
+1 format('This is a very long output literal edit descriptor for a FORMAT statement, and it will require statement continuation.')
+
+
+
+
+
+
+
+
+
+
+
+
+!CHECK: #line 25
+!CHECK:       end PROGRAM Main
+end PROGRAM Main

flang/test/Preprocessing/fixed-rescan.F

 ! RUN: %flang -E %s | FileCheck %s
-! CHECK: callbar
+! CHECK: call bar
 ! Ensure that rescanned lines after macro replacement are not
 ! misinterpreted as fixed-form comments when they start with C or D.
 #define foo bar

flang/test/Preprocessing/hollerith.f

-! RUN: %flang -E %s 2>&1 | FileCheck %s
+! RUN: %flang -E -Xflang -fno-reformat %s 2>&1 | FileCheck %s
 ! CHECK: character*1hi
 ! CHECK: dataa/1*1h /
 ! CHECK: datab/1*1h /

flang/test/Preprocessing/pp001.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
+! CHECK: if (777 .eq. 777) then
 * keyword macros
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp002.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm.eq.777)then
+! CHECK: if (KWM .eq. 777) then
 * #undef
       integer, parameter :: KWM = 777
 #define KWM 666

flang/test/Preprocessing/pp003.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * function-like macros
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp004.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm.eq.777)then
+! CHECK: if (kwm .eq. 777) then
 * KWMs case-sensitive
       integer, parameter :: KWM = 777
 #define KWM 666

flang/test/Preprocessing/pp005.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=777
+! CHECK: res = 777
 * KWM split across continuation, implicit padding
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp006.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=777
+! CHECK: res = 777
 * ditto, but with intervening *comment line
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp007.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=kwm
+! CHECK: res = KWM
 * KWM split across continuation, clipped after column 72
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp008.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=kwm
+! CHECK: res = K W M
 * KWM with spaces in name at invocation NOT replaced
       integer, parameter :: KWM = 777
 #define KWM 666

flang/test/Preprocessing/pp009.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call split across continuation, implicit padding
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp010.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * ditto, but with intervening *comment line
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp011.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=iflm(666)
+! CHECK: res = IFLM(666)
 * FLM call name split across continuation, clipped
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp012.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call name split across continuation
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp013.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call split between name and (
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp014.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call split between name and (, with intervening *comment
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp015.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call split between name and (, clipped
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp016.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call split between name and ( and in argument
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp017.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
+! CHECK: if (777 .eq. 777) then
 * KLM rescan
       integer, parameter :: KWM = 666, KWM2 = 667
 #define KWM2 777

flang/test/Preprocessing/pp018.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm2.eq.777)then
+! CHECK: if (KWM2 .eq. 777) then
 * KLM rescan with #undef (so rescan is after expansion)
       integer, parameter :: KWM2 = 777, KWM = 667
 #define KWM2 666
 #define KWM KWM2
 #undef KWM2
-      if (KWM .eq. 777) then
+      if (KWM  .eq. 777) then
         print *, 'pp018.F yes'
       else
         print *, 'pp018.F no: ', KWM

flang/test/Preprocessing/pp019.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM rescan
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp020.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((111)+666)
+! CHECK: res = ((111)+666)
 * FLM expansion of argument
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp021.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: ch='KWM'
-! CHECK: if(ch.eq.'KWM')then
+! CHECK: ch = 'KWM'
+! CHECK: if (ch .eq. 'KWM') then
 * KWM NOT expanded in 'literal'
 #define KWM 666
       character(len=3) :: ch

flang/test/Preprocessing/pp022.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: ch="KWM"
-! CHECK: if(ch.eq.'KWM')then
+! CHECK: ch = "KWM"
+! CHECK: if (ch .eq. 'KWM') then
 * KWM NOT expanded in "literal"
 #define KWM 666
       character(len=3) :: ch

flang/test/Preprocessing/pp023.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: ch=3hKWM
-! CHECK: if(ch.eq.'KWM')then
+! CHECK: ch = 3HKWM
+! CHECK: if (ch .eq. 'KWM') then
 * KWM NOT expanded in 9HHOLLERITH literal
 #define KWM 666
 #define HKWM 667

flang/test/Preprocessing/pp024.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: 100format(3hKWM)
-! CHECK: if(ch.eq.'KWM')then
+! CHECK: 100   format(3HKWM)
+! CHECK: if (ch .eq. 'KWM') then
 * KWM NOT expanded in Hollerith in FORMAT
 #define KWM 666
 #define HKWM 667

flang/test/Preprocessing/pp025.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=ikwm2z
+! CHECK: res = I KWM2 Z
 * KWM expansion is before token pasting due to fixed-form space removal
       integer, parameter :: IKWM2Z = 777
 #define KWM KWM2
       integer :: res
-      res = I KWM Z
+      res = I KWM  Z
       if (res .eq. 777) then
         print *, 'pp025.F yes'
       else

flang/test/Preprocessing/pp026.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((111)+666)
+! CHECK: res = ((111)+666)
 * ## token pasting works in FLM
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp027.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: kwm=666
-! CHECK: if(777.eq.777)then
+! CHECK: KWM = 666
+! CHECK: if (777 .eq. 777) then
 * #DEFINE works in fixed form
       integer, parameter :: KWM = 666
 #DEFINE KWM 777

flang/test/Preprocessing/pp028.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=kw
+! CHECK: res =                                                           KW
 * fixed-form clipping done before KWM expansion on source line
       integer, parameter :: KW = 777
 #define KWM 666

flang/test/Preprocessing/pp029.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
+! CHECK: if (777 .eq. 777) then
 * \ newline allowed in #define
       integer, parameter :: KWM = 666
 #define KWM 77\

flang/test/Preprocessing/pp030.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
+! CHECK: if (777 .eq. 777) then
 * /* C comment */ erased from #define
       integer, parameter :: KWM = 666
 #define KWM 777 /* C comment */

flang/test/Preprocessing/pp031.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777//ccomment.eq.777)then
-! CHECK: print*,'pp031.F no: ',777//ccomment
+! CHECK: if (777//Ccomment.eq.777)then
+! CHECK: print *, 'pp031.F no: ', 777//Ccomment
 *  // C++ comment NOT erased from #define
       integer, parameter :: KWM = 666
 #define KWM 777 // C comment

flang/test/Preprocessing/pp032.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
-! CHECK: print*,'pp032.F no: ',777
+! CHECK: if (777 .eq. 777) then
+! CHECK: print *, 'pp032.F no: ', 777
 * /* C comment */ \ newline erased from #define
       integer, parameter :: KWM = 666
 #define KWM 77/* C comment */\

flang/test/Preprocessing/pp033.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
-! CHECK: print*,'pp033.F no: ',777
+! CHECK: if (777 .eq. 777) then
+! CHECK: print *, 'pp033.F no: ', 777
 * /* C comment \ newline */ erased from #define
       integer, parameter :: KWM = 666
 #define KWM 77/* C comment \

flang/test/Preprocessing/pp034.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
-! CHECK: print*,'pp034.F no: ',777
+! CHECK: if (777 .eq. 777) then
+! CHECK: print *, 'pp034.F no: ', 777
 * \ newline allowed in name on KWM definition
       integer, parameter :: KWMC = 666
 #define KWM\

flang/test/Preprocessing/pp035.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777.eq.777)then
-! CHECK: print*,'pp035.F no: ',777
+! CHECK: if (777 .eq. 777) then
+! CHECK: print *, 'pp035.F no: ', 777
 * #if 2 .LT. 3 works
       integer, parameter :: KWM = 666
 #if 2 .LT. 3

flang/test/Preprocessing/pp036.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(.true.)then
-! CHECK: print*,'pp036.F no: ',.true.
+! CHECK: if (.TRUE .) then
+! CHECK: print *, 'pp036.F no: ', .TRUE .
 * #define FALSE TRUE ...  .FALSE. -> .TRUE.
 #define FALSE TRUE
       if (.FALSE.) then

flang/test/Preprocessing/pp037.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(7777.eq.777)then
-! CHECK: print*,'pp037.F no: ',7777
+! CHECK: if (7777 .eq. 777) then
+! CHECK: print *, 'pp037.F no: ', 7777
 * fixed-form clipping NOT applied to #define
       integer, parameter :: KWM = 666
 *        1         2         3         4         5         6         7
 *234567890123456789012345678901234567890123456789012345678901234567890123
 #define KWM                                                          7777
-      if (KWM .eq. 777) then
+      if (KWM  .eq. 777) then
         print *, 'pp037.F yes'
       else
         print *, 'pp037.F no: ', KWM

flang/test/Preprocessing/pp038.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=((666)+111)
+! CHECK: res = ((666)+111)
 * FLM call with closing ')' on next line (not a continuation)
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp039.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res=iflm
+! CHECK: res = IFLM
 ! CHECK: (666)
-! CHECK-NOT: res=((666)+111)
+! CHECK-NOT: res = ((666)+111)
 * FLM call with '(' on next line (not a continuation)
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp041.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: j=666wmj=j+1wm211
+! CHECK: j = 666WMj=j+1WM211
 * use KWM expansion as continuation indicators
 #define KWM 0
 #define KWM2 1

flang/test/Preprocessing/pp043.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm.eq.777)then
+! CHECK: if (KWM .eq. 777) then
 * #define with # in column 6 is a continuation line in fixed-form
       integer, parameter :: defineKWM666 = 555
       integer, parameter :: KWM =

flang/test/Preprocessing/pp044.F

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK-NOT:z=111
+! CHECK-NOT:z = 111
 * #define directive amid continuations
       integer, parameter :: KWM = 222, KWM111 = 333, KWM222 = 555
       integer, parameter :: KWMKWM = 333

flang/test/Preprocessing/pp101.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK:  if(777 .eq. 777) then
+! CHECK:  if (777 .eq. 777) then
 ! keyword macros
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp102.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm .eq. 777) then
+! CHECK: if (KWM .eq. 777) then
 ! #undef
       integer, parameter :: KWM = 777
 #define KWM 666

flang/test/Preprocessing/pp104.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm .eq. 777) then
+! CHECK: if (kwm .eq. 777) then
 ! KWMs case-sensitive
       integer, parameter :: KWM = 777
 #define KWM 666

flang/test/Preprocessing/pp107.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = kwm
+! CHECK: res = KWM
 ! KWM call name split across continuation, no leading &, with & ! comment
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp108.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = kwm
+! CHECK: res = KWM
 ! ditto, but without & ! comment
       integer, parameter :: KWM = 666
 #define KWM 777

flang/test/Preprocessing/pp111.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = iflm (666)
+! CHECK: res = IFLM (666)
 ! FLM call name split across continuation, no leading &, with & ! comment
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp112.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = iflm (666)
+! CHECK: res = IFLM (666)
 ! ditto, but without & ! comment
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp115.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = iflm (666)
+! CHECK: res = IFLM (666)
 ! ditto, with & ! comment, no leading &
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp116.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = iflm (666)
+! CHECK: res = IFLM (666)
 ! FLM call split between name and (, no leading &
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp117.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777 .eq. 777) then
+! CHECK: if (777 .eq. 777) then
 ! KWM rescan
       integer, parameter :: KWM = 666, KWM2 = 667
 #define KWM2 777

flang/test/Preprocessing/pp118.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(kwm2 .eq. 777) then
+! CHECK: if (KWM2 .eq. 777) then
 ! KWM rescan with #undef, proving rescan after expansion
       integer, parameter :: KWM2 = 777, KWM = 667
 #define KWM2 666

flang/test/Preprocessing/pp121.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
 ! CHECK: ch = 'KWM'
-! CHECK: if(ch .eq. 'KWM') then
+! CHECK: if (ch .eq. 'KWM') then
 ! KWM NOT expanded in 'literal'
 #define KWM 666
       character(len=3) :: ch

flang/test/Preprocessing/pp123.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: ch = 3hKWM
+! CHECK: ch = 3HKWM
 ! KWM NOT expanded in Hollerith literal
 #define KWM 666
 #define HKWM 667

flang/test/Preprocessing/pp124.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: 100 format(3hKWM)
+! CHECK: 100 format(3HKWM)
 ! KWM NOT expanded in Hollerith in FORMAT
 #define KWM 666
 #define HKWM 667

flang/test/Preprocessing/pp125.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777 .eq. 777) then
+! CHECK: if (777 .eq. 777) then
 ! #DEFINE works in free form
       integer, parameter :: KWM = 666
 #DEFINE KWM 777

flang/test/Preprocessing/pp126.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: if(777 .eq. 777) then
+! CHECK: if (777 .eq. 777) then
 ! \ newline works in #define
       integer, parameter :: KWM = 666
 #define KWM 77\

flang/test/Preprocessing/pp127.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = iflm(666 )
+! CHECK: res = IFLM(666 )
 ! FLM call with closing ')' on next line (not a continuation)
       integer function IFLM(x)
         integer :: x

flang/test/Preprocessing/pp128.F90

 ! RUN: %flang -E %s 2>&1 | FileCheck %s
-! CHECK: res = iflm
+! CHECK: res = IFLM
 ! FLM call with '(' on next line (not a continuation)
       integer function IFLM(x)
         integer :: x

flang/tools/f18-parse-demo/f18-parse-demo.cpp

   bool warnOnNonstandardUsage{false}; // -Mstandard
   bool warningsAreErrors{false}; // -Werror
   Fortran::parser::Encoding encoding{Fortran::parser::Encoding::LATIN_1};
+  bool lineDirectives{true}; // -P disables
   bool syntaxOnly{false};
   bool dumpProvenance{false};
-  bool dumpCookedChars{false};
+  bool noReformat{false}; // -E -fno-reformat
   bool dumpUnparse{false};
   bool dumpParseTree{false};
   bool timeParse{false};

     parsing.DumpProvenance(llvm::outs());
     return {};
   }
-  if (driver.dumpCookedChars) {
-    parsing.DumpCookedChars(llvm::outs());
+  if (options.prescanAndReformat) {
+    parsing.messages().Emit(llvm::errs(), allCookedSources);
+    if (driver.noReformat) {
+      parsing.DumpCookedChars(llvm::outs());
+    } else {
+      parsing.EmitPreprocessedSource(llvm::outs(), driver.lineDirectives);
+    }
     return {};
   }
   parsing.Parse(llvm::outs());

   while (!args.empty()) {
     std::string arg{std::move(args.front())};
     args.pop_front();
-    if (arg.empty()) {
+    if (arg.empty() || arg == "-Xflang") {
     } else if (arg.at(0) != '-') {
       anyFiles = true;
       auto dot{arg.rfind(".")};

       driver.warningsAreErrors = true;
     } else if (arg == "-ed") {
       options.features.Enable(Fortran::common::LanguageFeature::OldDebugLines);
-    } else if (arg == "-E" || arg == "-fpreprocess-only") {
-      driver.dumpCookedChars = true;
+    } else if (arg == "-E") {
+      options.prescanAndReformat = true;
+    } else if (arg == "-P") {
+      driver.lineDirectives = false;
+    } else if (arg == "-fno-reformat") {
+      driver.noReformat = true;
     } else if (arg == "-fbackslash") {
       options.features.Enable(
           Fortran::common::LanguageFeature::BackslashEscapes);

flang/tools/f18/f18.cpp

   bool warningsAreErrors{false}; // -Werror
   bool byteswapio{false}; // -byteswapio
   Fortran::parser::Encoding encoding{Fortran::parser::Encoding::UTF_8};
+  bool lineDirectives{true}; // -P disables
   bool syntaxOnly{false};
   bool dumpProvenance{false};
-  bool dumpCookedChars{false};
+  bool noReformat{false}; // -E -fno-reformat
   bool dumpUnparse{false};
   bool dumpUnparseWithSymbols{false};
   bool dumpParseTree{false};

     parsing.DumpProvenance(llvm::outs());
     return {};
   }
-  if (driver.dumpCookedChars) {
+  if (options.prescanAndReformat) {
     parsing.messages().Emit(llvm::errs(), allCookedSources);
-    parsing.DumpCookedChars(llvm::outs());
+    if (driver.noReformat) {
+      parsing.DumpCookedChars(llvm::outs());
+    } else {
+      parsing.EmitPreprocessedSource(llvm::outs(), driver.lineDirectives);
+    }
     return {};
   }
   parsing.Parse(llvm::outs());

     std::string suffix{arg.substr(dot + 1)};
     std::string prefix{arg.substr(0, 2)};
     args.pop_front();
-    if (arg.empty()) {
+    if (arg.empty() || arg == "-Xflang") {
     } else if (arg.at(0) != '-') {
       anyFiles = true;
       if (dot == std::string::npos) {

     } else if (arg == "-Munlimited" || arg == "-ffree-line-length-none" ||
         arg == "-ffree-line-length-0" || arg == "-ffixed-line-length-none" ||
         arg == "-ffixed-line-length-0") {
-      // For reparsing f18's -E output of fixed-form cooked character stream
+      // For reparsing f18's -E -fno-reformat output of fixed-form
+      // cooked character stream
       options.fixedFormColumns = 1000000;
     } else if (arg == "-Mbackslash") {
       options.features.Enable(

     } else if (arg == "-ed") {
       options.features.Enable(Fortran::common::LanguageFeature::OldDebugLines);
     } else if (arg == "-E") {
-      driver.dumpCookedChars = true;
+      options.prescanAndReformat = true;
+    } else if (arg == "-P") {
+      driver.lineDirectives = false;
+    } else if (arg == "-fno-reformat") {
+      driver.noReformat = true;
     } else if (arg == "-fbackslash" || arg == "-fno-backslash") {
       options.features.Enable(
           Fortran::common::LanguageFeature::BackslashEscapes,

flang/unittests/Frontend/FrontendActionTest.cpp

-//===- unittests/Frontend/PrintPreprocessedTest.cpp  FrontendAction tests--===//
+//===- unittests/Frontend/FrontendActionTest.cpp  FrontendAction tests-----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.

 
   // Set-up the action kind.
   compInst_.invocation().frontendOpts().programAction_ = PrintPreprocessedInput;
+  compInst_.invocation().preprocessorOpts().noReformat = true;
 
   // Set-up the output stream. We are using output buffer wrapped as an output
   // stream, as opposed to an actual file (or a file descriptor).