[flang][driver] Add -Xflang and make -test-io a frontend-only flag

[awarzynski]

Summary

This patch adds support for -Xflang in flang-new. The semantics are
identical to -Xclang.

With the addition of -Xflang, we can modify -test-io to be a
compiler-frontend only flag. This makes more sense, this flag is:

• very frontend specific
• to be used for development and testing only
• not to be exposed to the end user

Originally we added it to the compiler driver, flang-new, in order to
facilitate testing. With -Xflang this is no longer needed. Tests are
updated accordingly.

[SouraVX]

Thanks! for the patch @awarzynski . I support this enhancement. LGTM(Not accepting formally :) ), Let's wait and hear from others also, what's their take on this.

[SouraVX]

flang/test/Frontend/input-output-file.f90

6

I didn't follow this completely, Could you please elaborate a bit.

[awarzynski]

flang/test/Frontend/input-output-file.f90

6

Sorry, I'm doing a poor job documenting this here.

Basically, we want to instruct flang-new to stop after the preprocessing phase. -E is a phase control flag. -c is another phase control flag. You can also skip phase flags all together:

$ bin/flang-new -ccc-print-phases file.f90
            +- 0: input, "file.f90", f95-cpp-input
         +- 1: preprocessor, {0}, f95
      +- 2: compiler, {1}, ir
   +- 3: backend, {2}, assembler
+- 4: assembler, {3}, object
5: linker, {4}, image

As you can see, without -E the driver, flang-new, would attempt to run multiple phases which would result in multiple compiler _jobs_. But we only want the compiler job, i.e. we are only interested the the first phase. (IIRC, compiler jobs map 1-1 to compiler phases).

So, -E is added here to make sure that flang-new doesn't try to do too much here.

[SouraVX]

flang/test/Frontend/input-output-file.f90

6

Great - thanks for explaining the intent!

[awarzynski]

Thanks for reviewing @SouraVX ! I did update that comment before merging. Hopefully it _will_ make sense when we come back to it at later time :)

clang/include/clang/Driver/Options.td

defm sycl : BoolOption<"f", "sycl",
   BothFlags<[CoreOption], " SYCL kernels compilation for device">>,
   Group<sycl_Group>;
 def sycl_std_EQ : Joined<["-"], "sycl-std=">, Group<sycl_Group>, Flags<[CC1Option, NoArgumentUnused, CoreOption]>,
   HelpText<"SYCL language standard to compile for.">, Values<"2017,121,1.2.1,sycl-1.2.1">,
   NormalizedValues<["SYCL_2017", "SYCL_2017", "SYCL_2017", "SYCL_2017"]>, NormalizedValuesScope<"LangOptions">,
   MarshallingInfoString<LangOpts<"SYCLVersion">, "SYCL_None">, ShouldParseIf<fsycl.KeyPath>, AutoNormalizeEnum;
 
 //===----------------------------------------------------------------------===//
-// FlangOption and FC1 Options
+// FLangOption + CoreOption + NoXarchOption
 //===----------------------------------------------------------------------===//
-def test_io : Flag<["-"], "test-io">, Flags<[HelpHidden, FlangOption, FC1Option, FlangOnlyOption]>, Group<Action_Group>,
-  HelpText<"Run the InputOuputTest action. Use for development and testing only.">;
+let Flags = [FlangOption, FlangOnlyOption, NoXarchOption, CoreOption] in {
+def Xflang : Separate<["-"], "Xflang">,
+  HelpText<"Pass <arg> to the flang compiler">, MetaVarName<"<arg>">,
+  Flags<[NoXarchOption, CoreOption]>, Group<CompileOnly_Group>;
+}
 
+//===----------------------------------------------------------------------===//
+// FlangOption and FC1 Options
+//===----------------------------------------------------------------------===//
 let Flags = [FC1Option, FlangOption, FlangOnlyOption] in {
 
 def ffixed_form : Flag<["-"], "ffixed-form">, Group<f_Group>,
   HelpText<"Process source files in fixed form">;
 def ffree_form : Flag<["-"], "ffree-form">, Group<f_Group>,
   HelpText<"Process source files in free form">;
 def ffixed_line_length_EQ : Joined<["-"], "ffixed-line-length=">, Group<f_Group>,
   HelpText<"Use <value> as character line width in fixed mode">,

 def flogical_abbreviations : Flag<["-"], "flogical-abbreviations">, Group<f_Group>,
   HelpText<"Enable logical abbreviations">;
 def fno_logical_abbreviations : Flag<["-"], "fno-logical-abbreviations">, Group<f_Group>;
 def fimplicit_none : Flag<["-"], "fimplicit-none">, Group<f_Group>,
   HelpText<"No implicit typing allowed unless overridden by IMPLICIT statements">;
 def fno_implicit_none : Flag<["-"], "fno-implicit-none">, Group<f_Group>;
 def falternative_parameter_statement : Flag<["-"], "falternative-parameter-statement">, Group<f_Group>,
   HelpText<"Enable the old style PARAMETER statement">;
-
 }
 
 //===----------------------------------------------------------------------===//
 // FC1 Options
 //===----------------------------------------------------------------------===//
 let Flags = [FC1Option, FlangOnlyOption] in {
 
+def test_io : Flag<["-"], "test-io">, Group<Action_Group>,
+  HelpText<"Run the InputOuputTest action. Use for development and testing only.">;
 def fdebug_unparse : Flag<["-"], "fdebug-unparse">, Group<Action_Group>,
   HelpText<"Unparse and stop.">;
 def fdebug_unparse_with_symbols : Flag<["-"], "fdebug-unparse-with-symbols">, Group<Action_Group>,
   HelpText<"Unparse and stop.">;
 def fdebug_dump_symbols : Flag<["-"], "fdebug-dump-symbols">, Group<Action_Group>,
   HelpText<"Dump symbols after the semantic analysis">;
 def fdebug_dump_parse_tree : Flag<["-"], "fdebug-dump-parse-tree">, Group<Action_Group>,
   HelpText<"Dump the parse tree">;

clang/lib/Driver/ToolChains/Flang.cpp

void Flang::ConstructJob(Compilation &C, const JobAction &JA,
 
   // TODO: Once code-generation is available, this will need to be commented
   // out.
   // Add the "effective" target triple.
   // CmdArgs.push_back("-triple");
   // CmdArgs.push_back(Args.MakeArgString(TripleStr));
 
   if (isa<PreprocessJobAction>(JA)) {
-    if (C.getArgs().hasArg(options::OPT_test_io))
-      CmdArgs.push_back("-test-io");
-    else
       CmdArgs.push_back("-E");
   } else if (isa<CompileJobAction>(JA) || isa<BackendJobAction>(JA)) {
     if (JA.getType() == types::TY_Nothing) {
       CmdArgs.push_back("-fsyntax-only");
     } else if (JA.getType() == types::TY_AST) {
       CmdArgs.push_back("-emit-ast");
     } else if (JA.getType() == types::TY_LLVM_IR ||
                JA.getType() == types::TY_LTO_IR) {
void Flang::ConstructJob(Compilation &C, const JobAction &JA,
   if (types::getPreprocessedType(InputType) != types::TY_INVALID)
     AddPreprocessingOptions(Args, CmdArgs);
 
   AddFortranDialectOptions(Args, CmdArgs);
 
   // Add other compile options
   AddOtherOptions(Args, CmdArgs);
 
+  // Forward -Xflang arguments to -fc1
+  Args.AddAllArgValues(CmdArgs, options::OPT_Xflang);
+
   if (Output.isFilename()) {
     CmdArgs.push_back("-o");
     CmdArgs.push_back(Output.getFilename());
   } else {
     assert(Output.isNothing() && "Invalid output.");
   }
 
   assert(Input.isFilename() && "Invalid input.");

clang/lib/Driver/Types.cpp

 llvm::SmallVector<phases::ID, phases::MaxNumberOfPhases>
 types::getCompilationPhases(const clang::driver::Driver &Driver,
                             llvm::opt::DerivedArgList &DAL, ID Id) {
   phases::ID LastPhase;
 
   // Filter to compiler mode. When the compiler is run as a preprocessor then
   // compilation is not an option.
   // -S runs the compiler in Assembly listing mode.
-  // -test-io is used by Flang to run InputOutputTest action
   if (Driver.CCCIsCPP() || DAL.getLastArg(options::OPT_E) ||
       DAL.getLastArg(options::OPT__SLASH_EP) ||
       DAL.getLastArg(options::OPT_M, options::OPT_MM) ||
-      DAL.getLastArg(options::OPT__SLASH_P) ||
-      DAL.getLastArg(options::OPT_test_io))
+      DAL.getLastArg(options::OPT__SLASH_P))
     LastPhase = phases::Preprocess;
 
   // --precompile only runs up to precompilation.
   // This is a clang extension and is not compatible with GCC.
   else if (DAL.getLastArg(options::OPT__precompile))
     LastPhase = phases::Precompile;
 
   // -{fsyntax-only,-analyze,emit-ast} only run up to the compiler.

flang/test/Flang-Driver/driver-help-hidden.f90

 ! CHECK-NEXT: -fno-color-diagnostics Disable colors in diagnostics
 ! CHECK-NEXT: -fopenacc              Enable OpenACC
 ! CHECK-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! CHECK-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! CHECK-NEXT: -help     Display available options
 ! CHECK-NEXT: -I <dir>               Add directory to the end of the list of include search paths
 ! CHECK-NEXT: -module-dir <dir>      Put MODULE files in <dir>
 ! CHECK-NEXT: -o <file> Write output to <file>
-! CHECK-NEXT: -test-io  Run the InputOuputTest action. Use for development and testing only.
 ! CHECK-NEXT: -U <macro>             Undefine macro <macro>
 ! CHECK-NEXT: --version Print version information
+! CHECK-NEXT: -Xflang <arg>          Pass <arg> to the flang compiler
 
 !-------------------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG DRIVER (flang-new)
 !-------------------------------------------------------------
 ! ERROR-FLANG: error: unknown argument '-help-hidden'; did you mean '--help-hidden'?
 
 !-------------------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG FRONTEND DRIVER (flang-new -fc1)
 !-------------------------------------------------------------
 ! Frontend driver -help-hidden is not supported
 ! ERROR-FLANG-FC1: error: unknown argument: '{{.*}}'
 

flang/test/Flang-Driver/driver-help.f90

 ! HELP-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! HELP-NEXT: -help                  Display available options
 ! HELP-NEXT: -I <dir>               Add directory to the end of the list of include search paths
 ! HELP-NEXT: -module-dir <dir>      Put MODULE files in <dir>
 ! HELP-NEXT: -o <file>              Write output to <file>
 ! HELP-NEXT: -U <macro>             Undefine macro <macro>
 ! HELP-NEXT: --version              Print version information
+! HELP-NEXT: -Xflang <arg>          Pass <arg> to the flang compiler
 
 !-------------------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG FRONTEND DRIVER (flang-new -fc1)
 !-------------------------------------------------------------
 ! HELP-FC1:USAGE: flang-new
 ! HELP-FC1-EMPTY:
 ! HELP-FC1-NEXT:OPTIONS:
 ! HELP-FC1-NEXT: -D <macro>=<value>     Define <macro> to <value> (or 1 if <value> omitted)

 ! HELP-FC1-NEXT: -flogical-abbreviations Enable logical abbreviations
 ! HELP-FC1-NEXT: -fopenacc              Enable OpenACC
 ! HELP-FC1-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-FC1-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! HELP-FC1-NEXT: -help                  Display available options
 ! HELP-FC1-NEXT: -I <dir>               Add directory to the end of the list of include search paths
 ! HELP-FC1-NEXT: -module-dir <dir>      Put MODULE files in <dir>
 ! HELP-FC1-NEXT: -o <file>              Write output to <file>
+! HELP-FC1-NEXT: -test-io               Run the InputOuputTest action. Use for development and testing only.
 ! HELP-FC1-NEXT: -U <macro>             Undefine macro <macro>
 ! HELP-FC1-NEXT: --version              Print version information
 
 !---------------
 ! EXPECTED ERROR
 !---------------
 ! ERROR: error: unknown argument '-helps'; did you mean '-help'

flang/test/Frontend/input-output-file.f90

-! RUN: rm -rf %S/input-output-file.txt
-
 ! REQUIRES: new-flang-driver
 
 !--------------------------
 ! FLANG DRIVER (flang-new)
 !--------------------------
+! NOTE: Use `-E` so that the compiler driver stops after the 1st compilation phase, preprocessing. That's all we need.
+
 ! TEST 1: Print to stdout (implicit)
-! RUN: %flang-new -test-io %s  2>&1 | FileCheck %s --match-full-lines
+! RUN: %flang-new -E -Xflang -test-io %s  2>&1 | FileCheck %s --match-full-lines
+
 ! TEST 2: Print to stdout (explicit)
-! RUN: %flang-new -test-io -o - %s  2>&1 | FileCheck %s --match-full-lines
+! RUN: %flang-new -E -Xflang -test-io -o - %s  2>&1 | FileCheck %s --match-full-lines
+
 ! TEST 3: Print to a file
-! RUN: %flang-new -test-io -o %t %s 2>&1 && FileCheck %s --match-full-lines --input-file=%t
+! RUN: %flang-new -E -Xflang -test-io -o %t %s 2>&1 && FileCheck %s --match-full-lines --input-file=%t
 
 !----------------------------------------
 ! FLANG FRONTEND DRIVER (flang-new -fc1)
 !----------------------------------------
 ! TEST 4: Write to a file (implicit)
 ! This particular test case generates an output file in the same directory as the input file. We need to copy the input file into a
 ! temporary directory to avoid polluting the source directory.
 ! RUN: rm -rf %t-dir && mkdir -p %t-dir && cd %t-dir
 ! RUN: cp %s .
 ! RUN: %flang-new -fc1 -test-io input-output-file.f90  2>&1 && FileCheck %s --match-full-lines --input-file=input-output-file.txt
+
 ! TEST 5: Write to a file (explicit)
 ! RUN: %flang-new -fc1 -test-io  -o %t %s 2>&1 && FileCheck %s --match-full-lines --input-file=%t
 
 !-----------------------
 ! EXPECTED OUTPUT
 !-----------------------
 ! CHECK-LABEL: Program arithmetic
 ! CHECK-NEXT:    Integer :: i, j
 ! CHECK-NEXT:    i = 2; j = 3; i= i * j;
 ! CHECK-NEXT:  End Program arithmetic
 
 Program arithmetic
   Integer :: i, j
   i = 2; j = 3; i= i * j;
 End Program arithmetic

flang/test/Frontend/multiple-input-files.f90

-! RUN: rm -rf %S/multiple-input-files.txt  %S/Inputs/hello-world.txt
-
 ! REQUIRES: new-flang-driver
 
 !--------------------------
 ! FLANG DRIVER (flang-new)
 !--------------------------
+! NOTE: Use `-E` so that the compiler driver stops after the 1st compilation phase, preprocessing. That's all we need.
+
 ! TEST 1: Both input files are processed (output is printed to stdout)
-! RUN: %flang-new -test-io %s %S/Inputs/hello-world.f90 | FileCheck %s --match-full-lines -check-prefix=FLANG
+! RUN: %flang-new -E -Xflang -test-io %s %S/Inputs/hello-world.f90 | FileCheck %s --match-full-lines -check-prefix=FLANG
 
 ! TEST 2: None of the files is processed (not possible to specify the output file when multiple input files are present)
-! RUN: not %flang-new -test-io -o - %S/Inputs/hello-world.f90 %s  2>&1 | FileCheck %s --match-full-lines -check-prefix=ERROR
-! RUN: not %flang-new -test-io -o %t %S/Inputs/hello-world.f90 %s 2>&1 | FileCheck %s --match-full-lines -check-prefix=ERROR
+! RUN: not %flang-new -E -Xflang -test-io -o - %S/Inputs/hello-world.f90 %s  2>&1 | FileCheck %s --match-full-lines -check-prefix=ERROR
+! RUN: not %flang-new -E -Xflang -test-io -o %t %S/Inputs/hello-world.f90 %s 2>&1 | FileCheck %s --match-full-lines -check-prefix=ERROR
 
 !----------------------------------------
 ! FLANG FRONTEND DRIVER (flang-new -fc1)
 !----------------------------------------
 ! TEST 3: Both input files are processed
 ! This particular test case generates output files in the same directory as the input files. We need to copy the input files into a
 ! temporary directory to avoid polluting the source directory.
 ! RUN: rm -rf %t-dir && mkdir -p %t-dir && cd %t-dir