[flang][driver] Add support for `-c` and `-emit-obj`

[awarzynski]

Summary

This patch adds a frontend action for emitting object files. While Flang
does not support code-generation, this action remains a placeholder.
This patch simply provides glue-code to connect the compiler driver
with the appropriate frontend action.

The new action is triggered with the -c compiler driver flag, i.e.
flang-new -c. This is then translated to flang-new -fc1 -emit-obj,
so -emit-obj has to be marked as supported as well.

As code-generation is not available yet, flang-new -c results in a
driver error:

error: code-generation is not available yet

Hopefully this will help communicating the level of available
functionality within Flang.

The definition of emit-obj is updated so that it can be shared between
Clang and Flang. As the original definition was enclosed within a
Clang-specific TableGen let statement, it is extracted into a new let
statement. That felt like the cleanest option.

I also commented out -triple in Flang::ConstructJob and updated some
comments there. This is similar to https://reviews.llvm.org/D93027. I
wanted to make sure that it's clear that we can't support -triple
until we have code-generation. However, once code-generation is
available we _will need_ -triple.

As this patch adds -emit-obj, the emit-obj.f90 becomes irrelevant and
is deleted. Instead, phases.f90 is added to demonstrate that users can
control compilation phases (indeed, -c is a phase control flag).

Depends on: D92854

[awarzynski]

@dang , I would appreciate if you could take a look at the diff in Options.td - you introduced the most recent changes there: https://reviews.llvm.org/D82574. I got a bit confused with the nested let statements there. I couldn't find a better way of marking -emit-obj as a both CC1Option and FC1Option, so I just extracted it out.

[clementval]

clang/include/clang/Driver/Options.td

5228

Is it intended to modify the SYCL options?

flang/lib/Frontend/CompilerInvocation.cpp

106–107

Should you remove this line since you have the case for it?

flang/test/Flang-Driver/code-gen.f90

12

than instead of then?

[SouraVX]

Thanks! for the patch, some minor comments inlined. Rest LGTM. Let others also have a look.

clang/include/clang/Driver/Options.td

4857

Please correct if I've misunderstood this change ?
You're removing this option from here(clang) and defining again at line 4631 as a common option to both clang flang ?
+1 to that.
However this seems out of the purview of this patch. Do you think having this as a separate patch(with specific intent) would be good(For tracking/isolating changes) ?
Or the least you can do is convey this intent in this patch Summary too.
I'm happy with either of those :)

flang/test/Flang-Driver/code-gen.f90

16

Since it's an error NOT a string(or similar) generated, I would rather have ERROR as a string check. This way it is self-evident for end reader.
Sort of:

! ERROR: code-generation is not available yet

[awarzynski]

@clementval & @SouraVX - thank you for your comments! I will upload an updated version shortly.

clang/include/clang/Driver/Options.td

4857

You're removing this option from here(clang) and defining again at line 4631 as a common option to both clang flang ?

Correct :)

However this seems out of the purview of this patch.

Not quite. Without this change you will get an error when using -emit-obj:

bin/flang-new -fc1 -emit-obj file.f90
error: unknown argument: '-emit-obj'

That's because it wouldn't be parsed here:
https://github.com/llvm/llvm-project/blob/eba09a2db9eab46832cb7ec7ef0d2c227747772a/flang/lib/Frontend/CompilerInvocation.cpp#L166-L167
(note the definition of includedFlagsBitmask).

I do agree that the Summary should be updated, thanks for pointing that out!

5228

This is just updating the comment (so it's just an nfc, SYCL options are not affected).

This closing } corresponds to the opening { specified here:: https://github.com/llvm/llvm-project/blob/eba09a2db9eab46832cb7ec7ef0d2c227747772a/clang/include/clang/Driver/Options.td#L4040. So, currently we have this:

let Flags = [CC1Option, NoDriverOption] in {
// A lot of code HERE
} // let Flags = [CC1Option]

Instead it should be:

let Flags = [CC1Option, NoDriverOption] in {
// A lot of code HERE
} // let Flags = [CC1Option, NoDriverOption]

I was mislead by this comment while working on this patch, so to me it seemed like a _related_ change :) But it isn't and should be extracted into a separate patch: https://github.com/llvm/llvm-project/commit/7f19712a6a9e65bdc9a9843ea488030bc12f3acc.

flang/lib/Frontend/CompilerInvocation.cpp

106–107

Yes, thanks for pointing out!

flang/test/Flang-Driver/code-gen.f90

16

Yeah, ERROR would be better. I wish that we had a prospect of Clang's -verify.

[awarzynski]

Address PR comments

[awarzynski]

Ping

[clementval]

LGTM

[SouraVX]

LGTM! - Thanks!

clang/include/clang/Driver/Options.td

 def config_user_dir_EQ : Joined<["--"], "config-user-dir=">, Flags<[NoXarchOption, HelpHidden]>,
   HelpText<"User directory for configuration files">;
 def coverage : Flag<["-", "--"], "coverage">, Group<Link_Group>, Flags<[CoreOption]>;
 def cpp_precomp : Flag<["-"], "cpp-precomp">, Group<clang_ignored_f_Group>;
 def current__version : JoinedOrSeparate<["-"], "current_version">;
 def cxx_isystem : JoinedOrSeparate<["-"], "cxx-isystem">, Group<clang_i_Group>,
   HelpText<"Add directory to the C++ SYSTEM include search path">, Flags<[CC1Option]>,
   MetaVarName<"<directory>">;
-def c : Flag<["-"], "c">, Flags<[NoXarchOption]>, Group<Action_Group>,
+def c : Flag<["-"], "c">, Flags<[NoXarchOption, FlangOption]>, Group<Action_Group>,
   HelpText<"Only run preprocess, compile, and assemble steps">;
 def fconvergent_functions : Flag<["-"], "fconvergent-functions">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Assume functions may be convergent">;
 
 def gpu_use_aux_triple_only : Flag<["--"], "gpu-use-aux-triple-only">,
   InternalDriverOpt, HelpText<"Prepare '-aux-triple' only without populating "
                               "'-aux-target-cpu' and '-aux-target-feature'.">;
 def cuda_device_only : Flag<["--"], "cuda-device-only">,

 def emit_pch : Flag<["-"], "emit-pch">,
   HelpText<"Generate pre-compiled header file">;
 def emit_llvm_bc : Flag<["-"], "emit-llvm-bc">,
   HelpText<"Build ASTs then convert to LLVM, emit .bc file">;
 def emit_llvm_only : Flag<["-"], "emit-llvm-only">,
   HelpText<"Build ASTs and convert to LLVM, discarding output">;
 def emit_codegen_only : Flag<["-"], "emit-codegen-only">,
   HelpText<"Generate machine code, but discard output">;
-def emit_obj : Flag<["-"], "emit-obj">,
-  HelpText<"Emit native object files">;
 def rewrite_test : Flag<["-"], "rewrite-test">,
   HelpText<"Rewriter playground">;
 def rewrite_macros : Flag<["-"], "rewrite-macros">,
   HelpText<"Expand macros without full preprocessing">;
 def migrate : Flag<["-"], "migrate">,
   HelpText<"Migrate source code">;
 def compiler_options_dump : Flag<["-"], "compiler-options-dump">,
   HelpText<"Dump the compiler configuration options">;

 
 //===----------------------------------------------------------------------===//
 // SYCL Options
 //===----------------------------------------------------------------------===//
 
 def fsycl_is_device : Flag<["-"], "fsycl-is-device">,
   HelpText<"Generate code for SYCL device.">;
 
 } // let Flags = [CC1Option, NoDriverOption]
 
 //===----------------------------------------------------------------------===//
+// Frontend Options - cc1 + fc1
+//===----------------------------------------------------------------------===//
+let Flags = [CC1Option, FC1Option, NoDriverOption] in {
+let Group = Action_Group in {
+
+def emit_obj : Flag<["-"], "emit-obj">,
+  HelpText<"Emit native object files">;
+
+} // let Group = Action_Group
+} // let Flags = [CC1Option, FC1Option, NoDriverOption]
+
+//===----------------------------------------------------------------------===//
 // cc1as-only Options
 //===----------------------------------------------------------------------===//
 
 let Flags = [CC1AsOption, NoDriverOption] in {
 
 // Language Options
 def n : Flag<["-"], "n">,
   HelpText<"Don't automatically start assembly file with a text section">;

clang/lib/Driver/ToolChains/Flang.cpp

void Flang::AddPreprocessingOptions(const ArgList &Args,
                                     ArgStringList &CmdArgs) const {
   Args.AddAllArgs(CmdArgs, {options::OPT_D, options::OPT_U});
 }
 
 void Flang::ConstructJob(Compilation &C, const JobAction &JA,
                          const InputInfo &Output, const InputInfoList &Inputs,
                          const ArgList &Args, const char *LinkingOutput) const {
   const auto &TC = getToolChain();
-  const llvm::Triple &Triple = TC.getEffectiveTriple();
-  const std::string &TripleStr = Triple.getTriple();
+  // TODO: Once code-generation is available, this will need to be commented
+  // out.
+  // const llvm::Triple &Triple = TC.getEffectiveTriple();
+  // const std::string &TripleStr = Triple.getTriple();
 
   ArgStringList CmdArgs;
 
+  // Invoke ourselves in -fc1 mode.
   CmdArgs.push_back("-fc1");
 
-  // TODO: Eventually all actions will require a triple (e.g. `-triple
-  // aarch64-unknown-linux-gnu`). However, `-triple` is currently not supported
-  // by `flang-new -fc1`, so we only add it selectively to actions that we
-  // don't support/execute just yet.
+  // TODO: Once code-generation is available, this will need to be commented
+  // out.
+  // Add the "effective" target triple.
+  // CmdArgs.push_back("-triple");
+  // CmdArgs.push_back(Args.MakeArgString(TripleStr));
+
   if (isa<PreprocessJobAction>(JA)) {
     if (C.getArgs().hasArg(options::OPT_test_io))
       CmdArgs.push_back("-test-io");
     else
       CmdArgs.push_back("-E");
   } else if (isa<CompileJobAction>(JA) || isa<BackendJobAction>(JA)) {
     if (JA.getType() == types::TY_Nothing) {
       CmdArgs.push_back("-fsyntax-only");
     } else if (JA.getType() == types::TY_AST) {
       CmdArgs.push_back("-emit-ast");
     } else if (JA.getType() == types::TY_LLVM_IR ||
                JA.getType() == types::TY_LTO_IR) {
       CmdArgs.push_back("-emit-llvm");
     } else if (JA.getType() == types::TY_LLVM_BC ||
                JA.getType() == types::TY_LTO_BC) {
       CmdArgs.push_back("-emit-llvm-bc");
     } else if (JA.getType() == types::TY_PP_Asm) {
       CmdArgs.push_back("-S");
     } else {
       assert(false && "Unexpected output type!");
     }
   } else if (isa<AssembleJobAction>(JA)) {
-    CmdArgs.push_back("-triple");
-    CmdArgs.push_back(Args.MakeArgString(TripleStr));
     CmdArgs.push_back("-emit-obj");
   } else {
     assert(false && "Unexpected action class for Flang tool.");
   }
 
   const InputInfo &Input = Inputs[0];
   types::ID InputType = Input.getType();
 

flang/include/flang/Frontend/FrontendActions.h

 class PrintPreprocessedAction : public FrontendAction {
   void ExecuteAction() override;
 };
 
 class ParseSyntaxOnlyAction : public FrontendAction {
   void ExecuteAction() override;
 };
 
+class EmitObjAction : public FrontendAction {
+  void ExecuteAction() override;
+};
+
 } // namespace Fortran::frontend
 
 #endif // LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H

flang/include/flang/Frontend/FrontendOptions.h

enum ActionKind {
   InputOutputTest,
 
   /// -E mode.
   PrintPreprocessedInput,
 
   /// -fsyntax-only
   ParseSyntaxOnly,
 
+  /// Emit a .o file.
+  EmitObj,
+
   /// TODO: RunPreprocessor, EmitLLVM, EmitLLVMOnly,
   /// EmitCodeGenOnly, EmitAssembly, (...)
 };
 
 inline const char *GetActionKindName(const ActionKind ak) {
   switch (ak) {
   case InputOutputTest:
     return "InputOutputTest";

flang/lib/Frontend/CompilerInvocation.cpp

case clang::driver::options::OPT_test_io:
       opts.programAction_ = InputOutputTest;
       break;
     case clang::driver::options::OPT_E:
       opts.programAction_ = PrintPreprocessedInput;
       break;
     case clang::driver::options::OPT_fsyntax_only:
       opts.programAction_ = ParseSyntaxOnly;
       break;
+    case clang::driver::options::OPT_emit_obj:
+      opts.programAction_ = EmitObj;
+      break;
 
       // TODO:
-      // case clang::driver::options::OPT_emit_obj:
       // case calng::driver::options::OPT_emit_llvm:
       // case clang::driver::options::OPT_emit_llvm_only:
       // case clang::driver::options::OPT_emit_codegen_only:
       // case clang::driver::options::OPT_emit_module:
       // (...)
     }
   }
 
   opts.outputFile_ = args.getLastArgValue(clang::driver::options::OPT_o);

flang/lib/Frontend/FrontendActions.cpp

void ParseSyntaxOnlyAction::ExecuteAction() {
   semantics.EmitMessages(ci.semaOutputStream());
 
   if (semantics.AnyFatalError()) {
     unsigned DiagID = ci.diagnostics().getCustomDiagID(
         clang::DiagnosticsEngine::Error, "Semantic errors in %0");
     ci.diagnostics().Report(DiagID) << GetCurrentFileOrBufferName();
   }
 }
+
+void EmitObjAction::ExecuteAction() {
+  CompilerInstance &ci = this->instance();
+  unsigned DiagID = ci.diagnostics().getCustomDiagID(
+      clang::DiagnosticsEngine::Error, "code-generation is not available yet");
+  ci.diagnostics().Report(DiagID);
+}

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

static std::unique_ptr<FrontendAction> CreateFrontendBaseAction(
   case InputOutputTest:
     return std::make_unique<InputOutputTestAction>();
     break;
   case PrintPreprocessedInput:
     return std::make_unique<PrintPreprocessedAction>();
     break;
   case ParseSyntaxOnly:
     return std::make_unique<ParseSyntaxOnlyAction>();
+  case EmitObj:
+    return std::make_unique<EmitObjAction>();
     break;
   default:
     break;
     // TODO:
     // case RunPreprocessor:
     // case ParserSyntaxOnly:
     // case EmitLLVM:
     // case EmitLLVMOnly:

flang/test/Flang-Driver/code-gen.f90

+! RUN: not %flang-new %s 2>&1 | FileCheck %s --check-prefix=ERROR
+! RUN: not %flang-new -c %s 2>&1 | FileCheck %s --check-prefix=ERROR
+! RUN: not %flang-new -emit-obj %s 2>&1 | FileCheck %s --check-prefix=ERROR
+! RUN: not %flang-new -fc1 -emit-obj %s 2>&1 | FileCheck %s --check-prefix=ERROR
+
+! REQUIRES: new-flang-driver
+
+! Although code-generation is not yet available, we do have frontend actions
+! that correspond to `-c` and `-emit-obj`. For now these actions are just a
+! placeholder and running them leads to a driver error. This test makes sure
+! that these actions are indeed run (rather than `-c` or `-emit-obj` being
+! rejected earlier).
+! TODO: Replace this file with a proper test once code-generation is available.
+
+! ERROR: code-generation is not available yet

flang/test/Flang-Driver/driver-help-hidden.f90

 
 !----------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG DRIVER (flang-new)
 !----------------------------------------------------
 ! CHECK:USAGE: flang-new
 ! CHECK-EMPTY:
 ! CHECK-NEXT:OPTIONS:
 ! CHECK-NEXT: -###      Print (but do not run) the commands to run for this compilation
+! CHECK-NEXT: -c        Only run preprocess, compile, and assemble steps
 ! CHECK-NEXT: -D <macro>=<value>     Define <macro> to <value> (or 1 if <value> omitted)
 ! CHECK-NEXT: -E        Only run the preprocessor
 ! CHECK-NEXT: -fcolor-diagnostics    Enable colors in diagnostics
 ! CHECK-NEXT: -fno-color-diagnostics Disable colors in diagnostics
 ! CHECK-NEXT: -help     Display available options
 ! CHECK-NEXT: -o <file> Write output to <file>
 ! CHECK-NEXT: -test-io  Run the InputOuputTest action. Use for development and testing only.
 ! CHECK-NEXT: -U <macro>             Undefine macro <macro>

flang/test/Flang-Driver/driver-help.f90

 
 !----------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG DRIVER (flang-new)
 !----------------------------------------------------
 ! HELP:USAGE: flang-new
 ! HELP-EMPTY:
 ! HELP-NEXT:OPTIONS:
 ! HELP-NEXT: -###                   Print (but do not run) the commands to run for this compilation
+! HELP-NEXT: -c                     Only run preprocess, compile, and assemble steps
 ! HELP-NEXT: -D <macro>=<value>     Define <macro> to <value> (or 1 if <value> omitted)
 ! HELP-NEXT: -E                     Only run the preprocessor
 ! HELP-NEXT: -fcolor-diagnostics    Enable colors in diagnostics
 ! HELP-NEXT: -fno-color-diagnostics Disable colors in diagnostics
 ! HELP-NEXT: -help                  Display available options
 ! HELP-NEXT: -o <file>              Write output to <file>
 ! HELP-NEXT: -U <macro>             Undefine macro <macro>
 ! HELP-NEXT: --version              Print version information
 
 !-------------------------------------------------------------
 ! EXPECTED OUTPUT FOR FLANG FRONTEND DRIVER (flang-new -fc1)
 !-------------------------------------------------------------
 ! HELP-FC1:USAGE: flang-new
 ! HELP-FC1-EMPTY:
 ! HELP-FC1-NEXT:OPTIONS:
 ! HELP-FC1-NEXT: -D <macro>=<value>     Define <macro> to <value> (or 1 if <value> omitted)
+! HELP-FC1-NEXT: -emit-obj Emit native object files
 ! HELP-FC1-NEXT: -E                     Only run the preprocessor
 ! HELP-FC1-NEXT: -help                  Display available options
 ! HELP-FC1-NEXT: -o <file>              Write output to <file>
 ! HELP-FC1-NEXT: -U <macro>             Undefine macro <macro>
 ! HELP-FC1-NEXT: --version              Print version information
 
 !---------------
 ! EXPECTED ERROR
 !---------------
 ! ERROR: error: unknown argument '-helps'; did you mean '-help'

flang/test/Flang-Driver/emit-obj.f90

-! RUN: not %flang-new  %s 2>&1 | FileCheck %s --check-prefix=ERROR
-! RUN: not %flang-new  -emit-obj %s 2>&1 | FileCheck %s --check-prefix=ERROR
-! RUN: not %flang-new  -fc1 -emit-obj %s 2>&1 | FileCheck %s --check-prefix=ERROR-FC1
-
-! REQUIRES: new-flang-driver
-
-! By default (e.g. when no options like `-E` are passed) flang-new
-! creates a job that corresponds to `-emit-obj`. This option/action is
-! not yet supported. Verify that this is correctly reported as error.
-
-! ERROR: error: unknown argument: '-triple'
-! ERROR: error: unknown argument: '-emit-obj'
-
-! ERROR-FC1: error: unknown argument: '-emit-obj'

flang/test/Flang-Driver/phases.f90

+! RUN: %flang-new -E -ccc-print-phases %s 2>&1 | FileCheck %s --check-prefix=PP
+! RUN: %flang-new -fsyntax-only -ccc-print-phases %s 2>&1 | FileCheck %s --check-prefix=COMPILE
+! RUN: %flang-new -c -ccc-print-phases %s 2>&1 | FileCheck %s --check-prefix=EMIT_OBJ
+
+! REQUIRES: new-flang-driver
+
+! This test verifies the phase control in Flang compiler driver.
+
+! PP: +- 0: input, "{{.*}}phases.f90", f95-cpp-input
+! PP-NEXT: 1: preprocessor, {0}, f95
+
+! COMPILE: +- 0: input, "{{.*}}phases.f90", f95-cpp-input
+! COMPILE-NEXT: 1: preprocessor, {0}, f95
+! COMPILE-NEXT: 2: compiler, {1}, none
+
+! EMIT_OBJ: +- 0: input, "{{.*}}phases.f90", f95-cpp-input
+! EMIT_OBJ-NEXT: 1: preprocessor, {0}, f95
+! EMIT_OBJ-NEXT: 2: compiler, {1}, ir
+! EMIT_OBJ-NEXT: +- 3: backend, {2}, assembler
+! EMIT_OBJ-NEXT: 4: assembler, {3}, object