[flang][driver] Allow main program to be in an archive

[ekieri]

Summary

Add --undefined=_QQmain to the link line, so that a Fortran main program
will be included in the link job even if it is in an archive (unless we
are building a shared object). For now, this is only applied to the Gnu
toolchain.

We also add a section on the linker invocation to docs/FlangDriver.md.

The new tests require llvm-ar to construct an archive we can include in
the link job. This is a new dependency for flang/test (which already
depends on similar tools such as llvm-objdump).

See discussions in
https://github.com/llvm/llvm-project/issues/54787
which this patch fixes.

[ekieri]

CI fail noted, update will come.

[ekieri]

Fix test link-f90-main.f90, which was failing for statically linked builds.

[awarzynski]

Great stuff @ekieri , thanks for doing this!

You may want add a note in the summary that you've updated the docs as well. This is consistent with what we discussed in https://github.com/llvm/llvm-project/issues/54787. I've left a few minor comments, but otherwise LGTM.

Edit Please address my comments before landing :) Also, allow a couple more days for other reviewers to comment.

clang/lib/Driver/ToolChains/Gnu.cpp

601–603

I would add a reference to the GitHub issue too.

flang/docs/FlangDriver.md

152–158

Oh, good catch, completely forgot about this!

This is an unrelated change, but the noise is low and I'm happy for it to be included. You can also submit it as a separate patch if you want.

202

202–204

I used similar syntax in https://github.com/llvm/llvm-project/blob/release/14.x/flang/docs/FlangDriver.md. WDYT?

208

;-)

211

214

add default options

Perhaps clarify "where" (i.e. to the frontend driver and the linker invocations).

220

223

224

226

229

231

233

flang/test/Driver/link-c-main.c

8

Same for the other test. Let me know if you'd like somebody to test this for you on AArch64 ;-)

26–27

I'd be tempted to add a note that:

• main ought to be defined as it's specified in this file
• _QQmain should remain undefined as no Fortran-based object file generated by Flang is linked in

This is quite tricky stuff and in such cases "more is more" :)

flang/test/Driver/linker-flags.f90

63–67

How is this different to GNU-SAME?

[peixin]

Please fix the clang-format issue.

[ekieri]

Sorry to drop out on this. I got interrupted by other duties in life, and now I am no longer able to continue this work. If anybody wants to pick up the ball from here, please feel free to commandeer the patch.

[sunshaoce]

I would like to help land this patch.

[sunshaoce]

Address comments

[awarzynski]

Thanks for all the great effort, @ekieri !

@sunshaoce , mostly makes sense, just a few small suggestions. @peixin , wdyt?

clang/lib/Driver/ToolChains/Gnu.cpp

601–603

WDYT>

flang/test/Driver/link-c-main.c

4

6

flang/test/Driver/linker-flags.f90

15

Please, could you clarify that in this case _QQmain should not be flagged as undefined?

64

[sunshaoce]

Address @awarzynski's comments

[sunshaoce]

Any further suggestions?

[awarzynski]

In D134821#4301701, @sunshaoce wrote:

Any further suggestions?

I am happy for you to land this, thanks! LGTM

[awarzynski]

I've just reverted this patch - for context please see https://reviews.llvm.org/D149429.

@sunshaoce This was committed without acknowledging @ekieri 's contribution, so didn't follow the official guidelines: Attribution of Changes. IMHO, we should keep Emil as the author of this change (he has done the lion share of this highly non-trivial work) and add you as a co-author. Unless @ekieri has some other preference :) (I am happy as long as we follow the guidelines).

Either way, thank you both for contributing! Now, lets try to figure out what caused the issue with the bots.

[sunshaoce]

In D134821#4304690, @awarzynski wrote:

I've just reverted this patch - for context please see https://reviews.llvm.org/D149429.

@sunshaoce This was committed without acknowledging @ekieri 's contribution, so didn't follow the official guidelines: Attribution of Changes. IMHO, we should keep Emil as the author of this change (he has done the lion share of this highly non-trivial work) and add you as a co-author. Unless @ekieri has some other preference :) (I am happy as long as we follow the guidelines).

Either way, thank you both for contributing! Now, lets try to figure out what caused the issue with the bots.

Sorry for not following the official attribution guidelines. I agree with your suggestion to keep @ekieri as the main author. Thank you!

[peixin]

@peixin , wdyt?

@awarzynski Hi Andrzej, sorry for the late reply. I am distracted by several internal projects and other things in my life recently (just came back from vacation). My boss has not decided to let me continue working on LLVM Flang this year, yet.

[awarzynski]

In D134821#4317371, @peixin wrote:

@peixin , wdyt?

@awarzynski Hi Andrzej, sorry for the late reply. I am distracted by several internal projects and other things in my life recently (just came back from vacation). My boss has not decided to let me continue working on LLVM Flang this year, yet.

No worries, thank you for letting us know Peixin!

clang/lib/Driver/ToolChains/Gnu.cpp

void tools::gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
   // Silence warnings when linking C code with a C++ '-stdlib' argument.
   Args.ClaimAllArgs(options::OPT_stdlib_EQ);
 
   // Additional linker set-up and flags for Fortran. This is required in order
   // to generate executables. As Fortran runtime depends on the C runtime,
   // these dependencies need to be listed before the C runtime below (i.e.
   // AddRuntTimeLibs).
   if (D.IsFlangMode()) {
+    // A Fortran main program will be lowered to a function named _QQmain. Make
+    // _QQmain an undefined symbol, so that it's correctly resolved even when
+    // creating executable from archives. This is a workaround for how and where
+    // Flang's `main` is defined. For more context, see:
+    //   *  https://github.com/llvm/llvm-project/issues/54787
+    if (!Args.hasArg(options::OPT_shared))
+      CmdArgs.push_back("--undefined=_QQmain");
+
     addFortranRuntimeLibraryPath(ToolChain, Args, CmdArgs);
     addFortranRuntimeLibs(ToolChain, CmdArgs);
     CmdArgs.push_back("-lm");
   }
 
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_r)) {
     if (!Args.hasArg(options::OPT_nodefaultlibs)) {
       if (IsStatic || IsStaticPIE)

flang/docs/FlangDriver.md

 ```bash
 flang-new -ccc-print-phases -c file.f
          +- 0: input, "file.f", f95-cpp-input
       +- 1: preprocessor, {0}, f95
    +- 2: compiler, {1}, ir
 +- 3: backend, {2}, assembler
 4: assembler, {3}, object
 ```
-Note that currently Flang does not support code-generation and `flang-new` will
-fail during the second step above with the following error:
-```bash
-error: code-generation is not available yet
-```
-The other phases are printed nonetheless when using `-ccc-print-phases`, as
-that reflects what `clangDriver`, the library, will try to create and run.
 
 For actions specific to the frontend (e.g. preprocessing or code generation), a
 command to call the frontend driver is generated (more specifically, an
 instance of `clang::driver::Command`). Every command is bound to an instance of
 `clang::driver::Tool`. For Flang we introduced a specialisation of this class:
 `clang::driver::Flang`. This class implements the logic to either translate or
 forward compiler options to the frontend driver, `flang-new -fc1`.
 

 invoking the compiler driver, `flang-new`, with the `-fc1` flag.
 
 The frontend driver will only run one action at a time. If you specify multiple
 action flags, only the last one will be taken into account. The default action
 is `ParseSyntaxOnlyAction`, which corresponds to `-fsyntax-only`. In other
 words, `flang-new -fc1 <input-file>` is equivalent to `flang-new -fc1 -fsyntax-only
 <input-file>`.
 
+## Linker invocation
+> **_NOTE:_**  Linker invocation through the `flang-new` driver is so far
+> experimental. This section describes the currently intended design, and not
+> necessarily what is implemented.
+
+Calling
+```bash
+flang-new -flang-experimental-exec prog.f90
+```
+will, on a high level, do two things:
+* call the frontend driver, `flang-new -fc1`, to build the object file `prog.o`
+* call the system linker to build the executable `a.out`.
+
+In both invocations, `flang-new` will add default options to the frontend driver
+and the linker invocations. To see the exact invocations on your system, you can
+call
+```bash
+flang-new -### prog.f90
+```
+The link line will contain a fair number of options, which will depend on your
+system. Compared to when linking a C program with `clang`, the main additions
+are (on GNU/linux),
+* `--undefined=_QQmain`: A Fortran main program will appear in the corresponding
+  object file as a function called `_QQmain`. This flag makes sure that an
+  object file containing a Fortran main program (i.e., a symbol `_QQmain`) be
+  included in the linking also when it is bundled in an archive.
+* `-lFortran_main`: The Fortran_main archive is part of Flang's runtime. It
+  exports the symbol `main`, i.e., a c main function, which will make some
+  initial configuration before calling `_QQmain`, and clean up before returning.
+* `-lFortranRuntime`: Flang's Fortran runtime, containing, for example,
+  implementations of intrinsic functions.
+* `-lFortranDecimal`: Part of Flang's runtime, containing routines for parsing
+  and formatting decimal numbers.
+* `-lm`: Link with the math library, on which Flang's runtime depends.
+
 ## The `flang-to-external-fc` script
 The `flang-to-external-fc` wrapper script for `flang-new` was introduced as a
 development tool and to facilitate testing. The `flang-to-external-fc` wrapper
 script will:
 * use `flang-new` to unparse the input source file (i.e. it will run `flang-new
   -fc1 -fdebug-unparse <input-file>`), and then
 * call a host Fortran compiler, e.g. `gfortran`, to compile the unparsed file.
 

flang/test/CMakeLists.txt

set(FLANG_TEST_DEPENDS
   llvm-config
   FileCheck
   count
   not
   module_files
   fir-opt
   tco
   bbc
+  llvm-ar
   llvm-dis
   llvm-objdump
   llvm-readobj
   split-file
   FortranRuntime
   Fortran_main
   FortranDecimal
 )

flang/test/Driver/link-c-main.c

+/*
+Test that an object file with a C main function can be linked to an executable
+by Flang.
+
+For now, this test only covers the Gnu toolchain on Linux.
+
+REQUIRES: x86-registered-target || aarch64-registered-target || riscv64-registered-target
+REQUIRES: system-linux, c-compiler
+
+RUN: %cc -c %s -o %t.o
+RUN: %flang -target x86_64-unknown-linux-gnu %t.o -o %t.out -flang-experimental-exec
+RUN: llvm-objdump --syms %t.out | FileCheck %s --implicit-check-not Fortran
+
+Test that it also works if the c-main is bundled in an archive.
+
+RUN: llvm-ar -r %t.a %t.o
+RUN: %flang -target x86_64-unknown-linux-gnu %t.a -o %ta.out -flang-experimental-exec
+RUN: llvm-objdump --syms %ta.out | FileCheck %s --implicit-check-not Fortran
+*/
+
+int main(void) {
+    return 0;
+}
+
+/*
+CHECK-DAG: F .text {{[a-f0-9]+}} main
+CHECK-DAG: *UND* {{[a-f0-9]+}} _QQmain
+*/

flang/test/Driver/link-f90-main.f90

+! Test that a fortran main program can be linked to an executable
+! by flang.
+!
+! For now, this test only covers the Gnu toolchain on linux.
+
+!REQUIRES: x86-registered-target || aarch64-registered-target || riscv64-registered-target
+!REQUIRES: system-linux
+
+! RUN: %flang_fc1 -emit-obj %s -o %t.o
+! RUN: %flang -target x86_64-unknown-linux-gnu %t.o -o %t.out -flang-experimental-exec
+! RUN: llvm-objdump --syms %t.out | FileCheck %s
+
+! Test that it also works if the program is bundled in an archive.
+
+! RUN: llvm-ar -r %t.a %t.o
+! RUN: %flang -target x86_64-unknown-linux-gnu %t.a -o %ta.out -flang-experimental-exec
+! RUN: llvm-objdump --syms %ta.out | FileCheck %s
+
+end program
+
+! CHECK-DAG: F .text {{[a-f0-9]+}} main
+! CHECK-DAG: F .text {{[a-f0-9]+}} _QQmain
+! CHECK-DAG: _FortranAProgramStart

flang/test/Driver/linker-flags.f90

 ! Verify that the Fortran runtime libraries are present in the linker
 ! invocation. These libraries are added on top of other standard runtime
 ! libraries that the Clang driver will include.
 
 ! RUN: %flang -### -flang-experimental-exec -target ppc64le-linux-gnu %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,GNU
 ! RUN: %flang -### -flang-experimental-exec -target aarch64-apple-darwin %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,DARWIN
 ! RUN: %flang -### -flang-experimental-exec -target x86_64-windows-gnu %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,MINGW
 
 ! NOTE: Clang's driver library, clangDriver, usually adds 'libcmt' and
 !       'oldnames' on Windows, but they are not needed when compiling
 !       Fortran code and they might bring in additional dependencies.
 !       Make sure they're not added.
 ! RUN: %flang -### -flang-experimental-exec -target aarch64-windows-msvc %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,MSVC --implicit-check-not libcmt --implicit-check-not oldnames
 
+! Check linker invocation to generate shared object (only GNU toolchain for now)
+! Output should not contain any undefined reference to _QQmain since it is not
+! considered a valid entry point for shared objects, which are usually specified
+! using the bind attribute.
+! RUN: %flang -### -flang-experimental-exec -shared -target x86_64-linux-gnu %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,GNU-SHARED --implicit-check-not _QQmain
+! RUN: %flang -### -flang-experimental-exec -shared -target aarch64-linux-gnu %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,GNU-SHARED --implicit-check-not _QQmain
+! RUN: %flang -### -flang-experimental-exec -shared -target riscv64-linux-gnu %S/Inputs/hello.f90 2>&1 | FileCheck %s --check-prefixes=CHECK,GNU-SHARED --implicit-check-not _QQmain
+
 ! Compiler invocation to generate the object file
 ! CHECK-LABEL: {{.*}} "-emit-obj"
 ! CHECK-SAME:  "-o" "[[object_file:.*\.o]]" {{.*}}Inputs/hello.f90
 
 ! Linker invocation to generate the executable
 ! NOTE: Since we are cross-compiling, the host toolchain executables may
 !       run on any other platform, such as Windows that use a .exe
 !       suffix. Clang's driver will try to resolve the path to the ld
 !       executable and may find the GNU linker from MinGW or Cygwin.
 ! GNU-LABEL:  "{{.*}}ld{{(\.exe)?}}"
 ! GNU-SAME: "[[object_file]]"
+! GNU-SAME: --undefined=_QQmain
 ! GNU-SAME: -lFortran_main
 ! GNU-SAME: -lFortranRuntime
 ! GNU-SAME: -lFortranDecimal
 ! GNU-SAME: -lm
 
 ! DARWIN-LABEL:  "{{.*}}ld{{(\.exe)?}}"
 ! DARWIN-SAME: "[[object_file]]"
 ! DARWIN-SAME: -lFortran_main

 !       (lld-)link.exe on Windows platforms. The suffix may not be added
 !       when the executable is not found or on non-Windows platforms.
 ! MSVC-LABEL: link
 ! MSVC-SAME: Fortran_main.lib
 ! MSVC-SAME: FortranRuntime.lib
 ! MSVC-SAME: FortranDecimal.lib
 ! MSVC-SAME: /subsystem:console
 ! MSVC-SAME: "[[object_file]]"
+
+! Linker invocation to generate a shared object
+! GNU-SHARED-LABEL:  "{{.*}}ld"
+! GNU-SHARED-SAME: "[[object_file]]"
+! GNU-SHARED-SAME: -lFortranRuntime
+! GNU-SHARED-SAME: -lFortranDecimal