[Flang] Add support to use LTO specific pipelines

[mnadeem]

Summary

Thin and full LTO modes use different pre-link pipelines compared to regular compilation. This patch adds support for calling those pipelines.

This patch closely mimics clang implementation with the exception that I changed the codegen option name from PrepareForLTO to PrepareForFullLTO to be more precise.

With this patch:

• Compilation for full LTO should be as we expect (except possibly missing optimizations enabled by module summaries which we do not produce yet)
• thinLTO uses the correct prelink pipeline but will use the postlink backend for fullLTO due to missing metadata and summary in the llvm module. I have added a warning regarding this: flang-new: warning: the option '-flto=thin' is a work in progress

[mnadeem]

ping

[kiranchandramohan]

Thanks @mnadeem for this patch. Have you tried LTO and seen improvements? Or should flang emit additional info for this to work well?

This patch closely mimics clang behaviour with two exceptions:
No standalone option -flto for fc1. This is not needed if we always pass flto=xxx
Change the codegen option name from PrepareForLTO to PrepareForFullLTO to be more precise.

Is this a simplification/improvement or is there a strong case for these changes in Flang?

Are additional changes required for emitting assembly or bitcode?

flang/test/Driver/driver-help-hidden.f90

47

Does this option's description need a change since there are multiple LTO modes?

[mnadeem]

In D142420#4080402, @kiranchandramohan wrote:

Thanks @mnadeem for this patch. Have you tried LTO and seen improvements? Or should flang emit additional info for this to work well?

This patch closely mimics clang behaviour with two exceptions:
No standalone option -flto for fc1. This is not needed if we always pass flto=xxx
Change the codegen option name from PrepareForLTO to PrepareForFullLTO to be more precise.

Is this a simplification/improvement or is there a strong case for these changes in Flang?

Are additional changes required for emitting assembly or bitcode?

We have tried full LTO on Aarch64 (without this patch) and have seen a few 3-8% improvements in SPEC and a 100+% improvement in leslie3d.
There were a couple of additional failures in SPEC that we have yet to inspect but AFAICT no issue with respect to bitcode/assembly changes.

Is this a simplification/improvement or is there a strong case for these changes in Flang?

I have updated the change and now the only difference is the name for PrepareForFullLTO for clarity.

flang/test/Driver/driver-help-hidden.f90

47

No I dont think so. -flto is full lto.
-flto= can take either full or thin.

[kiranchandramohan]

We probably need tests for the following:
-> Additional linker invocation options are created. (Is this the plugin option?)
-> Test to ensure that the emitted object code contains the additional info require for LTO.

I see some additional info in the LLVM IR generated with LTO in clang. Example below. Isn't this info required for Flang as well?

^0 = module: (path: "", hash: (0, 0, 0, 0, 0))
^1 = gv: (name: "add", summaries: (function: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 1, live: 0, dsoLocal: 1, canAutoHide: 0), insts: 8, funcFlags: (readNone: 0, readOnly: 0, noRecurse: 0, returnDoesNotAlias: 0, noInline: 1, alwaysInline: 0, noUnwind: 1, mayThrow: 0, hasUnknownCall: 0, mustBeUnreachable: 0)))) ; guid = 2232412992676883508
^2 = flags: 8
^3 = blockcount: 1

flang/test/Driver/driver-help-hidden.f90

47

Ahh, OK. I got confused between -flto= and flto. Thanks.

[mnadeem]

In D142420#4081066, @kiranchandramohan wrote:

We probably need tests for the following:
-> Additional linker invocation options are created. (Is this the plugin option?)
-> Test to ensure that the emitted object code contains the additional info require for LTO.

I see some additional info in the LLVM IR generated with LTO in clang. Example below. Isn't this info required for Flang as well?

^0 = module: (path: "", hash: (0, 0, 0, 0, 0))
^1 = gv: (name: "add", summaries: (function: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 1, live: 0, dsoLocal: 1, canAutoHide: 0), insts: 8, funcFlags: (readNone: 0, readOnly: 0, noRecurse: 0, returnDoesNotAlias: 0, noInline: 1, alwaysInline: 0, noUnwind: 1, mayThrow: 0, hasUnknownCall: 0, mustBeUnreachable: 0)))) ; guid = 2232412992676883508
^2 = flags: 8
^3 = blockcount: 1

I'll plan to add a test for the linker invocation.

Regarding the additional info for LTO. I looked a bit and here is my understanding (I may be wrong):

• Module summary is not needed for full LTO, its only needed for thinLTO.
• Clang always emits a summary for full LTO (except for Apple triples) as it can enable some more optimizations. https://reviews.llvm.org/D34156 , https://reviews.llvm.org/D33922
• If the backend cannot find the module summary in thinLTO mode then the backend for fullLTO is run.
• ThinLTO can create a multi module bitcode files. Clang also does this for full LTO so that mixing thinlto+fulllto does not cause any errors https://reviews.llvm.org/D65009, https://reviews.llvm.org/D53891

The TODOs would be:

• Add support for -f[no]split-lto-unit
• Emit LTO summary
• Use ThinLTOBitcodeWriterPass for thin lto

To match clang behaviour I am thinking of just adding the functionality necessary for full LTO and leaving the thinLTO part for later and give a warning on using thinLTO. What do you think?

[awarzynski]

To match clang behaviour I am thinking of just adding the functionality necessary for full LTO and leaving the thinLTO part for later and give a warning on using thinLTO. What do you think?

Fine with me, provided this is clearly documented in commit summary.

clang/lib/Driver/ToolChains/Flang.cpp

306–312

Are these options parsed by the Clang driver library? If so, could you add a comment?

flang/lib/Frontend/CompilerInvocation.cpp

138–148

Please follow https://github.com/llvm/llvm-project/blob/main/flang/lib/Frontend/CompilerInvocation.cpp#L9 for the coding style. I see that that was not followed in the block below. Please feel encouraged to update that as well :) Preferably in a separate patch, but no problem if this happens here. And sorry for not pointing out before.

140–147

I wouldn't worry about the diagnostic in the frontend driver - it's intended for developers. You could add an assert instead.

flang/test/Driver/lto-flags.f90

2

How about -flto=jobserver, -flto=auto and -fno-lto?

15

[nit] Could you add an empty line to make this easier to parse.

18

[nit] You can probably skip one of the empty lines here.

[mnadeem]

• Address comments.
• Fix formatting for some old code as well.
• Update commit message.

[mnadeem]

flang/lib/Frontend/CompilerInvocation.cpp

138–148

Changed to camelBack

[mnadeem]

Add the new warning to a group to fix clang test failure.

[awarzynski]

Thanks for the updates! This mostly looks OK to me. I will be away for a week, but am happy for this to be merged once my comments are addressed.

thinLTO uses the correct prelink pipeline but will use the postlink backend for fullLTO due to missing information in the llvm module. I have added a warning regarding this: '-flto=thin' is only partially supported for flang

What does "partially" mean to the user? Bear in mind that you are exposing this to the end-user that will often be unfamiliar with the compiler internals. Perhaps rephrase as "work-in-progress" or "experimental"?

clang/include/clang/Basic/DiagnosticDriverKinds.td

116–118 ↗

(On Diff #492921)

1. I think that this diagnostic is unlikely to be re-used and you would probably just issue it inline.

2. What do you mean by`flang`? The driver is called flang-new and the subproject is called Flang. ATM, flang is merely a sub-directory within llvm-project.

3. You can skip "for flang" as the driver name will be displayed anyway (as in, this does not add any new information to the diagnostic).

clang/lib/Driver/ToolChains/Flang.cpp

306

It's "The Clang project" rather than "The clang project": https://clang.llvm.org/

[mnadeem]

clang/include/clang/Basic/DiagnosticDriverKinds.td

116–118 ↗

(On Diff #492921)

you would probably just issue it inline

Not sure what you mean by this, can you elaborate? Do you mean without adding it in DiagnosticDriverKinds.td? I don't know how to do that.

What do you mean by`flang`?
$ flang-new --help
OVERVIEW: flang LLVM compiler

You can skip "for flang" as the driver name will be displayed anyway (as in, this does not add any new information to the diagnostic).

Done

[tschuett]

Would it help to compile a lto.f90 to lto.o and check with llvm-objdump that there is a special section?

[mnadeem]

In D142420#4089859, @tschuett wrote:

Would it help to compile a lto.f90 to lto.o and check with llvm-objdump that there is a special section?

The .o is just llvm bitcode

clang/include/clang/Basic/DiagnosticDriverKinds.td

116–118 ↗

(On Diff #492921)

made it inline

[tschuett]

In D142420#4101422, @mnadeem wrote:

In D142420#4089859, @tschuett wrote:

Would it help to compile a lto.f90 to lto.o and check with llvm-objdump that there is a special section?

The .o is just llvm bitcode

Agreed. But you should be able to prove that LTO worked as expected. You should be able to distinguish LTO and non-LTO files. You only check the driver arguments.

[awarzynski]

In D142420#4101691, @tschuett wrote:

In D142420#4101422, @mnadeem wrote:

In D142420#4089859, @tschuett wrote:

Would it help to compile a lto.f90 to lto.o and check with llvm-objdump that there is a special section?

The .o is just llvm bitcode

Agreed. But you should be able to prove that LTO worked as expected. You should be able to distinguish LTO and non-LTO files. You only check the driver arguments.

I like this idea. And you could use llvm-dis instead that we already use in testing: https://github.com/llvm/llvm-project/blob/c7b1176e9afbfcc3da9482abbf7c1eb8793ff254/flang/test/Driver/emit-llvm-bc.f90#L3-L4

[kiranchandramohan]

ping @mnadeem. Do you need any inputs or helpf to complete this patch?

[mnadeem]

Add LTO bitcode test.

[kiranchandramohan]

LGTM. Please wait for @awarzynski.

flang/test/Driver/lto-bc.f90

3

Nit: Test that the output is LLVM bitcode for LTO and not a native objectfile by disassembling it to LLVM IR.

Nit: This probably can be taken up separately. There is metadata about ThinLTO being off in full mode.
!5 = !{i32 1, !"ThinLTO", i32 0}

22

Nit: newline

flang/test/Driver/lto-flags.f90

12

I see "-plugin-opt=mcpu=generic" for -flto in clang as well. Is it different in flang and only applicable to thin mode?

[awarzynski]

flang/test/Driver/lto-bc.f90

2–3

I thought that the purpose of this test was to demonstrate that adding -flto does change the output module. How come it does not?

[mnadeem]

update comments in the test and add newline at the end

[mnadeem]

flang/test/Driver/lto-bc.f90

3

Nit: This probably can be taken up separately. There is metadata about ThinLTO being off in full mode.

I put the TODOs in the commit message.

I thought that the purpose of this test was to demonstrate that adding -flto does change the output module. How come it does not?

More work is needed to support that, there is some discussion above.

flang/test/Driver/lto-flags.f90

12

I'm only checking for "-plugin-opt=thinlto" which is specific to thin lto and a differentiator between thin/full. The mcpu plugin option is present in both clang and flang and thin/full lto but I dont test for it.

[awarzynski]

[nit]

After this patch:

Do you mean "after this patch lands, I will work on this things ..." , or "after this patch lands, the following becomes available"? You could disambiguate by replacing with "With this patch:", wdyt?

flang/test/Driver/lto-bc.f90

3

I put the TODOs in the commit message.

IMHO, the commit message should focus on "what" and "why" is happening within this particular patch instead.

More work is needed to support that, there is some discussion above.

Please, could you make sure that the explanation is added to the commit message? As for the test - if there's no difference regardless of the flag, I would also add this RUN line:

! RUN: %flang %s -c -o - | llvm-dis -o - | FileCheck %s

[mnadeem]

In D142420#4164247, @awarzynski wrote:

[nit]

After this patch:

Do you mean "after this patch lands, I will work on this things ..." , or "after this patch lands, the following becomes available"? You could disambiguate by replacing with "With this patch:", wdyt?

Done. Replaced with "With this patch:"

Hi @awarzynski I think we are going in circles here. Or perhaps I am not understanding what you want here, either way it is frustrating for me because I don't have a clear picture of what I need to do to get this patch in.

Please let me know exactly what I need to get this patch in.

This patch "adds support to use LTO specific pipelines". Exceptions are mentioned in the commit message and can be worked on incrementally. There is more work needed to add LTO specific data in the module and it can also be added incrementally. The commit message already mentions that we are missing some information in the LLVM module.

I put the TODOs in the commit message.

IMHO, the commit message should focus on "what" and "why" is happening within this particular patch instead.

I have removed the TODOs from the commit message.

More work is needed to support that, there is some discussion above.

Please, could you make sure that the explanation is added to the commit message?

The current commit message already mentions it but I added a couple more words. Let me know if it is still unclear and I will be happy to update it.
To me, more work == todo. The TODOs which I have now removed said:

• Set module flags for thinlto and split lto unit
• Emit LTO summary for thin and/or full lto

As for the test - if there's no difference regardless of the flag, I would also add this RUN line:

! RUN: %flang %s -c -o - | llvm-dis -o - | FileCheck %s

It would produce an error: llvm-dis: error: file doesn't start with bitcode header
I am guessing non-lto compilation is probably already tested somewhere, but I can add this if it adds any value. Let me know.

I thought that the purpose of this test was to demonstrate that adding -flto does change the output module. How come it does not?

tschuett suggested that I add a test that compiles an x.f90 file and check that there is a special section. I responded that it is just a (normal) bitcode file, implying that there is no special section at this point.

Then they said that we need to be able distinguish LTO and non-LTO files and you supported the idea. So I added a test that compiles a file with LTO and pipes the output to llvm-dis to make sure that it is indeed LLVM bitcode and not an object file because that is is the only differentiator between LTO and non-LTO output at this time.

[awarzynski]

LGTM, many thanks for working on this!

I don't have a clear picture of what I need to do to get this patch in.

Please just ask for clarification if my comments are unclear. Also feel free to reach out on Flang's slack or via e-mail - I try to be responsive.

it is frustrating for me

I am sorry that you feel this way. I try to help with Flang driver as much as I can, but these days I'm distracted with other things and only review patches that others submit. And when there's a delay of more than a few days between revisions, I effectively have to remind myself the whole context. It's much easier for me when that's available in the summary.

Also, it seems that the LTO support is going to be split into multiple patches. IMHO it would make more sense to communicate the plan for this on Discourse rather than via a commit message (everything that's not explaining "what" and "why" is extra noise that could be avoided). This patch should implement the plan rather than define it.

tschuett suggested that I add a test that compiles an x.f90 file and check that there is a special section. I responded that it is just a (normal) bitcode file, implying that there is no special section at this point.

Yes, and together we have been able to come up with an alternative - that's great team work! You were able to address reviewer's comment and Flang gained a valuable regression tests ;-)

Testing the driver can be tricky, but it's not impossible.

As for the test - if there's no difference regardless of the flag, I would also add this RUN line:

! RUN: %flang %s -c -o - | llvm-dis -o - | FileCheck %s

It would produce an error: llvm-dis: error: file doesn't start with bitcode header

You can add -emit-llvm-bc. You can also add CHECK-NOT with stuff that is eventually going to appear in these modules at some point. That would help to understand/document what is coming. Usually there's a way to make things work in LLVM ;-) Feel free to ignore this suggestion.

[mnadeem]

Thanks awarzynski, I have been a bit busy but I will try to land this today after making the test changes.
In the future I'll try to communicate any planning beforehand, using discourse, to make reviewing easier. (I'm still a relatively new contributor to LLVM so still learning :) )

[tblah]

In D142420#4080619, @mnadeem wrote:

We have tried full LTO on Aarch64 (without this patch) and have seen a few 3-8% improvements in SPEC and a 100+% improvement in leslie3d.
There were a couple of additional failures in SPEC that we have yet to inspect but AFAICT no issue with respect to bitcode/assembly changes.

Hi @mnadeem, thanks for this patch. I'm having some trouble reproducing the improvement on leslie3d. Can you share more information about your configuration? Which flags did you use?

[mnadeem]

In D142420#4184825, @tblah wrote:

In D142420#4080619, @mnadeem wrote:

We have tried full LTO on Aarch64 (without this patch) and have seen a few 3-8% improvements in SPEC and a 100+% improvement in leslie3d.
There were a couple of additional failures in SPEC that we have yet to inspect but AFAICT no issue with respect to bitcode/assembly changes.

Hi @mnadeem, thanks for this patch. I'm having some trouble reproducing the improvement on leslie3d. Can you share more information about your configuration? Which flags did you use?

-O3 -flto -mllvm -fdynamic-heap-array on an Aarch64 device.
It was tested back in January, so if the improvement is not showing up right now then it may be possible that some other patch/optimization enabled better codegen in non-lto too.

clang/include/clang/Driver/Options.td

MarshallingInfoEnum<LangOpts<"LaxVectorConversions">,
                       open_cl.KeyPath #
                           " ? LangOptions::LaxVectorConversionKind::None" #
                           " : LangOptions::LaxVectorConversionKind::All">;
 def flax_vector_conversions : Flag<["-"], "flax-vector-conversions">, Group<f_Group>,
   Alias<flax_vector_conversions_EQ>, AliasArgs<["integer"]>;
 def flimited_precision_EQ : Joined<["-"], "flimited-precision=">, Group<f_Group>;
 def fapple_link_rtlib : Flag<["-"], "fapple-link-rtlib">, Group<f_Group>,
   HelpText<"Force linking the clang builtins runtime library">;
-def flto_EQ : Joined<["-"], "flto=">, Flags<[CoreOption, CC1Option]>, Group<f_Group>,
+def flto_EQ : Joined<["-"], "flto=">, Flags<[CoreOption, CC1Option, FC1Option, FlangOption]>, Group<f_Group>,
   HelpText<"Set LTO mode">, Values<"thin,full">;
 def flto_EQ_jobserver : Flag<["-"], "flto=jobserver">, Group<f_Group>,
   Alias<flto_EQ>, AliasArgs<["full"]>, HelpText<"Enable LTO in 'full' mode">;
 def flto_EQ_auto : Flag<["-"], "flto=auto">, Group<f_Group>,
   Alias<flto_EQ>, AliasArgs<["full"]>, HelpText<"Enable LTO in 'full' mode">;
-def flto : Flag<["-"], "flto">, Flags<[CoreOption, CC1Option]>, Group<f_Group>,
+def flto : Flag<["-"], "flto">, Flags<[CoreOption, CC1Option, FC1Option, FlangOption]>, Group<f_Group>,
   Alias<flto_EQ>, AliasArgs<["full"]>, HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Flags<[CoreOption, CC1Option]>, Group<f_Group>,
   HelpText<"Disable LTO mode (default)">;
 def foffload_lto_EQ : Joined<["-"], "foffload-lto=">, Flags<[CoreOption]>, Group<f_Group>,
   HelpText<"Set LTO mode for offload compilation">, Values<"thin,full">;
 def foffload_lto : Flag<["-"], "foffload-lto">, Flags<[CoreOption]>, Group<f_Group>,
   Alias<foffload_lto_EQ>, AliasArgs<["full"]>, HelpText<"Enable LTO in 'full' mode for offload compilation">;
 def fno_offload_lto : Flag<["-"], "fno-offload-lto">, Flags<[CoreOption]>, Group<f_Group>,

clang/lib/Driver/ToolChains/Flang.cpp

void Flang::ConstructJob(Compilation &C, const JobAction &JA,
                          const InputInfo &Output, const InputInfoList &Inputs,
                          const ArgList &Args, const char *LinkingOutput) const {
   const auto &TC = getToolChain();
   const llvm::Triple &Triple = TC.getEffectiveTriple();
   const std::string &TripleStr = Triple.getTriple();
 
   const Driver &D = TC.getDriver();
   ArgStringList CmdArgs;
+  DiagnosticsEngine &Diags = D.getDiags();
 
   // Invoke ourselves in -fc1 mode.
   CmdArgs.push_back("-fc1");
 
   // Add the "effective" target triple.
   CmdArgs.push_back("-triple");
   CmdArgs.push_back(Args.MakeArgString(TripleStr));
 
void Flang::ConstructJob(Compilation &C, const JobAction &JA,
 
   addFortranDialectOptions(Args, CmdArgs);
 
   // Color diagnostics are parsed by the driver directly from argv and later
   // re-parsed to construct this job; claim any possible color diagnostic here
   // to avoid warn_drv_unused_argument.
   Args.getLastArg(options::OPT_fcolor_diagnostics,
                   options::OPT_fno_color_diagnostics);
-  if (D.getDiags().getDiagnosticOptions().ShowColors)
+  if (Diags.getDiagnosticOptions().ShowColors)
     CmdArgs.push_back("-fcolor-diagnostics");
 
+  // LTO mode is parsed by the Clang driver library.
+  LTOKind LTOMode = D.getLTOMode(/* IsOffload */ false);
+  assert(LTOMode != LTOK_Unknown && "Unknown LTO mode.");
+  if (LTOMode == LTOK_Full)
+    CmdArgs.push_back("-flto=full");
+  else if (LTOMode == LTOK_Thin) {
+    Diags.Report(
+        Diags.getCustomDiagID(DiagnosticsEngine::Warning,
+                              "the option '-flto=thin' is a work in progress"));
+    CmdArgs.push_back("-flto=thin");
+  }
+
   // -fPIC and related options.
   addPicOptions(Args, CmdArgs);
 
   // Floating point related options
   addFloatingPointOptions(D, Args, CmdArgs);
 
   // Add target args, features, etc.
   addTargetOptions(Args, CmdArgs);

flang/include/flang/Frontend/CodeGenOptions.def

 CODEGENOPT(Name, Bits, Default)
 #endif
 
 CODEGENOPT(OptimizationLevel, 2, 0) ///< The -O[0-3] option specified.
 
 CODEGENOPT(DebugPassManager, 1, 0) ///< Prints debug information for the new
                                    ///< pass manager.
 
-CODEGENOPT(PICLevel, 2, 0) ///< PIC level of the LLVM module.
 CODEGENOPT(IsPIE, 1, 0) ///< PIE level is the same as PIC Level.
+CODEGENOPT(PICLevel, 2, 0) ///< PIC level of the LLVM module.
+CODEGENOPT(PrepareForFullLTO , 1, 0) ///< Set when -flto is enabled on the
+                                     ///< compile step.
+CODEGENOPT(PrepareForThinLTO , 1, 0) ///< Set when -flto=thin is enabled on the
+                                     ///< compile step.
 CODEGENOPT(StackArrays, 1, 0) ///< -fstack-arrays (enable the stack-arrays pass)
 
 CODEGENOPT(Underscoring, 1, 1)
 ENUM_CODEGENOPT(RelocationModel, llvm::Reloc::Model, 3, llvm::Reloc::PIC_) ///< Name of the relocation model to use.
 
 #undef CODEGENOPT
 #undef ENUM_CODEGENOPT

flang/lib/Frontend/CompilerInvocation.cpp

if (args.hasFlag(clang::driver::options::OPT_fstack_arrays,
                    clang::driver::options::OPT_fno_stack_arrays, false)) {
     opts.StackArrays = 1;
   }
 
   for (auto *a : args.filtered(clang::driver::options::OPT_fpass_plugin_EQ))
     opts.LLVMPassPlugins.push_back(a->getValue());
 
   // -fembed-offload-object option
   for (auto *a :
        args.filtered(clang::driver::options::OPT_fembed_offload_object_EQ))
     opts.OffloadObjects.push_back(a->getValue());
 
+  // -flto=full/thin option.
+  if (const llvm::opt::Arg *a =
+          args.getLastArg(clang::driver::options::OPT_flto_EQ)) {
+    llvm::StringRef s = a->getValue();
+    assert((s == "full" || s == "thin") && "Unknown LTO mode.");
+    if (s == "full")
+      opts.PrepareForFullLTO = true;
+    else
+      opts.PrepareForThinLTO = true;
+  }
+
   // -mrelocation-model option.
-  if (const llvm::opt::Arg *A =
+  if (const llvm::opt::Arg *a =
           args.getLastArg(clang::driver::options::OPT_mrelocation_model)) {
-    llvm::StringRef ModelName = A->getValue();
-    auto RM = llvm::StringSwitch<std::optional<llvm::Reloc::Model>>(ModelName)
-                  .Case("static", llvm::Reloc::Static)
-                  .Case("pic", llvm::Reloc::PIC_)
-                  .Case("dynamic-no-pic", llvm::Reloc::DynamicNoPIC)
-                  .Case("ropi", llvm::Reloc::ROPI)
-                  .Case("rwpi", llvm::Reloc::RWPI)
-                  .Case("ropi-rwpi", llvm::Reloc::ROPI_RWPI)
-                  .Default(std::nullopt);
-    if (RM.has_value())
-      opts.setRelocationModel(*RM);
+    llvm::StringRef modelName = a->getValue();
+    auto relocModel =
+        llvm::StringSwitch<std::optional<llvm::Reloc::Model>>(modelName)
+    if (relocModel.has_value())
+      opts.setRelocationModel(*relocModel);
     else
       diags.Report(clang::diag::err_drv_invalid_value)
-          << A->getAsString(args) << ModelName;
+          << a->getAsString(args) << modelName;
   }
 
   // -pic-level and -pic-is-pie option.
-  if (int PICLevel = getLastArgIntValue(
+  if (int picLevel = getLastArgIntValue(
           args, clang::driver::options::OPT_pic_level, 0, diags)) {
-    if (PICLevel > 2)
+    if (picLevel > 2)
       diags.Report(clang::diag::err_drv_invalid_value)
           << args.getLastArg(clang::driver::options::OPT_pic_level)
                  ->getAsString(args)
-          << PICLevel;
+          << picLevel;
 
-    opts.PICLevel = PICLevel;
+    opts.PICLevel = picLevel;
     if (args.hasArg(clang::driver::options::OPT_pic_is_pie))
       opts.IsPIE = 1;
   }
 
   // This option is compatible with -f[no-]underscoring in gfortran.
   if (args.hasFlag(clang::driver::options::OPT_fno_underscoring,
                    clang::driver::options::OPT_funderscoring, false)) {
     opts.Underscoring = 0;

flang/lib/Frontend/FrontendActions.cpp

#include "llvm/Support/Extension.def"
   pb.registerCGSCCAnalyses(cgam);
   pb.registerFunctionAnalyses(fam);
   pb.registerLoopAnalyses(lam);
   pb.crossRegisterProxies(lam, fam, cgam, mam);
 
   // Create the pass manager.
   llvm::ModulePassManager mpm;
   if (opts.OptimizationLevel == 0)
-    mpm = pb.buildO0DefaultPipeline(level, false);
+    mpm = pb.buildO0DefaultPipeline(level, opts.PrepareForFullLTO ||
+                                               opts.PrepareForThinLTO);
+  else if (opts.PrepareForFullLTO)
+    mpm = pb.buildLTOPreLinkDefaultPipeline(level);
+  else if (opts.PrepareForThinLTO)
+    mpm = pb.buildThinLTOPreLinkDefaultPipeline(level);
   else
     mpm = pb.buildPerModuleDefaultPipeline(level);
 
   if (action == BackendActionTy::Backend_EmitBC)
     mpm.addPass(llvm::BitcodeWriterPass(os));
 
   // Run the passes.
   mpm.run(*llvmModule, mam);

flang/test/Driver/default-optimization-pipelines.f90

 ! Verify that`-O{n}` is indeed taken into account when defining the LLVM optimization/middle-end pass pipeline.
 
 ! RUN: %flang -S -O0 %s -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0
+! RUN: %flang -S -O0 %s -flto -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0-ANYLTO
+! RUN: %flang -S -O0 %s -flto=thin -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0-ANYLTO
 ! RUN: %flang_fc1 -S -O0 %s -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0
+! RUN: %flang_fc1 -S -O0 %s -flto=full -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0-ANYLTO
+! RUN: %flang_fc1 -S -O0 %s -flto=thin -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0-ANYLTO
 
 ! RUN: %flang -S -O2 %s -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O2
+! RUN: %flang -S -O2 %s -flto -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O2-LTO
+! RUN: %flang -S -O2 %s -flto=thin -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O2-THINLTO
 ! RUN: %flang_fc1 -S -O2 %s -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O2
+! RUN: %flang_fc1 -S -O2 %s -flto=full -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O2-LTO
+! RUN: %flang_fc1 -S -O2 %s -flto=thin -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O2-THINLTO
 
 ! Verify that only the left-most `-O{n}` is used
 ! RUN: %flang -S -O2 -O0 %s -Xflang -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0
 ! RUN: %flang_fc1 -S -O2 -O0 %s -fdebug-pass-manager -o /dev/null 2>&1 | FileCheck %s --check-prefix=CHECK-O0
 
 ! CHECK-O0-NOT: Running pass: SimplifyCFGPass on simple_loop_
 ! CHECK-O0: Running analysis: TargetLibraryAnalysis on simple_loop_
+! CHECK-O0-ANYLTO: Running pass: CanonicalizeAliasesPass on [module]
+! CHECK-O0-ANYLTO: Running pass: NameAnonGlobalPass on [module]
 
 ! CHECK-O2: Running pass: SimplifyCFGPass on simple_loop_
 
+! CHECK-O2-LTO-NOT: Running pass: EliminateAvailableExternallyPass
+! CHECK-O2-LTO: Running pass: CanonicalizeAliasesPass on [module]
+! CHECK-O2-LTO: Running pass: NameAnonGlobalPass on [module]
+
+! CHECK-O2-THINLTO-NOT: Running pass: LoopVectorizePass
+! CHECK-O2-THINLTO: Running pass: CanonicalizeAliasesPass on [module]
+! CHECK-O2-THINLTO: Running pass: NameAnonGlobalPass on [module]
+
 subroutine simple_loop
   integer :: i
   do i=1,5
   end do
 end subroutine

flang/test/Driver/driver-help-hidden.f90

 ! CHECK-NEXT: -fimplicit-none        No implicit typing allowed unless overridden by IMPLICIT statements
 ! CHECK-NEXT: -finput-charset=<value> Specify the default character set for source files
 ! CHECK-NEXT: -fintrinsic-modules-path <dir>
 ! CHECK-NEXT:                        Specify where to find the compiled intrinsic modules
 ! CHECK-NEXT: -flang-experimental-exec
 ! CHECK-NEXT:                        Enable support for generating executables (experimental)
 ! CHECK-NEXT: -flarge-sizes          Use INTEGER(KIND=8) for the result type in size-related intrinsics
 ! CHECK-NEXT: -flogical-abbreviations Enable logical abbreviations
+! CHECK-NEXT: -flto=<value> Set LTO mode
+! CHECK-NEXT: -flto Enable LTO in 'full' mode
 ! CHECK-NEXT: -fno-automatic         Implies the SAVE attribute for non-automatic local objects in subprograms unless RECURSIVE
 ! CHECK-NEXT: -fno-color-diagnostics  Disable colors in diagnostics
 ! CHECK-NEXT: -fno-integrated-as     Disable the integrated assembler
 ! CHECK-NEXT: -fno-signed-zeros      Allow optimizations that ignore the sign of floating point zeros
 ! CHECK-NEXT: -fno-stack-arrays      Allocate array temporaries on the heap (default)
 ! CHECK-NEXT: -fopenacc              Enable OpenACC
 ! CHECK-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! CHECK-NEXT: -fpass-plugin=<dsopath> Load pass plugin from a dynamic shared object file (only with new pass manager).

flang/test/Driver/driver-help.f90

 ! HELP-NEXT: -ffp-contract=<value> Form fused FP ops (e.g. FMAs)
 ! HELP-NEXT: -ffree-form            Process source files in free form
 ! HELP-NEXT: -fimplicit-none        No implicit typing allowed unless overridden by IMPLICIT statements
 ! HELP-NEXT: -finput-charset=<value> Specify the default character set for source files
 ! HELP-NEXT: -fintrinsic-modules-path <dir>
 ! HELP-NEXT:                        Specify where to find the compiled intrinsic modules
 ! HELP-NEXT: -flarge-sizes          Use INTEGER(KIND=8) for the result type in size-related intrinsics
 ! HELP-NEXT: -flogical-abbreviations Enable logical abbreviations
+! HELP-NEXT: -flto=<value> Set LTO mode
+! HELP-NEXT: -flto Enable LTO in 'full' mode
 ! HELP-NEXT: -fno-automatic         Implies the SAVE attribute for non-automatic local objects in subprograms unless RECURSIVE
 ! HELP-NEXT: -fno-color-diagnostics  Disable colors in diagnostics
 ! HELP-NEXT: -fno-integrated-as      Disable the integrated assembler
 ! HELP-NEXT: -fno-signed-zeros      Allow optimizations that ignore the sign of floating point zeros
 ! HELP-NEXT: -fno-stack-arrays      Allocate array temporaries on the heap (default)
 ! HELP-NEXT: -fopenacc              Enable OpenACC
 ! HELP-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-NEXT: -fpass-plugin=<dsopath> Load pass plugin from a dynamic shared object file (only with new pass manager).

 ! HELP-FC1-NEXT:                        Get the symbol definition from <line> <start-column> <end-column>
 ! HELP-FC1-NEXT: -fget-symbols-sources   Dump symbols and their source code locations
 ! HELP-FC1-NEXT: -fimplicit-none        No implicit typing allowed unless overridden by IMPLICIT statements
 ! HELP-FC1-NEXT: -finput-charset=<value> Specify the default character set for source files
 ! HELP-FC1-NEXT: -fintrinsic-modules-path <dir>
 ! HELP-FC1-NEXT:                        Specify where to find the compiled intrinsic modules
 ! HELP-FC1-NEXT: -flarge-sizes          Use INTEGER(KIND=8) for the result type in size-related intrinsics
 ! HELP-FC1-NEXT: -flogical-abbreviations Enable logical abbreviations
+! HELP-FC1-NEXT: -flto=<value>           Set LTO mode
+! HELP-FC1-NEXT: -flto                   Enable LTO in 'full' mode
 ! HELP-FC1-NEXT: -fno-analyzed-objects-for-unparse
 ! HELP-FC1-NEXT:                        Do not use the analyzed objects when unparsing
 ! HELP-FC1-NEXT: -fno-automatic         Implies the SAVE attribute for non-automatic local objects in subprograms unless RECURSIVE
 ! HELP-FC1-NEXT: -fno-debug-pass-manager Disables debug printing for the new pass manager
 ! HELP-FC1-NEXT: -fno-reformat          Dump the cooked character stream in -E mode
 ! HELP-FC1-NEXT: -fno-signed-zeros      Allow optimizations that ignore the sign of floating point zeros
 ! HELP-FC1-NEXT: -fno-stack-arrays      Allocate array temporaries on the heap (default)
 ! HELP-FC1-NEXT: -fopenacc              Enable OpenACC

flang/test/Driver/lto-bc.f90

+! Test that the output is LLVM bitcode for LTO and not a native objectfile by
+! disassembling it to LLVM IR.
+! Right now there is nothing special about it and it is similar to non-lto IR,
+! more work is needed to add things like module summaries.
+
+! RUN: %flang %s -c -o - | not llvm-dis -o %t
+! RUN: %flang_fc1 %s -emit-llvm-bc -o - | llvm-dis -o - | FileCheck %s
+
+! RUN: %flang -flto %s -c -o - | llvm-dis -o - | FileCheck %s
+! RUN: %flang -flto=thin %s -c -o - | llvm-dis -o - | FileCheck %s
+
+! CHECK: define void @_QQmain()
+! CHECK-NEXT:  ret void
+! CHECK-NEXT: }
+
+! CHECK-NOT: ^0 = module:
+! CHECK-NOT: ^1 = gv: (name:
+! CHECK-NOT: ^2 = flags:
+! CHECK-NOT: ^3 = blockcount:
+
+end program

flang/test/Driver/lto-flags.f90

+! UNSUPPORTED: system-windows
+! RUN: %flang -### -S %s 2>&1 | FileCheck %s --check-prefix=NO-LTO
+! RUN: %flang -### -S -fno-lto %s 2>&1 | FileCheck %s --check-prefix=NO-LTO
+
+! Full LTO and aliases.
+! RUN: %flang -### -S -flto %s 2>&1 | FileCheck %s --check-prefix=FULL-LTO
+! RUN: %flang -### -S -flto=full %s 2>&1 | FileCheck %s --check-prefix=FULL-LTO
+! RUN: %flang -### -S -flto=auto %s 2>&1 | FileCheck %s --check-prefix=FULL-LTO
+! RUN: %flang -### -S -flto=jobserver %s 2>&1 | FileCheck %s --check-prefix=FULL-LTO
+
+! Also check linker plugin opt for Thin LTO
+! RUN: %flang -### -flto=thin %s 2>&1 | FileCheck %s --check-prefix=THIN-LTO
+
+! RUN: %flang -### -S -flto=somelto %s 2>&1 | FileCheck %s --check-prefix=ERROR
+
+! FC1 tests. Check that it does not crash.
+! RUN: %flang_fc1 -S %s -flto -o /dev/null
+! RUN: %flang_fc1 -S %s -flto=full -o /dev/null
+! RUN: %flang_fc1 -S %s -flto=thin -o /dev/null
+
+! NO-LTO: "-fc1"
+! NO-LTO-NOT: flto
+
+! FULL-LTO: "-fc1"
+! FULL-LTO-SAME: "-flto=full"
+
+! THIN-LTO: flang-new: warning: the option '-flto=thin' is a work in progress
+! THIN-LTO: "-fc1"
+! THIN-LTO-SAME: "-flto=thin"
+! THIN-LTO: "-plugin-opt=thinlto"
+
+! ERROR: error: unsupported argument 'somelto' to option '-flto=