[lld] Add a flag to enable split machine functions for LTO.

[snehasish]

Summary

The machine function splitter is a codegen optimization pass which is
only available on X86 ELF and not turned on by default. To allow LTO
builds to take advantage of this optimization we add a flag -
--lto-split-machine-functions.

[tejohnson]

Is it expected that the user will want to specify this only at link time and not during compile time? If it is normally specified in the compile step, you should consider adding it to the IR in some fashion (e.g. function attributes). The advantage is that the user doesn't need to pass different options for the LTO and non-LTO cases.

lld/test/ELF/lto/split-machine-functions.ll

10

Check that the opposite occurs by default and with the "no" variant

[snehasish]

In D88041#2287900, @tejohnson wrote:

Is it expected that the user will want to specify this only at link time and not during compile time? If it is normally specified in the compile step, you should consider adding it to the IR in some fashion (e.g. function attributes). The advantage is that the user doesn't need to pass different options for the LTO and non-LTO cases.

Ideally we should only have to specify this once. However, using function attributes doesn't seem ideal since the pass will be scheduled and repeatedly invoked only to return without actually running the pass. It would be cleaner to marshal the codegen specific options from the compile invocation and restore them for the LTO step. There are a couple of other codegen options which would also benefit from this approach --lto-unique-basic-block-section-names, --lto-basic-block-sections=<value>. @mtrofin pointed out that -fembed-bitcode saves the invocation in .llvmcmd. A similar approach to stash codegen specific options always for LTO to pick up and enable might be less intrusive. However, this is a larger effort and for current LTO builds it would be nice to have a command line option to enable it. WDYT about this alternative?

[tejohnson]

In D88041#2288858, @snehasish wrote:

In D88041#2287900, @tejohnson wrote:

Is it expected that the user will want to specify this only at link time and not during compile time? If it is normally specified in the compile step, you should consider adding it to the IR in some fashion (e.g. function attributes). The advantage is that the user doesn't need to pass different options for the LTO and non-LTO cases.

Ideally we should only have to specify this once. However, using function attributes doesn't seem ideal since the pass will be scheduled and repeatedly invoked only to return without actually running the pass. It would be cleaner to marshal the codegen specific options from the compile invocation and restore them for the LTO step. There are a couple of other codegen options which would also benefit from this approach --lto-unique-basic-block-section-names, --lto-basic-block-sections=<value>. @mtrofin pointed out that -fembed-bitcode saves the invocation in .llvmcmd. A similar approach to stash codegen specific options always for LTO to pick up and enable might be less intrusive. However, this is a larger effort and for current LTO builds it would be nice to have a command line option to enable it. WDYT about this alternative?

If you want it determined at the module level, then you can use a module level flag metadata. See Module::addModuleFlag(). You would just need to figure out how to combine flags from different modules when merging them for *LTO. See ModFlagBehavior. The advantage is that you wouldn't need to check this for every function. But the disadvantage is that you lose the ability to mix and match different behavior for functions from different modules after any kind of LTO merging.

[snehasish]

Let me look into module flags a little bit and I'll come back to this patch once I have a better understanding. Thanks for the comments!

lld/ELF/Config.h

struct Configuration {
   bool hasDynSymTab;
   bool ignoreDataAddressEquality;
   bool ignoreFunctionAddressEquality;
   bool ltoCSProfileGenerate;
   bool ltoDebugPassManager;
   bool ltoEmitAsm;
   bool ltoNewPassManager;
   bool ltoUniqueBasicBlockSectionNames;
+  bool ltoSplitMachineFunctions;
   bool ltoWholeProgramVisibility;
   bool mergeArmExidx;
   bool mipsN32Abi = false;
   bool mmapOutputFile;
   bool nmagic;
   bool noDynamicLinker = false;
   bool noinhibitExec;
   bool nostdlib;

lld/ELF/Driver.cpp

static void readConfigs(opt::InputArgList &args) {
   config->ltoObjPath = args.getLastArgValue(OPT_lto_obj_path_eq);
   config->ltoPartitions = args::getInteger(args, OPT_lto_partitions, 1);
   config->ltoSampleProfile = args.getLastArgValue(OPT_lto_sample_profile);
   config->ltoBasicBlockSections =
       args.getLastArgValue(OPT_lto_basic_block_sections);
   config->ltoUniqueBasicBlockSectionNames =
       args.hasFlag(OPT_lto_unique_basic_block_section_names,
                    OPT_no_lto_unique_basic_block_section_names, false);
+  config->ltoSplitMachineFunctions =
+      args.hasFlag(OPT_lto_split_machine_functions,
+                   OPT_no_lto_split_machine_functions, false);
   config->mapFile = args.getLastArgValue(OPT_Map);
   config->mipsGotSize = args::getInteger(args, OPT_mips_got_size, 0xfff0);
   config->mergeArmExidx =
       args.hasFlag(OPT_merge_exidx_entries, OPT_no_merge_exidx_entries, true);
   config->mmapOutputFile =
       args.hasFlag(OPT_mmap_output_file, OPT_no_mmap_output_file, true);
   config->nmagic = args.hasFlag(OPT_nmagic, OPT_no_nmagic, false);
   config->noinhibitExec = args.hasArg(OPT_noinhibit_exec);

lld/ELF/LTO.cpp

if (config->ltoBasicBlockSections == "all") {
       }
       c.Options.BBSections = BasicBlockSection::List;
     }
   }
 
   c.Options.UniqueBasicBlockSectionNames =
       config->ltoUniqueBasicBlockSectionNames;
 
+  c.Options.EnableMachineFunctionSplitter = config->ltoSplitMachineFunctions;
+
   if (auto relocModel = getRelocModelFromCMModel())
     c.RelocModel = *relocModel;
   else if (config->relocatable)
     c.RelocModel = None;
   else if (config->isPic)
     c.RelocModel = Reloc::PIC_;
   else
     c.RelocModel = Reloc::Static;

lld/ELF/Options.td

 def opt_remarks_format: Separate<["--"], "opt-remarks-format">,
   HelpText<"The format used for serializing remarks (default: YAML)">;
 def save_temps: F<"save-temps">;
 def lto_basic_block_sections: JJ<"lto-basic-block-sections=">,
   HelpText<"Enable basic block sections for LTO">;
 defm lto_unique_basic_block_section_names: BB<"lto-unique-basic-block-section-names",
     "Give unique names to every basic block section for LTO",
     "Do not give unique names to every basic block section for LTO (default)">;
+defm lto_split_machine_functions: BB<"lto-split-machine-functions",
+     "Enable late function splitting using profile information for LTO (x86 ELF)",
+     "Disable late function splitting using profile information for LTO (x86 ELF)">;
 def shuffle_sections: JJ<"shuffle-sections=">, MetaVarName<"<seed>">,
   HelpText<"Shuffle input sections using the given seed. If 0, use a random seed">;
 def thinlto_cache_dir: JJ<"thinlto-cache-dir=">,
   HelpText<"Path to ThinLTO cached object file directory">;
 defm thinlto_cache_policy: EEq<"thinlto-cache-policy", "Pruning policy for the ThinLTO cache">;
 def thinlto_emit_imports_files: FF<"thinlto-emit-imports-files">;
 def thinlto_index_only: FF<"thinlto-index-only">;
 def thinlto_index_only_eq: JJ<"thinlto-index-only=">;

lld/test/ELF/lto/split-machine-functions.ll

+; REQUIRES: x86
+; RUN: llvm-as %s -o %t.o
+; RUN: ld.lld %t.o -o %t --lto-split-machine-functions --lto-O0 --save-temps
+; RUN: llvm-nm %t.lto.o | FileCheck %s
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @foo1(i1 zeroext %0) nounwind !prof !14 !section_prefix !15 {
+;; Check that cold block is split out with new symbol. 
+; CHECK:  foo1.cold
+  br i1 %0, label %2, label %4, !prof !17
+
+2:                                                ; preds = %1
+  %3 = call i32 @bar()
+  br label %6
+
+4:                                                ; preds = %1
+  %5 = call i32 @baz()
+  br label %6
+
+6:                                                ; preds = %4, %2
+  ret void
+}
+
+define i32 @bar() noinline optnone { 
+  ret i32 1; 
+} 
+
+define i32 @baz() noinline optnone { 
+  ret i32 1; 
+} 
+
+!llvm.module.flags = !{!0}
+!0 = !{i32 1, !"ProfileSummary", !1}
+!1 = !{!2, !3, !4, !5, !6, !7, !8, !9}
+!2 = !{!"ProfileFormat", !"InstrProf"}
+!3 = !{!"TotalCount", i64 10000}
+!4 = !{!"MaxCount", i64 10}
+!5 = !{!"MaxInternalCount", i64 1}
+!6 = !{!"MaxFunctionCount", i64 1000}
+!7 = !{!"NumCounts", i64 3}
+!8 = !{!"NumFunctions", i64 5}
+!9 = !{!"DetailedSummary", !10}
+!10 = !{!11, !12, !13}
+!11 = !{i32 10000, i64 100, i32 1}
+!12 = !{i32 999900, i64 100, i32 1}
+!13 = !{i32 999999, i64 1, i32 2}
+!14 = !{!"function_entry_count", i64 7000}
+!15 = !{!"function_section_prefix", !".hot"}
+!16 = !{!"function_section_prefix", !".unlikely"}
+!17 = !{!"branch_weights", i32 7000, i32 0}