[lld] add options for context-sensitive PGO.

[xur]

Summary

Add lld options for CSPGO (context-sensitive PGO).

The main patch for CSPGO is here:
https://reviews.llvm.org/D54175

[ruiu]

You can submit this as long as the main patch is approved and submitted.

[ruiu]

ELF/Options.td

472–475 ↗

(On Diff #181590)

Actually, do you have to add these options? -plugin-opt=foo is an option for backward compatibility with gold. Since lld has a native LTO support, "plugin opt" doesn't make sense anymore. If they are new options, you shouldn't add -plugin-opt aliases.

[xur]

ELF/Options.td

472–475 ↗

(On Diff #181590)

Sorry I missed this comment.

I actually do use -plugin-opt options. If I remove these two options, I got errors in my test.
ld.lld: error: --plugin-opt: ld.lld: Unknown command line argument 'cs-profile-generate'.

The command line options are " .. -fcs-profile-generate=pass2 -flto=thin -fuse-ld=lld".

[xur]

ELF/Options.td

472–475 ↗

(On Diff #181590)

CSPGO options are handled in tools::AddGoldPlugin() (in lib/Driver/ToolChains/CommonArgs.cpp) for LTO/ThinLTO, and we add -plugin-opt there.
tools::AddGoldPlugin() is called regardless of what linker is used.
So I think lld needs to handle these -plugin-ops.

[xur]

Ping.

[ruiu]

Sorry for the belated response. (I was on vacation last week.)

ELF/Driver.cpp

801–802 ↗

(On Diff #181590)

Please sort ASCIIbetically.

ELF/Options.td

440–443 ↗

(On Diff #181590)

Are these new options, or is there a linker that already have these options? If they are new, is there any reason to choose a new name lto-cs-pgo-gen instead of lto-cs-profile-generate?

[xur]

Integrated Ruiu's review comments.

Thanks,

-Rong

[ruiu]

LGTM

ELF/Config.h

155 ↗

(On Diff #190169)

Remove = false as the default value would always be overwritten.

[dmajor]

Are there any bugs/reviews/etc. that I can subscribe to for the implementation of this in COFF?

[xur]

In D56675#1426458, @dmajor wrote:

Are there any bugs/reviews/etc. that I can subscribe to for the implementation of this in COFF?

Feature of the main implementation is reviewed here:
https://reviews.llvm.org/D54175
The change has been committed.

I'm not familiar with COFF. But I think you can just do similar things as in ELF: it just passes the plugin-opts to backend compilation.

ELF/Options.td

440–443 ↗

(On Diff #181590)

They are new options. I will change the name to -lto-cs-profile-generate.

lld/trunk/ELF/Config.h

struct Configuration {
   llvm::StringRef Chroot;
   llvm::StringRef DynamicLinker;
   llvm::StringRef DwoDir;
   llvm::StringRef Entry;
   llvm::StringRef Emulation;
   llvm::StringRef Fini;
   llvm::StringRef Init;
   llvm::StringRef LTOAAPipeline;
+  llvm::StringRef LTOCSProfileFile;
   llvm::StringRef LTONewPmPasses;
   llvm::StringRef LTOObjPath;
   llvm::StringRef LTOSampleProfile;
   llvm::StringRef MapFile;
   llvm::StringRef OutputFile;
   llvm::StringRef OptRemarksFilename;
   llvm::StringRef ProgName;
   llvm::StringRef SoName;
struct Configuration {
   bool GcSections;
   bool GdbIndex;
   bool GnuHash = false;
   bool GnuUnique;
   bool HasDynamicList = false;
   bool HasDynSymTab;
   bool IgnoreDataAddressEquality;
   bool IgnoreFunctionAddressEquality;
+  bool LTOCSProfileGenerate;
   bool LTODebugPassManager;
   bool LTONewPassManager;
   bool MergeArmExidx;
   bool MipsN32Abi = false;
   bool NoinhibitExec;
   bool Nostdlib;
   bool OFormatBinary;
   bool Omagic;

lld/trunk/ELF/Driver.cpp

void LinkerDriver::readConfigs(opt::InputArgList &Args) {
   Config->GdbIndex = Args.hasFlag(OPT_gdb_index, OPT_no_gdb_index, false);
   Config->ICF = getICF(Args);
   Config->IgnoreDataAddressEquality =
       Args.hasArg(OPT_ignore_data_address_equality);
   Config->IgnoreFunctionAddressEquality =
       Args.hasArg(OPT_ignore_function_address_equality);
   Config->Init = Args.getLastArgValue(OPT_init, "_init");
   Config->LTOAAPipeline = Args.getLastArgValue(OPT_lto_aa_pipeline);
+  Config->LTOCSProfileGenerate = Args.hasArg(OPT_lto_cs_profile_generate);
+  Config->LTOCSProfileFile = Args.getLastArgValue(OPT_lto_cs_profile_file);
   Config->LTODebugPassManager = Args.hasArg(OPT_lto_debug_pass_manager);
   Config->LTONewPassManager = Args.hasArg(OPT_lto_new_pass_manager);
   Config->LTONewPmPasses = Args.getLastArgValue(OPT_lto_newpm_passes);
   Config->LTOO = args::getInteger(Args, OPT_lto_O, 2);
   Config->LTOObjPath = Args.getLastArgValue(OPT_plugin_opt_obj_path_eq);
   Config->LTOPartitions = args::getInteger(Args, OPT_lto_partitions, 1);
   Config->LTOSampleProfile = Args.getLastArgValue(OPT_lto_sample_profile);
   Config->MapFile = Args.getLastArgValue(OPT_Map);

lld/trunk/ELF/LTO.cpp

static lto::Config createConfig() {
   C.RemarksFilename = Config->OptRemarksFilename;
   C.RemarksWithHotness = Config->OptRemarksWithHotness;
 
   C.SampleProfile = Config->LTOSampleProfile;
   C.UseNewPM = Config->LTONewPassManager;
   C.DebugPassManager = Config->LTODebugPassManager;
   C.DwoDir = Config->DwoDir;
 
+  C.CSIRProfile = Config->LTOCSProfileFile;
+  C.RunCSIRInstr = Config->LTOCSProfileGenerate;
+
   if (Config->EmitLLVM) {
     C.PostInternalizeModuleHook = [](size_t Task, const Module &M) {
       if (std::unique_ptr<raw_fd_ostream> OS = openFile(Config->OutputFile))
         WriteBitcodeToFile(M, *OS, false);
       return false;
     };
   }
 

lld/trunk/ELF/Options.td

 def lto_new_pass_manager: F<"lto-new-pass-manager">,
   HelpText<"Use new pass manager">;
 def lto_newpm_passes: J<"lto-newpm-passes=">,
   HelpText<"Passes to run during LTO">;
 def lto_O: J<"lto-O">, MetaVarName<"<opt-level>">,
   HelpText<"Optimization level for LTO">;
 def lto_partitions: J<"lto-partitions=">,
   HelpText<"Number of LTO codegen partitions">;
+def lto_cs_profile_generate: F<"lto-cs-profile-generate">,
+  HelpText<"Perform context senstive PGO instrumentation">;
+def lto_cs_profile_file: J<"lto-cs-profile-file=">,
+  HelpText<"Context sensitive profile file path">;
 def lto_sample_profile: J<"lto-sample-profile=">,
   HelpText<"Sample profile file path">;
 def disable_verify: F<"disable-verify">;
 defm mllvm: Eq<"mllvm", "Additional arguments to forward to LLVM's option processing">;
 def opt_remarks_filename: Separate<["--"], "opt-remarks-filename">,
   HelpText<"YAML output file for optimization remarks">;
 def opt_remarks_with_hotness: Flag<["--"], "opt-remarks-with-hotness">,
   HelpText<"Include hotness information in the optimization remarks file">;
def plugin_opt_dwo_dir_eq: J<"plugin-opt=dwo_dir=">,
   HelpText<"Directory to store .dwo files when LTO and debug fission are used">;
 def plugin_opt_emit_llvm: F<"plugin-opt=emit-llvm">;
 def: J<"plugin-opt=jobs=">, Alias<thinlto_jobs>, HelpText<"Alias for -thinlto-jobs">;
 def: J<"plugin-opt=lto-partitions=">, Alias<lto_partitions>, HelpText<"Alias for -lto-partitions">;
 def plugin_opt_mcpu_eq: J<"plugin-opt=mcpu=">;
 def: F<"plugin-opt=new-pass-manager">,
   Alias<lto_new_pass_manager>, HelpText<"Alias for -lto-new-pass-manager">;
 def plugin_opt_obj_path_eq: J<"plugin-opt=obj-path=">;
+def: F<"plugin-opt=cs-profile-generate">,
+  Alias<lto_cs_profile_generate>, HelpText<"Alias for -lto-cs-profile-generate">;
+def: J<"plugin-opt=cs-profile-path=">,
+  Alias<lto_cs_profile_file>, HelpText<"Alias for -lto-cs-profile-file">;
 def: J<"plugin-opt=sample-profile=">,
   Alias<lto_sample_profile>, HelpText<"Alias for -lto-sample-profile">;
 def: F<"plugin-opt=save-temps">, Alias<save_temps>, HelpText<"Alias for -save-temps">;
 def plugin_opt_thinlto_emit_imports_files: F<"plugin-opt=thinlto-emit-imports-files">;
 def plugin_opt_thinlto_index_only: F<"plugin-opt=thinlto-index-only">;
 def plugin_opt_thinlto_index_only_eq: J<"plugin-opt=thinlto-index-only=">;
 def plugin_opt_thinlto_object_suffix_replace_eq: J<"plugin-opt=thinlto-object-suffix-replace=">;
 def plugin_opt_thinlto_prefix_replace_eq: J<"plugin-opt=thinlto-prefix-replace=">;