[LTO] Add stats-file option to LTO/Config.h.

[fhahn]

Summary

This patch adds a StatsFile option to LTO/Config.h and updates both
LLVMGold and llvm-lto2 to set it.

[tejohnson]

A few thoughts/suggestions:

It would be better to add the calls to EnableStatistics and PrintStatisticsJSON to the LTO library, specifically inside LTO::run in LTO.cpp, so that all LTO clients can get the support with just an lto::Config change. E.g. the llvm-lto2 tool, lld.

What about -save-stats=cwd? It seems that both "obj" and "cwd" should be supported. But better than that:

• The -save-stats clang Driver option causes the -stats-file= cc1 option to be set. Better to support a -stats-file= option in gold-plugin and corresponding Config setting LTO, which is more generic than -save-stats.
• Once that is done, you can automatically set the gold-plugin option when -save-stats is passed to the cc1 invocation, in tools::AddGoldPlugin in clang. That simplifies the user model, and the option you need to pass in your corresponding CMake change, which can just be -save-stats=cwd.

Needs a test (assuming it is added to the LTO code, add both a gold and an llvm-lto2 based test).

[fhahn]

Thank you very much for your suggestions Teresa! I've updated the patch to handle a new StatsFile config option in LTO::run. And added stats-file support to LLVMgold.so and llvm-lto2. I will link the patch that updated AddGoldPlugin.

[tejohnson]

tools/gold/gold-plugin.cpp

202

document

1047

I think there is going to be a strange interaction with this code. I added this so that we still get the stats printed to stderr under the -stats llvm internal option when we exit early because we are only doing the ThinLTO indexing step (for distrubuted ThinLTO builds), since we don't execute ~StatisticInfo() in that case. The problem is that AreStatisticsEnabled() returns "Enabled || Stats", and Enabled will be true with this patch if the new stats-file plugin option is specified. So I think what would happen for distributed builds, when we exit early here, if the new plugin option is passed we will incorrectly dump stats to stderr as well.

Hmm, looks like we should probably just call llvm_shutdown() here before we exit, which would take care of destructing StatisticInfo. Let me try that and get back...

[tejohnson]

LGTM but please wait until after D45820 goes in.

tools/gold/gold-plugin.cpp

1047

Just mailed you D45820 that fixes this.

[fhahn]

Add comment to option in gold-plugin.cpp

tools/gold/gold-plugin.cpp

#endif
   // as plugin exclusive to pass to the code generator.
   static std::vector<const char *> extra;
   // Sample profile file path
   static std::string sample_profile;
   // New pass manager
   static bool new_pass_manager = false;
   // Debug new pass manager
   static bool debug_pass_manager = false;
+  static bool save_stats_obj = false;
 
   static void process_plugin_option(const char *opt_)
   {
     if (opt_ == nullptr)
       return;
     llvm::StringRef opt = opt_;
 
     if (opt.startswith("mcpu=")) {
static void process_plugin_option(const char *opt_)
     } else if (opt == "disable-verify") {
       DisableVerify = true;
     } else if (opt.startswith("sample-profile=")) {
       sample_profile= opt.substr(strlen("sample-profile="));
     } else if (opt == "new-pass-manager") {
       new_pass_manager = true;
     } else if (opt == "debug-pass-manager") {
       debug_pass_manager = true;
+    } else if (opt == "-save-stats=obj") {
+      save_stats_obj = true;
     } else {
       // Save this option to pass to the code generator.
       // ParseCommandLineOptions() expects argv[0] to be program name. Lazily
       // add that.
       if (extra.empty())
         extra.push_back("LLVMgold");
 
       extra.push_back(opt_);

 /// native object file. Then, perform optimization and codegen.
 static ld_plugin_status allSymbolsReadHook() {
   if (Modules.empty())
     return LDPS_OK;
 
   if (unsigned NumOpts = options::extra.size())
     cl::ParseCommandLineOptions(NumOpts, &options::extra[0]);
 
+  if (options::save_stats_obj)
+    llvm::EnableStatistics(false);
+
   std::vector<std::pair<SmallString<128>, bool>> Files = runLTO();
 
   if (options::TheOutputType == options::OT_DISABLE ||
       options::TheOutputType == options::OT_BC_ONLY)
     return LDPS_OK;
 
   if (options::thinlto_index_only) {
     if (llvm::AreStatisticsEnabled())
       llvm::PrintStatistics();
     cleanup_hook();
     exit(0);
   }
 
   for (const auto &F : Files)
     if (!F.first.empty())
       recordFile(F.first.str(), F.second);
 
   if (!options::extra_library_path.empty() &&
       set_extra_library_path(options::extra_library_path.c_str()) != LDPS_OK)
     message(LDPL_FATAL, "Unable to set the extra library path.");
 
+  if (options::save_stats_obj) {
+    std::error_code EC;
+    raw_fd_ostream OS(output_name+".stats", EC, sys::fs::OpenFlags::F_None);
+    if (EC)
+      message(LDPL_FATAL, "Failed to write the stats file.");
+    else
+      llvm::PrintStatisticsJSON(OS);
+  }
   return LDPS_OK;
 }
 
 static ld_plugin_status all_symbols_read_hook(void) {
   ld_plugin_status Ret = allSymbolsReadHook();
   llvm_shutdown();
 
   if (options::TheOutputType == options::OT_BC_ONLY ||