[Driver] Allow to collect `-save-stats` data to a file specified in the environment variable.

[vsapsai]

Summary

Using two environment variables CC_PRINT_INTERNAL_STAT and
CC_PRINT_INTERNAL_STAT_FILE to work like CC_PRINT_PROC_STAT.

The purpose of the change is to allow collecting the internal stats
without modifying the build scripts. Write all stats to a single file
to simplify aggregating the data.

[ahatanak]

clang/lib/Frontend/CompilerInstance.cpp

1095

OF_Append is necessary when the results are written to a single file specified via CC_PRINT_INTERNAL_STAT_FILE, but do we want this change for users passing -save-stats to the command line?

[vsapsai]

clang/lib/Frontend/CompilerInstance.cpp

1095

For implicit modules appending seems to be correct and desirable as we store information about all CompilerInstance and not only for the last one. But as llvm-test-suite https://github.com/llvm/llvm-test-suite/blob/main/litsupport/modules/stats.py doesn't expect appended stats, I'll look into doing it only for environment-driven stats collection.

[vsapsai]

Append to stats file when using CC_PRINT_INTERNAL_STAT and for LTO when reusing the same stats file.

[ahatanak]

clang/test/Driver/save-stats.c

26

Doesn't this require stats-file-append to be supported by the plugin just like stats-file is supported?

https://github.com/llvm/llvm-project/blob/main/llvm/tools/gold/gold-plugin.cpp#L309

[ahatanak]

clang/test/Driver/save-stats.c

26

Do we want to pass stats-file-append in tools::addLTOOptions?

[vsapsai]

clang/test/Driver/save-stats.c

26

You are right with the gold-plugin. I think I'll drop any changes for LTO now because that's not my purpose and I don't want to change emission of stats for something I don't work on. Also I believe it creates discrepancies for different linker plugins and I really don't want to introduce this inconsistency.

[vsapsai]

Don't touch LTO pipeline to avoid breaking existing workflows.

[vsapsai]

Adding folks involved with LTO. The problem is that stats emitted during the first round of compilation are likely to be overwritten by stats emitted during [Thin]LTO itself. Not touching it now to preserve the existing behavior. But wanted to inform you about this shortcoming in case it is important for you to fix.

Original change adding -save-stats for LTO is D45531.

[ahatanak]

LGTM

[vsapsai]

Thanks for the review!

clang/docs/CommandGuide/clang.rst

.. option:: -save-temps
   Save intermediate compilation results.
 
 .. option:: -save-stats, -save-stats=cwd, -save-stats=obj
 
   Save internal code generation (LLVM) statistics to a file in the current
   directory (:option:`-save-stats`/"-save-stats=cwd") or the directory
   of the output file ("-save-state=obj").
 
+  You can also use environment variables to control the statistics reporting.
+  Setting ``CC_PRINT_INTERNAL_STAT`` to ``1`` enables the feature, the report
+  goes to stdout in JSON format.
+
+  Setting ``CC_PRINT_INTERNAL_STAT_FILE`` to a file path makes it report
+  statistics to the given file in the JSON format.
+
+  Note that ``-save-stats`` take precedence over ``CC_PRINT_INTERNAL_STAT``
+  and ``CC_PRINT_INTERNAL_STAT_FILE``.
+
 .. option:: -integrated-as, -no-integrated-as
 
   Used to enable and disable, respectively, the use of the integrated
   assembler. Whether the integrated assembler is on by default is target
   dependent.
 
 .. option:: -time
 

clang/include/clang/Driver/Driver.h

public:
   std::string DriverTitle;
 
   /// Information about the host which can be overridden by the user.
   std::string HostBits, HostMachine, HostSystem, HostRelease;
 
   /// The file to log CC_PRINT_PROC_STAT_FILE output to, if enabled.
   std::string CCPrintStatReportFilename;
 
+  /// The file to log CC_PRINT_INTERNAL_STAT_FILE output to, if enabled.
+  std::string CCPrintInternalStatReportFilename;
+
   /// The file to log CC_PRINT_OPTIONS output to, if enabled.
   std::string CCPrintOptionsFilename;
 
   /// The file to log CC_PRINT_HEADERS output to, if enabled.
   std::string CCPrintHeadersFilename;
 
   /// The file to log CC_LOG_DIAGNOSTICS output to, if enabled.
   std::string CCLogDiagnosticsFilename;
public:
 
   /// Whether the driver is generating diagnostics for debugging purposes.
   unsigned CCGenDiagnostics : 1;
 
   /// Set CC_PRINT_PROC_STAT mode, which causes the driver to dump
   /// performance report to CC_PRINT_PROC_STAT_FILE or to stdout.
   unsigned CCPrintProcessStats : 1;
 
+  /// Set CC_PRINT_INTERNAL_STAT mode, which causes the driver to dump internal
+  /// performance report to CC_PRINT_INTERNAL_STAT_FILE or to stdout.
+  unsigned CCPrintInternalStats : 1;
+
   /// Pointer to the ExecuteCC1Tool function, if available.
   /// When the clangDriver lib is used through clang.exe, this provides a
   /// shortcut for executing the -cc1 command-line directly, in the same
   /// process.
   using CC1ToolFunc =
       llvm::function_ref<int(SmallVectorImpl<const char *> &ArgV)>;
   CC1ToolFunc CC1Main = nullptr;
 

clang/include/clang/Driver/Options.td

def opt_record_format : Separate<["-"], "opt-record-format">,
   HelpText<"The format used for serializing remarks (default: YAML)">;
 
 def print_stats : Flag<["-"], "print-stats">,
   HelpText<"Print performance metrics and statistics">,
   MarshallingInfoFlag<FrontendOpts<"ShowStats">>;
 def stats_file : Joined<["-"], "stats-file=">,
   HelpText<"Filename to write statistics to">,
   MarshallingInfoString<FrontendOpts<"StatsFile">>;
+def stats_file_append : Flag<["-"], "stats-file-append">,
+  HelpText<"If stats should be appended to stats-file instead of overwriting it">,
+  MarshallingInfoFlag<FrontendOpts<"AppendStats">>;
 def fdump_record_layouts_simple : Flag<["-"], "fdump-record-layouts-simple">,
   HelpText<"Dump record layout information in a simple form used for testing">,
   MarshallingInfoFlag<LangOpts<"DumpRecordLayoutsSimple">>;
 def fdump_record_layouts_canonical : Flag<["-"], "fdump-record-layouts-canonical">,
   HelpText<"Dump record layout information with canonical field types">,
   MarshallingInfoFlag<LangOpts<"DumpRecordLayoutsCanonical">>;
 def fdump_record_layouts_complete : Flag<["-"], "fdump-record-layouts-complete">,
   HelpText<"Dump record layout information for all complete types">,

clang/include/clang/Frontend/FrontendOptions.h

public:
   unsigned RelocatablePCH : 1;
 
   /// Show the -help text.
   unsigned ShowHelp : 1;
 
   /// Show frontend performance metrics and statistics.
   unsigned ShowStats : 1;
 
+  unsigned AppendStats : 1;
+
   /// print the supported cpus for the current target
   unsigned PrintSupportedCPUs : 1;
 
   /// Output time trace profile.
   unsigned TimeTrace : 1;
 
   /// Show the -version text.
   unsigned ShowVersion : 1;
public:
   unsigned TimeTraceGranularity;
 
   /// Path which stores the output files for -ftime-trace
   std::string TimeTracePath;
 
 public:
   FrontendOptions()
       : DisableFree(false), RelocatablePCH(false), ShowHelp(false),
-        ShowStats(false), TimeTrace(false), ShowVersion(false),
-        FixWhatYouCan(false), FixOnlyWarnings(false), FixAndRecompile(false),
-        FixToTemporaries(false), ARCMTMigrateEmitARCErrors(false),
-        SkipFunctionBodies(false), UseGlobalModuleIndex(true),
-        GenerateGlobalModuleIndex(true), ASTDumpDecls(false),
-        ASTDumpLookups(false), BuildingImplicitModule(false),
-        BuildingImplicitModuleUsesLock(true), ModulesEmbedAllFiles(false),
-        IncludeTimestamps(true), UseTemporary(true),
-        AllowPCMWithCompilerErrors(false), ModulesShareFileManager(true),
-        TimeTraceGranularity(500) {}
+        ShowStats(false), AppendStats(false), TimeTrace(false),
+        ShowVersion(false), FixWhatYouCan(false), FixOnlyWarnings(false),
+        FixAndRecompile(false), FixToTemporaries(false),
+        ARCMTMigrateEmitARCErrors(false), SkipFunctionBodies(false),
+        UseGlobalModuleIndex(true), GenerateGlobalModuleIndex(true),
+        ASTDumpDecls(false), ASTDumpLookups(false),
+        BuildingImplicitModule(false), BuildingImplicitModuleUsesLock(true),
+        ModulesEmbedAllFiles(false), IncludeTimestamps(true),
+        UseTemporary(true), AllowPCMWithCompilerErrors(false),
+        ModulesShareFileManager(true), TimeTraceGranularity(500) {}
 
   /// getInputKindForExtension - Return the appropriate input kind for a file
   /// extension. For example, "c" would return Language::C.
   ///
   /// \return The input kind for the extension, or Language::Unknown if the
   /// extension is not recognized.
   static InputKind getInputKindForExtension(StringRef Extension);
 };
 
 } // namespace clang
 
 #endif // LLVM_CLANG_FRONTEND_FRONTENDOPTIONS_H

clang/lib/Driver/Driver.cpp

Driver::Driver(StringRef ClangExecutable, StringRef TargetTriple,
                IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS)
     : Diags(Diags), VFS(std::move(VFS)), Mode(GCCMode),
       SaveTemps(SaveTempsNone), BitcodeEmbed(EmbedNone),
       Offload(OffloadHostDevice), CXX20HeaderType(HeaderMode_None),
       ModulesModeCXX20(false), LTOMode(LTOK_None),
       ClangExecutable(ClangExecutable), SysRoot(DEFAULT_SYSROOT),
       DriverTitle(Title), CCCPrintBindings(false), CCPrintOptions(false),
       CCLogDiagnostics(false), CCGenDiagnostics(false),
-      CCPrintProcessStats(false), TargetTriple(TargetTriple), Saver(Alloc),
-      PrependArg(nullptr), CheckInputsExist(true), ProbePrecompiled(true),
+      CCPrintProcessStats(false), CCPrintInternalStats(false),
+      TargetTriple(TargetTriple), Saver(Alloc), PrependArg(nullptr),
+      CheckInputsExist(true), ProbePrecompiled(true),
       SuppressMissingInputWarning(false) {
   // Provide a sane fallback if no VFS is specified.
   if (!this->VFS)
     this->VFS = llvm::vfs::getRealFileSystem();
 
   Name = std::string(llvm::sys::path::filename(ClangExecutable));
   Dir = std::string(llvm::sys::path::parent_path(ClangExecutable));
   InstalledDir = Dir; // Provide a sensible default installed dir.

clang/lib/Driver/ToolChains/Clang.cpp

   for (const Arg *A : Args.filtered(options::OPT_vfsoverlay)) {
     CmdArgs.push_back("--vfsoverlay");
     CmdArgs.push_back(A->getValue());
     A->claim();
   }
 
   // Setup statistics file output.
   SmallString<128> StatsFile = getStatsFileName(Args, Output, Input, D);
-  if (!StatsFile.empty())
+  if (!StatsFile.empty()) {
     CmdArgs.push_back(Args.MakeArgString(Twine("-stats-file=") + StatsFile));
+    if (D.CCPrintInternalStats)
+      CmdArgs.push_back("-stats-file-append");
+  }
 
   // Forward -Xclang arguments to -cc1, and -mllvm arguments to the LLVM option
   // parser.
   for (auto Arg : Args.filtered(options::OPT_Xclang)) {
     Arg->claim();
     // -finclude-default-header flag is for preprocessor,
     // do not pass it to other cc1 commands when save-temps is enabled
     if (C.getDriver().isSaveTempsEnabled() &&

clang/lib/Driver/ToolChains/CommonArgs.cpp

#endif
 
   if (Args.hasFlag(options::OPT_fstack_size_section,
                    options::OPT_fno_stack_size_section, false))
     CmdArgs.push_back(
         Args.MakeArgString(Twine(PluginOptPrefix) + "-stack-size-section"));
 
   // Setup statistics file output.
   SmallString<128> StatsFile = getStatsFileName(Args, Output, Input, D);
-  if (!StatsFile.empty())
+  if (!StatsFile.empty()) {
     CmdArgs.push_back(
         Args.MakeArgString(Twine(PluginOptPrefix) + "stats-file=" + StatsFile));
+    // Don't overwrite the same stats file but append to it.
+    CmdArgs.push_back(
+        Args.MakeArgString(Twine(PluginOptPrefix) + "-stats-file-append"));
+  }
 
   // Setup crash diagnostics dir.
   if (Arg *A = Args.getLastArg(options::OPT_fcrash_diagnostics_dir))
     CmdArgs.push_back(Args.MakeArgString(
         Twine(PluginOptPrefix) + "-crash-diagnostics-dir=" + A->getValue()));
 
   addX86AlignBranchArgs(D, Args, CmdArgs, /*IsLTO=*/true, PluginOptPrefix);
 
if (TC.getTriple().isAndroid() && !Args.hasArg(options::OPT_static) &&
     CmdArgs.push_back("-ldl");
 }
 
 SmallString<128> tools::getStatsFileName(const llvm::opt::ArgList &Args,
                                          const InputInfo &Output,
                                          const InputInfo &Input,
                                          const Driver &D) {
   const Arg *A = Args.getLastArg(options::OPT_save_stats_EQ);
-  if (!A)
+  if (!A && !D.CCPrintInternalStats)
     return {};
 
-  StringRef SaveStats = A->getValue();
   SmallString<128> StatsFile;
+  if (A) {
+    StringRef SaveStats = A->getValue();
     if (SaveStats == "obj" && Output.isFilename()) {
       StatsFile.assign(Output.getFilename());
       llvm::sys::path::remove_filename(StatsFile);
     } else if (SaveStats != "cwd") {
       D.Diag(diag::err_drv_invalid_value) << A->getAsString(Args) << SaveStats;
       return {};
     }
 
     StringRef BaseName = llvm::sys::path::filename(Input.getBaseInput());
     llvm::sys::path::append(StatsFile, BaseName);
     llvm::sys::path::replace_extension(StatsFile, "stats");
+  } else {
+    assert(D.CCPrintInternalStats);
+    StatsFile.assign(D.CCPrintInternalStatReportFilename.empty()
+                         ? "-"
+                         : D.CCPrintInternalStatReportFilename);
+  }
   return StatsFile;
 }
 
 void tools::addMultilibFlag(bool Enabled, const char *const Flag,
                             Multilib::flags_list &Flags) {
   Flags.push_back(std::string(Enabled ? "+" : "-") + Flag);
 }
 

clang/lib/Frontend/CompilerInstance.cpp

if (getFrontendOpts().ShowStats) {
     if (hasFileManager()) {
       getFileManager().PrintStats();
       OS << '\n';
     }
     llvm::PrintStatistics(OS);
   }
   StringRef StatsFile = getFrontendOpts().StatsFile;
   if (!StatsFile.empty()) {
+    llvm::sys::fs::OpenFlags FileFlags = llvm::sys::fs::OF_TextWithCRLF;
+    if (getFrontendOpts().AppendStats)
+      FileFlags |= llvm::sys::fs::OF_Append;
     std::error_code EC;
-    auto StatS = std::make_unique<llvm::raw_fd_ostream>(
-        StatsFile, EC, llvm::sys::fs::OF_TextWithCRLF);
+    auto StatS =
+        std::make_unique<llvm::raw_fd_ostream>(StatsFile, EC, FileFlags);
     if (EC) {
       getDiagnostics().Report(diag::warn_fe_unable_to_open_stats_file)
           << StatsFile << EC.message();
     } else {
       llvm::PrintStatisticsJSON(*StatS);
     }
   }
 

clang/test/Driver/save-stats.c

 
 // RUN: %clang -target x86_64-apple-darwin -save-stats=bla -c %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-INVALID
 // CHECK-INVALID: invalid value 'bla' in '-save-stats=bla'
 
 // RUN: %clang -target x86_64-linux-unknown -save-stats -flto -o obj/dir/save-stats.exe %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-LTO
 // CHECK-LTO: "-stats-file=save-stats.stats"
 // CHECK-LTO: "-o" "obj/dir{{/|\\\\}}save-stats.exe"
 // CHECK-LTO: "-plugin-opt=stats-file=save-stats.stats"
+// CHECK-LTO: "-plugin-opt=-stats-file-append"
 
 // RUN: %clang -target x86_64-linux-unknown -save-stats=obj -flto -o obj/dir/save-stats.exe %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-LTO-OBJ
 // CHECK-LTO-OBJ: "-plugin-opt=stats-file=obj/dir{{/|\\\\}}save-stats.stats"
+// CHECK-LTO-OBJ: "-plugin-opt=-stats-file-append"
+
+// RUN: env CC_PRINT_INTERNAL_STAT=1 \
+// RUN: %clang -target x86_64-apple-darwin %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-ENV
+// CHECK-ENV: "-stats-file=-"
+// CHECK-ENV-NO: "stats-file-append"
+
+// RUN: env CC_PRINT_INTERNAL_STAT=1 \
+// RUN:     CC_PRINT_INTERNAL_STAT_FILE=/tmp/stats.json \
+// RUN: %clang -target x86_64-apple-darwin %s -### 2>&1 | FileCheck %s -check-prefix=CHECK-ENV-FILE
+// CHECK-ENV-FILE: "-stats-file=/tmp/stats.json"
+// CHECK-ENV-FILE: "-stats-file-append"

clang/tools/driver/driver.cpp

static bool SetBackdoorDriverOutputsFromEnvVars(Driver &TheDriver) {
   }
 
   TheDriver.CCLogDiagnostics =
       checkEnvVar<bool>("CC_LOG_DIAGNOSTICS", "CC_LOG_DIAGNOSTICS_FILE",
                         TheDriver.CCLogDiagnosticsFilename);
   TheDriver.CCPrintProcessStats =
       checkEnvVar<bool>("CC_PRINT_PROC_STAT", "CC_PRINT_PROC_STAT_FILE",
                         TheDriver.CCPrintStatReportFilename);
+  TheDriver.CCPrintInternalStats =
+      checkEnvVar<bool>("CC_PRINT_INTERNAL_STAT", "CC_PRINT_INTERNAL_STAT_FILE",
+                        TheDriver.CCPrintInternalStatReportFilename);
 
   return true;
 }
 
 static void FixupDiagPrefixExeName(TextDiagnosticPrinter *DiagClient,
                                    const std::string &Path) {
   // If the clang binary happens to be named cl.exe for compatibility reasons,
   // use clang-cl.exe as the prefix to avoid confusion between clang and MSVC.