Add an option to save the backend-produced YAML optimization record to a file

[hfinkel]

Summary

The backend now has the capability to save information from optimizations, the same information that can be used to generate optimization diagnostics but in machine-consumable form, into an output file. This can be enabled when using opt (see r282539), and this patch will enable it when using clang. The idea is that other tools will be able to consume these files, and perhaps in combination with the original source code, produce various kinds of optimization reports for users (and for compiler developers).

This patch proposes the name -fsave-optimization-record (and -fsave-optimization-record=filename). Bikeshedding welcome.

[hfinkel]

Note: This depends on D25224.

[hfinkel]

@rsmith @rjmccall - I chatted with @anemet about this on IRC, and he's happy with it. Please look this over, in part to make sure you're happy with the option name.

On the name, two of my thoughts behind using -fsave-optimization-record were: 1) I did not want to call it a "report", because it is YAML output and not something for a human to use directly and 2) I thought that record, the noun, fit well, but not necessarily the verb, and by putting 'save' in the name it seems clear (at least to me) that record is the noun.

[hfinkel]

I reworked the way that the automatic file-name selection works so that it will work with offloading (e.g. CUDA), and added more tests for the automatic file-name selection. It now also bases the file name on the output file name if you use -S (not just -c). When using an offload device, it makes the output file name for the device different from the host in a way that mirrors other parts of the driver.

I also changed the default extension from .yaml to .opt.yaml to make it a little more descriptive (there are obviously lots of different kinds of yaml files).

[anemet]

test/CodeGen/opt-record.c

17–25 ↗

(On Diff #73874)

Wouldn't this be a good place to also check that we have -gline-tables-only properly hooked up, i.e. CHECK for DebugLoc: as well?

[anemet]

lib/CodeGen/CodeGenAction.cpp

198 ↗

(On Diff #73874)

Sorry, one more thing: if PGO is available, I think we want to set Ctx.setDiagnosticHotnessRequested as well. Without that, you'd have to pass -fsave-optimization-record and -fdiagnostics-show-hotness to get hotness info into the YAML file which feels strange. I am certainly fine if we do this later but I wanted to bring it up since it's seems related.

[hfinkel]

lib/CodeGen/CodeGenAction.cpp

198 ↗

(On Diff #73874)

I agree. We shouldn't require -fdiagnostics-show-hotness for that to work.

test/CodeGen/opt-record.c

17–25 ↗

(On Diff #73874)

Yes; will do.

[hfinkel]

Addressed review comments (DebugLoc is tested, and we enable hotness computation when saving the optimization record and also using PGO).

[anemet]

LGTM, thanks. I also like the option name but will let the the other reviewers official approve that part.

[rsmith]

As discussed on IRC, I have a mild concern about using -fsave-optimization-record (with no argument) to enable the feature, and -fsave-optimization-record=X to enable the feature and specify a filename; in most (but not all) cases, -option arg and -option=arg mean the same thing. Other than that, this looks good to me.

[hfinkel]

In D25225#566854, @rsmith wrote:

As discussed on IRC, I have a mild concern about using -fsave-optimization-record (with no argument) to enable the feature, and -fsave-optimization-record=X to enable the feature and specify a filename; in most (but not all) cases, -option arg and -option=arg mean the same thing. Other than that, this looks good to me.

For the record (pun intended, I suppose), we discussed on IRC adding a separate -foptimization-record-file=filename to set the file name.

[twoh]

Is there a particular reason why "opt_record_file" is defined in CC1Options.td, not Options.td? If -opt-record-file=<filename> is provided by the command line, line 829-831 in CompilerInvocation.cpp won't handle it because opt_record_file is not a CodeGenArg.

[hfinkel]

In D25225#584010, @twoh wrote:

Is there a particular reason why "opt_record_file" is defined in CC1Options.td, not Options.td? If -opt-record-file=<filename> is provided by the command line, line 829-831 in CompilerInvocation.cpp won't handle it because opt_record_file is not a CodeGenArg.

It is not intended to be used directly. The user should use -fsave-optimization-record (along with -foptimization-record-file= if they don't like the default name selected). We definitely need to write end-user docs for this; it is on my TODO list.

[twoh]

In D25225#584012, @hfinkel wrote:

In D25225#584010, @twoh wrote:

Is there a particular reason why "opt_record_file" is defined in CC1Options.td, not Options.td? If -opt-record-file=<filename> is provided by the command line, line 829-831 in CompilerInvocation.cpp won't handle it because opt_record_file is not a CodeGenArg.

It is not intended to be used directly. The user should use -fsave-optimization-record (along with -foptimization-record-file= if they don't like the default name selected). We definitely need to write end-user docs for this; it is on my TODO list.

Got it. Thanks!

cfe/trunk/include/clang/Driver/CC1Options.td

def mt_migrate_directory : Separate<["-"], "mt-migrate-directory">,
   HelpText<"Directory for temporary files produced during ARC or ObjC migration">;
 def arcmt_check : Flag<["-"], "arcmt-check">,
   HelpText<"Check for ARC migration issues that need manual handling">;
 def arcmt_modify : Flag<["-"], "arcmt-modify">,
   HelpText<"Apply modifications to files to conform to ARC">;
 def arcmt_migrate : Flag<["-"], "arcmt-migrate">,
   HelpText<"Apply modifications and produces temporary files that conform to ARC">;
 
+def opt_record_file : Separate<["-"], "opt-record-file">,
+  HelpText<"File name to use for YAML optimization record output">;
+
 def print_stats : Flag<["-"], "print-stats">,
   HelpText<"Print performance metrics and statistics">;
 def stats_file : Joined<["-"], "stats-file=">,
   HelpText<"Filename to write statistics to">;
 def fdump_record_layouts : Flag<["-"], "fdump-record-layouts">,
   HelpText<"Dump record layout information">;
 def fdump_record_layouts_simple : Flag<["-"], "fdump-record-layouts-simple">,
   HelpText<"Dump record layout information in a simple form used for testing">;

cfe/trunk/include/clang/Driver/Options.td

def fsyntax_only : Flag<["-"], "fsyntax-only">,
   Flags<[DriverOption,CoreOption,CC1Option]>, Group<Action_Group>;
 def ftabstop_EQ : Joined<["-"], "ftabstop=">, Group<f_Group>;
 def ftemplate_depth_EQ : Joined<["-"], "ftemplate-depth=">, Group<f_Group>;
 def ftemplate_depth_ : Joined<["-"], "ftemplate-depth-">, Group<f_Group>;
 def ftemplate_backtrace_limit_EQ : Joined<["-"], "ftemplate-backtrace-limit=">,
                                    Group<f_Group>;
 def foperator_arrow_depth_EQ : Joined<["-"], "foperator-arrow-depth=">,
                                Group<f_Group>;
+
+def fsave_optimization_record : Flag<["-"], "fsave-optimization-record">,
+  Group<f_Group>, HelpText<"Generate a YAML optimization record file">;
+def fno_save_optimization_record : Flag<["-"], "fno-save-optimization-record">,
+  Group<f_Group>, Flags<[NoArgumentUnused]>;
+def foptimization_record_file_EQ : Joined<["-"], "foptimization-record-file=">,
+  Group<f_Group>,
+  HelpText<"Specify the file name of any generated YAML optimization record">;
+
 def ftest_coverage : Flag<["-"], "ftest-coverage">, Group<f_Group>;
 def fvectorize : Flag<["-"], "fvectorize">, Group<f_Group>,
   HelpText<"Enable the loop vectorization passes">;
 def fno_vectorize : Flag<["-"], "fno-vectorize">, Group<f_Group>;
 def : Flag<["-"], "ftree-vectorize">, Alias<fvectorize>;
 def : Flag<["-"], "fno-tree-vectorize">, Alias<fno_vectorize>;
 def fslp_vectorize : Flag<["-"], "fslp-vectorize">, Group<f_Group>,
   HelpText<"Enable the superword-level parallelism vectorization passes">;

cfe/trunk/include/clang/Frontend/CodeGenOptions.h

public:
   /// importing.
   std::string ThinLTOIndexFile;
 
   /// A list of file names passed with -fcuda-include-gpubinary options to
   /// forward to CUDA runtime back-end for incorporating them into host-side
   /// object file.
   std::vector<std::string> CudaGpuBinaryFileNames;
 
+  /// The name of the file to which the backend should save YAML optimization
+  /// records.
+  std::string OptRecordFile;
+
   /// Regular expression to select optimizations for which we should enable
   /// optimization remarks. Transformation passes whose name matches this
   /// expression (and support this feature), will emit a diagnostic
   /// whenever they perform a transformation. This is enabled by the
   /// -Rpass=regexp flag.
   std::shared_ptr<llvm::Regex> OptimizationRemarkPattern;
 
   /// Regular expression to select optimizations for which we should enable

cfe/trunk/lib/CodeGen/CodeGenAction.cpp

 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IRReader/IRReader.h"
 #include "llvm/Linker/Linker.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/Timer.h"
+#include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/YAMLTraits.h"
 #include <memory>
 using namespace clang;
 using namespace llvm;
 
 namespace clang {
   class BackendConsumer : public ASTConsumer {
     virtual void anchor();
     DiagnosticsEngine &Diags;
void HandleTranslationUnit(ASTContext &C) override {
       Ctx.setInlineAsmDiagnosticHandler(InlineAsmDiagHandler, this);
 
       LLVMContext::DiagnosticHandlerTy OldDiagnosticHandler =
           Ctx.getDiagnosticHandler();
       void *OldDiagnosticContext = Ctx.getDiagnosticContext();
       Ctx.setDiagnosticHandler(DiagnosticHandler, this);
       Ctx.setDiagnosticHotnessRequested(CodeGenOpts.DiagnosticsWithHotness);
 
+      std::unique_ptr<llvm::tool_output_file> OptRecordFile;
+      if (!CodeGenOpts.OptRecordFile.empty()) {
+        std::error_code EC;
+        OptRecordFile =
+          llvm::make_unique<llvm::tool_output_file>(CodeGenOpts.OptRecordFile,
+                                                    EC, sys::fs::F_None);
+        if (EC) {
+          Diags.Report(diag::err_cannot_open_file) <<
+            CodeGenOpts.OptRecordFile << EC.message();
+          return;
+        }
+
+        Ctx.setDiagnosticsOutputFile(new yaml::Output(OptRecordFile->os()));
+
+        if (CodeGenOpts.getProfileUse() != CodeGenOptions::ProfileNone)
+          Ctx.setDiagnosticHotnessRequested(true);
+      }
+
       // Link LinkModule into this module if present, preserving its validity.
       for (auto &I : LinkModules) {
         unsigned LinkFlags = I.first;
         CurLinkModule = I.second.get();
         if (Linker::linkModules(*getModule(), std::move(I.second), LinkFlags))
           return;
       }
 
       EmbedBitcode(getModule(), CodeGenOpts, llvm::MemoryBufferRef());
 
       EmitBackendOutput(Diags, CodeGenOpts, TargetOpts, LangOpts,
                         C.getTargetInfo().getDataLayout(),
                         getModule(), Action, std::move(AsmOutStream));
 
       Ctx.setInlineAsmDiagnosticHandler(OldHandler, OldContext);
 
       Ctx.setDiagnosticHandler(OldDiagnosticHandler, OldDiagnosticContext);
+
+      if (OptRecordFile)
+        OptRecordFile->keep();
     }
 
     void HandleTagDeclDefinition(TagDecl *D) override {
       PrettyStackTraceDecl CrashInfo(D, SourceLocation(),
                                      Context->getSourceManager(),
                                      "LLVM IR generation of declaration");
       Gen->HandleTagDeclDefinition(D);
     }

cfe/trunk/lib/Driver/Tools.cpp

 
   // le32-specific flags:
   //  -fno-math-builtin: clang should not convert math builtins to intrinsics
   //                     by default.
   if (getToolChain().getArch() == llvm::Triple::le32) {
     CmdArgs.push_back("-fno-math-builtin");
   }
 
+  if (Args.hasFlag(options::OPT_fsave_optimization_record,
+                   options::OPT_fno_save_optimization_record, false)) {
+    CmdArgs.push_back("-opt-record-file");
+
+    const Arg *A = Args.getLastArg(options::OPT_foptimization_record_file_EQ);
+    if (A) {
+      CmdArgs.push_back(A->getValue());
+    } else {
+      SmallString<128> F;
+      if (Output.isFilename() && (Args.hasArg(options::OPT_c) ||
+                                  Args.hasArg(options::OPT_S))) {
+        F = Output.getFilename();
+      } else {
+        // Use the compilation directory.
+        F = llvm::sys::path::stem(Input.getBaseInput());
+
+        // If we're compiling for an offload architecture (i.e. a CUDA device),
+        // we need to make the file name for the device compilation different
+        // from the host compilation.
+        if (!JA.isDeviceOffloading(Action::OFK_None) &&
+            !JA.isDeviceOffloading(Action::OFK_Host)) {
+          llvm::sys::path::replace_extension(F, "");
+          F += JA.getOffloadingFileNamePrefix(Triple.normalize());
+          F += "-";
+          F += JA.getOffloadingArch();
+        }
+      }
+
+      llvm::sys::path::replace_extension(F, "opt.yaml");
+      CmdArgs.push_back(Args.MakeArgString(F));
+    }
+  }
+
 // Default to -fno-builtin-str{cat,cpy} on Darwin for ARM.
 //
 // FIXME: Now that PR4941 has been fixed this can be enabled.
 #if 0
   if (getToolChain().getTriple().isOSDarwin() &&
       (getToolChain().getArch() == llvm::Triple::arm ||
        getToolChain().getArch() == llvm::Triple::thumb)) {
     if (!Args.hasArg(options::OPT_fbuiltin_strcat))

cfe/trunk/lib/Frontend/CompilerInvocation.cpp

if (A->getOption().matches(OPT_fpcc_struct_return)) {
       Opts.setStructReturnConvention(CodeGenOptions::SRCK_InRegs);
     }
   }
 
   Opts.DependentLibraries = Args.getAllArgValues(OPT_dependent_lib);
   Opts.LinkerOptions = Args.getAllArgValues(OPT_linker_option);
   bool NeedLocTracking = false;
 
+  Opts.OptRecordFile = Args.getLastArgValue(OPT_opt_record_file);
+  if (!Opts.OptRecordFile.empty())
+    NeedLocTracking = true;
+
   if (Arg *A = Args.getLastArg(OPT_Rpass_EQ)) {
     Opts.OptimizationRemarkPattern =
         GenerateOptimizationRemarkRegex(Diags, Args, A);
     NeedLocTracking = true;
   }
 
   if (Arg *A = Args.getLastArg(OPT_Rpass_missed_EQ)) {
     Opts.OptimizationRemarkMissedPattern =

cfe/trunk/test/CodeGen/Inputs/opt-record.proftext

+foo
+# Func Hash:
+0
+# Num Counters:
+1
+# Counter Values:
+30
+
+bar
+# Func Hash:
+0
+# Num Counters:
+1
+# Counter Values:
+30
+
+Test
+# Func Hash:
+269
+# Num Counters:
+3
+# Counter Values:
+1
+30
+15
+

cfe/trunk/test/CodeGen/opt-record.c

+// RUN: %clang_cc1 -O3 -triple x86_64-unknown-linux-gnu -target-cpu x86-64 %s -o %t -dwarf-column-info -opt-record-file %t.yaml -emit-obj
+// RUN: cat %t.yaml | FileCheck %s
+// RUN: llvm-profdata merge %S/Inputs/opt-record.proftext -o %t.profdata
+// RUN: %clang_cc1 -O3 -triple x86_64-unknown-linux-gnu -target-cpu x86-64 -fprofile-instrument-use-path=%t.profdata %s -o %t -dwarf-column-info -opt-record-file %t.yaml -emit-obj
+// RUN: cat %t.yaml | FileCheck -check-prefix=CHECK -check-prefix=CHECK-PGO %s
+// REQUIRES: x86-registered-target
+
+void bar();
+void foo() { bar(); }
+
+void Test(int *res, int *c, int *d, int *p, int n) {
+  int i;
+
+#pragma clang loop vectorize(assume_safety)
+  for (i = 0; i < 1600; i++) {
+    res[i] = (p[i] == 0) ? res[i] : res[i] + d[i];
+  }
+}
+
+// CHECK: --- !Missed
+// CHECK: Pass:            inline
+// CHECK: Name:            NoDefinition
+// CHECK: DebugLoc:
+// CHECK: Function:        foo
+// CHECK-PGO: Hotness:
+
+// CHECK: --- !Passed
+// CHECK: Pass:            loop-vectorize
+// CHECK: Name:            Vectorized
+// CHECK: DebugLoc:
+// CHECK: Function:        Test
+// CHECK-PGO: Hotness:
+

cfe/trunk/test/Driver/opt-record.c

+// RUN: %clang -### -S -o FOO -fsave-optimization-record %s 2>&1 | FileCheck %s
+// RUN: %clang -### -c -o FOO -fsave-optimization-record %s 2>&1 | FileCheck %s
+// RUN: %clang -### -c -fsave-optimization-record %s 2>&1 | FileCheck %s -check-prefix=CHECK-NO-O
+// RUN: %clang -### -fsave-optimization-record %s 2>&1 | FileCheck %s -check-prefix=CHECK-NO-O
+// RUN: %clang -### -S -fsave-optimization-record -x cuda -nocudainc -nocudalib %s 2>&1 | FileCheck %s -check-prefix=CHECK-NO-O -check-prefix=CHECK-CUDA-DEV
+// RUN: %clang -### -fsave-optimization-record -x cuda -nocudainc -nocudalib %s 2>&1 | FileCheck %s -check-prefix=CHECK-NO-O -check-prefix=CHECK-CUDA-DEV
+// RUN: %clang -### -S -o FOO -fsave-optimization-record -foptimization-record-file=BAR.txt %s 2>&1 | FileCheck %s -check-prefix=CHECK-EQ
+
+// CHECK: "-cc1"
+// CHECK: "-opt-record-file" "FOO.opt.yaml"
+
+// CHECK-NO-O: "-cc1"
+// CHECK-NO-O-DAG: "-opt-record-file" "opt-record.opt.yaml"
+// CHECK-CUDA-DEV-DAG: "-opt-record-file" "opt-record-device-cuda-nvptx64-nvidia-cuda-sm_20.opt.yaml"
+
+// CHECK-EQ: "-cc1"
+// CHECK-EQ: "-opt-record-file" "BAR.txt"
+