[llvm-exegesis] RFC: don't default to running&dumping all analyses to '-'

[lebedev.ri]

Summary

Up until the point i have looked in the source, i didn't even understood that
i can disable 'cluster' output. I have always silenced it via &> /dev/null.
(And hoped it wasn't contributing much of the run time.)

While i expect that it has it's use-cases i never once needed it so far.
If i forget to silence it, console is completely flooded with that output.

How about not expecting users to opt-out of analyses,
but to explicitly specify the analyses that should be performed?

[courbet]

Sounds very reasonable.

tools/llvm-exegesis/llvm-exegesis.cpp

410 ↗

(On Diff #184942)

[nit] remove = after flag names.

[lebedev.ri]

In D57648#1382737, @courbet wrote:

Sounds very reasonable.

Great! Thank you for the review.

llvm/trunk/docs/CommandGuide/llvm-exegesis.rst

 
  .. option:: -snippets-file=<filename>
 
   Specify the custom code snippet to measure. See example 2 for details.
   Either `opcode-index`, `opcode-name` or `snippets-file` must be set.
 
 .. option:: -mode=[latency|uops|inverse_throughput|analysis]
 
- Specify the run mode.
+ Specify the run mode. Note that if you pick `analysis` mode, you also need
+ to specify at least one of the `-analysis-clusters-output-file=` and
+ `-analysis-inconsistencies-output-file=`.
 
 .. option:: -num-repetitions=<Number of repetition>
 
  Specify the number of repetitions of the asm snippet.
  Higher values lead to more accurate measurements but lengthen the benchmark.
 
 .. option:: -benchmarks-file=</path/to/file>
 
  File to read (`analysis` mode) or write (`latency`/`uops`/`inverse_throughput`
  modes) benchmark results. "-" uses stdin/stdout.
 
 .. option:: -analysis-clusters-output-file=</path/to/file>
 
  If provided, write the analysis clusters as CSV to this file. "-" prints to
- stdout.
+ stdout. By default, this analysis is not run.
 
 .. option:: -analysis-inconsistencies-output-file=</path/to/file>
 
  If non-empty, write inconsistencies found during analysis to this file. `-`
- prints to stdout.
+ prints to stdout. By default, this analysis is not run.
 
 .. option:: -analysis-numpoints=<dbscan numPoints parameter>
 
  Specify the numPoints parameters to be used for DBSCAN clustering
  (`analysis` mode).
 
 .. option:: -analysis-espilon=<dbscan epsilon parameter>
 

llvm/trunk/test/tools/llvm-exegesis/X86/analysis-uops-backwards.test

 # This tests backwards-compatibility of the yaml schema (see PR39082).
-# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-inconsistencies-output-file="" -analysis-numpoints=1 | FileCheck %s
+# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-clusters-output-file=- -analysis-numpoints=1 | FileCheck %s
 
 # CHECK: cluster_id,opcode_name,config,sched_class,HWPort0,HWPort1,HWPort2,HWPort3,HWPort4,HWPort5,HWPort6,HWPort7,NumMicroOps
 # CHECK-NEXT: vzeroall
 
 ---
 mode:            uops
 key:
   instructions:

llvm/trunk/test/tools/llvm-exegesis/X86/analysis-uops-variant.test

-# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-inconsistencies-output-file="" -analysis-numpoints=1 | FileCheck %s
+# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-clusters-output-file=- -analysis-numpoints=1 | FileCheck %s
 
 # REQUIRES: asserts
 # (sched class name is NDEBUG only)
 
 # CHECK: cluster_id,opcode_name,config,sched_class,SBPort0,SBPort1,SBPort23,SBPort4,SBPort5,NumMicroOps
 # CHECK-NEXT: SBWriteZeroLatency
 
 ---

llvm/trunk/test/tools/llvm-exegesis/X86/analysis-uops.test

-# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-inconsistencies-output-file="" -analysis-numpoints=1 | FileCheck %s
+# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-clusters-output-file=- -analysis-numpoints=1 | FileCheck %s
 
 # CHECK: cluster_id,opcode_name,config,sched_class,HWPort0,HWPort1,HWPort2,HWPort3,HWPort4,HWPort5,HWPort6,HWPort7,NumMicroOps
 # CHECK-NEXT: vzeroall
 
 ---
 mode:            uops
 key:
   instructions:

llvm/trunk/tools/llvm-exegesis/llvm-exegesis.cpp

 
 static cl::opt<float>
     AnalysisEpsilon("analysis-epsilon",
                     cl::desc("dbscan epsilon for analysis clustering"),
                     cl::init(0.1));
 
 static cl::opt<std::string>
     AnalysisClustersOutputFile("analysis-clusters-output-file", cl::desc(""),
-                               cl::init("-"));
+                               cl::init(""));
 static cl::opt<std::string>
     AnalysisInconsistenciesOutputFile("analysis-inconsistencies-output-file",
-                                      cl::desc(""), cl::init("-"));
+                                      cl::desc(""), cl::init(""));
 
 static cl::opt<std::string>
     CpuName("mcpu",
             cl::desc(
                 "cpu name to use for pfm counters, leave empty to autodetect"),
             cl::init(""));
 
 
static void maybeRunAnalysis(const Analysis &Analyzer, const std::string &Name,
   if (auto Err = Analyzer.run<Pass>(ClustersOS))
     llvm::report_fatal_error(std::move(Err));
 }
 
 static void analysisMain() {
   if (BenchmarkFile.empty())
     llvm::report_fatal_error("--benchmarks-file must be set.");
 
+  if (AnalysisClustersOutputFile.empty() &&
+      AnalysisInconsistenciesOutputFile.empty()) {
+    llvm::report_fatal_error(
+        "At least one of --analysis-clusters-output-file and "
+        "--analysis-inconsistencies-output-file must be specified.");
+  }
+
   llvm::InitializeNativeTarget();
   llvm::InitializeNativeTargetAsmPrinter();
   llvm::InitializeNativeTargetDisassembler();
   // Read benchmarks.
   const LLVMState State("");
   const std::vector<InstructionBenchmark> Points =
       ExitOnErr(InstructionBenchmark::readYamls(State, BenchmarkFile));
   llvm::outs() << "Parsed " << Points.size() << " benchmark points\n";