[llvm-exegesis] Allow measuring several instructions in a single run.

[courbet]

Summary

We try to recover gracefully on instructions that would crash the
program.

This includes some refactoring of runMeasurement() implementations.

[courbet]

revert debug code

[lebedev.ri]

How do you plan to deal with opcodes that crash (assertion failure) the llvm as a whole?
I.e. not when the snippet crashes, but the generation of the snippet itself crashes.
I *believe* there are some cases.

[gchatelet]

In D53371#1267732, @lebedev.ri wrote:

How do you plan to deal with opcodes that crash (assertion failure) the llvm as a whole?
I.e. not when the snippet crashes, but the generation of the snippet itself crashes.
I *believe* there are some cases.

We were looking at those as I speak. The plan is to not crash :)

tools/llvm-exegesis/lib/BenchmarkRunner.cpp

59 ↗

(On Diff #169999)

This is not very useful, is it?

63 ↗

(On Diff #169999)

Probably more than 2 counter names.

77 ↗

(On Diff #169999)

As discussed, lets store the result in a bool and return after the state is restored

CrashRecoveryContext CRC;
CrashRecoveryContext::Enable();
Counter.start();
const bool IsSuccess = CRC.RunSafely(...
Counter.stop();
CrashRecoveryContext::Disable();
if(!IsSuccess) return ....

[lebedev.ri]

In D53371#1267734, @gchatelet wrote:

In D53371#1267732, @lebedev.ri wrote:

How do you plan to deal with opcodes that crash (assertion failure) the llvm as a whole?
I.e. not when the snippet crashes, but the generation of the snippet itself crashes.
I *believe* there are some cases.

We were looking at those as I speak. The plan is to not crash :)

Ok, sounds great :)

[courbet]

address comments.

[courbet]

tools/llvm-exegesis/lib/BenchmarkRunner.cpp

63 ↗

(On Diff #169999)

The current max is 2, for SNB.

[gchatelet]

tools/llvm-exegesis/lib/BenchmarkRunner.cpp

58 ↗

(On Diff #169999)

Return an int64_t here.

tools/llvm-exegesis/lib/Latency.cpp

91 ↗

(On Diff #169999)

Can you please rename MinLatency to MinValue?

96 ↗

(On Diff #169999)

ExpectedCounterValue? so it's easier to understand the next lines

97 ↗

(On Diff #169999)

remove the braces

101 ↗

(On Diff #169999)

Now you don't need to cast anymore.

tools/llvm-exegesis/lib/Uops.cpp

236 ↗

(On Diff #169999)

remove the braces

245 ↗

(On Diff #169999)

ditto

tools/llvm-exegesis/llvm-exegesis.cpp

121 ↗

(On Diff #169999)

Let's go with a for loop so it's easier to reason about the last element.

125 ↗

(On Diff #169999)

can you specify the lambda return type so it's explicit.

[courbet]

address comments

[courbet]

Start/stop the counter inside the lambda to avoid measuring events from RunSafely().

[courbet]

rebase

llvm/trunk/docs/CommandGuide/llvm-exegesis.rst

 
  Print a summary of command line options.
 
 .. option:: -opcode-index=<LLVM opcode index>
 
  Specify the opcode to measure, by index. See example 1 for details.
  Either `opcode-index`, `opcode-name` or `snippets-file` must be set.
 
-.. option:: -opcode-name=<LLVM opcode name>
+.. option:: -opcode-name=<opcode name 1>,<opcode name 2>,...
 
- Specify the opcode to measure, by name. See example 1 for details.
+ Specify the opcode to measure, by name. Several opcodes can be specified as
+ a comma-separated list. See example 1 for details.
  Either `opcode-index`, `opcode-name` or `snippets-file` must be set.
 
  .. option:: -snippets-file=<filename>
 
   Specify the custom code snippet to measure. See example 2 for details.
   Either `opcode-index`, `opcode-name` or `snippets-file` must be set.
 
 .. option:: -mode=[latency|uops|analysis]

llvm/trunk/tools/llvm-exegesis/lib/BenchmarkRunner.h

struct ScratchSpace {
     char *ptr() const { return AlignedPtr; }
     void clear() { std::memset(ptr(), 0, kSize); }
 
   private:
     const std::unique_ptr<char[]> UnalignedPtr;
     char *const AlignedPtr;
   };
 
+  // A helper to measure counters while executing a function in a sandboxed
+  // context.
+  class FunctionExecutor {
+  public:
+    ~FunctionExecutor();
+    virtual llvm::Expected<int64_t>
+    runAndMeasure(const char *Counters) const = 0;
+  };
+
 protected:
   const LLVMState &State;
 
 private:
-  virtual std::vector<BenchmarkMeasure>
-  runMeasurements(const ExecutableFunction &EF,
-                  ScratchSpace &Scratch) const = 0;
+  virtual llvm::Expected<std::vector<BenchmarkMeasure>>
+  runMeasurements(const FunctionExecutor &Executor) const = 0;
 
   llvm::Expected<std::string>
   writeObjectFile(const BenchmarkCode &Configuration,
                   llvm::ArrayRef<llvm::MCInst> Code) const;
 
   const InstructionBenchmark::ModeE Mode;
 
   const std::unique_ptr<ScratchSpace> Scratch;
 };
 
 } // namespace exegesis
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_BENCHMARKRUNNER_H

llvm/trunk/tools/llvm-exegesis/lib/BenchmarkRunner.cpp

 //===-- BenchmarkRunner.cpp -------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include <array>
 #include <string>
 
 #include "Assembler.h"
 #include "BenchmarkRunner.h"
 #include "MCInstrDescView.h"
+#include "PerfHelper.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
+#include "llvm/Support/CrashRecoveryContext.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Program.h"
 
 namespace exegesis {
 
 BenchmarkFailure::BenchmarkFailure(const llvm::Twine &S)
     : llvm::StringError(S, llvm::inconvertibleErrorCode()) {}
GenerateInstructions(const BenchmarkCode &BC, const size_t MinInstructions) {
   if (BC.Instructions.empty())
     return {};
   std::vector<llvm::MCInst> Code = BC.Instructions;
   for (int I = 0; Code.size() < MinInstructions; ++I)
     Code.push_back(BC.Instructions[I % BC.Instructions.size()]);
   return Code;
 }
 
+namespace {
+class FunctionExecutorImpl : public BenchmarkRunner::FunctionExecutor {
+public:
+  FunctionExecutorImpl(const LLVMState &State,
+                       llvm::object::OwningBinary<llvm::object::ObjectFile> Obj,
+                       BenchmarkRunner::ScratchSpace *Scratch)
+      : Function(State.createTargetMachine(), std::move(Obj)),
+        Scratch(Scratch) {}
+
+private:
+  llvm::Expected<int64_t> runAndMeasure(const char *Counters) const override {
+    // We sum counts when there are several counters for a single ProcRes
+    // (e.g. P23 on SandyBridge).
+    int64_t CounterValue = 0;
+    llvm::SmallVector<llvm::StringRef, 2> CounterNames;
+    llvm::StringRef(Counters).split(CounterNames, ',');
+    char *const ScratchPtr = Scratch->ptr();
+    for (const auto &CounterName : CounterNames) {
+      pfm::PerfEvent PerfEvent(CounterName);
+      if (!PerfEvent.valid())
+        llvm::report_fatal_error(
+            llvm::Twine("invalid perf event ").concat(Counters));
+      pfm::Counter Counter(PerfEvent);
+      Scratch->clear();
+      {
+        llvm::CrashRecoveryContext CRC;
+        llvm::CrashRecoveryContext::Enable();
+        const bool Crashed = !CRC.RunSafely([this, &Counter, ScratchPtr]() {
+          Counter.start();
+          Function(ScratchPtr);
+          Counter.stop();
+        });
+        llvm::CrashRecoveryContext::Disable();
+        // FIXME: Better diagnosis.
+        if (Crashed)
+          return llvm::make_error<BenchmarkFailure>(
+              "snippet crashed while running");
+      }
+      CounterValue += Counter.read();
+    }
+    return CounterValue;
+  }
+
+  const ExecutableFunction Function;
+  BenchmarkRunner::ScratchSpace *const Scratch;
+};
+} // namespace
+
 InstructionBenchmark
 BenchmarkRunner::runConfiguration(const BenchmarkCode &BC,
                                   unsigned NumRepetitions) const {
   InstructionBenchmark InstrBenchmark;
   InstrBenchmark.Mode = Mode;
   InstrBenchmark.CpuName = State.getTargetMachine().getTargetCPU();
   InstrBenchmark.LLVMTriple =
       State.getTargetMachine().getTargetTriple().normalize();
BenchmarkRunner::runConfiguration(const BenchmarkCode &BC,
   auto ObjectFilePath = writeObjectFile(
       BC, GenerateInstructions(BC, InstrBenchmark.NumRepetitions));
   if (llvm::Error E = ObjectFilePath.takeError()) {
     InstrBenchmark.Error = llvm::toString(std::move(E));
     return InstrBenchmark;
   }
   llvm::outs() << "Check generated assembly with: /usr/bin/objdump -d "
                << *ObjectFilePath << "\n";
-  const ExecutableFunction EF(State.createTargetMachine(),
-                              getObjectFromFile(*ObjectFilePath));
-  InstrBenchmark.Measurements = runMeasurements(EF, *Scratch);
+  const FunctionExecutorImpl Executor(State, getObjectFromFile(*ObjectFilePath),
+                                      Scratch.get());
+  auto Measurements = runMeasurements(Executor);
+  if (llvm::Error E = Measurements.takeError()) {
+    InstrBenchmark.Error = llvm::toString(std::move(E));
+    return InstrBenchmark;
+  }
+  InstrBenchmark.Measurements = std::move(*Measurements);
   assert(InstrBenchmark.NumRepetitions > 0 && "invalid NumRepetitions");
   for (BenchmarkMeasure &BM : InstrBenchmark.Measurements) {
     // Scale the measurements by instruction.
     BM.PerInstructionValue /= InstrBenchmark.NumRepetitions;
     // Scale the measurements by snippet.
     BM.PerSnippetValue *= static_cast<double>(BC.Instructions.size()) /
                           InstrBenchmark.NumRepetitions;
   }
 
   return InstrBenchmark;
 }
 
 llvm::Expected<std::string>
 BenchmarkRunner::writeObjectFile(const BenchmarkCode &BC,
                                  llvm::ArrayRef<llvm::MCInst> Code) const {
   int ResultFD = 0;
   llvm::SmallString<256> ResultPath;
   if (llvm::Error E = llvm::errorCodeToError(llvm::sys::fs::createTemporaryFile(
           "snippet", "o", ResultFD, ResultPath)))
     return std::move(E);
   llvm::raw_fd_ostream OFS(ResultFD, true /*ShouldClose*/);
   assembleToStream(State.getExegesisTarget(), State.createTargetMachine(),
                    BC.LiveIns, BC.RegisterInitialValues, Code, OFS);
   return ResultPath.str();
 }
 
+BenchmarkRunner::FunctionExecutor::~FunctionExecutor() {}
+
 } // namespace exegesis

llvm/trunk/tools/llvm-exegesis/lib/Latency.h

 
 class LatencyBenchmarkRunner : public BenchmarkRunner {
 public:
   LatencyBenchmarkRunner(const LLVMState &State)
       : BenchmarkRunner(State, InstructionBenchmark::Latency) {}
   ~LatencyBenchmarkRunner() override;
 
 private:
-  std::vector<BenchmarkMeasure>
-  runMeasurements(const ExecutableFunction &EF,
-                  ScratchSpace &Scratch) const override;
+  llvm::Expected<std::vector<BenchmarkMeasure>>
+  runMeasurements(const FunctionExecutor &Executor) const override;
 
   virtual const char *getCounterName() const;
 };
 } // namespace exegesis
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_LATENCY_H

llvm/trunk/tools/llvm-exegesis/lib/Latency.cpp

 //===-- Latency.cpp ---------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "Latency.h"
 
 #include "Assembler.h"
 #include "BenchmarkRunner.h"
 #include "MCInstrDescView.h"
-#include "PerfHelper.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstBuilder.h"
 #include "llvm/Support/FormatVariadic.h"
 
 namespace exegesis {
 
 struct ExecutionClass {
const char *CounterName = State.getSubtargetInfo()
                                 .PfmCounters.CycleCounter;
   if (!CounterName)
     llvm::report_fatal_error("sched model does not define a cycle counter");
   return CounterName;
 }
 
 LatencyBenchmarkRunner::~LatencyBenchmarkRunner() = default;
 
-std::vector<BenchmarkMeasure>
-LatencyBenchmarkRunner::runMeasurements(const ExecutableFunction &Function,
-                                        ScratchSpace &Scratch) const {
+llvm::Expected<std::vector<BenchmarkMeasure>>
+LatencyBenchmarkRunner::runMeasurements(
+    const FunctionExecutor &Executor) const {
   // Cycle measurements include some overhead from the kernel. Repeat the
   // measure several times and take the minimum value.
   constexpr const int NumMeasurements = 30;
-  int64_t MinLatency = std::numeric_limits<int64_t>::max();
+  int64_t MinValue = std::numeric_limits<int64_t>::max();
   const char *CounterName = getCounterName();
   if (!CounterName)
     llvm::report_fatal_error("could not determine cycle counter name");
-  const pfm::PerfEvent CyclesPerfEvent(CounterName);
-  if (!CyclesPerfEvent.valid())
-    llvm::report_fatal_error("invalid perf event");
   for (size_t I = 0; I < NumMeasurements; ++I) {
-    pfm::Counter Counter(CyclesPerfEvent);
-    Scratch.clear();
-    Counter.start();
-    Function(Scratch.ptr());
-    Counter.stop();
-    const int64_t Value = Counter.read();
-    if (Value < MinLatency)
-      MinLatency = Value;
-  }
-  return {BenchmarkMeasure::Create("latency", MinLatency)};
+    auto ExpectedCounterValue = Executor.runAndMeasure(CounterName);
+    if (!ExpectedCounterValue)
+      return ExpectedCounterValue.takeError();
+    if (*ExpectedCounterValue < MinValue)
+      MinValue = *ExpectedCounterValue;
+  }
+  std::vector<BenchmarkMeasure> Result = {
+      BenchmarkMeasure::Create("latency", MinValue)};
+  return std::move(Result);
 }
 
 } // namespace exegesis

llvm/trunk/tools/llvm-exegesis/lib/Uops.h

 public:
   UopsBenchmarkRunner(const LLVMState &State)
       : BenchmarkRunner(State, InstructionBenchmark::Uops) {}
   ~UopsBenchmarkRunner() override;
 
   static constexpr const size_t kMinNumDifferentAddresses = 6;
 
 private:
-  std::vector<BenchmarkMeasure>
-  runMeasurements(const ExecutableFunction &EF,
-                  ScratchSpace &Scratch) const override;
+  llvm::Expected<std::vector<BenchmarkMeasure>>
+  runMeasurements(const FunctionExecutor &Executor) const override;
 };
 
 } // namespace exegesis
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_UOPS_H

llvm/trunk/tools/llvm-exegesis/lib/Uops.cpp

 //===-- Uops.cpp ------------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "Uops.h"
 
 #include "Assembler.h"
 #include "BenchmarkRunner.h"
 #include "MCInstrDescView.h"
-#include "PerfHelper.h"
 #include "Target.h"
 
 // FIXME: Load constants into registers (e.g. with fld1) to not break
 // instructions like x87.
 
 // Ideally we would like the only limitation on executing uops to be the issue
 // ports. Maximizing port pressure increases the likelihood that the load is
 // distributed evenly across possible ports.
UopsSnippetGenerator::generateCodeTemplates(const Instruction &Instr) const {
   }
   CT.Info =
       "instruction has no tied variables picking Uses different from defs";
   CT.Instructions.push_back(std::move(IT));
   instantiateMemoryOperands(CT.ScratchSpacePointerInReg, CT.Instructions);
   return getSingleton(std::move(CT));
 }
 
-std::vector<BenchmarkMeasure>
-UopsBenchmarkRunner::runMeasurements(const ExecutableFunction &Function,
-                                     ScratchSpace &Scratch) const {
+llvm::Expected<std::vector<BenchmarkMeasure>>
+UopsBenchmarkRunner::runMeasurements(const FunctionExecutor &Executor) const {
   const auto &SchedModel = State.getSubtargetInfo().getSchedModel();
 
-  const auto RunMeasurement = [&Function,
-                               &Scratch](const char *const Counters) {
-    // We sum counts when there are several counters for a single ProcRes
-    // (e.g. P23 on SandyBridge).
-    int64_t CounterValue = 0;
-    llvm::SmallVector<llvm::StringRef, 2> CounterNames;
-    llvm::StringRef(Counters).split(CounterNames, ',');
-    for (const auto &CounterName : CounterNames) {
-      pfm::PerfEvent UopPerfEvent(CounterName);
-      if (!UopPerfEvent.valid())
-        llvm::report_fatal_error(
-            llvm::Twine("invalid perf event ").concat(Counters));
-      pfm::Counter Counter(UopPerfEvent);
-      Scratch.clear();
-      Counter.start();
-      Function(Scratch.ptr());
-      Counter.stop();
-      CounterValue += Counter.read();
-    }
-    return CounterValue;
-  };
-
   std::vector<BenchmarkMeasure> Result;
   const auto &PfmCounters = SchedModel.getExtraProcessorInfo().PfmCounters;
   // Uops per port.
   for (unsigned ProcResIdx = 1;
        ProcResIdx < SchedModel.getNumProcResourceKinds(); ++ProcResIdx) {
     const char *const Counters = PfmCounters.IssueCounters[ProcResIdx];
     if (!Counters)
       continue;
-    const double CounterValue = RunMeasurement(Counters);
+    auto ExpectedCounterValue = Executor.runAndMeasure(Counters);
+    if (!ExpectedCounterValue)
+      return ExpectedCounterValue.takeError();
     Result.push_back(BenchmarkMeasure::Create(
-        SchedModel.getProcResource(ProcResIdx)->Name, CounterValue));
+        SchedModel.getProcResource(ProcResIdx)->Name, *ExpectedCounterValue));
   }
   // NumMicroOps.
   if (const char *const UopsCounter = PfmCounters.UopsCounter) {
-    const double CounterValue = RunMeasurement(UopsCounter);
-    Result.push_back(BenchmarkMeasure::Create("NumMicroOps", CounterValue));
+    auto ExpectedCounterValue = Executor.runAndMeasure(UopsCounter);
+    if (!ExpectedCounterValue)
+      return ExpectedCounterValue.takeError();
+    Result.push_back(
+        BenchmarkMeasure::Create("NumMicroOps", *ExpectedCounterValue));
   }
-  return Result;
+  return std::move(Result);
 }
 
 constexpr const size_t UopsSnippetGenerator::kMinNumDifferentAddresses;
 
 } // namespace exegesis

llvm/trunk/tools/llvm-exegesis/llvm-exegesis.cpp

 #include "llvm/Support/Format.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include <algorithm>
 #include <string>
 
-static llvm::cl::opt<unsigned>
+static llvm::cl::opt<int>
     OpcodeIndex("opcode-index", llvm::cl::desc("opcode to measure, by index"),
                 llvm::cl::init(0));
 
-static llvm::cl::opt<std::string>
-    OpcodeName("opcode-name", llvm::cl::desc("opcode to measure, by name"),
-               llvm::cl::init(""));
+static llvm::cl::opt<std::string> OpcodeNames(
+    "opcode-name",
+    llvm::cl::desc("comma-separated list of opcodes to measure, by name"),
 
 static llvm::cl::opt<std::string>
     SnippetsFile("snippets-file", llvm::cl::desc("code snippets to measure"),
                  llvm::cl::init(""));
 
 static llvm::cl::opt<std::string>
     BenchmarkFile("benchmarks-file", llvm::cl::desc(""), llvm::cl::init(""));
 

 namespace exegesis {
 
 static llvm::ExitOnError ExitOnErr;
 
 #ifdef LLVM_EXEGESIS_INITIALIZE_NATIVE_TARGET
 void LLVM_EXEGESIS_INITIALIZE_NATIVE_TARGET();
 #endif
 
-// Checks that only one of OpcodeName, OpcodeIndex or SnippetsFile is provided,
-// and returns the opcode index or 0 if snippets should be read from
+// Checks that only one of OpcodeNames, OpcodeIndex or SnippetsFile is provided,
+// and returns the opcode indices or {} if snippets should be read from
 // `SnippetsFile`.
-static unsigned getOpcodeOrDie(const llvm::MCInstrInfo &MCInstrInfo) {
-  const size_t NumSetFlags = (OpcodeName.empty() ? 0 : 1) +
+static std::vector<unsigned>
+getOpcodesOrDie(const llvm::MCInstrInfo &MCInstrInfo) {
+  const size_t NumSetFlags = (OpcodeNames.empty() ? 0 : 1) +
                              (OpcodeIndex == 0 ? 0 : 1) +
                              (SnippetsFile.empty() ? 0 : 1);
   if (NumSetFlags != 1)
     llvm::report_fatal_error(
         "please provide one and only one of 'opcode-index', 'opcode-name' or "
         "'snippets-file'");
   if (!SnippetsFile.empty())
-    return 0;
+    return {};
   if (OpcodeIndex > 0)
-    return OpcodeIndex;
+    return {static_cast<unsigned>(OpcodeIndex)};
+  if (OpcodeIndex < 0) {
+    std::vector<unsigned> Result;
+    for (unsigned I = 1, E = MCInstrInfo.getNumOpcodes(); I <= E; ++I)
+      Result.push_back(I);
+    return Result;
+  }
   // Resolve opcode name -> opcode.
-  for (unsigned I = 0, E = MCInstrInfo.getNumOpcodes(); I < E; ++I)
+  const auto ResolveName =
+      [&MCInstrInfo](llvm::StringRef OpcodeName) -> unsigned {
+    for (unsigned I = 1, E = MCInstrInfo.getNumOpcodes(); I < E; ++I)
       if (MCInstrInfo.getName(I) == OpcodeName)
         return I;
-  llvm::report_fatal_error(llvm::Twine("unknown opcode ").concat(OpcodeName));
+    return 0u;
+  };
+  llvm::SmallVector<llvm::StringRef, 2> Pieces;
+  llvm::StringRef(OpcodeNames.getValue())
+      .split(Pieces, ",", /* MaxSplit */ -1, /* KeepEmpty */ false);
+  std::vector<unsigned> Result;
+  for (const llvm::StringRef OpcodeName : Pieces) {
+    if (unsigned Opcode = ResolveName(OpcodeName))
+      Result.push_back(Opcode);
+    else
+      llvm::report_fatal_error(
+          llvm::Twine("unknown opcode ").concat(OpcodeName));
+  }
+  return Result;
 }
 
 // Generates code snippets for opcode `Opcode`.
 static llvm::Expected<std::vector<BenchmarkCode>>
 generateSnippets(const LLVMState &State, unsigned Opcode) {
   const Instruction Instr(State, Opcode);
   const llvm::MCInstrDesc &InstrDesc = *Instr.Description;
   // Ignore instructions that we cannot run.
void benchmarkMain() {
   llvm::InitializeNativeTarget();
   llvm::InitializeNativeTargetAsmPrinter();
   llvm::InitializeNativeTargetAsmParser();
 #ifdef LLVM_EXEGESIS_INITIALIZE_NATIVE_TARGET
   LLVM_EXEGESIS_INITIALIZE_NATIVE_TARGET();
 #endif
 
   const LLVMState State;
-  const auto Opcode = getOpcodeOrDie(State.getInstrInfo());
+  const auto Opcodes = getOpcodesOrDie(State.getInstrInfo());
 
   std::vector<BenchmarkCode> Configurations;
-  if (Opcode > 0) {
-    // Ignore instructions without a sched class if -ignore-invalid-sched-class
-    // is passed.
+  if (!Opcodes.empty()) {
+    for (const unsigned Opcode : Opcodes) {
+      // Ignore instructions without a sched class if
+      // -ignore-invalid-sched-class is passed.
       if (IgnoreInvalidSchedClass &&
           State.getInstrInfo().get(Opcode).getSchedClass() == 0) {
-      llvm::errs() << "ignoring instruction without sched class\n";
-      return;
+        llvm::errs() << State.getInstrInfo().getName(Opcode)
+                     << ": ignoring instruction without sched class\n";
+        continue;
+      }
+      auto ConfigsForInstr = generateSnippets(State, Opcode);
+      if (!ConfigsForInstr) {
+        llvm::logAllUnhandledErrors(
+            ConfigsForInstr.takeError(), llvm::errs(),
+            llvm::Twine(State.getInstrInfo().getName(Opcode)).concat(": "));
+        continue;
+      }
+      std::move(ConfigsForInstr->begin(), ConfigsForInstr->end(),
+                std::back_inserter(Configurations));
     }
-    Configurations = ExitOnErr(generateSnippets(State, Opcode));
   } else {
     Configurations = ExitOnErr(readSnippets(State, SnippetsFile));
   }
 
   const std::unique_ptr<BenchmarkRunner> Runner =
       State.getExegesisTarget().createBenchmarkRunner(BenchmarkMode, State);
   if (!Runner) {
     llvm::report_fatal_error("cannot create benchmark runner");