[exegesis] "Skip codegen" dry-run mode

[lebedev.ri]

Summary

While "skip measurements mode" is super useful for test coverage,
i've come to discover it's trade-offs. It still calls back-end
to actually codegen the target assembly, and that is what is taking
80%+ of the time regardless of whether or not we skip the measurements.

On the other hand, just being able to see that exegesis can come up
with a snippet to measure something, is already very useful,
and takes maybe a second for a all-opcode sweep.

[gchatelet]

llvm/docs/CommandGuide/llvm-exegesis.rst

213

Maybe add that --skip-codegen implies --skip-measurements as defined later in the code.

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h

39

[nit] It's probably better to restructure the BenchmarkRunner to only run the benchmark instead of also being responsible for doing codegen.

Also passing multiple consecutive bool arguments seems like a red flag. Using an enum seems more compelling (since one flag implies the other).

[lebedev.ri]

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h

39

[nit] It's probably better to restructure the BenchmarkRunner to only run the benchmark instead of also being responsible for doing codegen.

The separation of BenchmarkRunner::getRunnableConfiguration() and BenchmarkRunner::runConfiguration()
only appeared in refactorings towards parallelization: D140271
It might be doable, but i'm not sure if it makes sense yet.

Also passing multiple consecutive bool arguments seems like a red flag. Using an enum seems more compelling (since one flag implies the other).

Yeah, i've tried that. Let me try once more i guess.

[lebedev.ri]

@gchatelet thank you for taking a look.
After thinking more, i come up with this approach.

[lebedev.ri]

It has occurred to me that this is not what i actually want.
The *full* codegen is slow, but small snippet is fast to codegen,
and it's still useful to be able to to that. So more options..

[lebedev.ri]

ping

[gchatelet]

llvm/docs/CommandGuide/llvm-exegesis.rst

199–209

I'm confused by this documentation.
Can we list the options and describe what they mean exactly?
What is the difference between skip-codegen and skip-full-codegen?
What are the outputs for the different stages?
Maybe name the options according to what they do instead of what it doesn't do.

llvm/tools/llvm-exegesis/llvm-exegesis.cpp

125–141

It seems better to use more descriptive flags:

1. Only generate the minimal instruction sequence
2. Same as 1. but also dumps an excerpt of the sequence (hex encoded)
3. Same as 2. but also creates the full sequence that can be dumped to a file using --dump-object-to-disk
4. Same as 3. but also runs the measurement.

[lebedev.ri]

@gchatelet thank you for taking a look!
I acknowledge that the naming seems confusing, but i think it represents the internal flow quite well.

llvm/docs/CommandGuide/llvm-exegesis.rst

199–209

Is this better?
I'm effectively modelling them as stop-before/stop-after of the opt

[gchatelet]

llvm/docs/CommandGuide/llvm-exegesis.rst

199–209

This is better but I'd avoid using the skip- part as it's unclear what it's actually doing.
So yes the documentation is improved but changing the flags themselves into more meaningful commands would help as well I think.
How about:

• prepare-snippet
• prepare-and-dump-snippet
• prepare-measured-code
• measure

[lebedev.ri]

llvm/docs/CommandGuide/llvm-exegesis.rst

199–209

I don't see why that is an improvement.

[lebedev.ri]

llvm/docs/CommandGuide/llvm-exegesis.rst

199–209

In particular, we are really overloading the "snippet" term here,
and "dump" doesn't really make sense here.
Do we not dump the "measured code"? And we don't dump the actual mini-snippet either.

[lebedev.ri]

To be noted, i feel like it's a moot point to argue over the namings. It's mostly irrelevant in the long run.
What do @courbet @RKSimon think? I can change it any way that is agreed.

[gchatelet]

In D140702#4024640, @lebedev.ri wrote:

To be noted, i feel like it's a moot point to argue over the namings. It's mostly irrelevant in the long run.
What do @courbet @RKSimon think? I can change it any way that is agreed.

The naming was a concern for @courbet as well. We discussed this patch together.

[courbet]

In D140702#4024640, @lebedev.ri wrote:

To be noted, i feel like it's a moot point to argue over the namings. It's mostly irrelevant in the long run.
What do @courbet @RKSimon think? I can change it any way that is agreed.

I think it's better to express the flag in terms of what you *want* instead of what you *do not want*. You know what you want, but you might not know that you don't want what you don't know exists :)

[lebedev.ri]

In D140702#4025422, @gchatelet wrote:

In D140702#4024640, @lebedev.ri wrote:

To be noted, i feel like it's a moot point to argue over the namings. It's mostly irrelevant in the long run.
What do @courbet @RKSimon think? I can change it any way that is agreed.

The naming was a concern for @courbet as well. We discussed this patch together.

Good point. But none of the comments before this one mentioned it in any way.
Let me rename them then.

[lebedev.ri]

Ok, how about now?

[lebedev.ri]

Rebased.

[lebedev.ri]

@gchatelet thank you for the review!

[lebedev.ri]

llvm/tools/llvm-exegesis/lib/BenchmarkResult.h

37–42

I just realized, we only changed the user-facing side,
but kept the implementation detail. Is that fine?

[gchatelet]

llvm/tools/llvm-exegesis/lib/BenchmarkResult.h

37–42

I fixed it in https://github.com/llvm/llvm-project/commit/bb37cab8a564c0ee1a764e5464f7c8f41a86e504

llvm/docs/CommandGuide/llvm-exegesis.rst

  `latency` mode can be  make use of either RDTSC or LBR.
  `latency[LBR]` is only available on X86 (at least `Skylake`).
  To run in `latency` mode, a positive value must be specified
  for `x86-lbr-sample-period` and `--repetition-mode=loop`.
 
  In `analysis` mode, you also need to specify at least one of the
  `-analysis-clusters-output-file=` and `-analysis-inconsistencies-output-file=`.
 
-.. option:: --skip-measurements
+.. option:: --benchmark-phase=[prepare-snippet|prepare-and-assemble-snippet|assemble-measured-code|measure]
 
   By default, when `-mode=` is specified, the generated snippet will be executed
   and measured, and that requires that we are running on the hardware for which
   the snippet was generated, and that supports performance measurements.
-  But sometimes, you just want to generate snippets, and this is exactly what
-  this options allows one to do.
+  However, it is possible to stop at some stage before measuring. Choices are:
+  * ``prepare-snippet``: Only generate the minimal instruction sequence.
+  * ``prepare-and-assemble-snippet``: Same as ``prepare-snippet``, but also dumps an excerpt of the sequence (hex encoded).
+  * ``assemble-measured-code``: Same as ``prepare-and-assemble-snippet``. but also creates the full sequence that can be dumped to a file using ``--dump-object-to-disk``.
+  * ``measure``: Same as ``assemble-measured-code``, but also runs the measurement.
 
 .. option:: -x86-lbr-sample-period=<nBranches/sample>
 
   Specify the LBR sampling period - how many branches before we take a sample.
   When a positive value is specified for this option and when the mode is `latency`,
   we will use LBRs for measuring.
   On choosing the "right" sampling period, a small value is preferred, but throttling
   could occur if the sampling is too frequent. A prime number should be used to
   avoid consistently skipping certain blocks.
 
 .. option:: -x86-disable-upper-sse-registers
 
   Using the upper xmm registers (xmm8-xmm15) forces a longer instruction encoding

 .. option:: --analysis-override-benchmark-triple-and-cpu
 
   By default, llvm-exegesis will analyze the benchmarks for the triple/CPU they
   were measured for, but if you want to analyze them for some other combination
   (specified via `-mtriple`/`-mcpu`), you can pass this flag.
 
 .. option:: --dump-object-to-disk=true
 
- By default, llvm-exegesis will dump the generated code to a temporary file to
- enable code inspection. You may disable it to speed up the execution and save
- disk space.
+ If set,  llvm-exegesis will dump the generated code to a temporary file to
+ enable code inspection. Disabled by default.
 
 EXIT STATUS
 -----------
 
 :program:`llvm-exegesis` returns 0 on success. Otherwise, an error message is
 printed to standard error, and the tool returns a non 0 value.

llvm/test/tools/llvm-exegesis/PowerPC/latency-by-opcode-name.s

-# RUN: llvm-exegesis -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 --skip-measurements -mode=latency -opcode-name=ADD8 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 --benchmark-phase=assemble-measured-code -mode=latency -opcode-name=ADD8 | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD8
 CHECK-NEXT: config: ''
 CHECK-NEXT: register_initial_values:
 CHECK-DAG: - '[[REG1:[A-Z0-9]+]]=0x0'
 CHECK-LAST: ...

llvm/test/tools/llvm-exegesis/PowerPC/unsupported-opcode.s

-# RUN: llvm-exegesis -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 --skip-measurements -mode=latency -opcode-name=SELECT_I8 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 --benchmark-phase=assemble-measured-code -mode=latency -opcode-name=SELECT_I8 2>&1 | FileCheck %s
 
 CHECK: Unsupported opcode: isPseudo/usesCustomInserter

llvm/test/tools/llvm-exegesis/X86/inverse_throughput/inverse_throughput-PBLENDVBrr0.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --skip-measurements -x86-disable-upper-sse-registers -opcode-name=PBLENDVBrr0 -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --benchmark-phase=assemble-measured-code -x86-disable-upper-sse-registers -opcode-name=PBLENDVBrr0 -repetition-mode=loop | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode:            inverse_throughput
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     - 'PBLENDVBrr0 [[LHS0:XMM[1-7]]] [[LHS0]] [[RHS0:XMM[0-7]]]'
 CHECK-NEXT:     - 'PBLENDVBrr0 [[LHS1:XMM[1-7]]] [[LHS1]] [[RHS1:XMM[0-7]]]'
 CHECK-NEXT:     - 'PBLENDVBrr0 [[LHS2:XMM[1-7]]] [[LHS2]] [[RHS2:XMM[0-7]]]'
 CHECK-NEXT:     - 'PBLENDVBrr0 [[LHS3:XMM[1-7]]] [[LHS3]] [[RHS3:XMM[0-7]]]'

llvm/test/tools/llvm-exegesis/X86/inverse_throughput/inverse_throughput-by-opcode-name.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --skip-measurements -opcode-name=ADD32rr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --skip-measurements -opcode-name=ADD32rr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --benchmark-phase=assemble-measured-code -opcode-name=ADD32rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --benchmark-phase=assemble-measured-code -opcode-name=ADD32rr -repetition-mode=loop | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: inverse_throughput
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD32rr

llvm/test/tools/llvm-exegesis/X86/inverse_throughput/inverse_throughput-disable-upper-sse-registers.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --skip-measurements -x86-disable-upper-sse-registers -opcode-name=ADDPSrr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --benchmark-phase=assemble-measured-code -x86-disable-upper-sse-registers -opcode-name=ADDPSrr -repetition-mode=loop | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode:            inverse_throughput
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     - 'ADDPSrr [[LHS0:XMM[0-7]]] [[LHS0]] [[RHS0:XMM[0-7]]]'
 CHECK-NEXT:     - 'ADDPSrr [[LHS1:XMM[0-7]]] [[LHS1]] [[RHS1:XMM[0-7]]]'
 CHECK-NEXT:     - 'ADDPSrr [[LHS2:XMM[0-7]]] [[LHS2]] [[RHS2:XMM[0-7]]]'
 CHECK-NEXT:     - 'ADDPSrr [[LHS3:XMM[0-7]]] [[LHS3]] [[RHS3:XMM[0-7]]]'

llvm/test/tools/llvm-exegesis/X86/latency/dump-object-to-disk.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-ON
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-ON
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADDPSrr -repetition-mode=duplicate 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADDPSrr -repetition-mode=loop 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-ON
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-ON
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADDPSrr -repetition-mode=duplicate 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADDPSrr -repetition-mode=loop 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=ADDPSrr -repetition-mode=duplicate 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=ADDPSrr -repetition-mode=loop 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=1 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet -opcode-name=ADDPSrr -repetition-mode=duplicate -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet -opcode-name=ADDPSrr -repetition-mode=loop      -dump-object-to-disk=0 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet -opcode-name=ADDPSrr -repetition-mode=duplicate 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet -opcode-name=ADDPSrr -repetition-mode=loop 2>&1 | FileCheck %s --check-prefix=CHECK-OFF
 
 CHECK-ON: Check generated assembly with
 CHECK-OFF-NOT: Check generated assembly with

llvm/test/tools/llvm-exegesis/X86/latency/latency-CMOV32rr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=CMOV32rr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=CMOV32rr -repetition-mode=loop | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=CMOV32rr -repetition-mode=min | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=CMOV32rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=CMOV32rr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=CMOV32rr -repetition-mode=min | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     'CMOV32rr {{.*}} i_0x{{[0-9a-f]}}'
 CHECK-NEXT: config: ''
 CHECK-LAST: ...

llvm/test/tools/llvm-exegesis/X86/latency/latency-CVTSD2SI64rr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=CVTSD2SI64rr -repetition-mode=loop --max-configs-per-opcode=8192 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=CVTSD2SI64rr -repetition-mode=loop --max-configs-per-opcode=8192 | FileCheck %s
 
 # We used to fail to setup the snippet, and that disabled liveness tracking
 # which we'd then tried to access during this run-line.
 # Just check that we don't crash
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     CVTSD2SI64rr

llvm/test/tools/llvm-exegesis/X86/latency/latency-IN16rr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=IN16rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=IN16rr -repetition-mode=duplicate | FileCheck %s
 
 # FIXME: Sometimes fails with: 'unimplemented operand type'
 # ALLOW_RETRIES: 2
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     - 'IN16rr'
 CHECK-NEXT:     - {{.*}}
 CHECK-NEXT: config: ''
 CHECK-NEXT: register_initial_values:
 CHECK-DAG: - '[[REG1:[A-Z0-9]+]]=0x0'
 CHECK-LAST: ...

llvm/test/tools/llvm-exegesis/X86/latency/latency-LEA64_32r.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck --check-prefix CHECK-COUNTS %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck --check-prefix CHECK-COUNTS %s
 
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck --check-prefix CHECK-COUNTS %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck --check-prefix CHECK-COUNTS %s
 
 ## Intentionally run llvm-exegesis twice per output!
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 --benchmarks-file=%t.duplicate.yaml
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 --benchmarks-file=%t.duplicate.yaml
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 --benchmarks-file=%t.duplicate.yaml
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=duplicate -max-configs-per-opcode=2 --benchmarks-file=%t.duplicate.yaml
 # RUN: FileCheck --input-file %t.duplicate.yaml %s
 # RUN: FileCheck --input-file %t.duplicate.yaml --check-prefix CHECK-COUNTS %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 --benchmarks-file=%t.loop.yaml
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 --benchmarks-file=%t.loop.yaml
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 --benchmarks-file=%t.loop.yaml
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64_32r -repetition-mode=loop -max-configs-per-opcode=2 --benchmarks-file=%t.loop.yaml
 # RUN: FileCheck --input-file %t.loop.yaml %s
 # RUN: FileCheck --input-file %t.loop.yaml --check-prefix CHECK-COUNTS %s
 
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:

llvm/test/tools/llvm-exegesis/X86/latency/latency-LEA64r.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=LEA64r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=LEA64r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     LEA64r
 CHECK-NEXT: config: '0(%[[REG1:[A-Z0-9]+]], %[[REG1]], 1)'
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     LEA64r
 CHECK-NEXT: config: '42(%[[REG2:[A-Z0-9]+]], %[[REG2]], 1)'

llvm/test/tools/llvm-exegesis/X86/latency/latency-SBB8rr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SBB8rr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SBB8rr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SBB8rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SBB8rr -repetition-mode=loop | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     SBB8rr
 CHECK-NEXT: config: ''
 CHECK-NEXT: register_initial_values:
 CHECK-DAG: - '[[REG1:[A-Z0-9]+]]=0x0'
 CHECK-LAST: ...

llvm/test/tools/llvm-exegesis/X86/latency/latency-SETCCr-cond-codes-sweep.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SETCCr --max-configs-per-opcode=1 | FileCheck %s --check-prefix=CHECK
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SETCCr --max-configs-per-opcode=256 | FileCheck %s --check-prefix=SWEEP
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SETCCr --max-configs-per-opcode=1 | FileCheck %s --check-prefix=CHECK
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SETCCr --max-configs-per-opcode=256 | FileCheck %s --check-prefix=SWEEP
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     'SETCCr {{.*}} i_0x{{[0-9a-f]}}'
 
 SWEEP-DAG:      'SETCCr {{.*}} i_0x0'

llvm/test/tools/llvm-exegesis/X86/latency/latency-SQRTSSr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SQRTSSr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SQRTSSr -repetition-mode=loop | FileCheck %s
 
 # Check that the setup code for MXCSR does not crash the snippet.
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     SQRTSSr
 CHECK-NEXT: config: ''
 CHECK-NEXT: register_initial_values:
 CHECK-NOT: crashed
 CHECK-LAST: ...

llvm/test/tools/llvm-exegesis/X86/latency/latency-SYSENTER.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SYSENTER -repetition-mode=duplicate 2>&1 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SYSENTER -repetition-mode=loop 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SYSENTER -repetition-mode=duplicate 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SYSENTER -repetition-mode=loop 2>&1 | FileCheck %s
 
 CHECK: SYSENTER: unsupported opcode

llvm/test/tools/llvm-exegesis/X86/latency/latency-WRFSBASE.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=WRFSBASE -repetition-mode=duplicate 2>&1 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=WRFSBASE -repetition-mode=loop 2>&1 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=WRFSBASE64 -repetition-mode=duplicate 2>&1 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=WRFSBASE64 -repetition-mode=loop 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=WRFSBASE -repetition-mode=duplicate 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=WRFSBASE -repetition-mode=loop 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=WRFSBASE64 -repetition-mode=duplicate 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=WRFSBASE64 -repetition-mode=loop 2>&1 | FileCheck %s
 
 CHECK: WRFSBASE{{(64)?}}: unsupported opcode

llvm/test/tools/llvm-exegesis/X86/latency/latency-by-opcode-name.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADD32rr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=ADD32rr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADD32rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=ADD32rr -repetition-mode=loop | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD32rr
 CHECK-NEXT: config: ''
 CHECK-NEXT: register_initial_values:
 CHECK-DAG: - '[[REG1:[A-Z0-9]+]]=0x0'
 CHECK-LAST: ...

llvm/test/tools/llvm-exegesis/X86/latency/max-configs.test

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SBB8rr -max-configs-per-opcode=1 | FileCheck -check-prefixes=CHECK,CHECK1 %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --skip-measurements -opcode-name=SBB8rr -max-configs-per-opcode=2 | FileCheck -check-prefixes=CHECK,CHECK2 %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SBB8rr -max-configs-per-opcode=1 | FileCheck -check-prefixes=CHECK,CHECK1 %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code -opcode-name=SBB8rr -max-configs-per-opcode=2 | FileCheck -check-prefixes=CHECK,CHECK2 %s
 
 CHECK:      ---
 CHECK-NEXT: mode: latency
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     SBB8rr
 CHECK-NEXT: config: ''
 CHECK-NEXT: register_initial_values:

llvm/test/tools/llvm-exegesis/X86/latency/skip-codegen.s

+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=assemble-measured-code       -opcode-name=LEA64r -repetition-mode=duplicate | FileCheck %s --check-prefixes=CHECK,CHECK-CODEGEN
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-snippet              -opcode-name=LEA64r -repetition-mode=duplicate | FileCheck %s --check-prefixes=CHECK,CHECK-NO-CODEGEN
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=latency --benchmark-phase=prepare-and-assemble-snippet -opcode-name=LEA64r -repetition-mode=duplicate | FileCheck %s --check-prefixes=CHECK,CHECK-CODEGEN
+
+CHECK:      ---
+CHECK-NEXT: mode: latency
+CHECK-NEXT: key:
+CHECK-NEXT:   instructions:
+CHECK-NEXT:     LEA64r
+CHECK-CODEGEN: assembled_snippet: {{[A-Z0-9]+}}{{$}}
+CHECK-NO-CODEGEN: assembled_snippet: ''{{$}}

llvm/test/tools/llvm-exegesis/X86/uops/uops-ADD32mi8.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32mi8 -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32mi8 -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32mi8 -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32mi8 -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD32mi8

llvm/test/tools/llvm-exegesis/X86/uops/uops-ADD32mr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32mr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32mr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32mr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32mr -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD32mr

llvm/test/tools/llvm-exegesis/X86/uops/uops-ADD32rm.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32rm -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32rm -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32rm -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32rm -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD32rm

llvm/test/tools/llvm-exegesis/X86/uops/uops-ADD_F32m.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD_F32m -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD_F32m -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD_F32m -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD_F32m -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD_F32m
 CHECK:      register_initial_values:
 CHECK:      FPCW

llvm/test/tools/llvm-exegesis/X86/uops/uops-BEXTR32rm.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=BEXTR32rm -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=BEXTR32rm -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=BEXTR32rm -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=BEXTR32rm -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     BEXTR32rm

llvm/test/tools/llvm-exegesis/X86/uops/uops-BSF16rm.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=BSF16rm -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=BSF16rm -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=BSF16rm -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=BSF16rm -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     BSF16rm

llvm/test/tools/llvm-exegesis/X86/uops/uops-BTR64mr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=BTR64mr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=BTR64mr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=BTR64mr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=BTR64mr -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     BTR64mr

llvm/test/tools/llvm-exegesis/X86/uops/uops-CMOV16rm-noreg-serialization.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=CMOV16rm  -benchmarks-file=- | FileCheck %s -check-prefixes=CHECK-YAML
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=CMOV16rm  -benchmarks-file=- | FileCheck %s -check-prefixes=CHECK-YAML
 
 # https://bugs.llvm.org/show_bug.cgi?id=41448
 # Verify that we correctly serialize RegNo 0 as %noreg, not as an empty string!
 
 CHECK-YAML:      ---
 CHECK-YAML-NEXT: mode:            uops
 CHECK-YAML-NEXT: key:
 CHECK-YAML-NEXT:   instructions:
 CHECK-YAML-NEXT:     - 'CMOV16rm {{[A-Z0-9]+}} {{[A-Z0-9]+}} {{[A-Z0-9]+}} i_0x1 %noreg i_0x0 %noreg i_0x{{[0-9a-f]}}'
 CHECK-YAML-LAST: ...

llvm/test/tools/llvm-exegesis/X86/uops/uops-FLDENVm.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=FLDENVm,FLDL2E -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=FLDENVm,FLDL2E -repetition-mode=duplicate | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     FLDENVm

llvm/test/tools/llvm-exegesis/X86/uops/uops-HLT.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops -skip-measurements --dump-object-to-disk=0 --repetition-mode=loop --loop-body-size=1000 --result-aggregation-mode=min --opcode-name=HLT --max-configs-per-opcode=8192 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code --dump-object-to-disk=0 --repetition-mode=loop --loop-body-size=1000 --result-aggregation-mode=min --opcode-name=HLT --max-configs-per-opcode=8192 | FileCheck %s
 
 # By definition, loop repetitor can not be used for terminator instructions.
 # Just check that we do not crash.
 
 CHECK:      ---
 CHECK-NEXT: mode: uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     HLT

llvm/test/tools/llvm-exegesis/X86/uops/uops-LEA64r.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=LEA64r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=LEA64r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=LEA64r -repetition-mode=duplicate -max-configs-per-opcode=2 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=LEA64r -repetition-mode=loop -max-configs-per-opcode=2 | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     LEA64r
 CHECK-NEXT: config: '0(%[[REG1:[A-Z0-9]+]], %[[REG2:[A-Z0-9]+]], 1)'
 
 CHECK:      ---
 CHECK-NEXT: mode: uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     LEA64r
 CHECK-NEXT: config: '42(%[[REG3:[A-Z0-9]+]], %[[REG4:[A-Z0-9]+]], 1)'

llvm/test/tools/llvm-exegesis/X86/uops/uops-POPCNT32rr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=POPCNT32rr 2>&1 | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=POPCNT32rr 2>&1 | FileCheck %s
 
 CHECK:      ---
 CHECK-NEXT: mode: uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     - 'POPCNT32rr

llvm/test/tools/llvm-exegesis/X86/uops/uops-STD.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=STD -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=STD -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=STD -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=STD -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     STD

llvm/test/tools/llvm-exegesis/X86/uops/uops-VFMADDSS4rm.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=VFMADDSS4rm -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=VFMADDSS4rm -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=VFMADDSS4rm -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=VFMADDSS4rm -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     VFMADDSS4rm
 CHECK:      register_initial_values:
 CHECK:      MXCSR

llvm/test/tools/llvm-exegesis/X86/uops/uops-XCHG64rr.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=XCHG64rr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=XCHG64rr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=XCHG64rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=XCHG64rr -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     XCHG64rr

llvm/test/tools/llvm-exegesis/X86/uops/uops-by-opcode-name.s

-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32rr -repetition-mode=duplicate | FileCheck %s
-# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -opcode-name=ADD32rr -repetition-mode=loop | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32rr -repetition-mode=duplicate | FileCheck %s
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -opcode-name=ADD32rr -repetition-mode=loop | FileCheck %s
 
 CHECK:      mode:            uops
 CHECK-NEXT: key:
 CHECK-NEXT:   instructions:
 CHECK-NEXT:     ADD32rr

llvm/test/tools/llvm-exegesis/X86/uops/uops-misspelled-div.s

-# RUN: not llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --skip-measurements -snippets-file=%s 2>&1 | FileCheck %s
+# RUN: not llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=uops --benchmark-phase=assemble-measured-code -snippets-file=%s 2>&1 | FileCheck %s
 
 # llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64-DEFREG CL 1
 # llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64-DEFREG AX 1
 div8r cl
 
 CHECK: error: invalid instruction mnemonic 'div8r'
 CHECK: llvm-exegesis error: cannot parse asm file

llvm/tools/llvm-exegesis/lib/BenchmarkResult.h

 #include <unordered_map>
 #include <vector>
 
 namespace llvm {
 class Error;
 
 namespace exegesis {
 
+enum class BenchmarkPhaseSelectorE {
+  StopBeforeAllCodegen,
+  StopBeforeFullCodegen,
+  StopBeforeMeasurements,
+  Measure,
+};
+
 enum class InstructionBenchmarkFilter { All, RegOnly, WithMem };
 
 struct InstructionBenchmarkKey {
   // The LLVM opcode name.
   std::vector<MCInst> Instructions;
   // The initial values of the registers.
   std::vector<RegisterValue> RegisterInitialValues;
   // An opaque configuration, that can be used to separate several benchmarks of

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h

 namespace llvm {
 namespace exegesis {
 
 // Common code for all benchmark modes.
 class BenchmarkRunner {
 public:
   explicit BenchmarkRunner(const LLVMState &State,
                            InstructionBenchmark::ModeE Mode,
-                           bool BenchmarkSkipMeasurements);
+                           BenchmarkPhaseSelectorE BenchmarkPhaseSelector);
 
   virtual ~BenchmarkRunner();
 
   class RunnableConfiguration {
     friend class BenchmarkRunner;
 
   public:
     ~RunnableConfiguration() = default;
public:
 
     virtual Expected<llvm::SmallVector<int64_t, 4>>
     runAndSample(const char *Counters) const = 0;
   };
 
 protected:
   const LLVMState &State;
   const InstructionBenchmark::ModeE Mode;
-  const bool BenchmarkSkipMeasurements;
+  const BenchmarkPhaseSelectorE BenchmarkPhaseSelector;
 
 private:
   virtual Expected<std::vector<BenchmarkMeasure>>
   runMeasurements(const FunctionExecutor &Executor) const = 0;
 
   Expected<SmallString<0>> assembleSnippet(const BenchmarkCode &BC,
                                            const SnippetRepetitor &Repetitor,
                                            unsigned MinInstructions,

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp

 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Program.h"
 
 namespace llvm {
 namespace exegesis {
 
 BenchmarkRunner::BenchmarkRunner(const LLVMState &State,
                                  InstructionBenchmark::ModeE Mode,
-                                 bool BenchmarkSkipMeasurements)
-    : State(State), Mode(Mode),
-      BenchmarkSkipMeasurements(BenchmarkSkipMeasurements),
+                                 BenchmarkPhaseSelectorE BenchmarkPhaseSelector)
+    : State(State), Mode(Mode), BenchmarkPhaseSelector(BenchmarkPhaseSelector),
       Scratch(std::make_unique<ScratchSpace>()) {}
 
 BenchmarkRunner::~BenchmarkRunner() = default;
 
 namespace {
 class FunctionExecutorImpl : public BenchmarkRunner::FunctionExecutor {
 public:
   FunctionExecutorImpl(const LLVMState &State,
BenchmarkRunner::getRunnableConfiguration(
 
   const std::vector<MCInst> &Instructions = BC.Key.Instructions;
 
   InstrBenchmark.Key = BC.Key;
 
   // Assemble at least kMinInstructionsForSnippet instructions by repeating
   // the snippet for debug/analysis. This is so that the user clearly
   // understands that the inside instructions are repeated.
+  if (BenchmarkPhaseSelector > BenchmarkPhaseSelectorE::StopBeforeAllCodegen) {
     const int MinInstructionsForSnippet = 4 * Instructions.size();
     const int LoopBodySizeForSnippet = 2 * Instructions.size();
-  {
     auto Snippet = assembleSnippet(BC, Repetitor, MinInstructionsForSnippet,
                                    LoopBodySizeForSnippet);
     if (Error E = Snippet.takeError())
       return std::move(E);
     const ExecutableFunction EF(State.createTargetMachine(),
                                 getObjectFromBuffer(*Snippet));
     const auto FnBytes = EF.getFunctionBytes();
     llvm::append_range(InstrBenchmark.AssembledSnippet, FnBytes);
   }
 
   // Assemble NumRepetitions instructions repetitions of the snippet for
   // measurements.
-  {
+  if (BenchmarkPhaseSelector > BenchmarkPhaseSelectorE::StopBeforeFullCodegen) {
     auto Snippet = assembleSnippet(BC, Repetitor, InstrBenchmark.NumRepetitions,
                                    LoopBodySize);
     if (Error E = Snippet.takeError())
       return std::move(E);
     RC.ObjectFile = getObjectFromBuffer(*Snippet);
   }
 
   return std::move(RC);
 }
 
 Expected<InstructionBenchmark>
 BenchmarkRunner::runConfiguration(RunnableConfiguration &&RC,
                                   bool DumpObjectToDisk) const {
   InstructionBenchmark &InstrBenchmark = RC.InstrBenchmark;
   object::OwningBinary<object::ObjectFile> &ObjectFile = RC.ObjectFile;
 
-  if (DumpObjectToDisk) {
+  if (DumpObjectToDisk &&
+      BenchmarkPhaseSelector > BenchmarkPhaseSelectorE::StopBeforeFullCodegen) {
     auto ObjectFilePath = writeObjectFile(ObjectFile.getBinary()->getData());
     if (Error E = ObjectFilePath.takeError()) {
       InstrBenchmark.Error = toString(std::move(E));
       return std::move(InstrBenchmark);
     }
     outs() << "Check generated assembly with: /usr/bin/objdump -d "
            << *ObjectFilePath << "\n";
   }
 
-  if (BenchmarkSkipMeasurements) {
-    InstrBenchmark.Error =
-        "in --skip-measurements mode, actual measurements skipped.";
+  if (BenchmarkPhaseSelector < BenchmarkPhaseSelectorE::Measure) {
+    InstrBenchmark.Error = "actual measurements skipped.";
     return std::move(InstrBenchmark);
   }
 
   const FunctionExecutorImpl Executor(State, std::move(ObjectFile),
                                       Scratch.get());
   auto NewMeasurements = runMeasurements(Executor);
   if (Error E = NewMeasurements.takeError()) {
     if (!E.isA<SnippetCrash>())

llvm/tools/llvm-exegesis/lib/LatencyBenchmarkRunner.h

 
 namespace llvm {
 namespace exegesis {
 
 class LatencyBenchmarkRunner : public BenchmarkRunner {
 public:
   LatencyBenchmarkRunner(
       const LLVMState &State, InstructionBenchmark::ModeE Mode,
-      bool BenchmarkSkipMeasurements,
+      BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
       InstructionBenchmark::ResultAggregationModeE ResultAggMode);
   ~LatencyBenchmarkRunner() override;
 
 private:
   Expected<std::vector<BenchmarkMeasure>>
   runMeasurements(const FunctionExecutor &Executor) const override;
 
   InstructionBenchmark::ResultAggregationModeE ResultAggMode;
 };
 } // namespace exegesis
 } // namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_LATENCY_H

llvm/tools/llvm-exegesis/lib/LatencyBenchmarkRunner.cpp

 #include <algorithm>
 #include <cmath>
 
 namespace llvm {
 namespace exegesis {
 
 LatencyBenchmarkRunner::LatencyBenchmarkRunner(
     const LLVMState &State, InstructionBenchmark::ModeE Mode,
-    bool BenchmarkSkipMeasurements,
+    BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
     InstructionBenchmark::ResultAggregationModeE ResultAgg)
-    : BenchmarkRunner(State, Mode, BenchmarkSkipMeasurements) {
+    : BenchmarkRunner(State, Mode, BenchmarkPhaseSelector) {
   assert((Mode == InstructionBenchmark::Latency ||
           Mode == InstructionBenchmark::InverseThroughput) &&
          "invalid mode");
   ResultAggMode = ResultAgg;
 }
 
 LatencyBenchmarkRunner::~LatencyBenchmarkRunner() = default;
 

llvm/tools/llvm-exegesis/lib/Target.h

public:
   // Creates a snippet generator for the given mode.
   std::unique_ptr<SnippetGenerator>
   createSnippetGenerator(InstructionBenchmark::ModeE Mode,
                          const LLVMState &State,
                          const SnippetGenerator::Options &Opts) const;
   // Creates a benchmark runner for the given mode.
   Expected<std::unique_ptr<BenchmarkRunner>> createBenchmarkRunner(
       InstructionBenchmark::ModeE Mode, const LLVMState &State,
-      bool BenchmarkSkipMeasurements,
+      BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
       InstructionBenchmark::ResultAggregationModeE ResultAggMode =
           InstructionBenchmark::Min) const;
 
   // Returns the ExegesisTarget for the given triple or nullptr if the target
   // does not exist.
   static const ExegesisTarget *lookup(Triple TT);
   // Returns the default (unspecialized) ExegesisTarget.
   static const ExegesisTarget &getDefault();
private:
   // Targets can implement their own snippet generators/benchmarks runners by
   // implementing these.
   std::unique_ptr<SnippetGenerator> virtual createSerialSnippetGenerator(
       const LLVMState &State, const SnippetGenerator::Options &Opts) const;
   std::unique_ptr<SnippetGenerator> virtual createParallelSnippetGenerator(
       const LLVMState &State, const SnippetGenerator::Options &Opts) const;
   std::unique_ptr<BenchmarkRunner> virtual createLatencyBenchmarkRunner(
       const LLVMState &State, InstructionBenchmark::ModeE Mode,
-      bool BenchmarkSkipMeasurements,
+      BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
       InstructionBenchmark::ResultAggregationModeE ResultAggMode) const;
   std::unique_ptr<BenchmarkRunner> virtual createUopsBenchmarkRunner(
-      const LLVMState &State, bool BenchmarkSkipMeasurements,
+      const LLVMState &State, BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
       InstructionBenchmark::ResultAggregationModeE ResultAggMode) const;
 
   const ExegesisTarget *Next = nullptr;
   const ArrayRef<CpuAndPfmCounters> CpuPfmCounters;
 };
 
 } // namespace exegesis
 } // namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_TARGET_H

llvm/tools/llvm-exegesis/lib/Target.cpp

case InstructionBenchmark::InverseThroughput:
     return createParallelSnippetGenerator(State, Opts);
   }
   return nullptr;
 }
 
 Expected<std::unique_ptr<BenchmarkRunner>>
 ExegesisTarget::createBenchmarkRunner(
     InstructionBenchmark::ModeE Mode, const LLVMState &State,
-    bool BenchmarkSkipMeasurements,
+    BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
     InstructionBenchmark::ResultAggregationModeE ResultAggMode) const {
   PfmCountersInfo PfmCounters = State.getPfmCounters();
   switch (Mode) {
   case InstructionBenchmark::Unknown:
     return nullptr;
   case InstructionBenchmark::Latency:
   case InstructionBenchmark::InverseThroughput:
-    if (!BenchmarkSkipMeasurements && !PfmCounters.CycleCounter) {
+    if (BenchmarkPhaseSelector == BenchmarkPhaseSelectorE::Measure &&
+        !PfmCounters.CycleCounter) {
       const char *ModeName = Mode == InstructionBenchmark::Latency
                                  ? "latency"
                                  : "inverse_throughput";
       return make_error<Failure>(
           Twine("can't run '")
               .concat(ModeName)
               .concat(
                   "' mode, sched model does not define a cycle counter. You "
                   "can pass --skip-measurements to skip the actual "
                   "benchmarking."));
     }
-    return createLatencyBenchmarkRunner(State, Mode, BenchmarkSkipMeasurements,
+    return createLatencyBenchmarkRunner(State, Mode, BenchmarkPhaseSelector,
                                         ResultAggMode);
   case InstructionBenchmark::Uops:
-    if (!BenchmarkSkipMeasurements && !PfmCounters.UopsCounter &&
-        !PfmCounters.IssueCounters)
+    if (BenchmarkPhaseSelector == BenchmarkPhaseSelectorE::Measure &&
+        !PfmCounters.UopsCounter && !PfmCounters.IssueCounters)
       return make_error<Failure>(
           "can't run 'uops' mode, sched model does not define uops or issue "
           "counters. You can pass --skip-measurements to skip the actual "
           "benchmarking.");
-    return createUopsBenchmarkRunner(State, BenchmarkSkipMeasurements,
+    return createUopsBenchmarkRunner(State, BenchmarkPhaseSelector,
                                      ResultAggMode);
   }
   return nullptr;
 }
 
 std::unique_ptr<SnippetGenerator> ExegesisTarget::createSerialSnippetGenerator(
     const LLVMState &State, const SnippetGenerator::Options &Opts) const {
   return std::make_unique<SerialSnippetGenerator>(State, Opts);
 }
 
 std::unique_ptr<SnippetGenerator> ExegesisTarget::createParallelSnippetGenerator(
     const LLVMState &State, const SnippetGenerator::Options &Opts) const {
   return std::make_unique<ParallelSnippetGenerator>(State, Opts);
 }
 
 std::unique_ptr<BenchmarkRunner> ExegesisTarget::createLatencyBenchmarkRunner(
     const LLVMState &State, InstructionBenchmark::ModeE Mode,
-    bool BenchmarkSkipMeasurements,
+    BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
     InstructionBenchmark::ResultAggregationModeE ResultAggMode) const {
   return std::make_unique<LatencyBenchmarkRunner>(
-      State, Mode, BenchmarkSkipMeasurements, ResultAggMode);
+      State, Mode, BenchmarkPhaseSelector, ResultAggMode);
 }
 
 std::unique_ptr<BenchmarkRunner> ExegesisTarget::createUopsBenchmarkRunner(
-    const LLVMState &State, bool BenchmarkSkipMeasurements,
+    const LLVMState &State, BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
     InstructionBenchmark::ResultAggregationModeE /*unused*/) const {
-  return std::make_unique<UopsBenchmarkRunner>(State,
-                                               BenchmarkSkipMeasurements);
+  return std::make_unique<UopsBenchmarkRunner>(State, BenchmarkPhaseSelector);
 }
 
 static_assert(std::is_pod<PfmCountersInfo>::value,
               "We shouldn't have dynamic initialization here");
 const PfmCountersInfo PfmCountersInfo::Default = {nullptr, nullptr, nullptr,
                                                   0u};
 
 const PfmCountersInfo &ExegesisTarget::getPfmCounters(StringRef CpuName) const {

llvm/tools/llvm-exegesis/lib/UopsBenchmarkRunner.h

 
 #include "BenchmarkRunner.h"
 
 namespace llvm {
 namespace exegesis {
 
 class UopsBenchmarkRunner : public BenchmarkRunner {
 public:
-  UopsBenchmarkRunner(const LLVMState &State, bool BenchmarkSkipMeasurements)
+  UopsBenchmarkRunner(const LLVMState &State,
+                      BenchmarkPhaseSelectorE BenchmarkPhaseSelector)
       : BenchmarkRunner(State, InstructionBenchmark::Uops,
-                        BenchmarkSkipMeasurements) {}
+                        BenchmarkPhaseSelector) {}
   ~UopsBenchmarkRunner() override;
 
   static constexpr const size_t kMinNumDifferentAddresses = 6;
 
 private:
   Expected<std::vector<BenchmarkMeasure>>
   runMeasurements(const FunctionExecutor &Executor) const override;
 };
 
 } // namespace exegesis
 } // namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_UOPSBENCHMARKRUNNER_H

llvm/tools/llvm-exegesis/llvm-exegesis.cpp

cl::values(
                    "All of the above and take the minimum of measurements")),
     cl::init(exegesis::InstructionBenchmark::Duplicate));
 
 static cl::opt<bool> BenchmarkMeasurementsPrintProgress(
     "measurements-print-progress",
     cl::desc("Produce progress indicator when performing measurements"),
     cl::cat(BenchmarkOptions), cl::init(false));
 
-static cl::opt<bool> BenchmarkSkipMeasurements(
-    "skip-measurements",
-    cl::desc("do everything except actually performing the measurements"),
-    cl::cat(BenchmarkOptions), cl::init(false));
+static cl::opt<exegesis::BenchmarkPhaseSelectorE> BenchmarkPhaseSelector(
+    "benchmark-phase",
+    cl::desc(
+        "it is possible to stop the benchmarking process after some phase"),
+    cl::cat(BenchmarkOptions),
+    cl::values(
+        clEnumValN(exegesis::BenchmarkPhaseSelectorE::StopBeforeAllCodegen,
+                   "prepare-snippet",
+                   "Only generate the minimal instruction sequence"),
+        clEnumValN(exegesis::BenchmarkPhaseSelectorE::StopBeforeFullCodegen,
+                   "prepare-and-assemble-snippet",
+                   "Same as prepare-snippet, but also dumps an excerpt of the "
+                   "sequence (hex encoded)"),
+        clEnumValN(exegesis::BenchmarkPhaseSelectorE::StopBeforeMeasurements,
+                   "assemble-measured-code",
+                   "Same as prepare-and-assemble-snippet, but also creates the "
+                   "full sequence "
+                   "that can be dumped to a file using --dump-object-to-disk"),
+        clEnumValN(
+            exegesis::BenchmarkPhaseSelectorE::Measure, "measure",
+            "Same as prepare-measured-code, but also runs the measurement "
+            "(default)")),
+    cl::init(exegesis::BenchmarkPhaseSelectorE::Measure));
 
 static cl::opt<unsigned>
     NumRepetitions("num-repetitions",
                    cl::desc("number of time to repeat the asm snippet"),
                    cl::cat(BenchmarkOptions), cl::init(10000));
 
 static cl::opt<unsigned>
     LoopBodySize("loop-body-size",
if (RepetitionMode == InstructionBenchmark::RepetitionModeE::AggregateMin) {
       }
     }
 
     ExitOnFileError(BenchmarkFile, Result.writeYamlTo(State, Ostr));
   }
 }
 
 void benchmarkMain() {
-  if (!BenchmarkSkipMeasurements) {
+  if (BenchmarkPhaseSelector == BenchmarkPhaseSelectorE::Measure) {
 #ifndef HAVE_LIBPFM
     ExitWithError(
         "benchmarking unavailable, LLVM was built without libpfm. You can pass "
         "--skip-measurements to skip the actual benchmarking.");
 #else
     if (exegesis::pfm::pfmInitialize())
       ExitWithError("cannot initialize libpfm");
 #endif
   }
 
   InitializeAllAsmPrinters();
   InitializeAllAsmParsers();
   InitializeAllExegesisTargets();
 
   const LLVMState State = ExitOnErr(LLVMState::Create(TripleName, MCPU));
 
   // Preliminary check to ensure features needed for requested
   // benchmark mode are present on target CPU and/or OS.
-  if (!BenchmarkSkipMeasurements)
+  if (BenchmarkPhaseSelector == BenchmarkPhaseSelectorE::Measure)
     ExitOnErr(State.getExegesisTarget().checkFeatureSupport());
 
   const std::unique_ptr<BenchmarkRunner> Runner =
       ExitOnErr(State.getExegesisTarget().createBenchmarkRunner(
-          BenchmarkMode, State, BenchmarkSkipMeasurements, ResultAggMode));
+          BenchmarkMode, State, BenchmarkPhaseSelector, ResultAggMode));
   if (!Runner) {
     ExitWithError("cannot create benchmark runner");
   }
 
   const auto Opcodes = getOpcodesOrDie(State);
 
   SmallVector<std::unique_ptr<const SnippetRepetitor>, 2> Repetitors;
   if (RepetitionMode != InstructionBenchmark::RepetitionModeE::AggregateMin)