[llvm-exegesis][x86] Add option to prevent use of xmm8-xmm15 upper SSE registers

[RKSimon]

Summary

Noticed while trying to use llvm-exegesis to get some accurate capture numbers on some old Atom/Silverment hardware as part of the work with D103695.

These targets' frontends are particularly poor and the use of the xmm8-xmm15 SSE registers results in longer instruction encodings which were affecting the latency/throughput estimates.

I'm not sure how best to add test coverage for this - any suggestions?

[tschuett]

llvm/docs/CommandGuide/llvm-exegesis.rst

218

registers

[lebedev.ri]

I'm not sure how best to add test coverage for this - any suggestions?

--mode=inverse_throughput --opcode-name=ADDPSrr will try to make use of every register, so just check that the assembled snippet does not mention those regs.

[courbet]

You can test by adding a test in test/tools/llvm-exegesis/X86/uops/ with something like:

# RUN: llvm-exegesis -mode=uops -opcode-name=ADDPSrr -repetition-mode=duplicate | FileCheck --check_prefix=ALL_REGS %s
# RUN: llvm-exegesis -mode=uops -opcode-name=ADDPSrr -repetition-mode=duplicate -x86-disable-upper-sse-registers | FileCheck --check_prefix=LOW_REGS %s

ALL_REGS:   XMM15
LOW_REGS-NOT:   XMM15

[RKSimon]

Cheers - sorry for the delay - I've been away from any machines that had libPFM

[lebedev.ri]

llvm/tools/llvm-exegesis/lib/X86/Target.cpp

41–42

FYI this doesn't work. You probably want to expose the declaration of
the normal BenchmarkOptions in a header and use it,
not add a new group, because it isn't being displayed.
Maybe other tools have some other example on how to handle this.

[RKSimon]

llvm/tools/llvm-exegesis/lib/X86/Target.cpp

41–42

Cheers, I'll take a look as a follow up - you've probably guessed I was just doing an copy+paste of the lbr option :)

[lebedev.ri]

llvm/tools/llvm-exegesis/lib/X86/Target.cpp

41–42

Yes.

llvm/docs/CommandGuide/llvm-exegesis.rst

.. option:: -x86-lbr-sample-period=<nBranches/sample>
 
   Specify the LBR sampling period - how many branches before we take a sample.
   When a positive value is specified for this option and when the mode is `latency`,
   we will use LBRs for measuring.
   On choosing the "right" sampling period, a small value is preferred, but throttling
   could occur if the sampling is too frequent. A prime number should be used to
   avoid consistently skipping certain blocks.
 
+.. option:: -x86-disable-upper-sse-registers
+
+  Using the upper xmm registers (xmm8-xmm15) forces a longer instruction encoding
+  which may put greater pressure on the frontend fetch and decode stages,
+  potentially reducing the rate that instructions are dispatched to the backend,
+  particularly on older hardware. Comparing baseline results with this mode
+  enabled can help determine the effects of the frontend and can be used to
+  improve latency and throughput estimates.
+
 .. option:: -repetition-mode=[duplicate|loop|min]
 
  Specify the repetition mode. `duplicate` will create a large, straight line
  basic block with `num-repetitions` instructions (repeating the snippet
  `num-repetitions`/`snippet size` times). `loop` will, optionally, duplicate the
  snippet until the loop body contains at least `loop-body-size` instructions,
  and then wrap the result in a loop which will execute `num-repetitions`
  instructions (thus, again, repeating the snippet

llvm/test/tools/llvm-exegesis/X86/inverse_throughput/inverse_throughput-disable-upper-sse-registers.s

+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -mode=inverse_throughput --skip-measurements -x86-disable-upper-sse-registers -opcode-name=ADDPSrr -repetition-mode=loop | FileCheck %s
+
+CHECK:      ---
+CHECK-NEXT: mode:            inverse_throughput
+CHECK-NEXT: key:
+CHECK-NEXT:   instructions:
+CHECK-NEXT:     - 'ADDPSrr [[LHS0:XMM[0-7]]] [[LHS0]] [[RHS0:XMM[0-7]]]'
+CHECK-NEXT:     - 'ADDPSrr [[LHS1:XMM[0-7]]] [[LHS1]] [[RHS1:XMM[0-7]]]'
+CHECK-NEXT:     - 'ADDPSrr [[LHS2:XMM[0-7]]] [[LHS2]] [[RHS2:XMM[0-7]]]'
+CHECK-NEXT:     - 'ADDPSrr [[LHS3:XMM[0-7]]] [[LHS3]] [[RHS3:XMM[0-7]]]'

llvm/tools/llvm-exegesis/lib/X86/Target.cpp

 #endif
 #if defined(__x86_64__) && defined(_MSC_VER)
 #include <float.h> // For _clearfp in ~X86SavedState().
 #endif
 
 namespace llvm {
 namespace exegesis {
 
 static cl::OptionCategory
     BenchmarkOptions("llvm-exegesis benchmark x86-options");
 
 // If a positive value is specified, we are going to use the LBR in
 // latency-mode.
 //
 // Note:
 //  -  A small value is preferred, but too low a value could result in
 //     throttling.
 //  -  A prime number is preferred to avoid always skipping certain blocks.
 //
 static cl::opt<unsigned> LbrSamplingPeriod(
     "x86-lbr-sample-period",
     cl::desc("The sample period (nbranches/sample), used for LBR sampling"),
     cl::cat(BenchmarkOptions), cl::init(0));
 
+static cl::opt<bool>
+    DisableUpperSSERegisters("x86-disable-upper-sse-registers",
+                             cl::desc("Disable XMM8-XMM15 register usage"),
+                             cl::cat(BenchmarkOptions), cl::init(false));
+
 // FIXME: Validates that repetition-mode is loop if LBR is requested.
 
 // Returns a non-null reason if we cannot handle the memory references in this
 // instruction.
 static const char *isInvalidMemoryInstr(const Instruction &Instr) {
   switch (Instr.Description.TSFlags & X86II::FormMask) {
   default:
     return "Unknown FormMask value";
private:
   void decrementLoopCounterAndJump(MachineBasicBlock &MBB,
                                    MachineBasicBlock &TargetMBB,
                                    const MCInstrInfo &MII) const override;
 
   std::vector<MCInst> setRegTo(const MCSubtargetInfo &STI, unsigned Reg,
                                const APInt &Value) const override;
 
   ArrayRef<unsigned> getUnavailableRegisters() const override {
+    if (DisableUpperSSERegisters)
+      return makeArrayRef(kUnavailableRegistersSSE,
+                          sizeof(kUnavailableRegistersSSE) /
+                              sizeof(kUnavailableRegistersSSE[0]));
+
     return makeArrayRef(kUnavailableRegisters,
                         std::size(kUnavailableRegisters));
   }
 
   bool allowAsBackToBack(const Instruction &Instr) const override {
     const unsigned Opcode = Instr.Description.Opcode;
     return !isInvalidOpcode(Instr) && Opcode != X86::LEA64r &&
            Opcode != X86::LEA64_32r && Opcode != X86::LEA16r;
return llvm::make_error<llvm::StringError>(
         llvm::errc::not_supported);
   }
 
   std::unique_ptr<SavedState> withSavedState() const override {
     return std::make_unique<X86SavedState>();
   }
 
   static const unsigned kUnavailableRegisters[4];
+  static const unsigned kUnavailableRegistersSSE[12];
 };
 
 // We disable a few registers that cannot be encoded on instructions with a REX
 // prefix.
 const unsigned ExegesisX86Target::kUnavailableRegisters[4] = {X86::AH, X86::BH,
                                                               X86::CH, X86::DH};
 
+// Optionally, also disable the upper (x86_64) SSE registers to reduce frontend
+// decoder load.
+const unsigned ExegesisX86Target::kUnavailableRegistersSSE[12] = {
+    X86::AH,    X86::BH,    X86::CH,    X86::DH,    X86::XMM8,  X86::XMM9,
+    X86::XMM10, X86::XMM11, X86::XMM12, X86::XMM13, X86::XMM14, X86::XMM15};
+
 // We're using one of R8-R15 because these registers are never hardcoded in
 // instructions (e.g. MOVS writes to EDI, ESI, EDX), so they have less
 // conflicts.
 constexpr const unsigned kLoopCounterReg = X86::R8;
 
 } // namespace
 
 void ExegesisX86Target::addTargetSpecificPasses(PassManagerBase &PM) const {