[llvm-exegesis] Use MCJIT only for execution

[kpdev42]

Summary

Initially, llvm-exegesis was generating the benchmark code for the
host CPU to execute it inside its own process. Thus, MCJIT was reused
for fetching function's bytes to fill the assembled_snippet field in
the benchmark report.

Later, the --mtriple and --benchmark-phase command line options were
introduced that are handy for testing snippet generation even if
snippet execution is not possible. In that setup, MCJIT is asked to
parse an object file for a foreign CPU or operating system that is
probably not guaranteed to succeed and was actually observed to fail
in https://reviews.llvm.org/D145763.

This commit implements a much simplified function's code fetching,
assuming the benchmark function is the only function in the object file
and it spans across the entire text section (note that MCJIT-based code
has more or less the same assumption - see TrackingSectionMemoryManager
class).

~~~

Huawei RRI, OS Lab

[courbet]

llvm/test/tools/llvm-exegesis/PowerPC/lit.local.cfg

4–6 ↗

(On Diff #515705)

Maybe do this in a separate patch in case it has to be reverted ? :)

llvm/tools/llvm-exegesis/lib/Assembler.cpp

345

[nit] Propagate the error ?

348

I'm not very familiar with the section API, is this the relative address of the symbol within the section or an absolute address ? If the latter should this be compared to SectionRef::getAddress() rather than 0 ?

[kpdev42]

Address review comments

[kpdev42]

llvm/tools/llvm-exegesis/lib/Assembler.cpp

348

You are most probably right, because these addresses are subtracted in https://github.com/llvm/llvm-project/blob/main/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp#L177

[courbet]

llvm/tools/llvm-exegesis/lib/Assembler.cpp

345

ping

llvm/test/tools/llvm-exegesis/AArch64/print-assembled-snippet.s

+# Check that "assembled_snippet" is fetched correctly from object file.
+# Feel free to update the snippet in this test if the code generated by the
+# snippet repetitor was changed intentionally and it is still fetched correctly.
+
+# RUN: llvm-exegesis --mtriple=aarch64-linux-gnu --mcpu=generic --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-LINUX
+
+# RUN: llvm-exegesis --mtriple=aarch64-windows-gnu --mcpu=generic --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-WINDOWS-GNU
+
+# RUN: llvm-exegesis --mtriple=aarch64-windows-msvc --mcpu=generic --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-WINDOWS-MSVC
+
+# LLVM-EXEGESIS-DEFREG X0 0
+# LLVM-EXEGESIS-DEFREG X1 0
+add x0, x0, x1
+
+# CHECK-LINUX: cpu_name:          generic
+# CHECK-LINUX: llvm_triple:       aarch64-unknown-linux-gnu
+# CHECK-LINUX: assembled_snippet: 000080D2010080D20000018B0000018B0000018B0000018BC0035FD6{{$}}
+
+# CHECK-WINDOWS-GNU: cpu_name:          generic
+# CHECK-WINDOWS-GNU: llvm_triple:       aarch64-unknown-windows-gnu
+# CHECK-WINDOWS-GNU: assembled_snippet: 000080D2010080D20000018B0000018B0000018B0000018BC0035FD6{{$}}
+
+# CHECK-WINDOWS-MSVC: cpu_name:          generic
+# CHECK-WINDOWS-MSVC: llvm_triple:       aarch64-unknown-windows-msvc
+# CHECK-WINDOWS-MSVC: assembled_snippet: 000080D2010080D20000018B0000018B0000018B0000018BC0035FD6{{$}}

llvm/test/tools/llvm-exegesis/Mips/print-assembled-snippet.s

+# Check that "assembled_snippet" is fetched correctly from object file.
+# Feel free to update the snippet in this test if the code generated by the
+# snippet repetitor was changed intentionally and it is still fetched correctly.
+
+# RUN: llvm-exegesis --mtriple=mips-linux-gnu --mcpu=generic --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-LINUX
+
+# RUN: llvm-exegesis --mtriple=mips64-linux-gnu --mcpu=generic --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-LINUX-64
+
+# LLVM-EXEGESIS-DEFREG A0 0
+# LLVM-EXEGESIS-DEFREG A1 0
+add $2, $5, $4
+
+# CHECK-LINUX: cpu_name:             generic
+# CHECK-LINUX: llvm_triple:          mips-unknown-linux-gnu
+# CHECK-LINUX: assembled_snippet:    340400003405000000A4102000A4102000A4102000A4102003E00008{{$}}
+
+# CHECK-LINUX-64: cpu_name:           generic
+# CHECK-LINUX-64: llvm_triple:        mips64-unknown-linux-gnu
+# CHECK-LINUX-64: assembled_snippet:  340400003405000000A4102000A4102000A4102000A4102003E00008{{$}}

llvm/test/tools/llvm-exegesis/PowerPC/print-assembled-snippet.s

+# Check that "assembled_snippet" is fetched correctly from object file.
+# Feel free to update the snippet in this test if the code generated by the
+# snippet repetitor was changed intentionally and it is still fetched correctly.
+
+# RUN: llvm-exegesis --mtriple=powerpc64le-linux-gnu --mcpu=pwr8 --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-LINUX-64-LE
+
+# LLVM-EXEGESIS-DEFREG R3 0
+# LLVM-EXEGESIS-DEFREG R4 0
+add 3, 4, 3
+
+# CHECK-LINUX-64-LE: cpu_name:           pwr8
+# CHECK-LINUX-64-LE: llvm_triple:        powerpc64le-unknown-linux-gnu
+# CHECK-LINUX-64-LE: assembled_snippet:  0000603800008038141A647C141A647C141A647C141A647C2000804E000000000000000000000000{{$}}

llvm/test/tools/llvm-exegesis/X86/print-assembled-snippet.s

+# Check that "assembled_snippet" is fetched correctly from object file.
+# Feel free to update the snippet in this test if the code generated by the
+# snippet repetitor was changed intentionally and it is still fetched correctly.
+
+# RUN: llvm-exegesis --mtriple=x86_64-linux-gnu --mcpu=x86-64 --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-LINUX
+
+# RUN: llvm-exegesis --mtriple=x86_64-windows-gnu --mcpu=x86-64 --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-WINDOWS-GNU
+
+# RUN: llvm-exegesis --mtriple=x86_64-windows-msvc --mcpu=x86-64 --benchmark-phase=prepare-and-assemble-snippet \
+# RUN:               --snippets-file=%s --mode=latency --repetition-mode=duplicate \
+# RUN: | FileCheck %s --check-prefix=CHECK-WINDOWS-MSVC
+
+# LLVM-EXEGESIS-DEFREG RDI 0
+# LLVM-EXEGESIS-DEFREG RSI 0
+# LLVM-EXEGESIS-DEFREG RAX 0
+leaq    (%rdi,%rsi), %rax
+
+# CHECK-LINUX: cpu_name:          x86-64
+# CHECK-LINUX: llvm_triple:       x86_64-unknown-linux-gnu
+# CHECK-LINUX: assembled_snippet: 48BF000000000000000048BE000000000000000048B80000000000000000488D0437488D0437488D0437488D0437C3{{$}}
+
+# CHECK-WINDOWS-GNU: cpu_name:          x86-64
+# CHECK-WINDOWS-GNU: llvm_triple:       x86_64-unknown-windows-gnu
+# CHECK-WINDOWS-GNU: assembled_snippet: 565748BF000000000000000048BE000000000000000048B80000000000000000488D0437488D0437488D0437488D04375F5EC3{{$}}
+
+# CHECK-WINDOWS-MSVC: cpu_name:          x86-64
+# CHECK-WINDOWS-MSVC: llvm_triple:       x86_64-unknown-windows-msvc
+# CHECK-WINDOWS-MSVC: assembled_snippet: 565748BF000000000000000048BE000000000000000048B80000000000000000488D0437488D0437488D0437488D04375F5EC3{{$}}

llvm/tools/llvm-exegesis/lib/Assembler.h

void operator()(char *Memory) const {
     ((void (*)(char *))(intptr_t)FunctionBytes.data())(Memory);
   }
 
   std::unique_ptr<LLVMContext> Context;
   std::unique_ptr<ExecutionEngine> ExecEngine;
   StringRef FunctionBytes;
 };
 
+// Copies benchmark function's bytes from benchmark object.
+Error getBenchmarkFunctionBytes(const StringRef InputData,
+                                std::vector<uint8_t> &Bytes);
+
 // Creates a void(int8*) MachineFunction.
 MachineFunction &createVoidVoidPtrMachineFunction(StringRef FunctionID,
                                                   Module *Module,
                                                   MachineModuleInfo *MMI);
 
 } // namespace exegesis
 } // namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_ASSEMBLER_H

llvm/tools/llvm-exegesis/lib/Assembler.cpp

 #include "llvm/CodeGen/TargetInstrInfo.h"
 #include "llvm/CodeGen/TargetPassConfig.h"
 #include "llvm/CodeGen/TargetSubtargetInfo.h"
 #include "llvm/ExecutionEngine/SectionMemoryManager.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/Support/Alignment.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/raw_ostream.h"
 
 namespace llvm {
 namespace exegesis {
 
 static constexpr const char ModuleID[] = "ExegesisInfoTest";
 static constexpr const char FunctionID[] = "foo";
 static const Align kFunctionAlignment(4096);
 
ExecutableFunction::ExecutableFunction(
   // Fetching function bytes.
   const uint64_t FunctionAddress = ExecEngine->getFunctionAddress(FunctionID);
   assert(isAligned(kFunctionAlignment, FunctionAddress) &&
          "function is not properly aligned");
   FunctionBytes =
       StringRef(reinterpret_cast<const char *>(FunctionAddress), CodeSize);
 }
 
+Error getBenchmarkFunctionBytes(const StringRef InputData,
+                                std::vector<uint8_t> &Bytes) {
+  const auto Holder = getObjectFromBuffer(InputData);
+  const auto *Obj = Holder.getBinary();
+  // See RuntimeDyldImpl::loadObjectImpl(Obj) for much more complete
+  // implementation.
+
+  // Find the only function in the object file.
+  SmallVector<object::SymbolRef, 1> Functions;
+  for (auto &Sym : Obj->symbols()) {
+    auto SymType = Sym.getType();
+    if (SymType && *SymType == object::SymbolRef::Type::ST_Function)
+      Functions.push_back(Sym);
+  }
+  if (Functions.size() != 1)
+    return make_error<Failure>("Exactly one function expected");
+
+  // Find the containing section - it is assumed to contain only this function.
+  auto SectionOrErr = Functions.front().getSection();
+  if (!SectionOrErr || *SectionOrErr == Obj->section_end())
+    return make_error<Failure>("Section not found");
+
+  auto Address = Functions.front().getAddress();
+  if (!Address || *Address != SectionOrErr.get()->getAddress())
+    return make_error<Failure>("Unexpected layout");
+
+  auto ContentsOrErr = SectionOrErr.get()->getContents();
+  if (!ContentsOrErr)
+    return ContentsOrErr.takeError();
+  Bytes.assign(ContentsOrErr->begin(), ContentsOrErr->end());
+  return Error::success();
+}
+
 } // namespace exegesis
 } // namespace llvm

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp

BenchmarkRunner::getRunnableConfiguration(
   // understands that the inside instructions are repeated.
   if (BenchmarkPhaseSelector > BenchmarkPhaseSelectorE::PrepareSnippet) {
     const int MinInstructionsForSnippet = 4 * Instructions.size();
     const int LoopBodySizeForSnippet = 2 * Instructions.size();
     auto Snippet = assembleSnippet(BC, Repetitor, MinInstructionsForSnippet,
                                    LoopBodySizeForSnippet);
     if (Error E = Snippet.takeError())
       return std::move(E);
-    const ExecutableFunction EF(State.createTargetMachine(),
-                                getObjectFromBuffer(*Snippet));
-    const auto FnBytes = EF.getFunctionBytes();
-    llvm::append_range(InstrBenchmark.AssembledSnippet, FnBytes);
+
+    if (auto Err = getBenchmarkFunctionBytes(*Snippet,
+                                             InstrBenchmark.AssembledSnippet))
+      return Err;
   }
 
   // Assemble NumRepetitions instructions repetitions of the snippet for
   // measurements.
   if (BenchmarkPhaseSelector > BenchmarkPhaseSelectorE::PrepareAndAssembleSnippet) {
     auto Snippet = assembleSnippet(BC, Repetitor, InstrBenchmark.NumRepetitions,
                                    LoopBodySize);
     if (Error E = Snippet.takeError())