[llvm-exegesis] Introduce Subprocess Executor Mode

[aidengrossman]

Summary

This patch introduces the subprocess executor mode. Currently, this new
mode doesn't do anything fancy, just executing the same code that the
inprocess executor would do, but within a subprocess. This sets up the
ability to add in many more memory-related features in the future.

[aidengrossman]

Rebase

[courbet]

llvm/test/tools/llvm-exegesis/X86/latency/subprocess-segfault.s

8

is this guaranteed to segfault ? maybe use 0 ?

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp

145

Please add a simple class description.

189

Add a comment here: // Note: terminates the child process..

222

[nit] create a temporary variable for this to avoid repetition below ? These macros are not exactly self-explanatory.

247

[[noreturn]] ?

278

[nit] createSubProcessAndRunBenchmark for consistency ?

[aidengrossman]

Address reviewer feedback.

[courbet]

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp

148

with

[aidengrossman]

Format

[aidengrossman]

Rebase, address reviewer feedback.

[aidengrossman]

Add preprocessor directives around libpfm specific defines.

[aidengrossman]

Fix build failure on Debian stable due to not having the pidfd_getfd system call.

[aidengrossman]

Rebase

[Hahnfeld]

(FWIW this was re-committed in rG4d618b52f6e05e41d35f56653cb36bf7d4dc794e, without mentioning this Differential Revision...)

@aidengrossman AFAICT this requires Linux 5.6 to pass tests on x86. Can you please reconsider this change without prior notice? Some long-term distros don't have this yet (I care about EL8, which has Linux 4.18)

[aidengrossman]

Sorry about the missing differential revision. arc patch refused to work (of course) and I forgot to add the differential revision to the commit message in the recommit.

I'll work on a patch to get tests passing on systems with a pre-5.6 kernel. I don't think any of the buildbots have libpfm/perf counters together so none of these tests end up executing in the main CI infrastructure (maybe something we should fix?). I might work on making this more kernel version agnostic at some point, but currently I believe we're the only downstream consumer of this functionality within an already somewhat esoteric tool.

[aidengrossman]

@Hahnfeld I just recommitted this with an additional lit check to ensure that the tests don't run in environments where they're not supported so you shouldn't observe any additional test failures. Please let me know if you do and I'll work on getting them fixed.

For what it's worth, I'm also probably going to write a follow up patch at some point to get rid of the usage of these two system calls as I found out recently that my new place of employment has most of its HPC infrastructure on RHEL8 which is Kernel 4.18 so most of the platform specific test running should be gone at some point.

[Hahnfeld]

In D151021#4447595, @aidengrossman wrote:

@Hahnfeld I just recommitted this with an additional lit check to ensure that the tests don't run in environments where they're not supported so you shouldn't observe any additional test failures. Please let me know if you do and I'll work on getting them fixed.

Thanks, works perfectly fine now!

For what it's worth, I'm also probably going to write a follow up patch at some point to get rid of the usage of these two system calls as I found out recently that my new place of employment has most of its HPC infrastructure on RHEL8 which is Kernel 4.18 so most of the platform specific test running should be gone at some point.

Yes, RHEL8 and its clones have that older kernel, long-term support also always means that they are moving slower...

[zixuan-wu]

I met an issue that ptrace.h missing PTRACE_SEIZE enum value. Anybody know the reason?

[abidh]

In D151021#4467846, @zixuan-wu wrote:

I met an issue that ptrace.h missing PTRACE_SEIZE enum value. Anybody know the reason?

We have seen this error too. In our case, it was due to using an old sysroot. The PTRACE_SEIZE was introduced in Linux 3.4.

[aidengrossman]

Sorry for the late response. As @abidh mentions, PTRACE_SEIZE was introduced in Linux 3.4. Given how old 3.4 is, I don't think it is too common of a configuration, but if it is a significant blocker to someone I can put up a patch to get things compiling. I probably need to upstream my debugging flag anyways.

[zixuan-wu]

In case such requirement of linux kernel version, should we put the linux kernel requirement in the hardware requirement section of get started page? https://llvm.org/docs/GettingStarted.html#hardware

@chandlerc

[danilaml]

Having the issues with PTRACE_SEIZE on CentOS 6 (will probably need to build LLVM where for around a year). Don't use llvm-exegesis there, but it's hard to cleanly disable it from build, since tests depend on it.

[abidh]

In D151021#4474519, @aidengrossman wrote:

Sorry for the late response. As @abidh mentions, PTRACE_SEIZE was introduced in Linux 3.4. Given how old 3.4 is, I don't think it is too common of a configuration, but if it is a significant blocker to someone I can put up a patch to get things compiling. I probably need to upstream my debugging flag anyways.

That will be greatly appreciated. It is a blocker for us and it is not easy to cleanly disable llvm-exegesis from the build as @danilaml also pointed out.

[aidengrossman]

75b5541fe5d096d115d213cb8dc8a10cf3d11fbd should fix the issue as PTRACE_ATTACH was introduced a lot earlier from what I understand. Let me know if there are any other issues.

llvm/test/tools/llvm-exegesis/X86/latency/subprocess-abnormal-exit-code.s

+# REQUIRES: exegesis-can-execute-in-subprocess, x86_64-linux
+
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mode=latency -snippets-file=%s -execution-mode=subprocess | FileCheck %s
+
+# CHECK: error: 'Child benchmarking process exited with non-zero exit code: Child process returned with unknown exit code'
+
+movl $60, %eax
+movl $127, %edi
+syscall

llvm/test/tools/llvm-exegesis/X86/latency/subprocess-segfault.s

+# REQUIRES: exegesis-can-execute-in-subprocess, x86_64-linux
+
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mode=latency -snippets-file=%s -execution-mode=subprocess | FileCheck %s
+
+# CHECK: error:           'The benchmarking subprocess sent unexpected signal: Segmentation fault'
+
+# LLVM-EXEGESIS-DEFREG RBX 0
+movq (%rbx), %rax

llvm/test/tools/llvm-exegesis/X86/latency/subprocess.s

+# REQUIRES: exegesis-can-execute-in-subprocess, x86_64-linux
+
+# RUN: llvm-exegesis -mtriple=x86_64-unknown-unknown -mode=latency -snippets-file=%s -execution-mode=subprocess | FileCheck %s
+
+# CHECK: measurements:
+# CHECK-NEXT: value: {{.*}}, per_snippet_value: {{.*}}
+
+# LLVM-EXEGESIS-DEFREG XMM1 42
+# LLVM-EXEGESIS-DEFREG XMM2 42
+# LLVM-EXEGESIS-DEFREG XMM3 42
+vhaddps       %xmm2, %xmm2, %xmm3

llvm/test/tools/llvm-exegesis/lit.local.cfg

 import subprocess
+import re
 import lit.util
 
 
 def can_execute_generated_snippets(arch):
     is_host_arch = arch in config.root.host_triple
     # 'native' feature is defined as "host arch == default triple arch"
     is_native_codegen = "native" in config.available_features
     return is_host_arch and is_native_codegen
try:
             stderr=subprocess.DEVNULL,
         )
         return return_code == 0
     except OSError:
         print("could not exec llvm-exegesis")
         return False
 
 
+def can_execute_in_subprocess():
+    # We need certain Linux system calls present in order to run the subprocess
+    # executor mode, so check that we can use the subprocess mode to prevent
+    # test failures on platforms running older kernels.
+    llvm_exegesis_exe = lit.util.which("llvm-exegesis", config.llvm_tools_dir)
+    if llvm_exegesis_exe is None:
+        print("could not find llvm-exegesis")
+        return False
+    try:
+        command_vector = [
+            llvm_exegesis_exe,
+            "-mode=latency",
+            "--execution-mode=subprocess",
+            "-snippets-file=/dev/null",
+        ]
+        with subprocess.Popen(
+            command_vector, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL
+        ) as exegesis_subprocess:
+            stdout, stderr = exegesis_subprocess.communicate()
+            exegesis_output = stdout.decode("utf-8")
+            # Return true if we have an empty error section as llvm-exegesis
+            # doesn't change the return code if there's a snippet crash.
+            return re.search("error:\s*''", exegesis_output) is not None
+
+    except OSError:
+        print("Could not execute llvm-exegesis in subprocess executor mode")
+        return False
+
+
 for arch in ["aarch64", "mips", "powerpc", "x86_64"]:
     if can_execute_generated_snippets(arch):
         config.available_features.add("exegesis-can-execute-%s" % arch)
 
 if can_use_perf_counters("latency"):
     config.available_features.add("exegesis-can-measure-latency")
 
 if can_use_perf_counters("uops"):
     config.available_features.add("exegesis-can-measure-uops")
 
 if can_execute_generated_snippets("x86_64"):
     # Check for support of LBR format with cycles.
     if can_use_perf_counters(
         "latency", ["-x86-lbr-sample-period", "123", "-repetition-mode", "loop"]
     ):
         config.available_features.add("exegesis-can-measure-latency-lbr")
+
+if can_execute_in_subprocess():
+    print("Passed thing")
+    config.available_features.add("exegesis-can-execute-in-subprocess")

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h

 #include <vector>
 
 namespace llvm {
 namespace exegesis {
 
 // Common code for all benchmark modes.
 class BenchmarkRunner {
 public:
-  enum ExecutionModeE { InProcess };
+  enum ExecutionModeE { InProcess, SubProcess };
 
   explicit BenchmarkRunner(const LLVMState &State, Benchmark::ModeE Mode,
                            BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
                            ExecutionModeE ExecutionMode);
 
   virtual ~BenchmarkRunner();
 
   class RunnableConfiguration {

llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp

 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Support/CrashRecoveryContext.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Program.h"
+#include "llvm/Support/Signals.h"
+
+#ifdef __linux__
+#ifdef HAVE_LIBPFM
+#include <perfmon/perf_event.h>
+#endif
+#include <sys/mman.h>
+#include <sys/ptrace.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#endif // __linux__
 
 namespace llvm {
 namespace exegesis {
 
 BenchmarkRunner::BenchmarkRunner(const LLVMState &State, Benchmark::ModeE Mode,
                                  BenchmarkPhaseSelectorE BenchmarkPhaseSelector,
                                  ExecutionModeE ExecutionMode)
     : State(State), Mode(Mode), BenchmarkPhaseSelector(BenchmarkPhaseSelector),
#endif
 
     return Counter->readOrError(Function.getFunctionBytes());
   }
 
   const LLVMState &State;
   const ExecutableFunction Function;
   BenchmarkRunner::ScratchSpace *const Scratch;
 };
+
+#ifdef __linux__
+// The following class implements a function executor that executes the
+// benchmark code within a subprocess rather than within the main llvm-exegesis
+// process. This allows for much more control over the execution context of the
+// snippet, particularly with regard to memory. This class performs all the
+// necessary functions to create the subprocess, execute the snippet in the
+// subprocess, and report results/handle errors.
+class SubProcessFunctionExecutorImpl
+    : public BenchmarkRunner::FunctionExecutor {
+public:
+  SubProcessFunctionExecutorImpl(const LLVMState &State,
+                                 object::OwningBinary<object::ObjectFile> Obj,
+                                 const BenchmarkKey &Key)
+      : State(State), Function(State.createTargetMachine(), std::move(Obj)),
+        Key(Key) {}
+
+private:
+  enum ChildProcessExitCodeE {
+    CounterFDReadFailed = 1,
+    TranslatingCounterFDFailed
+  };
+
+  StringRef childProcessExitCodeToString(int ExitCode) const {
+    switch (ExitCode) {
+    case ChildProcessExitCodeE::CounterFDReadFailed:
+      return "Counter file descriptor read failed";
+    case ChildProcessExitCodeE::TranslatingCounterFDFailed:
+      return "Translating counter file descriptor into a file descriptor in "
+             "the child process failed. This might be due running an older "
+             "Linux kernel that doesn't support the pidfd_getfd system call "
+             "(anything before Linux 5.6).";
+    default:
+      return "Child process returned with unknown exit code";
+    }
+  }
+
+  Error createSubProcessAndRunBenchmark(
+      StringRef CounterName, SmallVectorImpl<int64_t> &CounterValues) const {
+    int PipeFiles[2];
+    int PipeSuccessOrErr = pipe(PipeFiles);
+    if (PipeSuccessOrErr != 0) {
+      return make_error<Failure>(
+          "Failed to create a pipe for interprocess communication between "
+          "llvm-exegesis and the benchmarking subprocess");
+    }
+
+    pid_t ParentOrChildPID = fork();
+    if (ParentOrChildPID == 0) {
+      // We are in the child process, close the write end of the pipe
+      close(PipeFiles[1]);
+      // Unregister handlers, signal handling is now handled through ptrace in
+      // the host process
+      llvm::sys::unregisterHandlers();
+      prepareAndRunBenchmark(PipeFiles[0], Key);
+      // The child process terminates in the above function, so we should never
+      // get to this point.
+      llvm_unreachable("Child process didn't exit when expected.");
+    }
+
+    const ExegesisTarget &ET = State.getExegesisTarget();
+    auto CounterOrError =
+        ET.createCounter(CounterName, State, ParentOrChildPID);
+
+    if (!CounterOrError)
+      return CounterOrError.takeError();
+
+    pfm::Counter *Counter = CounterOrError.get().get();
+
+    close(PipeFiles[0]);
+
+    int CounterFileDescriptor = Counter->getFileDescriptor();
+    ssize_t BytesWritten =
+        write(PipeFiles[1], &CounterFileDescriptor, sizeof(int));
+
+    if (BytesWritten != sizeof(int))
+      return make_error<Failure>("Writing peformance counter file descriptor "
+                                 "to child process failed: " +
+                                 Twine(strerror(errno)));
+
+    if (ptrace(PTRACE_SEIZE, ParentOrChildPID, NULL, NULL) != 0)
+      return make_error<Failure>("Failed to seize the child process: " +
+                                 Twine(strerror(errno)));
+
+    int ChildStatus;
+    if (wait(&ChildStatus) == -1) {
+      return make_error<Failure>(
+          "Waiting for the child process to complete failed: " +
+          Twine(strerror(errno)));
+    }
+
+    if (WIFEXITED(ChildStatus)) {
+      int ChildExitCode = WEXITSTATUS(ChildStatus);
+      if (ChildExitCode == 0) {
+        // The child exited succesfully, read counter values and return
+        // success
+        CounterValues[0] = Counter->read();
+        return Error::success();
+      }
+      // The child exited, but not successfully
+      return make_error<SnippetCrash>(
+          "Child benchmarking process exited with non-zero exit code: " +
+          childProcessExitCodeToString(ChildExitCode));
+    }
+
+    // An error was encountered running the snippet, process it
+    siginfo_t ChildSignalInfo;
+    if (ptrace(PTRACE_GETSIGINFO, ParentOrChildPID, NULL, &ChildSignalInfo) ==
+        -1) {
+      return make_error<Failure>("Getting signal info from the child failed: " +
+                                 Twine(strerror(errno)));
+    }
+
+    return make_error<SnippetCrash>(
+        "The benchmarking subprocess sent unexpected signal: " +
+        Twine(strsignal(ChildSignalInfo.si_signo)));
+  }
+
+  [[noreturn]] void prepareAndRunBenchmark(int Pipe,
+                                           const BenchmarkKey &Key) const {
+    // The following occurs within the benchmarking subprocess
+
+    int ParentCounterFileDescriptor = -1;
+    ssize_t BytesRead = read(Pipe, &ParentCounterFileDescriptor, sizeof(int));
+
+    if (BytesRead != sizeof(int)) {
+      exit(ChildProcessExitCodeE::CounterFDReadFailed);
+    }
+
+    // Make sure the following two syscalls are defined on the platform that
+    // we're building on as they were introduced to the kernel fairly recently
+    // (v5.6 for the second one).
+#if defined SYS_pidfd_open && defined SYS_pidfd_getfd
+    pid_t ParentPID = getppid();
+
+    int ParentPIDFD = syscall(SYS_pidfd_open, ParentPID, 0);
+    int CounterFileDescriptor =
+        syscall(SYS_pidfd_getfd, ParentPIDFD, ParentCounterFileDescriptor, 0);
+#else
+    int CounterFileDescriptor = 0;
+    exit(ChildProcessExitCodeE::TranslatingCounterFDFailed);
+#endif
+
+    if (CounterFileDescriptor == -1) {
+      exit(ChildProcessExitCodeE::TranslatingCounterFDFailed);
+    }
+
+#ifdef HAVE_LIBPFM
+    ioctl(CounterFileDescriptor, PERF_EVENT_IOC_RESET);
+#endif
+    this->Function(nullptr);
+#ifdef HAVE_LIBPFM
+    ioctl(CounterFileDescriptor, PERF_EVENT_IOC_DISABLE);
+#endif
+
+    exit(0);
+  }
+
+  Expected<llvm::SmallVector<int64_t, 4>>
+  runWithCounter(StringRef CounterName) const override {
+    SmallVector<int64_t, 4> Value(1, 0);
+    Error PossibleBenchmarkError =
+        createSubProcessAndRunBenchmark(CounterName, Value);
+
+    if (PossibleBenchmarkError) {
+      return std::move(PossibleBenchmarkError);
+    }
+
+    return Value;
+  }
+
+  const LLVMState &State;
+  const ExecutableFunction Function;
+  const BenchmarkKey &Key;
+};
+#endif // __linux__
 } // namespace
 
 Expected<SmallString<0>> BenchmarkRunner::assembleSnippet(
     const BenchmarkCode &BC, const SnippetRepetitor &Repetitor,
     unsigned MinInstructions, unsigned LoopBodySize) const {
   const std::vector<MCInst> &Instructions = BC.Key.Instructions;
   SmallString<0> Buffer;
   raw_svector_ostream OS(Buffer);

 Expected<std::unique_ptr<BenchmarkRunner::FunctionExecutor>>
 BenchmarkRunner::createFunctionExecutor(
     object::OwningBinary<object::ObjectFile> ObjectFile,
     const BenchmarkKey &Key) const {
   switch (ExecutionMode) {
   case ExecutionModeE::InProcess:
     return std::make_unique<InProcessFunctionExecutorImpl>(
         State, std::move(ObjectFile), Scratch.get());
+  case ExecutionModeE::SubProcess:
+#ifdef __linux__
+    return std::make_unique<SubProcessFunctionExecutorImpl>(
+        State, std::move(ObjectFile), Key);
+#else
+    return make_error<Failure>(
+        "The subprocess execution mode is only supported on Linux");
+#endif
   }
   llvm_unreachable("ExecutionMode is outside expected range");
 }
 
 Expected<Benchmark> BenchmarkRunner::runConfiguration(
     RunnableConfiguration &&RC,
     const std::optional<StringRef> &DumpFile) const {
   Benchmark &InstrBenchmark = RC.InstrBenchmark;

llvm/tools/llvm-exegesis/llvm-exegesis.cpp

DumpObjectToDisk("dump-object-to-disk",
                      cl::ValueOptional, cl::cat(BenchmarkOptions));
 
 static cl::opt<BenchmarkRunner::ExecutionModeE> ExecutionMode(
     "execution-mode",
     cl::desc("Selects the execution mode to use for running snippets"),
     cl::cat(BenchmarkOptions),
     cl::values(clEnumValN(BenchmarkRunner::ExecutionModeE::InProcess,
                           "inprocess",
-                          "Executes the snippets within the same process")),
+                          "Executes the snippets within the same process"),
+               clEnumValN(BenchmarkRunner::ExecutionModeE::SubProcess,
+                          "subprocess",
+                          "Spawns a subprocess for each snippet execution, "
+                          "allows for the use of memory annotations")),
     cl::init(BenchmarkRunner::ExecutionModeE::InProcess));
 
 static ExitOnError ExitOnErr("llvm-exegesis error: ");
 
 // Helper function that logs the error(s) and exits.
 template <typename... ArgTs> static void ExitWithError(ArgTs &&... Args) {
   ExitOnErr(make_error<Failure>(std::forward<ArgTs>(Args)...));
 }