[llvm-exegesis] Add ability to assign perf counters to specific PID

[aidengrossman]

Summary

This patch gives the ability to assign performance counters within
llvm-exegesis to a specific process by passing its PID. This is needed
later on for implementing a subprocess executor. Defaults to zero, the
current process, for the InProcessFunctionExecutorImpl.

[aidengrossman]

Rebase.

[courbet]

llvm/tools/llvm-exegesis/lib/PerfHelper.h

79

[nit] ProcessId (P already stands for "process")

[aidengrossman]

Address reviewer feedback.

[aidengrossman]

Format, remove LLVM_HAVE_LIBPFM guard around FileDescriptor variable so the getter doesn't cause build errors on setups that don't have libpfm installed.

[aidengrossman]

Update, fix non Linux builds.

[aidengrossman]

Fix another issue in the Windows CI build.

[aidengrossman]

Add semicolon to fix non-libpfm builds.

[aidengrossman]

Rebase

[mstorsjo]

This broke compilation for mingw targets (in particular, for 64 bit targets - the issue seems to be benign for 32 bit targets). For those targets, compilation fails with this error:

/home/martin/code/llvm-project/llvm/tools/llvm-exegesis/lib/PerfHelper.h:27:13: error: typedef redefinition with different types ('int' vs '_pid_t' (aka 'long long'))
   27 | typedef int pid_t;
      |             ^
/home/martin/clang-nightly/aarch64-w64-mingw32/include/sys/types.h:68:16: note: previous definition is here
   68 | typedef _pid_t  pid_t;
      |                 ^
1 error generated.

(For 32 bit targets, there's no mismatch between int and long long so it doesn't error out.)

HAVE_LIBPFM isn't a good guard for whether we need to define pid_t ourselves, since we can't know whether a transitively included header has defined this type regardless of libpfm. From looking around in e.g. lldb sources, pid_t is probably generally available for all targets except for Windows/MSVC (while mingw does have it).

[mstorsjo]

In D151020#4440457, @mstorsjo wrote:

This broke compilation for mingw targets (in particular, for 64 bit targets - the issue seems to be benign for 32 bit targets). For those targets, compilation fails with this error:

/home/martin/code/llvm-project/llvm/tools/llvm-exegesis/lib/PerfHelper.h:27:13: error: typedef redefinition with different types ('int' vs '_pid_t' (aka 'long long'))
   27 | typedef int pid_t;
      |             ^
/home/martin/clang-nightly/aarch64-w64-mingw32/include/sys/types.h:68:16: note: previous definition is here
   68 | typedef _pid_t  pid_t;
      |                 ^
1 error generated.

(For 32 bit targets, there's no mismatch between int and long long so it doesn't error out.)

HAVE_LIBPFM isn't a good guard for whether we need to define pid_t ourselves, since we can't know whether a transitively included header has defined this type regardless of libpfm. From looking around in e.g. lldb sources, pid_t is probably generally available for all targets except for Windows/MSVC (while mingw does have it).

I would really like to unbreak the mingw build. If anybody is reading and listening - do you prefer reverting this commit for now, until we can sort out how to safely typedef int across various OSes, or just add an && !defined(__MINGW32__) into the condition? Or something else?

[aidengrossman]

Sorry about that. I probably should have traced the headers more closely. It looks like just checking for __pid_t_defined should work. I'll get a patch up soon.

[aidengrossman]

Or, looks like Mac doesn't define that in their unistd.h header. I'll revert for now and figure out a better solution before relanding.

[mstorsjo]

In D151020#4442095, @aidengrossman wrote:

Or, looks like Mac doesn't define that in their unistd.h header. I'll revert for now and figure out a better solution before relanding.

Thanks, much appreciated.

Yeah it seems less trivial than I would have wanted/expected to get this done, without adding e.g. a cmake level check for it. Alternatively, a strategy like "include all of <sys/types.h> and <unistd.h> and similar that are available, and then add a typedef if MSVC" might work too, but that's not exactly neat either.

[aidengrossman]

It looks like the approach that lldb takes in regards to pid_t is to include sys/types.h (which is where the type is defined according to POSIX) everywhere except where _MSC_VER is defined (so MSVC and Clang on Windows) and then typedeffing it as uint32_t (why that specific type I'm not sure since POSIX mandates it is a signed integer type, but 32 bits is fine). I'm going to take this approach and reland this commit after some testing. Hopefully there aren't any further regressions from this.

llvm/tools/llvm-exegesis/lib/PerfHelper.h

 #include "llvm/ADT/StringRef.h"
 #include "llvm/Config/config.h"
 #include "llvm/Support/Error.h"
 
 #include <cstdint>
 #include <functional>
 #include <memory>
 
+#ifdef _MSC_VER
+typedef int pid_t;
+#else
+#include <sys/types.h>
+#endif // HAVE_LIBPFM
+
 struct perf_event_attr;
 
 namespace llvm {
 namespace exegesis {
 namespace pfm {
 
 // Returns true on error.
 bool pfmInitialize();
protected:
   PerfEvent() = default;
   std::string EventString;
   std::string FullQualifiedEventString;
   perf_event_attr *Attr;
 
 private:
   void initRealEvent(StringRef PfmEventString);
 };
 
 // Uses a valid PerfEvent to configure the Kernel so we can measure the
 // underlying event.
 class Counter {
 public:
   // event: the PerfEvent to measure.
-  explicit Counter(PerfEvent &&event);
+  explicit Counter(PerfEvent &&event, pid_t ProcessID = 0);
 
   Counter(const Counter &) = delete;
   Counter(Counter &&other) = default;
 
   virtual ~Counter();
 
   /// Starts the measurement of the event.
   virtual void start();
public:
   /// within the benchmarked code.
   /// If empty (or not specified), then no filtering will be done.
   /// Not all counters choose to use this.
   virtual llvm::Expected<llvm::SmallVector<int64_t, 4>>
   readOrError(StringRef FunctionBytes = StringRef()) const;
 
   virtual int numValues() const;
 
+  int getFileDescriptor() const { return FileDescriptor; }
+
 protected:
   PerfEvent Event;
-#ifdef HAVE_LIBPFM
   int FileDescriptor = -1;
-#endif
   bool IsDummyEvent;
 
 private:
-  void initRealEvent(const PerfEvent &E);
+  void initRealEvent(const PerfEvent &E, pid_t ProcessID);
 };
 
 } // namespace pfm
 } // namespace exegesis
 } // namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_EXEGESIS_PERFHELPER_H

llvm/tools/llvm-exegesis/lib/PerfHelper.cpp

 bool PerfEvent::valid() const { return !FullQualifiedEventString.empty(); }
 
 const perf_event_attr *PerfEvent::attribute() const { return Attr; }
 
 StringRef PerfEvent::getPfmEventString() const {
   return FullQualifiedEventString;
 }
 
-Counter::Counter(PerfEvent &&E) : Event(std::move(E)){
+Counter::Counter(PerfEvent &&E, pid_t ProcessID) : Event(std::move(E)) {
   assert(Event.valid());
   IsDummyEvent = Event.name() == PerfEvent::DummyEventString;
   if (!IsDummyEvent)
-    initRealEvent(E);
+    initRealEvent(E, ProcessID);
 }
 
 #ifdef HAVE_LIBPFM
-void Counter::initRealEvent(const PerfEvent &E) {
-  const pid_t Pid = 0;    // measure current process/thread.
+void Counter::initRealEvent(const PerfEvent &E, pid_t ProcessID) {
   const int Cpu = -1;     // measure any processor.
   const int GroupFd = -1; // no grouping of counters.
   const uint32_t Flags = 0;
   perf_event_attr AttrCopy = *Event.attribute();
-  FileDescriptor = perf_event_open(&AttrCopy, Pid, Cpu, GroupFd, Flags);
+  FileDescriptor = perf_event_open(&AttrCopy, ProcessID, Cpu, GroupFd, Flags);
   if (FileDescriptor == -1) {
     errs() << "Unable to open event. ERRNO: " << strerror(errno)
            << ". Make sure your kernel allows user "
               "space perf monitoring.\nYou may want to try:\n$ sudo sh "
               "-c 'echo -1 > /proc/sys/kernel/perf_event_paranoid'.\n"
            << "If you are debugging and just want to execute the snippet "
               "without actually reading performance counters, "
               "pass --use-dummy-perf-counters command line option.\n";
Counter::readOrError(StringRef /*unused*/) const {
   llvm::SmallVector<int64_t, 4> Result;
   Result.push_back(Count);
   return Result;
 }
 
 int Counter::numValues() const { return 1; }
 #else
 
-void Counter::initRealEvent(const PerfEvent &) {}
+void Counter::initRealEvent(const PerfEvent &, pid_t ProcessID) {}
 
 Counter::~Counter() = default;
 
 void Counter::start() {}
 
 void Counter::stop() {}
 
 int64_t Counter::read() const { return 42; }

llvm/tools/llvm-exegesis/lib/Target.h

 
 class ExegesisTarget {
 public:
   explicit ExegesisTarget(ArrayRef<CpuAndPfmCounters> CpuPfmCounters)
       : CpuPfmCounters(CpuPfmCounters) {}
 
   // Targets can use this to create target-specific perf counters.
   virtual Expected<std::unique_ptr<pfm::Counter>>
-  createCounter(StringRef CounterName, const LLVMState &State) const;
+  createCounter(StringRef CounterName, const LLVMState &State,
+                const pid_t ProcessID = 0) const;
 
   // Targets can use this to add target-specific passes in assembleToStream();
   virtual void addTargetSpecificPasses(PassManagerBase &PM) const {}
 
   // Generates code to move a constant into a the given register.
   // Precondition: Value must fit into Reg.
   virtual std::vector<MCInst> setRegTo(const MCSubtargetInfo &STI, unsigned Reg,
                                        const APInt &Value) const = 0;

llvm/tools/llvm-exegesis/lib/Target.cpp

const ExegesisTarget *ExegesisTarget::lookup(Triple TT) {
   for (const ExegesisTarget *T = FirstTarget; T != nullptr; T = T->Next) {
     if (T->matchesArch(TT.getArch()))
       return T;
   }
   return nullptr;
 }
 
 Expected<std::unique_ptr<pfm::Counter>>
-ExegesisTarget::createCounter(StringRef CounterName, const LLVMState &) const {
+ExegesisTarget::createCounter(StringRef CounterName, const LLVMState &,
+                              const pid_t ProcessID) const {
   pfm::PerfEvent Event(CounterName);
   if (!Event.valid())
     return llvm::make_error<Failure>(
         llvm::Twine("Unable to create counter with name '")
             .concat(CounterName)
             .concat("'"));
 
-  return std::make_unique<pfm::Counter>(std::move(Event));
+  return std::make_unique<pfm::Counter>(std::move(Event), ProcessID);
 }
 
 void ExegesisTarget::registerTarget(ExegesisTarget *Target) {
   if (FirstTarget == nullptr) {
     FirstTarget = Target;
     return;
   }
   if (Target->Next != nullptr)

llvm/tools/llvm-exegesis/lib/X86/Target.cpp

 #endif
 };
 
 class ExegesisX86Target : public ExegesisTarget {
 public:
   ExegesisX86Target() : ExegesisTarget(X86CpuPfmCounters) {}
 
   Expected<std::unique_ptr<pfm::Counter>>
-  createCounter(StringRef CounterName, const LLVMState &State) const override {
+  createCounter(StringRef CounterName, const LLVMState &State,
+                const pid_t ProcessID) const override {
     // If LbrSamplingPeriod was provided, then ignore the
     // CounterName because we only have one for LBR.
     if (LbrSamplingPeriod > 0) {
       // Can't use LBR without HAVE_LIBPFM, LIBPFM_HAS_FIELD_CYCLES, or without
       // __linux__ (for now)
 #if defined(HAVE_LIBPFM) && defined(LIBPFM_HAS_FIELD_CYCLES) &&                \
     defined(__linux__)
       return std::make_unique<X86LbrCounter>(
           X86LbrPerfEvent(LbrSamplingPeriod));
 #else
       return llvm::make_error<llvm::StringError>(
           "LBR counter requested without HAVE_LIBPFM, LIBPFM_HAS_FIELD_CYCLES, "
           "or running on Linux.",
           llvm::errc::invalid_argument);
 #endif
     }
-    return ExegesisTarget::createCounter(CounterName, State);
+    return ExegesisTarget::createCounter(CounterName, State, ProcessID);
   }
 
 private:
   void addTargetSpecificPasses(PassManagerBase &PM) const override;
 
   unsigned getScratchMemoryRegister(const Triple &TT) const override;
 
   unsigned getLoopCounterRegister(const Triple &) const override;