Use sched_getaffinity instead of std::thread::hardware_concurrency when available

[• rafael]

Summary

The issue with std::thread::hardware_concurrency is that it forwards to libc and some implementations (like glibc) don't take thread affinity into consideration.

With this change a llvm program that can execute in only 2 cores will use 2 threads, even if the machine has 32 cores.

This makes benchmarking a lot easier, but should also help if someone doesn't want to use all cores for compilation for example.

[smeenai]

include/llvm/Support/Threading.h

135

Typo: nreturns

[ruiu]

Thank you for doing this.

Off-topic: there seems to be many things we can do utilize multiple cores more efficiently. One thing I want to fix is the large variance of lld's execution time. Currently, lld sometimes takes unreasonably long time to finish even though that rarely happens. I'd guess the kernel does a bad job at scheduling threads. I wonder if we could do something -- e.g. pinning each thread to an available core.

include/llvm/Support/Threading.h

140

It sounds like it returns a list of std::thread objects or something. How about hardware_concurrency? We have heavyweight_hardware_concurrency function just above this declaration, and compared to that name, I don't think that's a bad name.

[• rafael]

Renamed. Fixed typo.

[ruiu]

include/llvm/Support/ThreadPool.h

42

hardware_concurrency()

lib/Support/Threading.cpp

82

AFAIK std::thread::hardware_concurrency() is allowed to return 0 if it doesn't know anything about hardware concurrency. I don't know if there's actually such system that the function could return 0, but it is better to not rely on an undefined behavior. I'd explicitly check if std::thread::hardware_concurrency returns 0 and if that's the case, return 1.

[• rafael]

Rename the missing places. Guard against std::thread::hardware_concurrency returning 0.

[ruiu]

LGTM

lib/Fuzzer/FuzzerUtil.cpp

199–204

Since you are using your own hardware_concurrency, you don't need to guard against 0, no?

tools/llvm-profdata/llvm-profdata.cpp

214–215

You can remove std::max because std::min(hardware_concurrency(), unsigned(Inputs.size() / 2)) can now never return 0.

[• rafael]

r314809

[timshen]

Hi,

I reverted the change to FuzzerUtil.cpp, as it doesn't seem to depend on LLVM.

[dneilson]

Reverted: https://reviews.llvm.org/rL314922

cmake/config-ix.cmake

   add_definitions( -D_GNU_SOURCE )
 endif()
 # This check requires _GNU_SOURCE
+check_library_exists(c sched_getaffinity "" HAVE_SCHED_GETAFFINITY)
 if(HAVE_LIBPTHREAD)
   check_library_exists(pthread pthread_getname_np "" HAVE_PTHREAD_GETNAME_NP)
   check_library_exists(pthread pthread_setname_np "" HAVE_PTHREAD_SETNAME_NP)

include/llvm/Config/config.h.cmake

 /* Define to 1 if you have the `setenv' function. */
 #cmakedefine HAVE_SETENV ${HAVE_SETENV}
 
+/* Define to 1 if you have the `sched_getaffinity' function. */
+#cmakedefine HAVE_SCHED_GETAFFINITY ${HAVE_SCHED_GETAFFINITY}
+
 /* Define to 1 if you have the `setrlimit' function. */
 #cmakedefine HAVE_SETRLIMIT ${HAVE_SETRLIMIT}
 

include/llvm/Support/ThreadPool.h

   using TaskTy = std::function<void()>;
   using PackagedTaskTy = std::packaged_task<void()>;
 
-  /// Construct a pool with the number of core available on the system (or
-  /// whatever the value returned by std::thread::hardware_concurrency() is).
+  /// Construct a pool with the number of threads found by
+  /// hardware_concurrency().
   ThreadPool();
 
   /// Construct a pool of \p ThreadCount threads

include/llvm/Support/Threading.h

   /// Returns 1 when LLVM is configured with LLVM_ENABLE_THREADS=OFF
   unsigned heavyweight_hardware_concurrency();
 
+  /// Get the number of threads that the current program can execute
+  /// concurrently. On some systems std::thread::hardware_concurrency() returns
+  /// the total number of cores, without taking affinity into consideration.
+  /// Returns 1 when LLVM is configured with LLVM_ENABLE_THREADS=OFF.
+  /// Fallback to std::thread::hardware_concurrency() if sched_getaffinity is
+  /// not available.
+  unsigned hardware_concurrency();
+
   /// \brief Return the current thread id, as used in various OS system calls.
   /// Note that not all platforms guarantee that the value returned will be
   /// unique across the entire system, so portable code should not assume

lib/Fuzzer/FuzzerUtil.cpp

 }
 
 unsigned NumberOfCpuCores() {
-  unsigned N = std::thread::hardware_concurrency();
+  unsigned N = hardware_concurrency();
   if (!N) {
-    Printf("WARNING: std::thread::hardware_concurrency not well defined for "
+    Printf("WARNING: hardware_concurrency not well defined for "
            "your platform. Assuming CPU count of 1.\n");
     N = 1;
   }

lib/Support/Parallel.cpp

 
 #include "llvm/Support/Parallel.h"
 #include "llvm/Config/llvm-config.h"
+#include "llvm/Support/Threading.h"
 
 #include <atomic>
 #include <stack>

 ///   in filo order.
 class ThreadPoolExecutor : public Executor {
 public:
-  explicit ThreadPoolExecutor(
-      unsigned ThreadCount = std::thread::hardware_concurrency())
+  explicit ThreadPoolExecutor(unsigned ThreadCount = hardware_concurrency())
       : Done(ThreadCount) {
     // Spawn all but one of the threads in another thread as spawning threads
     // can take a while.

lib/Support/ThreadPool.cpp

 #include "llvm/Support/ThreadPool.h"
 
 #include "llvm/Config/llvm-config.h"
+#include "llvm/Support/Threading.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 
 #if LLVM_ENABLE_THREADS
 
-// Default to std::thread::hardware_concurrency
-ThreadPool::ThreadPool() : ThreadPool(std::thread::hardware_concurrency()) {}
+// Default to hardware_concurrency
+ThreadPool::ThreadPool() : ThreadPool(hardware_concurrency()) {}
 
 ThreadPool::ThreadPool(unsigned ThreadCount)
     : ActiveThreads(0), EnableFlag(true) {

lib/Support/Threading.cpp

 
 unsigned llvm::heavyweight_hardware_concurrency() { return 1; }
 
+unsigned llvm::hardware_concurrency() { return 1; }
+
 uint64_t llvm::get_threadid() { return 0; }
 
 uint32_t llvm::get_max_thread_name_length() { return 0; }

   return NumPhysical;
 }
 
+unsigned llvm::hardware_concurrency() {
+#ifdef HAVE_SCHED_GETAFFINITY
+  cpu_set_t Set;
+  if (sched_getaffinity(0, sizeof(Set), &Set))
+    return CPU_COUNT(&Set);
+#endif
+  // Guard against std::thread::hardware_concurrency() returning 0.
+  if (unsigned Val = std::thread::hardware_concurrency())
+    return Val;
+  return 1;
+}
+
 // Include the platform-specific parts of this class.
 #ifdef LLVM_ON_UNIX
 #include "Unix/Threading.inc"

tools/llvm-profdata/llvm-profdata.cpp

 
   // If NumThreads is not specified, auto-detect a good default.
   if (NumThreads == 0)
-    NumThreads = std::max(1U, std::min(std::thread::hardware_concurrency(),
-                                       unsigned(Inputs.size() / 2)));
+    NumThreads = std::max(
+        1U, std::min(hardware_concurrency(), unsigned(Inputs.size() / 2)));
 
   // Initialize the writer contexts.
   SmallVector<std::unique_ptr<WriterContext>, 4> Contexts;