HomePhabricator

[Support] Fix computeHostNumPhysicalCores() to respect affinity

Authored by MaskRay on Jul 31 2020, 11:20 AM.

Description

[Support] Fix computeHostNumPhysicalCores() to respect affinity

computeHostNumPhysicalCores() is designed to respect CPU affinity.
D84764 used sysconf(_SC_NPROCESSORS_ONLN) which does not respect
affinity.
SupportTests Threading.PhysicalConcurrency may fail if taskset -c is specified.

Details

Committed
MaskRayJul 31 2020, 11:20 AM
Parents
rG8830f1170dae: [flang]Verify C7107, C7108, C7109 from - Clause 7 constraint checks for f18.
Branches
Unknown
Tags
Unknown

Event Timeline

nemanjai added inline comments.
/llvm/lib/Support/Host.cpp
1277

This is not safe on systems with more than 1023 CPUs. It therefore causes failures on 3 PPC bots that run on a large machine. The only way to safely rely on this is by using CPU_ALLOC to dynamically allocate a set large enough.

Presumably something along these lines:

cpu_set_t *Affinity;
int NumCPUs = 1;
Affinity = CPU_ALLOC(NumCPUs);
while (sched_getaffinity(0, CPU_ALLOC_SIZE(NumCPUs), Affinity) != 0) {
  NumCPUs <<= 1;
  CPU_FREE(Affinity);
  Affinity = CPU_ALLOC(NumCPUs);
  // Give up if the number of CPUs is not reasonable (no PPC system can have more than 2k CPUs).
  if (NumCPUs > 2048)
    return -1;
}
NumCPUs = CPU_COUNT(Affinity);
CPU_FREE(Affinity);
return NumCPUs;
MaskRay added inline comments.Aug 3 2020, 9:35 AM
/llvm/lib/Support/Host.cpp
1277

Can you commit the change? I did the commit because I observed some ppc64le failures with the original code (which did not respect affinity). An ideal fix should respect disabled core like the x86 branch, but I don't know enough about ppc64 to commit such a change.