HomePhabricator

[OpenMP] Add Two-level Distributed Barrier
Concern Raised25073a4ecfc9

Authored by tlwilmar on May 21 2021, 3:06 PM.

Description

[OpenMP] Add Two-level Distributed Barrier

Two-level distributed barrier is a new experimental barrier designed
for Intel hardware that has better performance in some cases than the
default hyper barrier.

This barrier is designed to handle fine granularity parallelism where
barriers are used frequently with little compute and memory access
between barriers. There is no need to use it for codes with few
barriers and large granularity compute, or memory intensive
applications, as little difference will be seen between this barrier
and the default hyper barrier. This barrier is designed to work
optimally with a fixed number of threads, and has a significant setup
time, so should NOT be used in situations where the number of threads
in a team is varied frequently.

The two-level distributed barrier is off by default -- hyper barrier
is used by default. To use this barrier, you must set all barrier
patterns to use this type, because it will not work with other barrier
patterns. Thus, to turn it on, the following settings are required:

KMP_FORKJOIN_BARRIER_PATTERN=dist,dist
KMP_PLAIN_BARRIER_PATTERN=dist,dist
KMP_REDUCTION_BARRIER_PATTERN=dist,dist

Branching factors (set with KMP_FORKJOIN_BARRIER, KMP_PLAIN_BARRIER,
and KMP_REDUCTION_BARRIER) are ignored by the two-level distributed
barrier.

Differential Revision: https://reviews.llvm.org/D103121

Event Timeline

ggouaillardet raised a concern with this commit.Jun 19 2021, 10:03 PM
ggouaillardet added a subscriber: ggouaillardet.

This commit broke llvm (main branch) compilation on aarch64 because of undefined _mm_malloc() and _mm_free() functions.
(x86_64) is fine though.
The current workaround on aarch64 is to revert this commit.

Building CXX object projects/openmp/runtime/src/CMakeFiles/omp.dir/kmp_alloc.cpp.o
In file included from /home/users/u0001043/src/llvm-main/openmp/runtime/src/kmp.h:118,

from /home/users/u0001043/src/llvm-main/openmp/runtime/src/kmp_alloc.cpp:13:

/home/users/u0001043/src/llvm-main/openmp/runtime/src/kmp_barrier.h: In static member function ?static distributedBarrier* distributedBarrier::allocate(int)?:
/home/users/u0001043/src/llvm-main/openmp/runtime/src/kmp_barrier.h:82:51: error: ?_mm_malloc? was not declared in this scope; did you mean ?malloc??

82 |     distributedBarrier *d = (distributedBarrier *)_mm_malloc(
   |                                                   ^~~~~~~~~~
   |                                                   malloc

/home/users/u0001043/src/llvm-main/openmp/runtime/src/kmp_barrier.h: In static member function ?static void distributedBarrier::deallocate(distributedBarrier*)?:
/home/users/u0001043/src/llvm-main/openmp/runtime/src/kmp_barrier.h:99:52: error: ?_mm_free? was not declared in this scope

99 |   static void deallocate(distributedBarrier *db) { _mm_free(db); }
   |                                                    ^~~~~~~~
This commit now has outstanding concerns.Jun 19 2021, 10:03 PM