Page MenuHomePhabricator

Offer API for setting number of loop dispatch buffers
ClosedPublic

Authored by jlpeyton on May 26 2016, 12:29 PM.

Details

Summary

The problem is the lack of dispatch buffers when thousands of loops with nowait, about 10 iterations each, are executed by hundreds of threads. We only have built-in 7 dispatch buffers, but there is a need in dozens or hundreds of buffers.

The problem can be fixed by setting KMP_MAX_DISP_BUF to bigger value. In order to give users same possibility I changed build-time control into run-time one, adding API just in case.

This change adds an environment variable KMP_DISP_NUM_BUFFERS and a new API function kmp_set_disp_num_buffers(int num_buffers).

The KMP_DISP_NUM_BUFFERS envirable works only before serial initialization, because during the serial initialization we already allocate buffers for the hot team, so it is too late to change the number of buffers later (or we need to reallocate buffers for all teams which sounds too complicated). The kmp_set_defaults() routine does not work for this envirable, because it calls serial initialization before reading the parameter string. So a new routine, kmp_set_disp_num_buffers(), is created so that it can set our internal global variable before the library initialization. If both the envirable and API used the envirable wins.

Diff Detail

Repository
rL LLVM

Event Timeline

jlpeyton updated this revision to Diff 58664.May 26 2016, 12:29 PM
jlpeyton retitled this revision from to Offer API for setting number of loop dispatch buffers.
jlpeyton updated this object.
jlpeyton added reviewers: AndreyChurbanov, tlwilmar.
jlpeyton set the repository for this revision to rL LLVM.
jlpeyton added a subscriber: openmp-commits.
tlwilmar accepted this revision.May 31 2016, 10:25 AM
tlwilmar edited edge metadata.

LGTM

This revision is now accepted and ready to land.May 31 2016, 10:25 AM
This revision was automatically updated to reflect the committed changes.