Page MenuHomePhabricator

[OPENMP] Make -fopenmp to turn on OpenMP support by default.
ClosedPublic

Authored by ABataev on Oct 16 2015, 5:43 AM.

Details

Summary

Patch turns on OpenMP support in clang by default after fixing OpenMP buildbots.

Diff Detail

Repository
rL LLVM

Event Timeline

ABataev updated this revision to Diff 37572.Oct 16 2015, 5:43 AM
ABataev retitled this revision from to [OPENMP] Make -fopenmp to turn on OpenMP support by default..
ABataev updated this object.
ABataev added a reviewer: chandlerc.
emaste added a subscriber: emaste.Oct 16 2015, 6:09 AM
chandlerc edited edge metadata.Oct 21 2015, 5:13 AM

Some minor issues below. Also, Hans and Tom should have a glance at the release bits of this, I don't know any of that stuff.

Also, there are still some bugs with check-libomp. The immediate one I noticed was that when i run it in a clean build with enough parallelism, it fails to find the just-built clang binary. I suspect that the 'check-libomp' CMake target is missing dependencies on all of the binaries used when running the lit tests because it copied the libc++ CMake rules for it. While the libc++ CMake rules are *mostly* a good proxy, the *testing* strategy for libc++ is notably different here. In fact, libomp's testing is (IMO) quite a bit *better* here, and uncovers a nasty missing dependency.

The libc++ test suite tests libc++ with the *host* compiler, not the just-built compiler. This has some advantages and disadvantages, but for a library with close coupling to the compiler (such as compiler-rt or libomp) it is inappropriate. Your libomp tests very correctly use the just-built Clang, but the CMake needs to model this now. You can see in projects/compiler-rt/test/CMakeLists.txt lines 19-33 how compiler-rt sets up variables with these tools (clang, clang-headers, etc) which end up used on line 99 of projects/compiler-rt/test/asan/CMakeLists.txt for example.

I don't know what all tools you need (other than clang and probably clang-headers), probably not as many as the sanitizers do, so I'll let you put together a correct patch here. I've hacked around it locally to test further.

I've also had one test fail, and then start passing for me on Linux (after fixing the above). I haven't had it fail again, but I don't have a good way of running tests repeatedly (see below, llvm-lit doesn't yet work). It might be good to give the test suite a good 10 or 50 runs and see if anything starts failing. I'll do that overnight and report back if so.

configure
5953 ↗(On Diff #37572)

You should be modifying the configure.ac file and regenerating this?

utils/llvm-lit/llvm-lit.in
42–46 ↗(On Diff #37572)

This change is independent of enabling anything; can you break it out?

Also, manually patching this bit in order to test things didn't actually allow me to use llvm-lit with libomp, so I don't think this is quite working as intended yet.

ABataev marked an inline comment as done.Oct 25 2015, 11:40 PM

Hi Chandler, thanks for the review.

Some minor issues below. Also, Hans and Tom should have a glance at the release bits of this, I don't know any of that stuff.

Also, there are still some bugs with check-libomp. The immediate one I noticed was that when i run it in a clean build with enough parallelism, it fails to find the just-built clang binary. I suspect that the 'check-libomp' CMake target is missing dependencies on all of the binaries used when running the lit tests because it copied the libc++ CMake rules for it. While the libc++ CMake rules are *mostly* a good proxy, the *testing* strategy for libc++ is notably different here. In fact, libomp's testing is (IMO) quite a bit *better* here, and uncovers a nasty missing dependency.

The libc++ test suite tests libc++ with the *host* compiler, not the just-built compiler. This has some advantages and disadvantages, but for a library with close coupling to the compiler (such as compiler-rt or libomp) it is inappropriate. Your libomp tests very correctly use the just-built Clang, but the CMake needs to model this now. You can see in projects/compiler-rt/test/CMakeLists.txt lines 19-33 how compiler-rt sets up variables with these tools (clang, clang-headers, etc) which end up used on line 99 of projects/compiler-rt/test/asan/CMakeLists.txt for example.

I don't know what all tools you need (other than clang and probably clang-headers), probably not as many as the sanitizers do, so I'll let you put together a correct patch here. I've hacked around it locally to test further.

I created a separate patch for libomp, that fixes this issue.

I've also had one test fail, and then start passing for me on Linux (after fixing the above). I haven't had it fail again, but I don't have a good way of running tests repeatedly (see below, llvm-lit doesn't yet work). It might be good to give the test suite a good 10 or 50 runs and see if anything starts failing. I'll do that overnight and report back if so.

Actually, these tests are written so, that they repeats about 1000 times to be sure that they are correct. But some of them might be sensible to system load.

configure
5953 ↗(On Diff #37572)

Ok, done

utils/llvm-lit/llvm-lit.in
42–46 ↗(On Diff #37572)

Ok, reverted it

ABataev updated this revision to Diff 38374.Oct 25 2015, 11:41 PM
ABataev added reviewers: hans, tstellarAMD.
ABataev marked an inline comment as done.
hans edited edge metadata.Oct 26 2015, 9:14 AM

The release script change looks good to me. Better than good actually; it's a great improvement :-)

Hi Chandler, thanks for the review.

I've also had one test fail, and then start passing for me on Linux (after fixing the above). I haven't had it fail again, but I don't have a good way of running tests repeatedly (see below, llvm-lit doesn't yet work). It might be good to give the test suite a good 10 or 50 runs and see if anything starts failing. I'll do that overnight and report back if so.

Actually, these tests are written so, that they repeats about 1000 times to be sure that they are correct. But some of them might be sensible to system load.

Hmm, so I think we need to find some way to make some of these tests more reliable.

With the fix from D14055 patched in (minus the flag change in it) I am seeing tests fail pretty regularly:

libomp :: worksharing/for/omp_for_schedule_auto.c

This test seems to fail pretty frequently for me. Maybe as much as half the time.

libomp :: worksharing/sections/omp_sections_nowait.c

This test fails less frequently, but still have seen it a few times.

libomp :: flush/omp_flush.c

I've seen this test fail just once...

Hi Chandler, thanks for the review.

I've also had one test fail, and then start passing for me on Linux (after fixing the above). I haven't had it fail again, but I don't have a good way of running tests repeatedly (see below, llvm-lit doesn't yet work). It might be good to give the test suite a good 10 or 50 runs and see if anything starts failing. I'll do that overnight and report back if so.

Actually, these tests are written so, that they repeats about 1000 times to be sure that they are correct. But some of them might be sensible to system load.

Hmm, so I think we need to find some way to make some of these tests more reliable.

With the fix from D14055 patched in (minus the flag change in it) I am seeing tests fail pretty regularly:

libomp :: worksharing/for/omp_for_schedule_auto.c

Credit goes to a colleague of mine, Ray Loy, for spotting this. This test has a race condition. The updates to the global sum1 are not protected by any kind of synchronization.

sum1 = sum0;

the for pragma is missing a private(sum1) clause?

This test seems to fail pretty frequently for me. Maybe as much as half the time.

libomp :: worksharing/sections/omp_sections_nowait.c

In omp_testsuite.h, we have this:

/* following times are in seconds */
#define SLEEPTIME 0.1

and this test, and also omp_flush.c, are sensitive to this. If the machine is heavily loaded, or otherwise configured, so the threads involved don't all get scheduled within 0.1 seconds, this will certainly fail.

Also, does this test have a race condition? Specifically, should there be a

#pragma omp flush(count)

before

if (count == 0)

I think there should be.

This test fails less frequently, but still have seen it a few times.

libomp :: flush/omp_flush.c

I've seen this test fail just once...

This is trying to test the memory flush pragma by having one thread write and flush, and a second thread wait for a fixed time, flush and read. As mentioned above, this is sensitive to SLEEPTIME.

The proposed patches here to change the default behavior of -fopenmp from -fopenmp=libgomp to -fopenmp=libomp seem to only handle the configure-based build. The following change required to switch over the crake-based build to default -fopenmp to libomp is missing.

Index: CMakeLists.txt
===================================================================
--- CMakeLists.txt	(revision 251539)
+++ CMakeLists.txt	(working copy)
@@ -196,7 +196,7 @@ set(GCC_INSTALL_PREFIX "" CACHE PATH "Di
 set(DEFAULT_SYSROOT "" CACHE PATH
   "Default <path> to all compiler invocations for --sysroot=<path>." )
 
-set(CLANG_DEFAULT_OPENMP_RUNTIME "libgomp" CACHE STRING
+set(CLANG_DEFAULT_OPENMP_RUNTIME "libomp" CACHE STRING
   "Default OpenMP runtime used by -fopenmp.")
 
 set(CLANG_VENDOR "" CACHE STRING

The proposed patches here to change the default behavior of -fopenmp from -fopenmp=libgomp to -fopenmp=libomp seem to only handle the configure-based build. The following change required to switch over the crake-based build to default -fopenmp to libomp is missing.

Index: CMakeLists.txt
===================================================================
--- CMakeLists.txt	(revision 251539)
+++ CMakeLists.txt	(working copy)
@@ -196,7 +196,7 @@ set(GCC_INSTALL_PREFIX "" CACHE PATH "Di
 set(DEFAULT_SYSROOT "" CACHE PATH
   "Default <path> to all compiler invocations for --sysroot=<path>." )
 
-set(CLANG_DEFAULT_OPENMP_RUNTIME "libgomp" CACHE STRING
+set(CLANG_DEFAULT_OPENMP_RUNTIME "libomp" CACHE STRING
   "Default OpenMP runtime used by -fopenmp.")
 
 set(CLANG_VENDOR "" CACHE STRING

This patch is for clang and it is accepted already. But I'm going to land all patches for -fopenmp only when llvm and libomp changes are accepted.

Hi Chandler, thanks for the review.

I've also had one test fail, and then start passing for me on Linux (after fixing the above). I haven't had it fail again, but I don't have a good way of running tests repeatedly (see below, llvm-lit doesn't yet work). It might be good to give the test suite a good 10 or 50 runs and see if anything starts failing. I'll do that overnight and report back if so.

Actually, these tests are written so, that they repeats about 1000 times to be sure that they are correct. But some of them might be sensible to system load.

Hmm, so I think we need to find some way to make some of these tests more reliable.

With the fix from D14055 patched in (minus the flag change in it) I am seeing tests fail pretty regularly:

libomp :: worksharing/for/omp_for_schedule_auto.c

Credit goes to a colleague of mine, Ray Loy, for spotting this. This test has a race condition. The updates to the global sum1 are not protected by any kind of synchronization.

sum1 = sum0;

the for pragma is missing a private(sum1) clause?

This test seems to fail pretty frequently for me. Maybe as much as half the time.

libomp :: worksharing/sections/omp_sections_nowait.c

In omp_testsuite.h, we have this:

/* following times are in seconds */
#define SLEEPTIME 0.1

and this test, and also omp_flush.c, are sensitive to this. If the machine is heavily loaded, or otherwise configured, so the threads involved don't all get scheduled within 0.1 seconds, this will certainly fail.

Also, does this test have a race condition? Specifically, should there be a

#pragma omp flush(count)

before

if (count == 0)

I think there should be.

This test fails less frequently, but still have seen it a few times.

libomp :: flush/omp_flush.c

I've seen this test fail just once...

This is trying to test the memory flush pragma by having one thread write and flush, and a second thread wait for a fixed time, flush and read. As mentioned above, this is sensitive to SLEEPTIME.

Hal, thanks for pointing this. I'll take a look and try to fix these tests.

The CMakeLists.txt change cited is in top-level of the clang sources and not in the llvm top-level.

Chandler, gcc buildbot greenified, tests for clang are fixed. Hans Wennborg accepted this patch. Could accept this patch?

tstellarAMD resigned from this revision.Dec 2 2015, 10:35 AM
tstellarAMD removed a reviewer: tstellarAMD.

Hans already commented on the release script changes, and I agrre they look good.

hans added a comment.Dec 8 2015, 1:35 PM

Alexey & Chandler: Is there any ongoing communication here?

It would be great if we could resolve this, or at least figure out what the status is, before we branch for 3.8 in a month or so.

ABataev added a subscriber: ABataev.Dec 9 2015, 2:09 AM

Hi Hans,
I asked Chandler to look at this once again but seems to me he is very
busy. Maybe you could ping him also?

Best regards,

Alexey Bataev

Software Engineer
Intel Compiler Team

09.12.2015 0:35, Hans Wennborg пишет:

hans added a comment.

Alexey & Chandler: Is there any ongoing communication here?

It would be great if we could resolve this, or at least figure out what the status is, before we branch for 3.8 in a month or so.

http://reviews.llvm.org/D13802

chandlerc accepted this revision.Dec 9 2015, 6:56 AM
chandlerc edited edge metadata.

Sorry I was a bit slow -- was out when commented in Nov, and was slow catching back up the past two weeks.

I've done some more testing, and the check-libomp is passing very nicely now! =D Ship it! Also make sure you follow up with the broader set of build bots to have them include this repository in the checkout.

Minor documentation tweak below, but feel free to land everything. =D I'm excited, it'll be interesting to see if anything else breaks, but so far it seems to be working well for me.

docs/GettingStarted.rst
58–62 ↗(On Diff #38374)

I'd skip the optional tag -- as you've nicely documented, its required for OpenMP support.

This revision is now accepted and ready to land.Dec 9 2015, 6:56 AM
ABataev marked an inline comment as done.Dec 9 2015, 9:08 PM

Chandler, thank you very much for review!

docs/GettingStarted.rst
58–62 ↗(On Diff #38374)

Ok, removed

This revision was automatically updated to reflect the committed changes.
ABataev marked an inline comment as done.