[OMPT] Add annotations to testcases that are expected to fail when using certain compilers

[sconvent]

Summary

Add annotations to testcases that are expected to fail when using certain compilers

[protze.joachim]

LGTM

[Hahnfeld]

runtime/test/ompt/misc/control_tool.c

3

This works on LLVM's buildbot, so this can't be entirely correct: http://lab.llvm.org:8011/builders/openmp-ompt-gcc-x86_64-linux-debian/builds/14/steps/test-libomp/logs/stdio

Why did you want to deactivate this?

runtime/test/ompt/worksharing/sections.c

4

I thought versions 17 and 18 of the Intel Compiler pass in the data to differentiate between sections and loops?

[protze.joachim]

@omalyshe Which version of icc introduced the ident information for workshare constructs?

[omalyshe]

In D40384#938757, @protze.joachim wrote:

@omalyshe Which version of icc introduced the ident information for workshare constructs?

ICC with the ident information has not been released yet

[sconvent]

runtime/test/ompt/misc/control_tool.c

3

Which gcc version does the buildbot use? For me the testcase fails for versions up to 7.2 (I haven't tested more recent versions).
The problem is that in those versions, gcc has problems with labels inside of OpenMP regions. So when using the "print_current_address" macro (which defines a label) in a parallel region, the compilation fails.
Other testcases with the same problem also work in the buildbot so I believe this is a matter of the gcc version.

[Hahnfeld]

In D40384#938762, @omalyshe wrote:

In D40384#938757, @protze.joachim wrote:

@omalyshe Which version of icc introduced the ident information for workshare constructs?

ICC with the ident information has not been released yet

In that case cancel/cancel_worksharing.c should fail for icc-17, icc-18 too?

runtime/test/ompt/misc/control_tool.c

3

GCC 7.2.0, probably the system default in Debian: http://lab.llvm.org:8011/builders/openmp-ompt-gcc-x86_64-linux-debian/builds/16/steps/configure-openmp/logs/stdio

Maybe the distribution carries some downstream patches?!?

[sconvent]

In D40384#939025, @Hahnfeld wrote:

In D40384#938762, @omalyshe wrote:

In D40384#938757, @protze.joachim wrote:

@omalyshe Which version of icc introduced the ident information for workshare constructs?

ICC with the ident information has not been released yet

In that case cancel/cancel_worksharing.c should fail for icc-17, icc-18 too?

No, the ident information was only missing for the worksharing constructs. For cancellation constructs, the compiler generates a call to '__kmpc_cancel(...)' with the correct kind parameter. icc-16 however did not generate the correct call.

[protze.joachim]

runtime/test/ompt/misc/control_tool.c

3

The context is this (fixed) issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81687

[Hahnfeld]

runtime/test/ompt/misc/control_tool.c

3

Yes, so the fix has been backported to GCC 5, 6, 7. This has happened after 7.2.0 has been released, but probably Debian tracks a more recent commit (which most distributions do unfortunately). So although I don't like we might need to turn this into an UNSUPPORTED because a passing XFAIL is treated as error :-(

[Hahnfeld]

runtime/test/ompt/misc/control_tool.c

3

Yes, the fix has been included in gcc-7.2.0-5 (http://metadata.ftp-master.debian.org/changelogs/main/g/gcc-7/gcc-7_7.2.0-16_changelog)

[protze.joachim]

runtime/test/ompt/misc/control_tool.c

3

So
XFAIL gcc-5 gcc-6 gcc-7.0 gcc-7.1
UNSUPPORTED gcc-7.2

This would affect all tests, that have this line in this patch.

[Hahnfeld]

runtime/test/ompt/misc/control_tool.c

3

I think we don't have gcc-7.0 gcc-7.1 at the moment, only gcc-7. And we could possibly get in trouble with the older versions as well because the fix was backported to GCC 5 and 6, too.

[omalyshe]

In D40384#939079, @sconvent wrote:

In D40384#939025, @Hahnfeld wrote:

In D40384#938762, @omalyshe wrote:

In D40384#938757, @protze.joachim wrote:

@omalyshe Which version of icc introduced the ident information for workshare constructs?

ICC with the ident information has not been released yet

In that case cancel/cancel_worksharing.c should fail for icc-17, icc-18 too?

No, the ident information was only missing for the worksharing constructs. For cancellation constructs, the compiler generates a call to '__kmpc_cancel(...)' with the correct kind parameter. icc-16 however did not generate the correct call.

Right, the ident information is needed only to distinguish worksharing constructs. ICC 17.0 will NOT have ident flags implemented; as for 18.0 - probably Update 2 will have it.

[Hahnfeld]

So we need to treat compilers with "fixes" in minor versions as UNSUPPORTED because we don't know when distributions will backport patches and what version they give their "patched" compiler. To avoid any hassles with GCC 5 and 6 which might see a maintenance release or not, I'd propose the following:

• For https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81687: UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7, GCC 8 will certainly have the fix
• ident information:
  • XFAIL: icc-16, icc-17 (these will not be updated according to @omalyshe), UNSUPPORTED: icc-18 (might get them in update 2), later versions will have the information for sure.
  • XFAIL: clang-3, clang-4, clang-5 can stay because the patch was not backported from trunk which will become Clang 6 (I do really hope no downstream distribution will do that on their own)

Thoughts?

[Hahnfeld]

runtime/test/ompt/tasks/dependences.c

3

btw, this one must be failing for a different reason, unrelated to the problem with labels: http://lab.llvm.org:8011/builders/openmp-ompt-gcc-x86_64-linux-debian/builds/20/steps/test-libomp/logs/FAIL%3A%20libomp%3A%3Adependences.c

[sconvent]

The use of UNSUPPORTED seems to be necessary. I updated the Diff.

[sconvent]

runtime/test/ompt/tasks/dependences.c

3

You're right, this testcase also (currently) fails with a fixed compiler. I will publish a fix in another revision.

[Hahnfeld]

Two small things to considers, looks good otherwise.

Maybe we can commit all changes because of the GCC bug separately and add a meaningfull description? (no need to split this revision though)

runtime/test/ompt/cancel/cancel_taskgroup.c

3–5

Please add clang-4.0.0 if this test case triggers the hang on cancellation

runtime/test/ompt/tasks/dependences.c

3

Shouldn't this be UNSUPPORTED as well?

[sconvent]

Implemented suggestions.

cmake/OpenMPTesting.cmake

else()
   set(OPENMP_TEST_COMPILER_VERSION_MAJOR "${LLVM_MAJOR_VERSION}")
   set(OPENMP_TEST_COMPILER_OPENMP_FLAGS "-fopenmp")
 endif()
 
 # Function to set compiler features for use in lit.
 function(set_test_compiler_features)
   if ("${OPENMP_TEST_COMPILER_ID}" STREQUAL "GNU")
     set(comp "gcc")
+  elseif ("${OPENMP_TEST_COMPILER_ID}" STREQUAL "Intel")
+    set(comp "icc")
   else()
     # Just use the lowercase of the compiler ID as fallback.
     string(TOLOWER "${OPENMP_TEST_COMPILER_ID}" comp)
   endif()
   set(OPENMP_TEST_COMPILER_FEATURES "['${comp}', '${comp}-${OPENMP_TEST_COMPILER_VERSION_MAJOR}', '${comp}-${OPENMP_TEST_COMPILER_VERSION}']" PARENT_SCOPE)
 endfunction()
 set_test_compiler_features()
 

runtime/test/ompt/cancel/cancel_taskgroup.c

 // RUN:  %libomp-compile && env OMP_CANCELLATION=true %libomp-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
-// Current GOMP interface implementation does not support cancellation
-// XFAIL: gcc
+// UNSUPPORTED: clang-3, clang-4.0.0
+// Current GOMP interface implementation does not support cancellation; icc 16 has a bug
+// XFAIL: gcc, icc-16
 
 #include "callback.h"
 #include <unistd.h>  
 #include <stdio.h>
 
 int main()
 {
   int condition=0;

runtime/test/ompt/cancel/cancel_worksharing.c

 // RUN: %libomp-compile && env OMP_CANCELLATION=true %libomp-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
-// Current GOMP interface implementation does not support cancellation
-// XFAIL: gcc
-
+// Current GOMP interface implementation does not support cancellation; icc 16 does not distinguish between sections and loops
+// XFAIL: gcc, icc-16
 
 #include "callback.h"
 #include <unistd.h>
 
 int main()
 {
   int condition=0;
   #pragma omp parallel num_threads(2)

runtime/test/ompt/misc/control_tool.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   #pragma omp parallel num_threads(1)
   {
     print_frame(1);

runtime/test/ompt/parallel/max_active_levels_serialized.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=THREADS %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   omp_set_nested(1);
   omp_set_max_active_levels(1);
 

runtime/test/ompt/parallel/nested.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=THREADS %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 #include <unistd.h>
 
 int main()
 {
   int condition=0;
   omp_set_nested(1);

runtime/test/ompt/parallel/nested_lwt.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=THREADS %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 #include <unistd.h>
 
 int main()
 {
   omp_set_nested(1);
   int condition;

runtime/test/ompt/parallel/nested_serialized.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=THREADS %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   omp_set_nested(0);
 
   #pragma omp parallel num_threads(4)

runtime/test/ompt/parallel/parallel_if0.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 
 int main()
 {
 //  print_frame(0);
   #pragma omp parallel if(0)
   {
 //    print_frame(1);

runtime/test/ompt/parallel/serialized.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 
 int main()
 {
 //  print_frame(0);
   #pragma omp parallel num_threads(1)
   {
 //    print_frame(1);

runtime/test/ompt/synchronization/barrier/explicit.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   int x = 0;
 
   #pragma omp parallel num_threads(2)

runtime/test/ompt/synchronization/barrier/for_loop.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   int y[] = {0,1,2,3};
 
   #pragma omp parallel num_threads(2)

runtime/test/ompt/synchronization/barrier/for_simd.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // REQUIRES: ompt
+// XFAIL: gcc-4
 
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   int y[] = {0,1,2,3};
 

runtime/test/ompt/synchronization/barrier/single.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   int x = 0;
 
   #pragma omp parallel num_threads(2)

runtime/test/ompt/synchronization/critical.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   #pragma omp critical
   {
     print_current_address(1);

runtime/test/ompt/synchronization/ordered.c

 // RUN: %libomp-compile-and-run | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   #pragma omp ordered
   {
     print_current_address(1);

runtime/test/ompt/synchronization/taskgroup.c

 // RUN:  %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 
 #include "callback.h"
 #include <unistd.h>  
 #include <stdio.h>
 
 int main()
 {
   int condition=0;

runtime/test/ompt/synchronization/taskwait.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   int x = 0;
   #pragma omp parallel num_threads(2)
   {

runtime/test/ompt/synchronization/test_nest_lock_parallel.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   omp_nest_lock_t nest_lock;
   omp_init_nest_lock(&nest_lock);
 

runtime/test/ompt/tasks/dependences.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 
 #include "callback.h"
 #include <omp.h>   
 #include <math.h>
 #include <unistd.h>
 
 int main()
 {

runtime/test/ompt/tasks/explicit_task.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h> 
 
 int main()
 {
   int condition=0;
   omp_set_nested(0);
   print_frame(0);

runtime/test/ompt/tasks/serialized.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 #include "callback.h"
 #include <omp.h>   
 #include <math.h>
 
 int main()
 {
   omp_set_nested(0);
   print_frame(0);

runtime/test/ompt/worksharing/for/dynamic_split.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %S/base_split.h
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=CHECK-LOOP %S/base_split.h
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 
 #define SCHEDULE dynamic
 #include "base_split.h"

runtime/test/ompt/worksharing/for/guided_split.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %S/base_split.h
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=CHECK-LOOP %S/base_split.h
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 
 #define SCHEDULE guided
 #include "base_split.h"

runtime/test/ompt/worksharing/for/runtime_split.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %S/base_split.h
 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck --check-prefix=CHECK-LOOP %S/base_split.h
 // REQUIRES: ompt
+// UNSUPPORTED: gcc-4, gcc-5, gcc-6, gcc-7
 
 #define SCHEDULE runtime
 #include "base_split.h"

runtime/test/ompt/worksharing/sections.c

 // RUN: %libomp-compile-and-run | %sort-threads | FileCheck %s
 // REQUIRES: ompt
-// GCC generates code that does not distinguish between sections and loops
-// XFAIL: gcc
+// Some compilers generate code that does not distinguish between sections and loops
+// XFAIL: gcc, clang-3, clang-4, clang-5, icc-16, icc-17
+// UNSUPPORTED: icc-18
 
 #include "callback.h"
 #include <omp.h>
 
 int main()
 {
   #pragma omp parallel sections num_threads(2)
   {