Add missing memory barrier for queuing locks

[Hahnfeld]

Summary

Otherwise I see hangs in the omp_single_copyprivate test when
compiling in release mode. With the debug assertions, I get a
failure head > 0 && tail > 0.

[Hahnfeld]

Looks like this also fixes https://bugs.llvm.org/show_bug.cgi?id=26199.

[pawosm01]

+1, although I ran this omp_single_copyprivate.c example in a loop 10000 times on two different AArch64 machines (ARMv8 and ARMv8.1) and didn't observe any hangs, before and after applying this patch.

[Hahnfeld]

In D40722#949468, @pawosm01 wrote:

+1, although I ran this omp_single_copyprivate.c example in a loop 10000 times on two different AArch64 machines (ARMv8 and ARMv8.1) and didn't observe any hangs, before and after applying this patch.

That's the problem with memory consistency... Did you add the num_threads clause when testing without the memory barrier? This made it much more likely for me

[pawosm01]

I tried various number of threads with OMP_NUM_THREADS (in range between 4 and 96 on 96 cores dual socket ARMv8.1 machine, and between 2 and 4 on 4 cores single socket ARMv8 machine) and enforced affinity with OMP_PROC_BIND=spread, so I guess it answers your question.

[simoatze]

In D40722#942168, @Hahnfeld wrote:

Looks like this also fixes https://bugs.llvm.org/show_bug.cgi?id=26199.

@Hahnfeld it does, I was looking into this bug here at PGI and I came across your last fix.
Looks like the hang does not happen anymore, thanks!

openmp/trunk/runtime/src/kmp_lock.cpp

 
 #if OMPT_SUPPORT
 /* nothing to do - no other thread is trying to shift blame */
 #endif
         return KMP_LOCK_RELEASED;
       }
       dequeued = FALSE;
     } else {
+      KMP_MB();
       tail = *tail_id_p;
       if (head == tail) { /* only one thread on the queue */
 #ifdef DEBUG_QUEUING_LOCKS
         if (head <= 0)
           __kmp_dump_queuing_lock(this_thr, gtid, lck, head, tail);
 #endif
         KMP_DEBUG_ASSERT(head > 0);
 

openmp/trunk/runtime/test/worksharing/single/omp_single_copyprivate.c

 // RUN: %libomp-compile-and-run
 #include "omp_testsuite.h"
 
 #define DEBUG_TEST 0
 
 int j;
 #pragma omp threadprivate(j)
 
 int test_omp_single_copyprivate()
 {
   int result;
   int nr_iterations;
 
   result = 0;
   nr_iterations = 0;
-  #pragma omp parallel
+  #pragma omp parallel num_threads(4)
   {
     int i;
     for (i = 0; i < LOOPCOUNT; i++)
     {
 #if DEBUG_TEST
          int thread;
          thread = omp_get_thread_num ();
 #endif