Fix hierarchical barrier deadlock for infinite blocktime

[Nawrin]

Summary

This patch fixes the deadlock in hierarchical barrier for infinite blocktime by adjusting the offset value of leaf kids to match the value of leaf state.

[jdoerfert]

Can we have a test for this?

[Nawrin]

Add env in barrier test to use hierarchical barrier instead of the default barrier

[jlpeyton]

one comment.

openmp/runtime/test/barrier/omp_barrier.c

2 ↗

(On Diff #302412)

Can you add the KMP_BLOCKTIME=infinite envirable as well?

[Nawrin]

Patch integrated with https://reviews.llvm.org/D94241

openmp/runtime/src/kmp_barrier.cpp

if (thr_bar->my_level || __kmp_dflt_blocktime != KMP_MAX_BLOCKTIME ||
         !thr_bar->use_oncore_barrier) { // Parent is waiting on my b_arrived
       // flag; release it
       ANNOTATE_BARRIER_BEGIN(this_thr);
       kmp_flag_64 flag(&thr_bar->b_arrived, other_threads[thr_bar->parent_tid]);
       flag.release();
     } else {
       // Leaf does special release on "offset" bits of parent's b_arrived flag
       thr_bar->b_arrived = team->t.t_bar[bt].b_arrived + KMP_BARRIER_STATE_BUMP;
-      kmp_flag_oncore flag(&thr_bar->parent_bar->b_arrived, thr_bar->offset);
+      kmp_flag_oncore flag(&thr_bar->parent_bar->b_arrived,
+                           thr_bar->offset + 1);
       flag.set_waiter(other_threads[thr_bar->parent_tid]);
       flag.release();
     }
   } else { // Master thread needs to update the team's b_arrived value
     team->t.t_bar[bt].b_arrived = new_state;
     KA_TRACE(20, ("__kmp_hierarchical_barrier_gather: T#%d(%d:%d) set team %d "
                   "arrived(%p) = %llu\n",
                   gtid, team->t.t_id, tid, team->t.t_id,
if (!thr_bar->use_oncore_barrier ||
       ANNOTATE_BARRIER_END(this_thr);
       TCW_8(thr_bar->b_go,
             KMP_INIT_BARRIER_STATE); // Reset my b_go flag for next time
     } else { // Thread barrier data is initialized, this is a leaf, blocktime is
       // infinite, not nested
       // Wait on my "offset" bits on parent's b_go flag
       thr_bar->wait_flag = KMP_BARRIER_PARENT_FLAG;
       kmp_flag_oncore flag(&thr_bar->parent_bar->b_go, KMP_BARRIER_STATE_BUMP,
-                           thr_bar->offset, bt,
+                           thr_bar->offset + 1, bt,
                            this_thr USE_ITT_BUILD_ARG(itt_sync_obj));
       flag.wait(this_thr, TRUE);
       if (thr_bar->wait_flag ==
           KMP_BARRIER_SWITCHING) { // Thread was switched to own b_go
         TCW_8(thr_bar->b_go,
               KMP_INIT_BARRIER_STATE); // Reset my b_go flag for next time
       } else { // Reset my bits on parent's b_go flag
         (RCAST(volatile char *,
-               &(thr_bar->parent_bar->b_go)))[thr_bar->offset] = 0;
+               &(thr_bar->parent_bar->b_go)))[thr_bar->offset + 1] = 0;
       }
     }
     thr_bar->wait_flag = KMP_BARRIER_NOT_WAITING;
     // Early exit for reaping threads releasing forkjoin barrier
     if (bt == bs_forkjoin_barrier && TCR_4(__kmp_global.g.g_done))
       return;
     // The worker thread may now assume that the team is valid.
     team = __kmp_threads[gtid]->th.th_team;