[OPENMP][NVPTX]Fixed initialization of the data-sharing interface.

[ABataev]

Summary

Avoid using of the atomic loop to wait for the completion of the
data-sharing interface initialization, use __shfl_sync instead for the
communication within the warp to signal other threads in the warp about
completion of the initialization.

[grokos]

libomptarget/deviceRTLs/nvptx/src/data_sharing.cu

449

Isn't FrameP always 8-bytes long?

[ABataev]

libomptarget/deviceRTLs/nvptx/src/data_sharing.cu

449

What if the architecture is 32 bits?

[grokos]

libomptarget/deviceRTLs/nvptx/src/data_sharing.cu

449

That would be true if we compiled the RTL on a 32-bit host system, but - at least at the moment - only 64-bit hosts are supported by libomptarget (x86_64, ppc64, aarch64). So the 32-bit case is unreachable, although admittedly it doesn't hurt to have this discrimination here as a safeguard for the future. On the other hand, how probable is it that we will ever support 32-bit systems? Anyway, the patch looks good, I'm leaving it up to you to decide whether the 32-bit scenario should be considered here.

libomptarget/deviceRTLs/nvptx/src/data_sharing.cu

INLINE void* data_sharing_push_stack_common(size_t PushSize) {
   bool IsWarpMaster = (getThreadId() % WARPSIZE) == 0;
 
   // Add worst-case padding to DataSize so that future stack allocations are
   // correctly aligned.
   const size_t Alignment = 8;
   PushSize = (PushSize + (Alignment - 1)) / Alignment * Alignment;
 
   // Frame pointer must be visible to all workers in the same warp.
-  unsigned WID = getWarpId();
-  void *volatile &FrameP = DataSharingState.FramePtr[WID];
+  const unsigned WID = getWarpId();
+  void *FrameP = 0;
+  const int32_t CurActive = getActiveThreadsMask();
 
   if (IsWarpMaster) {
     // SlotP will point to either the shared memory slot or an existing
     // global memory slot.
     __kmpc_data_sharing_slot *&SlotP = DataSharingState.SlotPtr[WID];
     void *&StackP = DataSharingState.StackPtr[WID];
 
     // Check if we have room for the data in the current slot.
if (EndAddress < RequestedEndAddress) {
 
       // Make previous slot point to the newly allocated slot.
       SlotP->Next = NewSlot;
       // The current slot becomes the new slot.
       SlotP = NewSlot;
       // The stack pointer always points to the next free stack frame.
       StackP = &NewSlot->Data[0] + PushSize;
       // The frame pointer always points to the beginning of the frame.
-      FrameP = &NewSlot->Data[0];
+      FrameP = DataSharingState.FramePtr[WID] = &NewSlot->Data[0];
     } else {
       // Add the data chunk to the current slot. The frame pointer is set to
       // point to the start of the new frame held in StackP.
-      FrameP = StackP;
+      FrameP = DataSharingState.FramePtr[WID] = StackP;
       // Reset stack pointer to the requested address.
       StackP = (void *)RequestedEndAddress;
     }
-  } else {
-    while (!FrameP);
   }
+  // Get address from lane 0.
+  ((int *)&FrameP)[0] = __SHFL_SYNC(CurActive, ((int *)&FrameP)[0], 0);
+  if (sizeof(FrameP) == 8)
+    ((int *)&FrameP)[1] = __SHFL_SYNC(CurActive, ((int *)&FrameP)[1], 0);
 
   return FrameP;
 }
 
 EXTERN void* __kmpc_data_sharing_coalesced_push_stack(size_t DataSize,
     int16_t UseSharedMemory) {
   return data_sharing_push_stack_common(DataSize);
 }