[OpenMP] libomptarget / CUDA plugin: Check for existing CUDA context before new context creation

[grokos]

Summary

This commit checks for an existing CUDA context (using cuCtxGetCurrent), before creating a new context. If an existing context exists, no new context is created. This change ensures that all allocations made in the program interacting with the openmp offload library are visible in the device context; especially important for CUDA managed allocations (using cudaMallocManaged).

For the existing version, if a context exists, the cuCtxCreate call fails, and hence all further allocations crash.

Thanks to Hashim Sharif, University of Illinois at Urbana, for spotting the problem and submitting the original patch.

[grokos]

It seems there is a bug but it's not related to context creation. False alarm, I'm cancelling the revision. Sorry for that...

[hfinkel]

In D34804#794830, @grokos wrote:

It seems there is a bug but it's not related to context creation. False alarm, I'm cancelling the revision. Sorry for that...

What do you mean?

libomptarget/plugins/cuda/src/rtl.cpp

257

Spaces needed before/after parens.

[grokos]

In D34804#795297, @hfinkel wrote:

What do you mean?

Hashim Sharif had some issues with managed allocations which went away when he applied this patch onto his local branch. However the patch is definitely wrong, it doesn't do what I thought it does. It actually sets the context on all devices to the same context (cuCtxGetCurrent does not retrieve the context from DeviceInfo.Contexts[device_id], rather it gets the context from the current host thread and assigns it to DeviceInfo.Contexts[device_id] - this way all devices get the context of the first device to be initialized...).

I have requested a test case demonstrating Hashim's problem, once I've got got I'll get back to you with an exact description of the problem and a proposed solution.

libomptarget/plugins/cuda/src/rtl.cpp

int32_t __tgt_rtl_init_device(int32_t device_id) {
   DP("Getting device %d\n", device_id);
   CUresult err = cuDeviceGet(&cuDevice, device_id);
   if (err != CUDA_SUCCESS) {
     DP("Error when getting CUDA device with id = %d\n", device_id);
     CUDA_ERR_STRING(err);
     return OFFLOAD_FAIL;
   }
 
+  err = cuCtxGetCurrent(&DeviceInfo.Contexts[device_id]);  
+  if(DeviceInfo.Contexts[device_id] == NULL){
     // Create the context and save it to use whenever this device is selected.
     err = cuCtxCreate(&DeviceInfo.Contexts[device_id], CU_CTX_SCHED_BLOCKING_SYNC,
                     cuDevice);
+  }
+
   if (err != CUDA_SUCCESS) {
     DP("Error when creating a CUDA context\n");
     CUDA_ERR_STRING(err);
     return OFFLOAD_FAIL;
   }
 
   // scan properties to determine number of threads/block and blocks/grid.
   struct cudaDeviceProp Properties;