[OpenMP][libomptarget] Enable usage of unified memory for declare target link variables

[gtbercea]

Summary

This patch enables the usage of a host variable on the device for declare target link variables when unified memory is available.

[gtbercea]

• Disable device copies for target link variables under unified memory.
• Add requires init on plugin side.

[gtbercea]

Moved some of the functionality from D60223 to here since it fits better with what this patch is trying to achieve.

[Hahnfeld]

libomptarget/include/omptargetplugin.h

34–36 ↗

(On Diff #200560)

According to init_requires_ty, this should have a return type. I think it's a good idea to have that from the beginning, this might be useful in the future without breaking compatibility.

libomptarget/plugins/generic-elf-64bit/src/rtl.cpp

73–75 ↗

(On Diff #200560)

As before in D60223: You don't need this here and the implementation is optional, so it can go away.

155–159 ↗

(On Diff #200560)

likewise

[gtbercea]

• Remove function from generic plugin.

[gtbercea]

ping

[Hahnfeld]

Looks mostly good, one last question about the return type of __tgt_rtl_init_requires that I noticed only now.

libomptarget/plugins/cuda/src/rtl.cpp

276 ↗

(On Diff #200771)

This will truncate int64_t RequiresFlags to int32_t. Does it make sense to change the return type to int64_t?

[grokos]

libomptarget/plugins/cuda/src/rtl.cpp

276 ↗

(On Diff #200771)

I would suggest that we change the return type to 64 bits. In the future we may need to add extra "requires" attributes that go beyond 32 bits and then we will have to modify this interface function. Let's be proactive and avoid such situations. The same thing happened before with map types and device IDs.

[gtbercea]

• Change return type to 64 bits.

[gtbercea]

libomptarget/plugins/cuda/src/rtl.cpp

276 ↗

(On Diff #200771)

Agreed. Just updated the patch to reflect that.

[gtbercea]

ping

[grokos]

libomptarget/plugins/exports

5 ↗

(On Diff #201728)

One last question because I'm not very familiar with exporting symbols. If a plugin does not implement __tgt_rtl_init_requires (as is the case with generic-elf-64), will there be any problem with exporting the symbol?

[gtbercea]

libomptarget/plugins/exports

5 ↗

(On Diff #201728)

Supporting that method is optional so the function will not get called in that case. In the next chunk of added code you can see that the call to the init_requires() function is guarded:

if (RTL->init_requires)
    RTL->init_requires(RTLRequiresFlags);

[gtbercea]

• Clean-up code.

[grokos]

libomptarget/plugins/exports

5 ↗

(On Diff #201728)

My question was different. I meant, while building libomptarget.rtl.x86_64.so the linker is instructed to export a specific symbol from the exports list but this symbol is not defined in that plugin. Will the linker complain about the missing symbol or just ignore it? I suspect it's the latter, but I just want to make sure.

[gtbercea]

libomptarget/plugins/exports

5 ↗

(On Diff #201728)

I've never run into that particular issue so I suspect it is ignored.

[gtbercea]

libomptarget/plugins/exports

5 ↗

(On Diff #201728)

I commented our the method for the cuda plugin and I didn't get any link errors so it just gets ignored.

[grokos]

OK, there don't seem to be any more comments, looks good.

openmp/trunk/libomptarget/include/omptargetplugin.h

 
 // Return an integer different from zero if the provided device image can be
 // supported by the runtime. The functionality is similar to comparing the
 // result of __tgt__rtl__load__binary to NULL. However, this is meant to be a
 // lightweight query to determine if the RTL is suitable for an image without
 // having to load the library, which can be expensive.
 int32_t __tgt_rtl_is_valid_binary(__tgt_device_image *Image);
 
+// Initialize the requires flags for the device.
+int64_t __tgt_rtl_init_requires(int64_t RequiresFlags);
+
 // Initialize the specified device. In case of success return 0; otherwise
 // return an error code.
 int32_t __tgt_rtl_init_device(int32_t ID);
 
 // Pass an executable image section described by image to the specified
 // device and prepare an address table of target entities. In case of error,
 // return NULL. Otherwise, return a pointer to the built address table.
 // Individual entries in the table may also be NULL, when the corresponding

openmp/trunk/libomptarget/plugins/cuda/src/rtl.cpp

public:
   // OpenMP properties
   std::vector<int> NumTeams;
   std::vector<int> NumThreads;
 
   // OpenMP Environment properties
   int EnvNumTeams;
   int EnvTeamLimit;
 
+  // OpenMP Requires Flags
+  int64_t RequiresFlags;
+
   //static int EnvNumThreads;
   static const int HardTeamLimit = 1<<16; // 64k
   static const int HardThreadLimit = 1024;
   static const int DefaultNumTeams = 128;
   static const int DefaultNumThreads = 128;
 
   // Record entry point associated with device
   void addOffloadEntry(int32_t device_id, __tgt_offload_entry entry) {
#endif // OMPTARGET_DEBUG
     envStr = getenv("OMP_NUM_TEAMS");
     if (envStr) {
       // OMP_NUM_TEAMS has been set
       EnvNumTeams = std::stoi(envStr);
       DP("Parsed OMP_NUM_TEAMS=%d\n", EnvNumTeams);
     } else {
       EnvNumTeams = -1;
     }
+
+    // Default state.
+    RequiresFlags = OMP_REQ_UNDEFINED;
   }
 
   ~RTLDeviceInfoTy() {
     // Close modules
     for (auto &module : Modules)
       if (module) {
         CUresult err = cuModuleUnload(module);
         if (err != CUDA_SUCCESS) {

 #endif
 
 int32_t __tgt_rtl_is_valid_binary(__tgt_device_image *image) {
   return elf_check_machine(image, 190); // EM_CUDA = 190.
 }
 
 int32_t __tgt_rtl_number_of_devices() { return DeviceInfo.NumberOfDevices; }
 
+int64_t __tgt_rtl_init_requires(int64_t RequiresFlags) {
+  DP("Init requires flags to %ld\n", RequiresFlags);
+  DeviceInfo.RequiresFlags = RequiresFlags;
+  return RequiresFlags;
+}
+
 int32_t __tgt_rtl_init_device(int32_t device_id) {
 
   CUdevice cuDevice;
   DP("Getting device %d\n", device_id);
   CUresult err = cuDeviceGet(&cuDevice, device_id);
   if (err != CUDA_SUCCESS) {
     DP("Error when getting CUDA device with id = %d\n", device_id);
     CUDA_ERR_STRING(err);
if (e->size) {
         CUDA_ERR_STRING(err);
         return NULL;
       }
 
       DP("Entry point " DPxMOD " maps to global %s (" DPxMOD ")\n",
           DPxPTR(e - HostBegin), e->name, DPxPTR(cuptr));
       entry.addr = (void *)cuptr;
 
+      if (DeviceInfo.RequiresFlags & OMP_REQ_UNIFIED_SHARED_MEMORY &&
+          e->flags & OMP_DECLARE_TARGET_LINK) {
+        // If unified memory is present any target link variables
+        // can access host addresses directly. There is no longer a
+        // need for device copies.
+        cuMemcpyHtoD(cuptr, e->addr, sizeof(void *));
+        DP("Copy linked variable host address (" DPxMOD ")"
+           "to device address (" DPxMOD ")\n",
+          DPxPTR(*((void**)e->addr)), DPxPTR(cuptr));
+      }
+
       DeviceInfo.addOffloadEntry(device_id, entry);
 
       continue;
     }
 
     CUfunction fun;
     err = cuModuleGetFunction(&fun, cumod, e->name);
 

openmp/trunk/libomptarget/plugins/exports

 VERS1.0 {
   global:
     __tgt_rtl_is_valid_binary;
     __tgt_rtl_number_of_devices;
+    __tgt_rtl_init_requires;
     __tgt_rtl_init_device;
     __tgt_rtl_load_binary;
     __tgt_rtl_data_alloc;
     __tgt_rtl_data_submit;
     __tgt_rtl_data_retrieve;
     __tgt_rtl_data_delete;
     __tgt_rtl_run_target_team_region;
     __tgt_rtl_run_target_region;
   local:
     *;
 };

openmp/trunk/libomptarget/src/device.cpp

int DeviceTy::deallocTgtPtr(void *HstPtrBegin, int64_t Size, bool ForceDelete) {
   }
 
   DataMapMtx.unlock();
   return rc;
 }
 
 /// Init device, should not be called directly.
 void DeviceTy::init() {
+  // Make call to init_requires if it exists for this plugin.
+  if (RTL->init_requires)
+    RTL->init_requires(RTLRequiresFlags);
   int32_t rc = RTL->init_device(RTLDeviceID);
   if (rc == OFFLOAD_SUCCESS) {
     IsInit = true;
   }
 }
 
 /// Thread-safe method to initialize the device only once.
 int32_t DeviceTy::initOnce() {

openmp/trunk/libomptarget/src/rtl.h

struct RTLInfoTy {
   typedef void *(data_alloc_ty)(int32_t, int64_t, void *);
   typedef int32_t(data_submit_ty)(int32_t, void *, void *, int64_t);
   typedef int32_t(data_retrieve_ty)(int32_t, void *, void *, int64_t);
   typedef int32_t(data_delete_ty)(int32_t, void *);
   typedef int32_t(run_region_ty)(int32_t, void *, void **, ptrdiff_t *,
                                  int32_t);
   typedef int32_t(run_team_region_ty)(int32_t, void *, void **, ptrdiff_t *,
                                       int32_t, int32_t, int32_t, uint64_t);
+  typedef int64_t(init_requires_ty)(int64_t);
 
   int32_t Idx;                     // RTL index, index is the number of devices
                                    // of other RTLs that were registered before,
                                    // i.e. the OpenMP index of the first device
                                    // to be registered with this RTL.
   int32_t NumberOfDevices;         // Number of devices this RTL deals with.
 
   void *LibraryHandler;
 
 #ifdef OMPTARGET_DEBUG
   std::string RTLName;
 #endif
 
   // Functions implemented in the RTL.
   is_valid_binary_ty *is_valid_binary;
   number_of_devices_ty *number_of_devices;
   init_device_ty *init_device;
   load_binary_ty *load_binary;
   data_alloc_ty *data_alloc;
   data_submit_ty *data_submit;
   data_retrieve_ty *data_retrieve;
   data_delete_ty *data_delete;
   run_region_ty *run_region;
   run_team_region_ty *run_team_region;
+  init_requires_ty *init_requires;
 
   // Are there images associated with this RTL.
   bool isUsed;
 
   // Mutex for thread-safety when calling RTL interface functions.
   // It is easier to enforce thread-safety at the libomptarget level,
   // so that developers of new RTLs do not have to worry about it.
   std::mutex Mtx;
 
   // The existence of the mutex above makes RTLInfoTy non-copyable.
   // We need to provide a copy constructor explicitly.
   RTLInfoTy()
       : Idx(-1), NumberOfDevices(-1), LibraryHandler(0),
 #ifdef OMPTARGET_DEBUG
         RTLName(),
 #endif
         is_valid_binary(0), number_of_devices(0), init_device(0),
         load_binary(0), data_alloc(0), data_submit(0), data_retrieve(0),
-        data_delete(0), run_region(0), run_team_region(0), isUsed(false),
-        Mtx() {}
+        data_delete(0), run_region(0), run_team_region(0),
+        init_requires(0), isUsed(false), Mtx() {}
 
   RTLInfoTy(const RTLInfoTy &r) : Mtx() {
     Idx = r.Idx;
     NumberOfDevices = r.NumberOfDevices;
     LibraryHandler = r.LibraryHandler;
 #ifdef OMPTARGET_DEBUG
     RTLName = r.RTLName;
 #endif
     is_valid_binary = r.is_valid_binary;
     number_of_devices = r.number_of_devices;
     init_device = r.init_device;
     load_binary = r.load_binary;
     data_alloc = r.data_alloc;
     data_submit = r.data_submit;
     data_retrieve = r.data_retrieve;
     data_delete = r.data_delete;
     run_region = r.run_region;
     run_team_region = r.run_team_region;
+    init_requires = r.init_requires;
     isUsed = r.isUsed;
   }
 };
 
 /// RTLs identified in the system.
 class RTLsTy {
 private:
   // Mutex-like object to guarantee thread-safety and unique initialization

openmp/trunk/libomptarget/src/rtl.cpp

if (!(*((void**) &R.data_delete) = dlsym(
       continue;
     if (!(*((void**) &R.run_region) = dlsym(
               dynlib_handle, "__tgt_rtl_run_target_region")))
       continue;
     if (!(*((void**) &R.run_team_region) = dlsym(
               dynlib_handle, "__tgt_rtl_run_target_team_region")))
       continue;
 
+    // Optional functions
+    *((void**) &R.init_requires) = dlsym(
+        dynlib_handle, "__tgt_rtl_init_requires");
+
     // No devices are supported by this RTL?
     if (!(R.NumberOfDevices = R.number_of_devices())) {
       DP("No devices supported in this RTL\n");
       continue;
     }
 
     DP("Registering RTL %s supporting %d devices!\n",
         R.RTLName.c_str(), R.NumberOfDevices);