[Libomptarget] Full implementation of the target-offload-icv

[grokos]

Summary

This patch implements full support for the target-offload-icv and regulates the library's behavior in case of offload failure. There are three offload kinds: DEFAULT, DISABLED and MANDATORY.

Initially, the offload kind is retrieved from libomp. If the offload kind is set to DEFAULT, then libomptarget behaves as if the ICV were set to MANDATORY if at least one device is detected in the system, otherwise it falls back to DISABLED.

The behavior of libomptarget is defined as follows:

1. If the ICV has been set to MANDATORY: If target offload fails, then libomptarget prints a warning and aborts the application. The warning is printed unconditionally, independently from debug/release configuration or the value of LIBOMPTARGET_DEBUG.
2. If the ICV has been set to DISABLED: Offloading is disabled and all calls to __tgt_* functions return immediately.
3. If the ICV is set to DEFAULT and an offload operation fails, then there are two cases: (i) If there have been previous successful offload operations then libomptarget prints out a warning and terminates the application. It is not safe to fall back onto the host as the host may not have the most up-to-date version of the application's data. (ii) If there are no previous successful offload operations, then the ICV is set to DISABLED and execution (including any subsequent offload operations) continues on the host.

[Hahnfeld]

Will this be defined in the next revisions of the standard? I remember OpenMP 4.5 being unclear about these cases.

[RaviNarayanaswamy]

Openmp has defined OMP_TARGET_OFFLOAD to control offloading, We should be using this

OMP_TARGET_OFFLOAD
6 The OMP_TARGET_OFFLOAD environment variable sets the initial value of the target-offload-var
7 ICV. The value of the OMP_TARGET_OFFLOAD environment variable may be set to one of these
8 values:
9 MANDATORY | DISABLED | DEFAULT
10 The MANDATORY value specifies that a device construct or a device memory routine must execute
11 on a target device. If the device construct cannot execute on its target device, or if a device memory
12 routine fails to execute, a warning is issued and the program execution aborts. Device constructs
13 are exempt from this behavior when an if-clause is present and the if-clause expression evaluates to
14 false.
15 The support of DISABLED is implementation defined. If an implementation supports it, the
16 behavior should be that a device construct must execute on the host. The behavior with this
17 environment value is equivalent to an if clause present on all device constructs, where each of these
18 if clause expressions evaluate to false. Device memory routines behave as if all device number
19 parameters are set to the value returned by omp_get_initial_device(). The
20 omp_get_initial_device() routine returns that no target device is available
21 The DEFAULT value specifies that when one or more target devices are available, the runtime
22 behaves as if this environment variable is set to MANDATORY; otherwise, the runtime behaves as if
23 this environment variable is set to DISABLED.

[grokos]

Right, I just read TR6 and everything is defined there. I'll revise the patch and upload a new diff.

[tlwilmar]

We have a patch coming with libomp-side addition of OMP_TARGET_OFFLOAD envirable plus an internal API so that libomptarget can get the value from libomp. Would it help if we get this patch out faster?

[grokos]

Waiting for the libomp-side patch for OMP_TARGET_OFFLOAD, I removed parsing this env var from within libomptarget and now this revision only implements the internal logic to handle whatever libomptarget will get by querying libomp. (there is already some code checking for OMP_TARGET_OFFLOAD==DISABLED, I'm leaving it there in order not to break systems which make use of this option right now)

[grokos]

Updated the patch to query the target offload kind from libomp instead of parsing OMP_TARGET_OFFLOAD.

[grokos]

Rebase after bug fix.

[Hahnfeld]

This has completely gone under my radar. Be sure to ping changes if there is no review and some reviewers are actively working on other parts of LLVM.

In D44522#1039056, @RaviNarayanaswamy wrote:

21 The DEFAULT value specifies that when one or more target devices are available, the runtime
22 behaves as if this environment variable is set to MANDATORY; otherwise, the runtime behaves as if
23 this environment variable is set to DISABLED.

After reviewing the code (see inline comments) I don't see this behavior implemented: It only falls back to DISABLED, but doesn't raise to MANDATORY. I'm not sure though if there have been changes to the spec after Ravi posted the quote.

Another general comment: This patch doesn't handle API routines, I think they should be affected as well?

libomptarget/src/interface.cpp

28–30

If we are exiting, this shouldn't say WARNING. I think the word failure is enough here?

33–35

Likewise

37

Remove dead code

45–59

I'd prefer eliminating these macros and just putting the check in all entries. This also has the advantage of having precise debugging output.

libomptarget/src/omptarget.cpp

31–38

No need to have this here, just put it in interface.cpp and make it static (don't forget to remove the declaration from the header file!)

[Hahnfeld]

Do we want to get this into 7.0?

[AndreyChurbanov]

In D44522#1175241, @Hahnfeld wrote:

Do we want to get this into 7.0?

Note that corresponding libomp change committed long ago (r328046).

[Hahnfeld]

@grokos do you plan to update this patch? If not I'll try to implement this variable according to TR7 which will be very close to what OpenMP 5.0 will have. It's really annoying that target execution falls back to the host if the device is busy or there is an error in the kernel :-(

[Hahnfeld]

Not needed anymore, superseeded by D50522.

libomptarget/src/interface.cpp

 
 #include "device.h"
 #include "private.h"
 #include "rtl.h"
 
 #include <cassert>
 #include <cstdlib>
 
+static inline void checkOffloadResult(int rc) {
+  if (rc == OFFLOAD_SUCCESS) {
+    HasSuccessfulRuns = true;
+  } else if (TargetOffloadKind == OMP_TGT_OFFLOAD_MANDATORY) {
+    fprintf(stderr, "OMP: WARNING: Target offload failure, terminating "
+        "application\n");
+    exit(OFFLOAD_FAIL);
+  } else if (TargetOffloadKind == OMP_TGT_OFFLOAD_DEFAULT) {
+    if (HasSuccessfulRuns) {
+      fprintf(stderr, "OMP: WARNING: Target offload failure after offload "
+          "success, terminating application due to possible inconsistent state "
+          "of data\n");
+      exit(OFFLOAD_FAIL);
+    } else { // if (!HasSuccessfulRuns)
+      fprintf(stderr, "OMP: WARNING: Target offload failure, falling back to "
+          "the host\n");
+      TargetOffloadKind = OMP_TGT_OFFLOAD_DISABLED;
+    }
+  }
+}
+
+#define CHECK_HOST_FALLBACK_DATA()                                             \
+  {                                                                            \
+    if (TargetOffloadKind == OMP_TGT_OFFLOAD_DISABLED) {                       \
+      DP("Target offload disabled, ignoring \"target data\", "                 \
+          "\"target enter/exit data\" or \"target update\"\n");                \
+      return;                                                                  \
+    }                                                                          \
+  }
+#define CHECK_HOST_FALLBACK_TARGET()                                           \
+  {                                                                            \
+    if (TargetOffloadKind == OMP_TGT_OFFLOAD_DISABLED) {                       \
+      DP("Target offload disabled, executing target region on the host\n");    \
+      return OFFLOAD_FAIL;                                                     \
+    }                                                                          \
+  }
+
 ////////////////////////////////////////////////////////////////////////////////
 /// adds a target shared library to the target execution image
 EXTERN void __tgt_register_lib(__tgt_bin_desc *desc) {
   RTLs.RegisterLib(desc);
 }
 
 ////////////////////////////////////////////////////////////////////////////////
 /// unloads a target shared library
static void cleanup_map(int32_t new_arg_num, void **new_args_base,
   }
 }
 
 /// creates host-to-target data mapping, stores it in the
 /// libomptarget.so internal structure (an entry in a stack of data maps)
 /// and passes the data to the device.
 EXTERN void __tgt_target_data_begin(int64_t device_id, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types) {
+  CHECK_HOST_FALLBACK_DATA();
   DP("Entering data begin region for device %ld with %d mappings\n", device_id,
      arg_num);
 
   // No devices available?
   if (device_id == OFFLOAD_DEVICE_DEFAULT) {
     device_id = omp_get_default_device();
     DP("Use default device id %ld\n", device_id);
   }
 
   if (CheckDeviceAndCtors(device_id) != OFFLOAD_SUCCESS) {
     DP("Failed to get device %ld ready\n", device_id);
+    checkOffloadResult(OFFLOAD_FAIL);
     return;
   }
 
   DeviceTy& Device = Devices[device_id];
 
   // Translate maps
   int32_t new_arg_num;
   void **new_args_base;
   void **new_args;
   int64_t *new_arg_sizes;
   int64_t *new_arg_types;
   translate_map(arg_num, args_base, args, arg_sizes, arg_types, new_arg_num,
       new_args_base, new_args, new_arg_sizes, new_arg_types, false);
 
   //target_data_begin(Device, arg_num, args_base, args, arg_sizes, arg_types);
-  target_data_begin(Device, new_arg_num, new_args_base, new_args, new_arg_sizes,
-      new_arg_types);
+  int rc = target_data_begin(Device, new_arg_num, new_args_base, new_args,
+      new_arg_sizes, new_arg_types);
 
   // Cleanup translation memory
   cleanup_map(new_arg_num, new_args_base, new_args, new_arg_sizes,
       new_arg_types, arg_num, args_base);
+
+  checkOffloadResult(rc);
 }
 
 EXTERN void __tgt_target_data_begin_nowait(int64_t device_id, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types,
     int32_t depNum, void *depList, int32_t noAliasDepNum,
     void *noAliasDepList) {
   if (depNum + noAliasDepNum > 0)
     __kmpc_omp_taskwait(NULL, 0);
 
   __tgt_target_data_begin(device_id, arg_num, args_base, args, arg_sizes,
                           arg_types);
 }
 
 /// passes data from the target, releases target memory and destroys
 /// the host-target mapping (top entry from the stack of data maps)
 /// created by the last __tgt_target_data_begin.
 EXTERN void __tgt_target_data_end(int64_t device_id, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types) {
+  CHECK_HOST_FALLBACK_DATA();
   DP("Entering data end region with %d mappings\n", arg_num);
 
   // No devices available?
   if (device_id == OFFLOAD_DEVICE_DEFAULT) {
     device_id = omp_get_default_device();
   }
 
   RTLsMtx.lock();
   size_t Devices_size = Devices.size();
   RTLsMtx.unlock();
   if (Devices_size <= (size_t)device_id) {
     DP("Device ID  %ld does not have a matching RTL.\n", device_id);
+    checkOffloadResult(OFFLOAD_FAIL);
     return;
   }
 
   DeviceTy &Device = Devices[device_id];
   if (!Device.IsInit) {
-    DP("uninit device: ignore");
+    DP("Uninit device: ignore");
     return;
   }
 
   // Translate maps
   int32_t new_arg_num;
   void **new_args_base;
   void **new_args;
   int64_t *new_arg_sizes;
   int64_t *new_arg_types;
   translate_map(arg_num, args_base, args, arg_sizes, arg_types, new_arg_num,
       new_args_base, new_args, new_arg_sizes, new_arg_types, false);
 
   //target_data_end(Device, arg_num, args_base, args, arg_sizes, arg_types);
-  target_data_end(Device, new_arg_num, new_args_base, new_args, new_arg_sizes,
-      new_arg_types);
+  int rc = target_data_end(Device, new_arg_num, new_args_base, new_args,
+      new_arg_sizes, new_arg_types);
 
   // Cleanup translation memory
   cleanup_map(new_arg_num, new_args_base, new_args, new_arg_sizes,
       new_arg_types, arg_num, args_base);
+
+  checkOffloadResult(rc);
 }
 
 EXTERN void __tgt_target_data_end_nowait(int64_t device_id, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types,
     int32_t depNum, void *depList, int32_t noAliasDepNum,
     void *noAliasDepList) {
   if (depNum + noAliasDepNum > 0)
     __kmpc_omp_taskwait(NULL, 0);
 
   __tgt_target_data_end(device_id, arg_num, args_base, args, arg_sizes,
                         arg_types);
 }
 
 EXTERN void __tgt_target_data_update(int64_t device_id, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types) {
+  CHECK_HOST_FALLBACK_DATA();
   DP("Entering data update with %d mappings\n", arg_num);
 
   // No devices available?
   if (device_id == OFFLOAD_DEVICE_DEFAULT) {
     device_id = omp_get_default_device();
   }
 
   if (CheckDeviceAndCtors(device_id) != OFFLOAD_SUCCESS) {
     DP("Failed to get device %ld ready\n", device_id);
+    checkOffloadResult(OFFLOAD_FAIL);
     return;
   }
 
   DeviceTy& Device = Devices[device_id];
-  target_data_update(Device, arg_num, args_base, args, arg_sizes, arg_types);
+  int rc = target_data_update(Device, arg_num, args_base, args, arg_sizes,
+      arg_types);
+
+  checkOffloadResult(rc);
 }
 
 EXTERN void __tgt_target_data_update_nowait(
     int64_t device_id, int32_t arg_num, void **args_base, void **args,
     int64_t *arg_sizes, int64_t *arg_types, int32_t depNum, void *depList,
     int32_t noAliasDepNum, void *noAliasDepList) {
   if (depNum + noAliasDepNum > 0)
     __kmpc_omp_taskwait(NULL, 0);
 
   __tgt_target_data_update(device_id, arg_num, args_base, args, arg_sizes,
                            arg_types);
 }
 
 EXTERN int __tgt_target(int64_t device_id, void *host_ptr, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types) {
+  CHECK_HOST_FALLBACK_TARGET();
   DP("Entering target region with entry point " DPxMOD " and device Id %ld\n",
      DPxPTR(host_ptr), device_id);
 
   if (device_id == OFFLOAD_DEVICE_DEFAULT) {
     device_id = omp_get_default_device();
   }
 
   if (CheckDeviceAndCtors(device_id) != OFFLOAD_SUCCESS) {
     DP("Failed to get device %ld ready\n", device_id);
+    checkOffloadResult(OFFLOAD_FAIL);
     return OFFLOAD_FAIL;
   }
 
   // Translate maps
   int32_t new_arg_num;
   void **new_args_base;
   void **new_args;
   int64_t *new_arg_sizes;
   int64_t *new_arg_types;
   translate_map(arg_num, args_base, args, arg_sizes, arg_types, new_arg_num,
       new_args_base, new_args, new_arg_sizes, new_arg_types, true);
 
   //return target(device_id, host_ptr, arg_num, args_base, args, arg_sizes,
   //    arg_types, 0, 0, false /*team*/, false /*recursive*/);
   int rc = target(device_id, host_ptr, new_arg_num, new_args_base, new_args,
       new_arg_sizes, new_arg_types, 0, 0, false /*team*/);
 
   // Cleanup translation memory
   cleanup_map(new_arg_num, new_args_base, new_args, new_arg_sizes,
       new_arg_types, arg_num, args_base);
 
+  checkOffloadResult(rc);
   return rc;
 }
 
 EXTERN int __tgt_target_nowait(int64_t device_id, void *host_ptr,
     int32_t arg_num, void **args_base, void **args, int64_t *arg_sizes,
     int64_t *arg_types, int32_t depNum, void *depList, int32_t noAliasDepNum,
     void *noAliasDepList) {
   if (depNum + noAliasDepNum > 0)
     __kmpc_omp_taskwait(NULL, 0);
 
   return __tgt_target(device_id, host_ptr, arg_num, args_base, args, arg_sizes,
                       arg_types);
 }
 
 EXTERN int __tgt_target_teams(int64_t device_id, void *host_ptr,
     int32_t arg_num, void **args_base, void **args, int64_t *arg_sizes,
     int64_t *arg_types, int32_t team_num, int32_t thread_limit) {
+  CHECK_HOST_FALLBACK_TARGET();
   DP("Entering target region with entry point " DPxMOD " and device Id %ld\n",
      DPxPTR(host_ptr), device_id);
 
   if (device_id == OFFLOAD_DEVICE_DEFAULT) {
     device_id = omp_get_default_device();
   }
 
   if (CheckDeviceAndCtors(device_id) != OFFLOAD_SUCCESS) {
     DP("Failed to get device %ld ready\n", device_id);
+    checkOffloadResult(OFFLOAD_FAIL);
     return OFFLOAD_FAIL;
   }
 
   // Translate maps
   int32_t new_arg_num;
   void **new_args_base;
   void **new_args;
   int64_t *new_arg_sizes;
   int64_t *new_arg_types;
   translate_map(arg_num, args_base, args, arg_sizes, arg_types, new_arg_num,
       new_args_base, new_args, new_arg_sizes, new_arg_types, true);
 
   //return target(device_id, host_ptr, arg_num, args_base, args, arg_sizes,
   //              arg_types, team_num, thread_limit, true /*team*/,
   //              false /*recursive*/);
   int rc = target(device_id, host_ptr, new_arg_num, new_args_base, new_args,
       new_arg_sizes, new_arg_types, team_num, thread_limit, true /*team*/);
 
   // Cleanup translation memory
   cleanup_map(new_arg_num, new_args_base, new_args, new_arg_sizes,
       new_arg_types, arg_num, args_base);
 
+  checkOffloadResult(rc);
   return rc;
 }
 
 EXTERN int __tgt_target_teams_nowait(int64_t device_id, void *host_ptr,
     int32_t arg_num, void **args_base, void **args, int64_t *arg_sizes,
     int64_t *arg_types, int32_t team_num, int32_t thread_limit, int32_t depNum,
     void *depList, int32_t noAliasDepNum, void *noAliasDepList) {
   if (depNum + noAliasDepNum > 0)
     __kmpc_omp_taskwait(NULL, 0);
 
   return __tgt_target_teams(device_id, host_ptr, arg_num, args_base, args,
                             arg_sizes, arg_types, team_num, thread_limit);
 }
 
 
 // The trip count mechanism will be revised - this scheme is not thread-safe.
 EXTERN void __kmpc_push_target_tripcount(int64_t device_id,
     uint64_t loop_tripcount) {
   if (device_id == OFFLOAD_DEVICE_DEFAULT) {
     device_id = omp_get_default_device();
   }
 
   if (CheckDeviceAndCtors(device_id) != OFFLOAD_SUCCESS) {
     DP("Failed to get device %ld ready\n", device_id);
+    checkOffloadResult(OFFLOAD_FAIL);
     return;
   }
 
   DP("__kmpc_push_target_tripcount(%ld, %" PRIu64 ")\n", device_id,
       loop_tripcount);
   Devices[device_id].loopTripCnt = loop_tripcount;
 }

libomptarget/src/omptarget.cpp

 
 #include <cassert>
 #include <vector>
 
 #ifdef OMPTARGET_DEBUG
 int DebugLevel = 0;
 #endif // OMPTARGET_DEBUG
 
+/// Target offload ICV - can be MANDATORY, DISABLED or DEFAULT.
+tgt_offload_kind TargetOffloadKind;
+
+// Variable that controls the behavior of libomptarget in case of offload
+// failure when the offload kind is DEFAULT. It is OK to always fail (we can
+// safely fall back to the host, since data on the host is up to date). It is
+// not OK to succeed at first and then fail, as the most up to date data might
+// be on the device, so falling back to the host is not guaranteed to yield
+// correct results. Once an offload has fallen back on the host, every
+// subsequent offload will also be executed on the host.
+bool HasSuccessfulRuns = false;
+
 /// Map global data and execute pending ctors
 static int InitLibrary(DeviceTy& Device) {
   /*
    * Map global data
    */
   int32_t device_id = Device.DeviceID;
   int rc = OFFLOAD_SUCCESS;
 
if (arg_types[i] & OMP_TGT_MAPTYPE_TO) {
 
       if (copy) {
         DP("Moving %" PRId64 " bytes (hst:" DPxMOD ") -> (tgt:" DPxMOD ")\n",
             arg_sizes[i], DPxPTR(HstPtrBegin), DPxPTR(TgtPtrBegin));
         int rt = Device.data_submit(TgtPtrBegin, HstPtrBegin, arg_sizes[i]);
         if (rt != OFFLOAD_SUCCESS) {
           DP("Copying data to device failed.\n");
           rc = OFFLOAD_FAIL;
+          break;
         }
       }
     }
 
     if (arg_types[i] & OMP_TGT_MAPTYPE_PTR_AND_OBJ) {
       DP("Update pointer (" DPxMOD ") -> [" DPxMOD "]\n",
           DPxPTR(Pointer_TgtPtrBegin), DPxPTR(TgtPtrBegin));
       uint64_t Delta = (uint64_t)HstPtrBegin - (uint64_t)HstPtrBase;
       void *TgtPtrBase = (void *)((uint64_t)TgtPtrBegin - Delta);
       int rt = Device.data_submit(Pointer_TgtPtrBegin, &TgtPtrBase,
           sizeof(void *));
       if (rt != OFFLOAD_SUCCESS) {
         DP("Copying data to device failed.\n");
         rc = OFFLOAD_FAIL;
+        break;
       }
       // create shadow pointers for this entry
       Device.ShadowMtx.lock();
       Device.ShadowPtrMap[Pointer_HstPtrBegin] = {HstPtrBase,
           Pointer_TgtPtrBegin, TgtPtrBase};
       Device.ShadowMtx.unlock();
     }
   }
if ((arg_types[i] & OMP_TGT_MAPTYPE_FROM) || DelEntry) {
 
         if (DelEntry || Always || CopyMember) {
           DP("Moving %" PRId64 " bytes (tgt:" DPxMOD ") -> (hst:" DPxMOD ")\n",
               arg_sizes[i], DPxPTR(TgtPtrBegin), DPxPTR(HstPtrBegin));
           int rt = Device.data_retrieve(HstPtrBegin, TgtPtrBegin, arg_sizes[i]);
           if (rt != OFFLOAD_SUCCESS) {
             DP("Copying data from device failed.\n");
             rc = OFFLOAD_FAIL;
+            break;
           }
         }
       }
 
       // If we copied back to the host a struct/array containing pointers, we
       // need to restore the original host pointer values from their shadow
       // copies. If the struct is going to be deallocated, remove any remaining
       // shadow pointer entries for this struct.
if ((arg_types[i] & OMP_TGT_MAPTYPE_FROM) || DelEntry) {
       Device.ShadowMtx.unlock();
 
       // Deallocate map
       if (DelEntry) {
         int rt = Device.deallocTgtPtr(HstPtrBegin, arg_sizes[i], ForceDelete);
         if (rt != OFFLOAD_SUCCESS) {
           DP("Deallocating data from device failed.\n");
           rc = OFFLOAD_FAIL;
+          break;
         }
       }
     }
   }
 
   return rc;
 }
 
 /// Internal function to pass data to/from the target.
-void target_data_update(DeviceTy &Device, int32_t arg_num,
-    void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types) {
+int target_data_update(DeviceTy &Device, int32_t arg_num, void **args_base,
+    void **args, int64_t *arg_sizes, int64_t *arg_types) {
+  int rc = OFFLOAD_SUCCESS;
   // process each input.
   for (int32_t i = 0; i < arg_num; ++i) {
     if ((arg_types[i] & OMP_TGT_MAPTYPE_LITERAL) ||
         (arg_types[i] & OMP_TGT_MAPTYPE_PRIVATE))
       continue;
 
     void *HstPtrBegin = args[i];
     int64_t MapSize = arg_sizes[i];
     bool IsLast;
     void *TgtPtrBegin = Device.getTgtPtrBegin(HstPtrBegin, MapSize, IsLast,
         false);
 
     if (arg_types[i] & OMP_TGT_MAPTYPE_FROM) {
       DP("Moving %" PRId64 " bytes (tgt:" DPxMOD ") -> (hst:" DPxMOD ")\n",
           arg_sizes[i], DPxPTR(TgtPtrBegin), DPxPTR(HstPtrBegin));
-      Device.data_retrieve(HstPtrBegin, TgtPtrBegin, MapSize);
+      int rt = Device.data_retrieve(HstPtrBegin, TgtPtrBegin, MapSize);
+      if (rt != OFFLOAD_SUCCESS) {
+        DP("Copying data from device failed.\n");
+        rc = OFFLOAD_FAIL;
+        break;
+      }
 
       uintptr_t lb = (uintptr_t) HstPtrBegin;
       uintptr_t ub = (uintptr_t) HstPtrBegin + MapSize;
       Device.ShadowMtx.lock();
       for (ShadowPtrListTy::iterator it = Device.ShadowPtrMap.begin();
           it != Device.ShadowPtrMap.end(); ++it) {
         void **ShadowHstPtrAddr = (void**) it->first;
         if ((uintptr_t) ShadowHstPtrAddr < lb)
           continue;
         if ((uintptr_t) ShadowHstPtrAddr >= ub)
           break;
         DP("Restoring original host pointer value " DPxMOD " for host pointer "
             DPxMOD "\n", DPxPTR(it->second.HstPtrVal),
             DPxPTR(ShadowHstPtrAddr));
         *ShadowHstPtrAddr = it->second.HstPtrVal;
       }
       Device.ShadowMtx.unlock();
     }
 
     if (arg_types[i] & OMP_TGT_MAPTYPE_TO) {
       DP("Moving %" PRId64 " bytes (hst:" DPxMOD ") -> (tgt:" DPxMOD ")\n",
           arg_sizes[i], DPxPTR(HstPtrBegin), DPxPTR(TgtPtrBegin));
-      Device.data_submit(TgtPtrBegin, HstPtrBegin, MapSize);
+      int rt = Device.data_submit(TgtPtrBegin, HstPtrBegin, MapSize);
+      if (rt != OFFLOAD_SUCCESS) {
+        DP("Copying data to device failed.\n");
+        rc = OFFLOAD_FAIL;
+        break;
+      }
 
       uintptr_t lb = (uintptr_t) HstPtrBegin;
       uintptr_t ub = (uintptr_t) HstPtrBegin + MapSize;
       Device.ShadowMtx.lock();
       for (ShadowPtrListTy::iterator it = Device.ShadowPtrMap.begin();
           it != Device.ShadowPtrMap.end(); ++it) {
         void **ShadowHstPtrAddr = (void**) it->first;
         if ((uintptr_t) ShadowHstPtrAddr < lb)
           continue;
         if ((uintptr_t) ShadowHstPtrAddr >= ub)
           break;
         DP("Restoring original target pointer value " DPxMOD " for target "
             "pointer " DPxMOD "\n", DPxPTR(it->second.TgtPtrVal),
             DPxPTR(it->second.TgtPtrAddr));
         Device.data_submit(it->second.TgtPtrAddr,
             &it->second.TgtPtrVal, sizeof(void *));
       }
       Device.ShadowMtx.unlock();
     }
   }
+
+  return rc;
 }
 
 /// performs the same actions as data_begin in case arg_num is
 /// non-zero and initiates run of the offloaded region on the target platform;
 /// if arg_num is non-zero after the region execution is done it also
 /// performs the same action as data_update and data_end above. This function
 /// returns 0 if it was able to transfer the execution to a target and an
 /// integer different from zero otherwise.

libomptarget/src/private.h

 #include <cstdint>
 
 extern int target_data_begin(DeviceTy &Device, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types);
 
 extern int target_data_end(DeviceTy &Device, int32_t arg_num, void **args_base,
     void **args, int64_t *arg_sizes, int64_t *arg_types);
 
-extern void target_data_update(DeviceTy &Device, int32_t arg_num,
+extern int target_data_update(DeviceTy &Device, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types);
 
 extern int target(int64_t device_id, void *host_ptr, int32_t arg_num,
     void **args_base, void **args, int64_t *arg_sizes, int64_t *arg_types,
     int32_t team_num, int32_t thread_limit, int IsTeamConstruct);
 
 extern int CheckDeviceAndCtors(int64_t device_id);
 
 // Implemented in libomp, they are called from within __tgt_* functions.
 #ifdef __cplusplus
 extern "C" {
 #endif
 int omp_get_default_device(void) __attribute__((weak));
 int32_t __kmpc_omp_taskwait(void *loc_ref, int32_t gtid) __attribute__((weak));
+int __kmpc_get_target_offload() __attribute__((weak));
 #ifdef __cplusplus
 }
 #endif
 
 #ifdef OMPTARGET_DEBUG
 extern int DebugLevel;
 
 #define DP(...) \
   do { \
     if (DebugLevel > 0) { \
       DEBUGP("Libomptarget", __VA_ARGS__); \
     } \
   } while (false)
 #else // OMPTARGET_DEBUG
 #define DP(...) {}
 #endif // OMPTARGET_DEBUG
 
+// Target offload ICV - must match definition in libomp
+enum tgt_offload_kind {
+  OMP_TGT_OFFLOAD_DISABLED = 0,
+  OMP_TGT_OFFLOAD_DEFAULT = 1,
+  OMP_TGT_OFFLOAD_MANDATORY = 2
+};
+extern tgt_offload_kind TargetOffloadKind;
+
+// Variable that controls the behavior in case of offload failure
+extern bool HasSuccessfulRuns;
+
 #endif

libomptarget/src/rtl.cpp

 
 void RTLsTy::LoadRTLs() {
 #ifdef OMPTARGET_DEBUG
   if (char *envStr = getenv("LIBOMPTARGET_DEBUG")) {
     DebugLevel = std::stoi(envStr);
   }
 #endif // OMPTARGET_DEBUG
 
-  // Parse environment variable OMP_TARGET_OFFLOAD (if set)
-  char *envStr = getenv("OMP_TARGET_OFFLOAD");
-  if (envStr && !strcmp(envStr, "DISABLED")) {
+  // Query libomp about the kind of target offload
+  TargetOffloadKind = (tgt_offload_kind) __kmpc_get_target_offload();
+  if (TargetOffloadKind == OMP_TGT_OFFLOAD_DISABLED) {
     DP("Target offloading disabled by environment\n");
     return;
   }
 
   DP("Loading RTLs...\n");
 
+  bool HasDevices = false;
+
   // Attempt to open all the plugins and, if they exist, check if the interface
   // is correct and if they are supporting any devices.
   for (auto *Name : RTLNames) {
     DP("Loading library '%s'...\n", Name);
     void *dynlib_handle = dlopen(Name, RTLD_NOW);
 
     if (!dynlib_handle) {
       // Library does not exist or cannot be found.
#endif
     if (!(R.NumberOfDevices = R.number_of_devices())) {
       DP("No devices supported in this RTL\n");
       continue;
     }
 
     DP("Registering RTL %s supporting %d devices!\n",
         R.RTLName.c_str(), R.NumberOfDevices);
 
+    HasDevices = true;
+
     // The RTL is valid! Will save the information in the RTLs list.
     AllRTLs.push_back(R);
   }
 
   DP("RTLs loaded!\n");
 
+  // At this point, if the target-offload-icv is DEFAULT and no devices have
+  // been found, the icv falls back to DISABLED.
+  if (TargetOffloadKind == OMP_TGT_OFFLOAD_DEFAULT && !HasDevices) {
+    TargetOffloadKind = OMP_TGT_OFFLOAD_DISABLED;
+    DP("No offload devices found, target-offload-icv set to DISABLED\n");
+  }
+
   return;
 }
 
 ////////////////////////////////////////////////////////////////////////////////
 // Functionality for registering libs
 
 static void RegisterImageIntoTranslationTable(TranslationTable &TT,
     RTLInfoTy &RTL, __tgt_device_image *image) {
for (int32_t i = 0; i < RTL->NumberOfDevices; ++i) {
     Device.PendingGlobalsMtx.unlock();
   }
 }
 
 void RTLsTy::RegisterLib(__tgt_bin_desc *desc) {
   // Attempt to load all plugins available in the system.
   std::call_once(initFlag, &RTLsTy::LoadRTLs, this);
 
+  // Return immediately if offloading has been disabled
+  if (TargetOffloadKind == OMP_TGT_OFFLOAD_DISABLED) {
+    DP("Target offload disabled, skipping registering library\n");
+    return;
+  }
+
   RTLsMtx.lock();
   // Register the images with the RTLs that understand them, if any.
   for (int32_t i = 0; i < desc->NumDeviceImages; ++i) {
     // Obtain the image.
     __tgt_device_image *img = &desc->DeviceImages[i];
 
     RTLInfoTy *FoundRTL = NULL;
 
void RTLsTy::RegisterLib(__tgt_bin_desc *desc) {
   }
   RTLsMtx.unlock();
 
 
   DP("Done registering entries!\n");
 }
 
 void RTLsTy::UnregisterLib(__tgt_bin_desc *desc) {
+  // Return immediately if offloading has been disabled
+  if (TargetOffloadKind == OMP_TGT_OFFLOAD_DISABLED) {
+    DP("Target offload disabled, skipping unregister library\n");
+    return;
+  }
+
   DP("Unloading target library!\n");
 
   RTLsMtx.lock();
   // Find which RTL understands each image, if any.
   for (int32_t i = 0; i < desc->NumDeviceImages; ++i) {
     // Obtain the image.
     __tgt_device_image *img = &desc->DeviceImages[i];