[OpenMP][libomptarget] Notify the plugins regarding new mapping/unmappings

[kevinsala]

Summary

The NextGen plugins use the information regarding new mapping/unmappings to lock/unlock the corresponding host buffer and speed up the host-device transfers. Handle also the buffers that were already locked externally, i.e., from the application.

[kevinsala]

Fix format.

[tianshilei1992]

mapped/unmapped is a confusing name, especially in the context of OpenMP, you can always check if a pointer is mapped in libomptarget. Of course I understand the "map" here is not OpenMP map.

[kevinsala]

In D142514#4078741, @tianshilei1992 wrote:

mapped/unmapped is a confusing name, especially in the context of OpenMP, you can always check if a pointer is mapped in libomptarget. Of course I understand the "map" here is not OpenMP map.

True. Any recommendation for the rename?

[tianshilei1992]

In D142514#4078818, @kevinsala wrote:

In D142514#4078741, @tianshilei1992 wrote:

mapped/unmapped is a confusing name, especially in the context of OpenMP, you can always check if a pointer is mapped in libomptarget. Of course I understand the "map" here is not OpenMP map.

True. Any recommendation for the rename?

I'm working on Nvidia GPUs, so IMO pinned memory makes sense. Not sure if AMD people would object, though in the implementation you are using that name. ;-)

[ye-luo]

openmp/libomptarget/include/omptargetplugin.h

219

The same as line 215

[jdoerfert]

@tianshilei1992 This is supposed to mean "mapped" in the OpenMP sense. So the plugins are informed if a mapping occurs.

[jdoerfert]

I think this is fine. Two nits and Ye's comment need to be addressed. @tianshilei1992 please accept or suggest a different name. Since we really mean "OpenMP mapped" here, I find the name fitting.

openmp/libomptarget/plugins-nextgen/amdgpu/src/rtl.cpp

1855

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.h

533

Docs

[tianshilei1992]

If it servers as notifying data mapping, __tgt_rtl_notify_data_map or something like that sounds more appropriate to me. __tgt_rtl_data_mapped sounds like checking whether it is mapped, and that's how it confused me initially.

[ye-luo]

In D142514#4078989, @tianshilei1992 wrote:

If it servers as notifying data mapping, __tgt_rtl_notify_data_map or something like that sounds more appropriate to me. __tgt_rtl_data_mapped sounds like checking whether it is mapped, and that's how it confused me initially.

Can be tgt_rtl_data_notify_mapped and tgt_rtl_data_notify_unmapped by keeping __tgt_rtl_data prefix

[ye-luo]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

887

I don't understand this part of code.
The implementation actually lock the host buffer instead of just notifying.

[jdoerfert]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

887

While it is mapped, we want to keep it (by default) pinned. The escape hatch (env var) is missing though.
This allows us to avoid the pin/unpin on the data transfer back and intermediate updates.

[ye-luo]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

887

Unconditionally pinning mapped host memory should be an option to opt-in rather than default.
If I just map 10 scalars, don't pin them, staging can be better.
if users want fine-grain control, use the lock API. For really lazy users, give them the env var to opt-in.

[jdoerfert]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

887

Mapping scalars as firstprivate or as tofrom? The former would not go through this mechanism. The latter makes reasonable sense to pin the page, no?

I agree we need an env var, default is the question.

[ye-luo]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

887

I was referring to map(tofrom) scalars. firstprivte is not considered a map technically. I don't think you want to pay the cost of pinning 10 pages due to 10 scalars. Pinning is very expensive (interacting with OS).

[jdoerfert]

Let's make it opt-in then. And we can use heuristics as well as we go.

[kevinsala]

Changes:

• Fixing comments.
• Re-writing the PinnedAllocationMapTy functions to be clearer and easier to read.
• Renaming RTL API functions.
• Disable automatic locking of mapped host buffers by default. Can be enabled by setting LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS envar.
• Although the previous envar is not set, mapped buffers that were externally locked (e.g, the app allocating HSA memory) are registered into the plugin's map and we use a single-step asynchronous HSA memory transfer. (@ye-luo)

This patch still detects invalid partial overlapping in offloading/complex_reduction.cpp, which I have to fix. The previous patch fails with OpenMC, I've to check if this also fails.

[kevinsala]

openmp/libomptarget/plugins-nextgen/amdgpu/src/rtl.cpp

1855

It's an output parameter. I'll add the proper comment.

[kevinsala]

Fix format.

[kevinsala]

Fixing bug. Now complex reduction tests work.

[tianshilei1992]

openmp/libomptarget/include/omptargetplugin.h

215

__tgt_rtl_notify_data_mapped?

[kevinsala]

openmp/libomptarget/include/omptargetplugin.h

215

This name was proposed by @ye-luo to be consistent with the other data-related API functions:

Can be tgt_rtl_data_notify_mapped and tgt_rtl_data_notify_unmapped by keeping __tgt_rtl_data prefix

[tianshilei1992]

LGTM

[ye-luo]

openmp/libomptarget/src/device.cpp

611

"Size=%" missing "%"

[kevinsala]

Fix debug print

[kevinsala]

The locking of mapped host buffers is disabled by default. Can be enabled through the LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS envar. The envar accepts boolean values (on/off) and a special option:

• off: Do not lock mapped host buffers (default).
• on: Lock mapped host buffers automatically, but do not report lock failures if the plugin fails to lock them.
• mandatory: Lock mapped host buffers automatically and treat locking failures in the plugins as fatal errors. This option may be useful for debugging purposes.

[DavidSpickett]

This is probably a very late report but I happened to be looking at openmp bots for a different reason, some of the error returns in this change are failing to compile:
https://lab.llvm.org/buildbot/#/builders/4/builds/32052/steps/5/logs/stdio

Despite the "gcc" name, this bot appears to use clang-11 which is still supported for building llvm.

This is a simplified version: https://godbolt.org/z/x91h9KcGq

I'm not sure how we're expected to use Expected (no pun intended) in this scenario. There is probably a helper function somewhere.

[jhuber6]

In D142514#4206591, @DavidSpickett wrote:

This is probably a very late report but I happened to be looking at openmp bots for a different reason, some of the error returns in this change are failing to compile:
https://lab.llvm.org/buildbot/#/builders/4/builds/32052/steps/5/logs/stdio

Despite the "gcc" name, this bot appears to use clang-11 which is still supported for building llvm.

This is a simplified version: https://godbolt.org/z/x91h9KcGq

I'm not sure how we're expected to use Expected (no pun intended) in this scenario. There is probably a helper function somewhere.

Needs an explicit move https://godbolt.org/z/Kj1s7od7e.

[jhuber6]

In D142514#4206591, @DavidSpickett wrote:

This is probably a very late report but I happened to be looking at openmp bots for a different reason, some of the error returns in this change are failing to compile:
https://lab.llvm.org/buildbot/#/builders/4/builds/32052/steps/5/logs/stdio

Despite the "gcc" name, this bot appears to use clang-11 which is still supported for building llvm.

This is a simplified version: https://godbolt.org/z/x91h9KcGq

I'm not sure how we're expected to use Expected (no pun intended) in this scenario. There is probably a helper function somewhere.

This should resolve it, thanks for bringing this to my attention https://reviews.llvm.org/rGedc03550063c.

openmp/libomptarget/include/device.h

int32_t submitData(void *TgtPtrBegin, void *HstPtrBegin, int64_t Size,
                      AsyncInfoTy &AsyncInfo);
   // Copy data from device back to host
   int32_t retrieveData(void *HstPtrBegin, void *TgtPtrBegin, int64_t Size,
                        AsyncInfoTy &AsyncInfo);
   // Copy data from current device to destination device directly
   int32_t dataExchange(void *SrcPtr, DeviceTy &DstDev, void *DstPtr,
                        int64_t Size, AsyncInfoTy &AsyncInfo);
 
+  /// Notify the plugin about a new mapping starting at the host address
+  /// \p HstPtr and \p Size bytes.
+  int32_t notifyDataMapped(void *HstPtr, int64_t Size);
+
+  /// Notify the plugin about an existing mapping being unmapped starting at
+  /// the host address \p HstPtr.
+  int32_t notifyDataUnmapped(void *HstPtr);
+
   // Launch the kernel identified by \p TgtEntryPtr with the given arguments.
   int32_t launchKernel(void *TgtEntryPtr, void **TgtVarsPtr,
                        ptrdiff_t *TgtOffsets, const KernelArgsTy &KernelArgs,
                        AsyncInfoTy &AsyncInfo);
 
   /// Synchronize device/queue/event based on \p AsyncInfo and return
   /// OFFLOAD_SUCCESS/OFFLOAD_FAIL when succeeds/fails.
   int32_t synchronize(AsyncInfoTy &AsyncInfo);

openmp/libomptarget/include/omptargetplugin.h

 
 // lock/pin host memory
 int32_t __tgt_rtl_data_lock(int32_t ID, void *HstPtr, int64_t Size,
                             void **LockedPtr);
 
 // unlock/unpin host memory
 int32_t __tgt_rtl_data_unlock(int32_t ID, void *HstPtr);
 
+// Notify the plugin about a new mapping starting at the host address \p HstPtr
+// and \p Size bytes. The plugin may lock/pin that buffer to achieve optimal
+// memory transfers involving that buffer.
+int32_t __tgt_rtl_data_notify_mapped(int32_t ID, void *HstPtr, int64_t Size);
+
+// Notify the plugin about an existing mapping being unmapped, starting at the
+// host address \p HstPtr and \p Size bytes.
+int32_t __tgt_rtl_data_notify_unmapped(int32_t ID, void *HstPtr);
+
 #ifdef __cplusplus
 }
 #endif
 
 #endif // _OMPTARGETPLUGIN_H_

openmp/libomptarget/include/rtl.h

struct RTLInfoTy {
   typedef int32_t(sync_event_ty)(int32_t, void *);
   typedef int32_t(destroy_event_ty)(int32_t, void *);
   typedef int32_t(release_async_info_ty)(int32_t, __tgt_async_info *);
   typedef int32_t(init_async_info_ty)(int32_t, __tgt_async_info **);
   typedef int64_t(init_device_into_ty)(int64_t, __tgt_device_info *,
                                        const char **);
   typedef int32_t(data_lock_ty)(int32_t, void *, int64_t, void **);
   typedef int32_t(data_unlock_ty)(int32_t, void *);
+  typedef int32_t(data_notify_mapped_ty)(int32_t, void *, int64_t);
+  typedef int32_t(data_notify_unmapped_ty)(int32_t, void *);
 
   int32_t Idx = -1;             // RTL index, index is the number of devices
                                 // of other RTLs that were registered before,
                                 // i.e. the OpenMP index of the first device
                                 // to be registered with this RTL.
   int32_t NumberOfDevices = -1; // Number of devices this RTL deals with.
 
   std::unique_ptr<llvm::sys::DynamicLibrary> LibraryHandler;
#endif
   wait_event_ty *wait_event = nullptr;
   sync_event_ty *sync_event = nullptr;
   destroy_event_ty *destroy_event = nullptr;
   init_async_info_ty *init_async_info = nullptr;
   init_device_into_ty *init_device_info = nullptr;
   release_async_info_ty *release_async_info = nullptr;
   data_lock_ty *data_lock = nullptr;
   data_unlock_ty *data_unlock = nullptr;
+  data_notify_mapped_ty *data_notify_mapped = nullptr;
+  data_notify_unmapped_ty *data_notify_unmapped = nullptr;
 
   // Are there images associated with this RTL.
   bool IsUsed = false;
 
   llvm::DenseSet<const __tgt_device_image *> UsedImages;
 
   // Mutex for thread-safety when calling RTL interface functions.
   // It is easier to enforce thread-safety at the libomptarget level,

openmp/libomptarget/plugins-nextgen/amdgpu/src/rtl.cpp

struct AMDGPUDeviceTy : public GenericDeviceTy, AMDGenericDeviceTy {
   }
 
   /// Unpin the host buffer.
   Error dataUnlockImpl(void *HstPtr) override {
     hsa_status_t Status = hsa_amd_memory_unlock(HstPtr);
     return Plugin::check(Status, "Error in hsa_amd_memory_unlock: %s\n");
   }
 
+  /// Check through the HSA runtime whether the \p HstPtr buffer is pinned.
+  Expected<bool> isPinnedPtrImpl(void *HstPtr, void *&BaseHstPtr,
+                                 void *&BaseDevAccessiblePtr,
+                                 size_t &BaseSize) const override {
+    hsa_amd_pointer_info_t Info;
+    Info.size = sizeof(hsa_amd_pointer_info_t);
+
+    hsa_status_t Status =
+        hsa_amd_pointer_info(HstPtr, &Info, /* Allocator */ nullptr,
+                             /* Number of accessible agents (out) */ nullptr,
+                             /* Accessible agents */ nullptr);
+    if (auto Err = Plugin::check(Status, "Error in hsa_amd_pointer_info: %s"))
+      return Err;
+
+    // The buffer may be locked or allocated through HSA allocators. Assume that
+    // the buffer is host pinned if the runtime reports a HSA type.
+    if (Info.type != HSA_EXT_POINTER_TYPE_LOCKED &&
+        Info.type != HSA_EXT_POINTER_TYPE_HSA)
+      return false;
+
+    assert(Info.hostBaseAddress && "Invalid host pinned address");
+    assert(Info.agentBaseAddress && "Invalid agent pinned address");
+    assert(Info.sizeInBytes > 0 && "Invalid pinned allocation size");
+
+    // Save the allocation info in the output parameters.
+    BaseHstPtr = Info.hostBaseAddress;
+    BaseDevAccessiblePtr = Info.agentBaseAddress;
+    BaseSize = Info.sizeInBytes;
+
+    return true;
+  }
+
   /// Submit data to the device (host to device transfer).
   Error dataSubmitImpl(void *TgtPtr, const void *HstPtr, int64_t Size,
                        AsyncInfoWrapperTy &AsyncInfoWrapper) override {
     // Use one-step asynchronous operation when host memory is already pinned.
     if (void *PinnedPtr =
             PinnedAllocs.getDeviceAccessiblePtrFromPinnedBuffer(HstPtr)) {
       AMDGPUStreamTy &Stream = getStream(AsyncInfoWrapper);
       return Stream.pushPinnedMemoryCopyAsync(TgtPtr, PinnedPtr, Size);

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.h

struct EntryTy {
     /// The pointer that devices' driver should use to transfer data from/to the
     /// pinned allocation. In most plugins, this pointer will be the same as the
     /// host pointer above.
     void *DevAccessiblePtr;
 
     /// The size of the pinned allocation.
     size_t Size;
 
+    /// Indicate whether the allocation was locked from outside the plugin, for
+    /// instance, from the application. The externally locked allocations are
+    /// not unlocked by the plugin when unregistering the last user.
+    bool ExternallyLocked;
+
     /// The number of references to the pinned allocation. The allocation should
     /// remain pinned and registered to the map until the number of references
     /// becomes zero.
     mutable size_t References;
 
-    /// Create an entry with the host and device acessible pointers, and the
-    /// buffer size.
-    EntryTy(void *HstPtr, void *DevAccessiblePtr, size_t Size)
+    /// Create an entry with the host and device acessible pointers, the buffer
+    /// size, and a boolean indicating whether the buffer was locked externally.
+    EntryTy(void *HstPtr, void *DevAccessiblePtr, size_t Size,
+            bool ExternallyLocked)
         : HstPtr(HstPtr), DevAccessiblePtr(DevAccessiblePtr), Size(Size),
-          References(1) {}
+          ExternallyLocked(ExternallyLocked), References(1) {}
 
     /// Utility constructor used for std::set searches.
     EntryTy(void *HstPtr)
-        : HstPtr(HstPtr), DevAccessiblePtr(nullptr), Size(0), References(0) {}
+        : HstPtr(HstPtr), DevAccessiblePtr(nullptr), Size(0),
+          ExternallyLocked(false), References(0) {}
   };
 
   /// Comparator of mep entries. Use the host pointer to enforce an order
   /// between entries.
   struct EntryCmpTy {
     bool operator()(const EntryTy &Left, const EntryTy &Right) const {
       return Left.HstPtr < Right.HstPtr;
     }
   };
 
   typedef std::set<EntryTy, EntryCmpTy> PinnedAllocSetTy;
 
   /// The map of host pinned allocations.
   PinnedAllocSetTy Allocs;
 
   /// The mutex to protect accesses to the map.
   mutable std::shared_mutex Mutex;
 
   /// Reference to the corresponding device.
   GenericDeviceTy &Device;
 
-  /// Find an allocation that intersects with \p Buffer pointer. Assume
-  /// the map's mutex is acquired.
-  PinnedAllocSetTy::iterator findIntersecting(const void *Buffer) const {
+  /// Envar that indicates whether mapped host bufffers should be locked
+  /// automatically.
+  BoolEnvar OMPX_LockMappedBuffers;
+
+  /// Find an allocation that intersects with \p HstPtr pointer. Assume the
+  /// map's mutex is acquired.
+  const EntryTy *findIntersecting(const void *HstPtr) const {
     if (Allocs.empty())
-      return Allocs.end();
+      return nullptr;
 
     // Search the first allocation with starting address that is not less than
     // the buffer address.
-    auto It = Allocs.lower_bound({const_cast<void *>(Buffer)});
+    auto It = Allocs.lower_bound({const_cast<void *>(HstPtr)});
 
     // Direct match of starting addresses.
-    if (It != Allocs.end() && It->HstPtr == Buffer)
-      return It;
+    if (It != Allocs.end() && It->HstPtr == HstPtr)
+      return &(*It);
 
     // Not direct match but may be a previous pinned allocation in the map which
     // contains the buffer. Return false if there is no such a previous
     // allocation.
     if (It == Allocs.begin())
-      return Allocs.end();
+      return nullptr;
 
     // Move to the previous pinned allocation.
     --It;
 
     // The buffer is not contained in the pinned allocation.
-    if (advanceVoidPtr(It->HstPtr, It->Size) > Buffer)
-      return It;
+    if (advanceVoidPtr(It->HstPtr, It->Size) > HstPtr)
+      return &(*It);
 
     // None found.
-    return Allocs.end();
+    return nullptr;
+  }
+
+  /// Insert an entry to the map representing a locked buffer. The number of
+  /// references is set to one.
+  Error insertEntry(void *HstPtr, void *DevAccessiblePtr, size_t Size,
+                    bool ExternallyLocked = false);
+
+  /// Erase an existing entry from the map.
+  Error eraseEntry(const EntryTy &Entry);
+
+  /// Register a new user into an entry that represents a locked buffer. Check
+  /// also that the registered buffer with \p HstPtr address and \p Size is
+  /// actually contained into the entry.
+  Error registerEntryUse(const EntryTy &Entry, void *HstPtr, size_t Size);
+
+  /// Unregister a user from the entry and return whether it is the last user.
+  /// If it is the last user, the entry will have to be removed from the map
+  /// and unlock the entry's host buffer (if necessary).
+  Expected<bool> unregisterEntryUse(const EntryTy &Entry);
+
+  /// Indicate whether the first range A fully contains the second range B.
+  static bool contains(void *PtrA, size_t SizeA, void *PtrB, size_t SizeB) {
+    void *EndA = advanceVoidPtr(PtrA, SizeA);
+    void *EndB = advanceVoidPtr(PtrB, SizeB);
+    return (PtrB >= PtrA && EndB <= EndA);
+  }
+
+  /// Indicate whether the first range A intersects with the second range B.
+  static bool intersects(void *PtrA, size_t SizeA, void *PtrB, size_t SizeB) {
+    void *EndA = advanceVoidPtr(PtrA, SizeA);
+    void *EndB = advanceVoidPtr(PtrB, SizeB);
+    return (PtrA <= EndB) and (EndA >= PtrB);
   }
 
 public:
   /// Create the map of pinned allocations corresponding to a specific device.
-  PinnedAllocationMapTy(GenericDeviceTy &Device) : Device(Device) {}
+  PinnedAllocationMapTy(GenericDeviceTy &Device)
+      : Device(Device),
+        OMPX_LockMappedBuffers("LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS", false) {
+  }
 
-  /// Register a host buffer that was recently locked. None of the already
-  /// registered pinned allocations should intersect with this new one. The
-  /// registration requires the host pointer in \p HstPtr, the pointer that the
-  /// devices should use when transferring data from/to the allocation in
-  /// \p DevAccessiblePtr, and the size of the allocation in \p Size. Notice
-  /// that some plugins may use the same pointer for the \p HstPtr and
-  /// \p DevAccessiblePtr. The allocation must be unregistered using the
+  /// Register a buffer that was recently allocated as a locked host buffer.
+  /// None of the already registered pinned allocations should intersect with
+  /// this new one. The registration requires the host pointer in \p HstPtr,
+  /// the device accessible pointer in \p DevAccessiblePtr, and the size of the
+  /// allocation in \p Size. The allocation must be unregistered using the
   /// unregisterHostBuffer function.
   Error registerHostBuffer(void *HstPtr, void *DevAccessiblePtr, size_t Size);
 
   /// Unregister a host pinned allocation passing the host pointer which was
   /// previously registered using the registerHostBuffer function. When calling
-  /// this function, the pinned allocation cannot have any other user.
+  /// this function, the pinned allocation cannot have any other user and will
+  /// not be unlocked by this function.
   Error unregisterHostBuffer(void *HstPtr);
 
   /// Lock the host buffer at \p HstPtr or register a new user if it intersects
   /// with an already existing one. A partial overlapping with extension is not
   /// allowed. The function returns the device accessible pointer of the pinned
   /// buffer. The buffer must be unlocked using the unlockHostBuffer function.
   Expected<void *> lockHostBuffer(void *HstPtr, size_t Size);
 
   /// Unlock the host buffer at \p HstPtr or unregister a user if other users
   /// are still using the pinned allocation. If this was the last user, the
   /// pinned allocation is removed from the map and the memory is unlocked.
   Error unlockHostBuffer(void *HstPtr);
 
+  /// Lock or register a host buffer that was recently mapped by libomptarget.
+  /// This behavior is applied if LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS is
+  /// enabled. Even if not enabled, externally locked buffers are registered
+  /// in order to optimize their transfers.
+  Error lockMappedHostBuffer(void *HstPtr, size_t Size);
+
+  /// Unlock or unregister a host buffer that was unmapped by libomptarget.
+  Error unlockUnmappedHostBuffer(void *HstPtr);
+
   /// Return the device accessible pointer associated to the host pinned
   /// allocation which the \p HstPtr belongs, if any. Return null in case the
   /// \p HstPtr does not belong to any host pinned allocation. The device
   /// accessible pointer is the one that devices should use for data transfers
   /// that involve a host pinned buffer.
   void *getDeviceAccessiblePtrFromPinnedBuffer(const void *HstPtr) const {
     std::shared_lock<std::shared_mutex> Lock(Mutex);
 
     // Find the intersecting allocation if any.
-    auto It = findIntersecting(HstPtr);
-    if (It == Allocs.end())
+    const EntryTy *Entry = findIntersecting(HstPtr);
+    if (!Entry)
       return nullptr;
 
-    const EntryTy &Entry = *It;
-    return advanceVoidPtr(Entry.DevAccessiblePtr,
-                          getPtrDiff(HstPtr, Entry.HstPtr));
+    return advanceVoidPtr(Entry->DevAccessiblePtr,
+                          getPtrDiff(HstPtr, Entry->HstPtr));
   }
 
   /// Check whether a buffer belongs to a registered host pinned allocation.
   bool isHostPinnedBuffer(const void *HstPtr) const {
     std::shared_lock<std::shared_mutex> Lock(Mutex);
 
     // Return whether there is an intersecting allocation.
-    return (findIntersecting(const_cast<void *>(HstPtr)) != Allocs.end());
+    return (findIntersecting(const_cast<void *>(HstPtr)) != nullptr);
   }
 };
 
 /// Class implementing common functionalities of offload devices. Each plugin
 /// should define the specific device class, derive from this generic one, and
 /// implement the necessary virtual function members.
 struct GenericDeviceTy : public DeviceAllocatorTy {
   /// Construct a device with its device id within the plugin, the number of
struct GenericDeviceTy : public DeviceAllocatorTy {
 
   /// Pin host memory to optimize transfers and return the device accessible
   /// pointer that devices should use for memory transfers involving the host
   /// pinned allocation.
   Expected<void *> dataLock(void *HstPtr, int64_t Size) {
     return PinnedAllocs.lockHostBuffer(HstPtr, Size);
   }
 
-  virtual Expected<void *> dataLockImpl(void *HstPtr, int64_t Size) = 0;
-
   /// Unpin a host memory buffer that was previously pinned.
   Error dataUnlock(void *HstPtr) {
     return PinnedAllocs.unlockHostBuffer(HstPtr);
   }
 
+  /// Lock the host buffer \p HstPtr with \p Size bytes with the vendor-specific
+  /// API and return the device accessible pointer.
+  virtual Expected<void *> dataLockImpl(void *HstPtr, int64_t Size) = 0;
+
+  /// Unlock a previously locked host buffer starting at \p HstPtr.
   virtual Error dataUnlockImpl(void *HstPtr) = 0;
 
+  /// Mark the host buffer with address \p HstPtr and \p Size bytes as a mapped
+  /// buffer. This means that libomptarget created a new mapping of that host
+  /// buffer (e.g., because a user OpenMP target map) and the buffer may be used
+  /// as source/destination of memory transfers. We can use this information to
+  /// lock the host buffer and optimize its memory transfers.
+  Error notifyDataMapped(void *HstPtr, int64_t Size) {
+    return PinnedAllocs.lockMappedHostBuffer(HstPtr, Size);
+  }
+
+  /// Mark the host buffer with address \p HstPtr as unmapped. This means that
+  /// libomptarget removed an existing mapping. If the plugin locked the buffer
+  /// in notifyDataMapped, this function should unlock it.
+  Error notifyDataUnmapped(void *HstPtr) {
+    return PinnedAllocs.unlockUnmappedHostBuffer(HstPtr);
+  }
+
+  /// Check whether the host buffer with address \p HstPtr is pinned by the
+  /// underlying vendor-specific runtime (if any). Retrieve the host pointer,
+  /// the device accessible pointer and the size of the original pinned buffer.
+  virtual Expected<bool> isPinnedPtrImpl(void *HstPtr, void *&BaseHstPtr,
+                                         void *&BaseDevAccessiblePtr,
+                                         size_t &BaseSize) const = 0;
+
   /// Submit data to the device (host to device transfer).
   Error dataSubmit(void *TgtPtr, const void *HstPtr, int64_t Size,
                    __tgt_async_info *AsyncInfo);
   virtual Error dataSubmitImpl(void *TgtPtr, const void *HstPtr, int64_t Size,
                                AsyncInfoWrapperTy &AsyncInfoWrapper) = 0;
 
   /// Retrieve data from the device (device to host transfer).
   Error dataRetrieve(void *HstPtr, const void *TgtPtr, int64_t Size,

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

GenericDeviceTy::getExecutionModeForKernel(StringRef Name,
   // Check that the retrieved execution mode is valid.
   if (!GenericKernelTy::isValidExecutionMode(ExecModeGlobal.getValue()))
     return Plugin::error("Invalid execution mode %d for '%s'",
                          ExecModeGlobal.getValue(), Name.data());
 
   return ExecModeGlobal.getValue();
 }
 
+Error PinnedAllocationMapTy::insertEntry(void *HstPtr, void *DevAccessiblePtr,
+                                         size_t Size, bool ExternallyLocked) {
+  // Insert the new entry into the map.
+  auto Res = Allocs.insert({HstPtr, DevAccessiblePtr, Size, ExternallyLocked});
+  if (!Res.second)
+    return Plugin::error("Cannot insert locked buffer entry");
+
+  // Check whether the next entry overlaps with the inserted entry.
+  auto It = std::next(Res.first);
+  if (It == Allocs.end())
+    return Plugin::success();
+
+  const EntryTy *NextEntry = &(*It);
+  if (intersects(NextEntry->HstPtr, NextEntry->Size, HstPtr, Size))
+    return Plugin::error("Partial overlapping not allowed in locked buffers");
+
+  return Plugin::success();
+}
+
+Error PinnedAllocationMapTy::eraseEntry(const EntryTy &Entry) {
+  // Erase the existing entry. Notice this requires an additional map lookup,
+  // but this should not be a performance issue. Using iterators would make
+  // the code more difficult to read.
+  size_t Erased = Allocs.erase({Entry.HstPtr});
+  if (!Erased)
+    return Plugin::error("Cannot erase locked buffer entry");
+  return Plugin::success();
+}
+
+Error PinnedAllocationMapTy::registerEntryUse(const EntryTy &Entry,
+                                              void *HstPtr, size_t Size) {
+  if (!contains(Entry.HstPtr, Entry.Size, HstPtr, Size))
+    return Plugin::error("Partial overlapping not allowed in locked buffers");
+
+  ++Entry.References;
+  return Plugin::success();
+}
+
+Expected<bool> PinnedAllocationMapTy::unregisterEntryUse(const EntryTy &Entry) {
+  if (Entry.References == 0)
+    return Plugin::error("Invalid number of references");
+
+  // Return whether this was the last user.
+  return (--Entry.References == 0);
+}
+
 Error PinnedAllocationMapTy::registerHostBuffer(void *HstPtr,
                                                 void *DevAccessiblePtr,
                                                 size_t Size) {
   assert(HstPtr && "Invalid pointer");
   assert(DevAccessiblePtr && "Invalid pointer");
+  assert(Size && "Invalid size");
 
   std::lock_guard<std::shared_mutex> Lock(Mutex);
 
   // No pinned allocation should intersect.
-  auto Res = Allocs.insert({HstPtr, DevAccessiblePtr, Size});
-  if (!Res.second)
-    return Plugin::error("Cannot register locked buffer");
+  const EntryTy *Entry = findIntersecting(HstPtr);
+  if (Entry)
+    return Plugin::error("Cannot insert entry due to an existing one");
 
-  return Plugin::success();
+  // Now insert the new entry.
+  return insertEntry(HstPtr, DevAccessiblePtr, Size);
 }
 
 Error PinnedAllocationMapTy::unregisterHostBuffer(void *HstPtr) {
   assert(HstPtr && "Invalid pointer");
 
   std::lock_guard<std::shared_mutex> Lock(Mutex);
 
-  // Find the pinned allocation starting at the host pointer address.
-  auto It = Allocs.find({HstPtr});
-  if (It == Allocs.end())
+  const EntryTy *Entry = findIntersecting(HstPtr);
+  if (!Entry)
     return Plugin::error("Cannot find locked buffer");
 
-  const EntryTy &Entry = *It;
+  // The address in the entry should be the same we are unregistering.
+  if (Entry->HstPtr != HstPtr)
+    return Plugin::error("Unexpected host pointer in locked buffer entry");
+
+  // Unregister from the entry.
+  auto LastUseOrErr = unregisterEntryUse(*Entry);
+  if (!LastUseOrErr)
+    return LastUseOrErr.takeError();
 
   // There should be no other references to the pinned allocation.
-  if (Entry.References > 1)
+  if (!(*LastUseOrErr))
     return Plugin::error("The locked buffer is still being used");
 
-  // Remove the entry from the map.
-  Allocs.erase(It);
-
-  return Plugin::success();
+  // Erase the entry from the map.
+  return eraseEntry(*Entry);
 }
 
 Expected<void *> PinnedAllocationMapTy::lockHostBuffer(void *HstPtr,
                                                        size_t Size) {
   assert(HstPtr && "Invalid pointer");
+  assert(Size && "Invalid size");
 
   std::lock_guard<std::shared_mutex> Lock(Mutex);
 
-  auto It = findIntersecting(HstPtr);
+  const EntryTy *Entry = findIntersecting(HstPtr);
 
-  // No intersecting registered allocation found in the map. We must lock and
-  // register the memory buffer into the map.
-  if (It == Allocs.end()) {
-    // First, lock the host buffer and retrieve the device accessible pointer.
-    auto PinnedPtrOrErr = Device.dataLockImpl(HstPtr, Size);
-    if (!PinnedPtrOrErr)
-      return PinnedPtrOrErr.takeError();
-
-    // Then, insert the host buffer entry into the map.
-    auto Res = Allocs.insert({HstPtr, *PinnedPtrOrErr, Size});
-    if (!Res.second)
-      return Plugin::error("Cannot register locked buffer");
+  if (Entry) {
+    // An already registered intersecting buffer was found. Register a new use.
+    if (auto Err = registerEntryUse(*Entry, HstPtr, Size))
+      return Err;
 
-    // Return the device accessible pointer.
-    return *PinnedPtrOrErr;
+    // Return the device accessible pointer with the correct offset.
+    return advanceVoidPtr(Entry->DevAccessiblePtr,
+                          getPtrDiff(HstPtr, Entry->HstPtr));
   }
 
-  const EntryTy &Entry = *It;
+  // No intersecting registered allocation found in the map. First, lock the
+  // host buffer and retrieve the device accessible pointer.
+  auto DevAccessiblePtrOrErr = Device.dataLockImpl(HstPtr, Size);
+  if (!DevAccessiblePtrOrErr)
+    return DevAccessiblePtrOrErr.takeError();
 
-#ifdef OMPTARGET_DEBUG
-  // Do not allow partial overlapping among host pinned buffers.
-  if (advanceVoidPtr(HstPtr, Size) > advanceVoidPtr(Entry.HstPtr, Entry.Size))
-    return Plugin::error("Partial overlapping not allowed in locked memory");
-#endif
-
-  // Increase the number of references.
-  Entry.References++;
+  // Now insert the new entry into the map.
+  if (auto Err = insertEntry(HstPtr, *DevAccessiblePtrOrErr, Size))
+    return Err;
 
-  // Return the device accessible pointer after applying the correct offset.
-  return advanceVoidPtr(Entry.DevAccessiblePtr,
-                        getPtrDiff(HstPtr, Entry.HstPtr));
+  // Return the device accessible pointer.
+  return *DevAccessiblePtrOrErr;
 }
 
 Error PinnedAllocationMapTy::unlockHostBuffer(void *HstPtr) {
   assert(HstPtr && "Invalid pointer");
 
   std::lock_guard<std::shared_mutex> Lock(Mutex);
 
-  auto It = findIntersecting(HstPtr);
-  if (It == Allocs.end())
+  const EntryTy *Entry = findIntersecting(HstPtr);
+  if (!Entry)
     return Plugin::error("Cannot find locked buffer");
 
-  const EntryTy &Entry = *It;
-
-  // Decrease the number of references. No need to do anything if there are
+  // Unregister from the locked buffer. No need to do anything if there are
   // others using the allocation.
-  if (--Entry.References > 0)
+  auto LastUseOrErr = unregisterEntryUse(*Entry);
+  if (!LastUseOrErr)
+    return LastUseOrErr.takeError();
+
+  // No need to do anything if there are others using the allocation.
+  if (!(*LastUseOrErr))
     return Plugin::success();
 
-  // This was the last user of the allocation. Unlock the original locked memory
-  // buffer, which is the host pointer stored in the entry.
-  if (auto Err = Device.dataUnlockImpl(Entry.HstPtr))
+  // This was the last user of the allocation. Unlock the original locked buffer
+  // if it was locked by the plugin. Do not unlock it if it was locked by an
+  // external entity. Unlock the buffer using the host pointer of the entry.
+  if (!Entry->ExternallyLocked)
+    if (auto Err = Device.dataUnlockImpl(Entry->HstPtr))
       return Err;
 
-  // Remove the entry from the map.
-  size_t Erased = Allocs.erase(Entry);
-  if (!Erased)
-    return Plugin::error("Cannot find locked buffer");
+  // Erase the entry from the map.
+  return eraseEntry(*Entry);
+}
 
+Error PinnedAllocationMapTy::lockMappedHostBuffer(void *HstPtr, size_t Size) {
+  assert(HstPtr && "Invalid pointer");
+  assert(Size && "Invalid size");
+
+  std::lock_guard<std::shared_mutex> Lock(Mutex);
+
+  // If previously registered, just register a new user on the entry.
+  const EntryTy *Entry = findIntersecting(HstPtr);
+  if (Entry)
+    return registerEntryUse(*Entry, HstPtr, Size);
+
+  size_t BaseSize;
+  void *BaseHstPtr, *BaseDevAccessiblePtr;
+
+  // Check if it was externally pinned by a vendor-specific API.
+  auto IsPinnedOrErr = Device.isPinnedPtrImpl(HstPtr, BaseHstPtr,
+                                              BaseDevAccessiblePtr, BaseSize);
+  if (!IsPinnedOrErr)
+    return IsPinnedOrErr.takeError();
+
+  // If pinned, just insert the entry representing the whole pinned buffer.
+  if (*IsPinnedOrErr)
+    return insertEntry(BaseHstPtr, BaseDevAccessiblePtr, BaseSize,
+                       /* Externally locked */ true);
+
+  // Not externally pinned. Do nothing if locking of mapped buffers is disabled.
+  if (!OMPX_LockMappedBuffers)
     return Plugin::success();
+
+  // Otherwise, lock the buffer and insert the new entry.
+  auto DevAccessiblePtrOrErr = Device.dataLockImpl(HstPtr, Size);
+  if (!DevAccessiblePtrOrErr)
+    return DevAccessiblePtrOrErr.takeError();
+
+  return insertEntry(HstPtr, *DevAccessiblePtrOrErr, Size);
+}
+
+Error PinnedAllocationMapTy::unlockUnmappedHostBuffer(void *HstPtr) {
+  assert(HstPtr && "Invalid pointer");
+
+  std::lock_guard<std::shared_mutex> Lock(Mutex);
+
+  // Check whether there is any intersecting entry.
+  const EntryTy *Entry = findIntersecting(HstPtr);
+
+  // No entry but automatic locking of mapped buffers is disabled, so
+  // nothing to do.
+  if (!Entry && !OMPX_LockMappedBuffers)
+    return Plugin::success();
+
+  // No entry, but the automatic locking is enabled, so this is an error.
+  if (!Entry)
+    return Plugin::error("Locked buffer not found");
+
+  // There is entry, so unregister a user and check whether it was the last one.
+  auto LastUseOrErr = unregisterEntryUse(*Entry);
+  if (!LastUseOrErr)
+    return LastUseOrErr.takeError();
+
+  // If it is not the last one, there is nothing to do.
+  if (!(*LastUseOrErr))
+    return Plugin::success();
+
+  // Otherwise, if it was the last and the buffer was locked by the plugin,
+  // unlock it.
+  if (!Entry->ExternallyLocked)
+    if (auto Err = Device.dataUnlockImpl(Entry->HstPtr))
+      return Err;
+
+  // Finally erase the entry from the map.
+  return eraseEntry(*Entry);
 }
 
 Error GenericDeviceTy::synchronize(__tgt_async_info *AsyncInfo) {
   if (!AsyncInfo || !AsyncInfo->Queue)
     return Plugin::error("Invalid async info queue");
 
   return synchronizeImpl(*AsyncInfo);
 }
if (Kind == TARGET_ALLOC_HOST)
     if (auto Err = PinnedAllocs.unregisterHostBuffer(TgtPtr))
       return Err;
 
   return Plugin::success();
 }
 
 Error GenericDeviceTy::dataSubmit(void *TgtPtr, const void *HstPtr,
                                   int64_t Size, __tgt_async_info *AsyncInfo) {
   auto Err = Plugin::success();
   AsyncInfoWrapperTy AsyncInfoWrapper(Err, *this, AsyncInfo);
   Err = dataSubmitImpl(TgtPtr, HstPtr, Size, AsyncInfoWrapper);
   return Err;
 }
 
 Error GenericDeviceTy::dataRetrieve(void *HstPtr, const void *TgtPtr,
                                     int64_t Size, __tgt_async_info *AsyncInfo) {
   auto Err = Plugin::success();
if (Err) {
     REPORT("Failure to unlock memory %p: %s\n", Ptr,
            toString(std::move(Err)).data());
     return OFFLOAD_FAIL;
   }
 
   return OFFLOAD_SUCCESS;
 }
 
+int32_t __tgt_rtl_data_notify_mapped(int32_t DeviceId, void *HstPtr,
+                                     int64_t Size) {
+  auto Err = Plugin::get().getDevice(DeviceId).notifyDataMapped(HstPtr, Size);
+  if (Err) {
+    REPORT("Failure to notify data mapped %p: %s\n", HstPtr,
+           toString(std::move(Err)).data());
+    return OFFLOAD_FAIL;
+  }
+
+  return OFFLOAD_SUCCESS;
+}
+
+int32_t __tgt_rtl_data_notify_unmapped(int32_t DeviceId, void *HstPtr) {
+  auto Err = Plugin::get().getDevice(DeviceId).notifyDataUnmapped(HstPtr);
+  if (Err) {
+    REPORT("Failure to notify data unmapped %p: %s\n", HstPtr,
+           toString(std::move(Err)).data());
+    return OFFLOAD_FAIL;
+  }
+
+  return OFFLOAD_SUCCESS;
+}
+
 int32_t __tgt_rtl_data_submit(int32_t DeviceId, void *TgtPtr, void *HstPtr,
                               int64_t Size) {
   return __tgt_rtl_data_submit_async(DeviceId, TgtPtr, HstPtr, Size,
                                      /* AsyncInfoPtr */ nullptr);
 }
 
 int32_t __tgt_rtl_data_submit_async(int32_t DeviceId, void *TgtPtr,
                                     void *HstPtr, int64_t Size,

openmp/libomptarget/plugins-nextgen/cuda/src/rtl.cpp

struct CUDADeviceTy : public GenericDeviceTy {
 
   Expected<void *> dataLockImpl(void *HstPtr, int64_t Size) override {
     // TODO: Register the buffer as CUDA host memory.
     return HstPtr;
   }
 
   Error dataUnlockImpl(void *HstPtr) override { return Plugin::success(); }
 
+  Expected<bool> isPinnedPtrImpl(void *HstPtr, void *&BaseHstPtr,
+                                 void *&BaseDevAccessiblePtr,
+                                 size_t &BaseSize) const override {
+    // TODO: Implement pinning feature for CUDA.
+    return false;
+  }
+
   /// Submit data to the device (host to device transfer).
   Error dataSubmitImpl(void *TgtPtr, const void *HstPtr, int64_t Size,
                        AsyncInfoWrapperTy &AsyncInfoWrapper) override {
     if (auto Err = setContext())
       return Err;
 
     CUstream Stream = getStream(AsyncInfoWrapper);
     if (!Stream)

openmp/libomptarget/plugins-nextgen/generic-elf-64bit/src/rtl.cpp

struct GenELF64DeviceTy : public GenericDeviceTy {
   /// return the same pointer as the device pointer.
   Expected<void *> dataLockImpl(void *HstPtr, int64_t Size) override {
     return HstPtr;
   }
 
   /// Nothing to do when unlocking the buffer.
   Error dataUnlockImpl(void *HstPtr) override { return Plugin::success(); }
 
+  /// Indicate that the buffer is not pinned.
+  Expected<bool> isPinnedPtrImpl(void *HstPtr, void *&BaseHstPtr,
+                                 void *&BaseDevAccessiblePtr,
+                                 size_t &BaseSize) const override {
+    return false;
+  }
+
   /// Submit data to the device (host to device transfer).
   Error dataSubmitImpl(void *TgtPtr, const void *HstPtr, int64_t Size,
                        AsyncInfoWrapperTy &AsyncInfoWrapper) override {
     std::memcpy(TgtPtr, HstPtr, Size);
     return Plugin::success();
   }
 
   /// Retrieve data from the device (device to host transfer).

openmp/libomptarget/src/device.cpp

int DeviceTy::associatePtr(void *HstPtrBegin, void *TgtPtrBegin, int64_t Size) {
   DP("Creating new map entry: HstBase=" DPxMOD ", HstBegin=" DPxMOD
      ", HstEnd=" DPxMOD ", TgtBegin=" DPxMOD ", DynRefCount=%s, "
      "HoldRefCount=%s\n",
      DPxPTR(NewEntry.HstPtrBase), DPxPTR(NewEntry.HstPtrBegin),
      DPxPTR(NewEntry.HstPtrEnd), DPxPTR(NewEntry.TgtPtrBegin),
      NewEntry.dynRefCountToStr().c_str(), NewEntry.holdRefCountToStr().c_str());
   (void)NewEntry;
 
-  return OFFLOAD_SUCCESS;
+  // Notify the plugin about the new mapping.
+  return notifyDataMapped(HstPtrBegin, Size);
 }
 
 int DeviceTy::disassociatePtr(void *HstPtrBegin) {
   HDTTMapAccessorTy HDTTMap = HostDataToTargetMap.getExclusiveAccessor();
 
   auto It = HDTTMap->find(HstPtrBegin);
   if (It != HDTTMap->end()) {
     HostDataToTargetTy &HDTT = *It->HDTT;
     // Mapping exists
     if (HDTT.getHoldRefCount()) {
       // This is based on OpenACC 3.1, sec 3.2.33 "acc_unmap_data", L3656-3657:
       // "It is an error to call acc_unmap_data if the structured reference
       // count for the pointer is not zero."
       REPORT("Trying to disassociate a pointer with a non-zero hold reference "
              "count\n");
     } else if (HDTT.isDynRefCountInf()) {
       DP("Association found, removing it\n");
       void *Event = HDTT.getEvent();
       delete &HDTT;
       if (Event)
         destroyEvent(Event);
       HDTTMap->erase(It);
-      return OFFLOAD_SUCCESS;
+
+      // Notify the plugin about the unmapped memory.
+      return notifyDataUnmapped(HstPtrBegin);
     } else {
       REPORT("Trying to disassociate a pointer which was not mapped via "
              "omp_target_associate_ptr\n");
     }
   } else {
     REPORT("Association not found\n");
   }
 
if (LR.Flags.IsContained ||
     INFO(OMP_INFOTYPE_MAPPING_CHANGED, DeviceID,
          "Creating new map entry with HstPtrBase=" DPxMOD
          ", HstPtrBegin=" DPxMOD ", TgtPtrBegin=" DPxMOD ", Size=%ld, "
          "DynRefCount=%s, HoldRefCount=%s, Name=%s\n",
          DPxPTR(HstPtrBase), DPxPTR(HstPtrBegin), DPxPTR(Ptr), Size,
          Entry->dynRefCountToStr().c_str(), Entry->holdRefCountToStr().c_str(),
          (HstPtrName) ? getNameFromMapping(HstPtrName).c_str() : "unknown");
     TargetPointer = (void *)Ptr;
+
+    // Notify the plugin about the new mapping.
+    if (notifyDataMapped(HstPtrBegin, Size))
+      return {{false /* IsNewEntry */, false /* IsHostPointer */},
+              nullptr /* Entry */,
+              nullptr /* TargetPointer */};
   } else {
     // This entry is not present and we did not create a new entry for it.
     IsPresent = false;
   }
 
   // If the target pointer is valid, and we need to transfer data, issue the
   // data transfer.
   if (TargetPointer && !IsHostPtr && HasFlagTo && (IsNew || HasFlagAlways)) {
int DeviceTy::deallocTgtPtrAndEntry(HostDataToTargetTy *Entry, int64_t Size) {
 
   void *Event = Entry->getEvent();
   if (Event && destroyEvent(Event) != OFFLOAD_SUCCESS) {
     REPORT("Failed to destroy event " DPxMOD "\n", DPxPTR(Event));
     return OFFLOAD_FAIL;
   }
 
   int Ret = deleteData((void *)Entry->TgtPtrBegin);
+
+  // Notify the plugin about the unmapped memory.
+  Ret |= notifyDataUnmapped((void *)Entry->HstPtrBegin);
+
   delete Entry;
 
   return Ret;
 }
 
 /// Init device, should not be called directly.
 void DeviceTy::init() {
   // Make call to init_requires if it exists for this plugin.
if (!AsyncInfo || !RTL->data_exchange_async || !RTL->synchronize) {
     assert(RTL->data_exchange && "RTL->data_exchange is nullptr");
     return RTL->data_exchange(RTLDeviceID, SrcPtr, DstDev.RTLDeviceID, DstPtr,
                               Size);
   }
   return RTL->data_exchange_async(RTLDeviceID, SrcPtr, DstDev.RTLDeviceID,
                                   DstPtr, Size, AsyncInfo);
 }
 
+int32_t DeviceTy::notifyDataMapped(void *HstPtr, int64_t Size) {
+  if (!RTL->data_notify_mapped)
+    return OFFLOAD_SUCCESS;
+
+  DP("Notifying about new mapping: HstPtr=" DPxMOD ", Size=" PRId64 "\n",
+     DPxPTR(HstPtr), Size);
+
+  if (RTL->data_notify_mapped(RTLDeviceID, HstPtr, Size)) {
+    REPORT("Notifiying about data mapping failed.\n");
+    return OFFLOAD_FAIL;
+  }
+  return OFFLOAD_SUCCESS;
+}
+
+int32_t DeviceTy::notifyDataUnmapped(void *HstPtr) {
+  if (!RTL->data_notify_unmapped)
+    return OFFLOAD_SUCCESS;
+
+  DP("Notifying about an unmapping: HstPtr=" DPxMOD "\n", DPxPTR(HstPtr));
+
+  if (RTL->data_notify_unmapped(RTLDeviceID, HstPtr)) {
+    REPORT("Notifiying about data unmapping failed.\n");
+    return OFFLOAD_FAIL;
+  }
+  return OFFLOAD_SUCCESS;
+}
+
 // Run region on device
 int32_t DeviceTy::launchKernel(void *TgtEntryPtr, void **TgtVarsPtr,
                                ptrdiff_t *TgtOffsets,
                                const KernelArgsTy &KernelArgs,
                                AsyncInfoTy &AsyncInfo) {
   return RTL->launch_kernel(RTLDeviceID, TgtEntryPtr, TgtVarsPtr, TgtOffsets,
                             &KernelArgs, AsyncInfo);
 }

openmp/libomptarget/src/omptarget.cpp

for (auto *HostEntriesBegin : PM->HostEntriesBeginRegistrationOrder) {
           HDTTMap->emplace(new HostDataToTargetTy(
               (uintptr_t)CurrHostEntry->addr /*HstPtrBase*/,
               (uintptr_t)CurrHostEntry->addr /*HstPtrBegin*/,
               (uintptr_t)CurrHostEntry->addr +
                   CurrHostEntry->size /*HstPtrEnd*/,
               (uintptr_t)CurrDeviceEntry->addr /*TgtPtrBegin*/,
               false /*UseHoldRefCount*/, CurrHostEntry->name,
               true /*IsRefCountINF*/));
+
+          // Notify about the new mapping.
+          if (Device.notifyDataMapped(CurrHostEntry->addr, CurrHostEntry->size))
+            return OFFLOAD_FAIL;
         }
       }
     }
   }
 
   if (Rc != OFFLOAD_SUCCESS) {
     return Rc;
   }

openmp/libomptarget/src/rtl.cpp

#endif
   *((void **)&RTL.init_async_info) =
       DynLibrary->getAddressOfSymbol("__tgt_rtl_init_async_info");
   *((void **)&RTL.init_device_info) =
       DynLibrary->getAddressOfSymbol("__tgt_rtl_init_device_info");
   *((void **)&RTL.data_lock) =
       DynLibrary->getAddressOfSymbol("__tgt_rtl_data_lock");
   *((void **)&RTL.data_unlock) =
       DynLibrary->getAddressOfSymbol("__tgt_rtl_data_unlock");
+  *((void **)&RTL.data_notify_mapped) =
+      DynLibrary->getAddressOfSymbol("__tgt_rtl_data_notify_mapped");
+  *((void **)&RTL.data_notify_unmapped) =
+      DynLibrary->getAddressOfSymbol("__tgt_rtl_data_notify_unmapped");
 
   RTL.LibraryHandler = std::move(DynLibrary);
 
   // Successfully loaded
   return true;
 }
 
 ////////////////////////////////////////////////////////////////////////////////

openmp/libomptarget/test/lit.cfg

 # Allow running the tests with omptarget debug output
 if 'LIBOMPTARGET_DEBUG' in os.environ:
     config.environment['LIBOMPTARGET_DEBUG'] = os.environ['LIBOMPTARGET_DEBUG']
 
 # Allow running the tests with nextgen plugins when available
 if 'LIBOMPTARGET_NEXTGEN_PLUGINS' in os.environ:
     config.environment['LIBOMPTARGET_NEXTGEN_PLUGINS'] = os.environ['LIBOMPTARGET_NEXTGEN_PLUGINS']
 
+if 'LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS' in os.environ:
+    config.environment['LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS'] = os.environ['LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS']
+
 if 'OMP_TARGET_OFFLOAD' in os.environ:
     config.environment['OMP_TARGET_OFFLOAD'] = os.environ['OMP_TARGET_OFFLOAD']
 
 def append_dynamic_library_path(name, value, sep):
     if name in config.environment:
         config.environment[name] = value + sep + config.environment[name]
     else:
         config.environment[name] = value