[RFC] Initial documentation for declare target indirect support.

[vzakhari]

Summary

Please review proposed implementation of the support for declare target indirect.

[vzakhari]

openmp/libomptarget/docs/declare_target_indirect.md

36

If needed, we can isolate the lookup implementation into a device RTL API, so that FE just generates a lookup call to map the original function address to the device function address.

[jdoerfert]

openmp/libomptarget/docs/declare_target_indirect.md

36

API, please: fp() -> (lookupDevPtr(fp))()

40

How would we translate/map all pointers? I fail to see how we could even identify all (function) pointers (see below). Or did we add wording to the standard to forbid this?

char data[sizeof(fp)];
memcpy(data, &fp, sizeof(fp));

#pragma omp target map(to:data[:sizeof(fp)])
{
  void (*fpc)();
  memcpy(&fpc, data, sizeof(fp));
  fpc(); 
}

Modulo my syntax errors ;)

[vzakhari]

openmp/libomptarget/docs/declare_target_indirect.md

36

Okay, I will add a definition for the lookup API.

40

I agree that in this case there will be no translation. I will remove the words about non-unified_shared_memory "optimization". Does it sound right?

[RaviNarayanaswamy]

openmp/libomptarget/docs/declare_target_indirect.md

40

I agree that in this case there will be no translation. I will remove the words about non-unified_shared_memory "optimization". Does it sound right?

We should get the spec to disallow copy/casting of host pointers and using it to allow the optimization.

[vzakhari]

openmp/libomptarget/docs/declare_target_indirect.md

36

If I just specify the device RTL API that FEs will be able to use, I think I should remove any mentioning of __openmp_offload_function_ptr_map symbol. This becomes an implementation detail of the API for a particular device, so different device compiler and device plugins will be able to implement this different ways.

I can describe __openmp_offload_function_ptr_map as one of the alternative implementations. Does it sound right?

40

I can move this to a TODO section at the end of the doc. Let's agree on the general case, first.

[jdoerfert]

openmp/libomptarget/docs/declare_target_indirect.md

36

sounds good to me.

[dreachem]

openmp/libomptarget/docs/declare_target_indirect.md

40

I agree that in this case there will be no translation. I will remove the words about non-unified_shared_memory "optimization". Does it sound right?

We should get the spec to disallow copy/casting of host pointers and using it to allow the optimization.

This restriction was added for 5.2: "A program must not rely on the value of a function address in a target region except for assignments, comparisons to zero and indirect calls."

So, you can copy pointers around, as in the example posted. You want to forbid this as well?

[vzakhari]

openmp/libomptarget/docs/declare_target_indirect.md

40

I think Ravi only talked about disallowing verbatim copies of function addresses at the boundary of target regions, which will allow the optimization. @RaviNarayanaswamy, can you please reply to the comment?

[RaviNarayanaswamy]

openmp/libomptarget/docs/declare_target_indirect.md

40

Correct. Don't was passing function pointers as non function pointers in map clause. Assigning function pointer to another function pointer should be ok.

[vzakhari]

If there are no more comments, I am going to merge this by the end of the week.

[JonChesterfield]

Let's have the docs change. They can always be updated during implementation.

openmp/libomptarget/docs/declare_target_indirect.md

+# Overview
+
+The indirect clause enables **indirect device invocation** for a procedure:
+> 19 An indirect call to the device version of a procedure on a device other than the host<br>
+> 20 device, through a function pointer (C/C++), a pointer to a member function (C++) or<br>
+> 21 a procedure pointer (Fortran) that refers to the host version of the procedure.
+
+# Compiler support
+### Offload entry metadata (C++ FE)
+
+For each function declared as **declare target indirect** C++ FE generates the following offload metadata:
+
+```c++
+// Entry 0 -> Kind of this type of metadata (2)
+// Entry 1 -> Mangled name of the function.
+// Entry 2 -> Order the entry was created.
+```
+
+The offloading metadata uses new `OffloadEntriesInfoManagerTy::OffloadingEntryInfoKinds::OffloadingEntryInfoDeviceIndirectFunc`  metadata kind.
+
+### Offload entries table
+
+The offload entries table that is created for the host and for each of the device images currently have entries for **declare target** global variables, **omp target** outlined functions and constructor/destructor thunks for **declare target** global variables.
+
+
+Compiler will also produce an entry for each procedure listed in **indirect** clause of **declare target** construct:
+```C++
+struct __tgt_offload_entry {
+  void *addr;       // Pointer to the function
+  char *name;       // Name of the function
+  size_t size;      // 0 for function
+  int32_t flags;    // OpenMPOffloadingDeclareTargetFlags::OMP_DECLARE_TARGET_FPTR
+  int32_t reserved; // Reserved
+};
+```
+
+### Run-time dispatch in device code
+
+When an indirect function call is generated by a FE in **device code** it translates the original function pointer (which may be an address of a host function) into the device function pointer using a translation API, and uses the resulting function pointer for the call.
+
+Original call code:
+
+```
+  %0 = load void ()*, void ()** %fptr.addr
+  call void %0()
+```
+
+Becomes this:
+
+```
+  %0 = load void ()*, void ()** %fptr.addr
+  %1 = bitcast void ()* %0 to i8*
+  %call = call i8* @__kmpc_target_translate_fptr(i8* %1)
+  %fptr_device = bitcast i8* %call to void ()*
+  call void %fptr_device()
+```
+
+Device RTLs must provide the translation API:
+
+```c++
+// Translate \p FnPtr identifying a host function into a function pointer
+// identifying its device counterpart.
+// If \p FnPtr matches an address of any host function
+// declared as 'declare target indirect', then the API
+// must return an address of the same function compiled
+// for the device. If \p FnPtr does not match an address
+// of any host function, then the API returns \p FnPtr
+// unchanged.
+EXTERN void *__kmpc_target_translate_fptr(void *FnPtr);
+```
+
+# Runtime handling of function pointers
+
+`OpenMPOffloadingDeclareTargetFlags::OMP_DECLARE_TARGET_FPTR` is a new flag to distinguish offload entries for function pointers from other function entries.  Unlike other function entries (with `size` equal to 0) `omptarget::InitLibrary()` will establish mapping for function pointer entries in `Device.HostDataToTargetMap`.
+
+For each `OMP_DECLARE_TARGET_FPTR` entry in the offload entries table `libomptarget` creates an entry of the following type:
+
+```c++
+struct __omp_offloading_fptr_map_ty {
+  int64_t host_ptr; // key
+  int64_t tgt_ptr;  // value
+};
+```
+
+Where `host_ptr` is `__tgt_offload_entry::addr` in a **host** offload entry, and `tgt_ptr` is `__tgt_offload_entry::addr` in the corresponding **device** offload entry (which may be found using the populated `Device.HostDataToTargetMap`).
+
+When all `__omp_offloading_function_ptr_map_ty` entries are collected in a single host array, `libomptarget` sorts the table by `host_ptr` values and passes it to the device plugin for registration, if plugin supports optional `__tgt_rtl_set_function_ptr_map` API.
+
+Plugins may provide the following API, if they want to support **declare target indirect** functionality:
+
+```c++
+// Register in a target implementation defined way a table
+// of __omp_offloading_function_ptr_map_ty entries providing
+// mapping between host and device addresses of 'declare target indirect'
+// functions. \p table_size is the number of elements in \p table_host_ptr
+// array.
+EXTERN void __tgt_rtl_set_function_ptr_map(
+    int32_t device_id, uint64_t table_size, __omp_offloading_fptr_map_ty *table_host_ptr);
+```
+
+# Sample implementation
+
+This section describes one of potential implementations.
+
+A FE may define the following global symbols for each translation module containing **declare target indirect**, when compiling this module for a device:
+
+```c++
+// Mapping between host and device functions declared as
+// 'declare target indirect'.
+__attribute__((weak)) struct __omp_offloading_fptr_map_ty {
+  int64_t host_ptr; // key
+  int64_t tgt_ptr;  // value
+} *__omp_offloading_fptr_map_p = 0;
+
+// Number of elements in __omp_offloading_fptr_map_p table.
+__attribute__((weak)) uint64_t __omp_offloading_fptr_map_size = 0;
+```
+
+`__tgt_rtl_set_function_ptr_map(int32_t device_id, uint64_t table_size, __omp_offloading_fptr_map_ty *table_host_ptr)` allocates device memory of size `sizeof(__omp_offloading_fptr_map_ty) * table_size`, and transfers the contents of `table_host_ptr` array into this device memory.  An address of the allocated device memory area is then assigned to `__omp_offloading_fptr_map_p` global variables on the device.  For example, in **CUDA**, a device address of `__omp_offloading_fptr_map_p` may be taken by calling `cuModuleGetGlobal`, and then a pointer-sized data transfer will initialize `__omp_offloading_fptr_map_p` to point to the device copy of `table_host_ptr` array.  `__omp_offloading_fptr_map_size` is assigned to `table_size` the same way.
+
+An alternative implementation of `__tgt_rtl_set_function_ptr_map` may invoke a device kernel that will do the assignments.
+
+`__kmpc_target_translate_fptr(void *FnPtr)` API uses binary search to match `FnPtr` against `host_ptr` inside the device table pointed to by `__omp_offloading_fptr_map_p`.  If the matching key is found, it returns the corresponding `tgt_ptr`, otherwise, it returns `FnPtr`.
+
+# TODO: Optimization for non-unified_shared_memory
+
+If a program does not use **required unified_shared_memory**, and all function pointers are mapped (not a requirement by OpenMP spec), then an implementation may avoid the runtime dispatch code for indirect function calls (i.e. `__kmpc_target_translate_fptr` is not needed) and also `__tgt_rtl_set_function_ptr_map` is not needed.  `libomptarget` will just map the function pointers as regular data pointers via `Device.HostDataToTargetMap`.