Current !callback metadata assumes that arguments of callback function
are encoded in broker function arguments, as the following example shows:
void callback(int, float, double); void broker(void(*cb)(int, float, double), int, float, double);
We call the argument encoding as "flat mode". However, it is not always the
case. For some programming model, such as CUDA and OpenMP offloading, arguments
are "stacked":
void callback(int *, float *, double *); void broker(void(*cb)(int *, float *, double *), void **args, size_t size) { // ... cb(args[0], args[1], args[2]); }
Let's call this encode as "stacked mode" (a more approporiate name is welcomed).
Current !callback metadata cannot handle this case very well. In fact, we can
establish connection between callback function arguments and args passed to
the broker function via the use chain of args. For example, before we call the
broker function, we need to construct args by setting its every element. In
this way, we can know which pointer eventually corresponds to which callback
function argument.
In order to do that, we need to tell apart the two different parameter encoding
modes. In this patch, the !callback metadata is updated in the following way:
- The first operand is an i64 value, representing parameter encoding mode. 0 is "flat mode", and 1 is "stacked mode".
- The second operand is the callback function index, same as the previous first operand.
- The next one or more operands are still parameter encoding with a little difference. If the encoding mode is 0, it is same as current way. If the mode is 1, there must only be one operand: the index of the base pointer (e.g. the index of void **args above).
- The last operand is still for the variadic arguments.
P.S. People might argue in CUDA or OpenMP offloading, the broker function
actually doesn't accept a callback function pointer. It's usually a global
symbol which can be used to find the entry kernel function during the runtime.
More important, the kernel function (or "callback" function) cannot be seen in
the host module at all. What's the point of doing this? Yes, that's true for now.
We're working on the heterogenous module, which basically will "merge" (or "link")
kernel moduel and host module together. In this way, we can optimize them with
the information from both host and kernel side. We can bridge the kernel function
and broker function via the global symbol somehow to make the kernel function a
"virtual" callback function. Although pointers in void **args will not be used
by the kernel function directly (as shown in the example above), the real
arguments passed to the kernel function are 1:1 mapped from pointers in args.
Therefore the correspondence still exists.
I'm not certain it's reasonable to turn this into an error given that this attribute already exists in the wild. This basically breaks all users of the attribute.