Clarify how musttail can be used to create forwarding thunks
Maybe worth explicitly stating that this includes registers which would not normally be used to pass arguments as part of a varargs argument list?
Maybe it's also worth adding a restriction that "thunk" functions aren't allowed to call va_start, so it's clear that the functions aren't true varargs?
This could probably be clarified.
Both "tail" and "musttail" imply that the callee doesn't access any va_list generated by a va_start call in the caller. This is analogous to the restriction on accessing allocas, applied to the memory implicitly allocated in the stack frame.
Beyond that, for "tail", the expected behavior for varargs is obvious: each call accesses its own argument list.
It's not clear what we expect to happen for musttail calls which are varargs, but not in a function marked "thunk". Is it legal to pass any varargs arguments? (I guess no, because we couldn't lower it in general.) If not, what happens if the callee calls va_start? Does it get variadic arguments from the caller's caller?
Is the "bitcasting" rule still relevant, given we now explicitly state that "thunk" functions have special rules?
I'll add something.
Why should we make it illegal for thunks to call va_start? I think as long as the registers and memory get forwarded, the thunk can examine the variadic arguments.
I think we mainly added "thunk" to stop instcombine from messing things up when it could devirtualize and see the return value types. In the backend, we don't look at the "thunk" attribute at all.
The problem is that the "variadic" arguments to a thunk aren't necessarily passed the same way a true variadic argument list would be passed. For example, on Windows, variadic functions never use floating-point registers for arguments. So even if it's legal to call va_start, we can't promise the result is meaningful.
The only reason the backend doesn't look at the "thunk" attribute is that it's using the presence of "musttail" as a proxy for whether the caller is a thunk. The result of getForwardedMustTailRegParms() isn't meaningful in non-thunk functions.