[WebAssembly] Fix conflict between ret legalization and sjlj

Authored by loladiro on Aug 5 2019, 2:36 PM.


[WebAssembly] Fix conflict between ret legalization and sjlj

When the WebAssembly backend encounters a return type that doesn't
fit within i32, SelectionDAG performs sret demotion, adding an
additional argument to the start of the function that contains
a pointer to an sret buffer to use instead. However, this conflicts
with the emscripten sjlj lowering pass. There we translate calls like:

	call {i32, i32} @foo()

into (in pseudo-llvm)

	%addr = @foo
	call {i32, i32} @__invoke_{i32,i32}(%addr)

i.e. we perform an indirect call through an extra function.
However, the sret transform now transforms this into
the equivalent of

%addr = @foo
%sret = alloca {i32, i32}
call {i32, i32} @__invoke_{i32,i32}(%sret, %addr)

(while simultaneously translation the implementation of @foo as well).
Unfortunately, this doesn't work out. The __invoke_ ABI expected
the function address to be the first argument, causing crashes.

There is several possible ways to fix this:

  1. Implementing the sret rewrite at the IR level as well and performing it as part of lowering to __invoke
  2. Fixing the wasm backend to recognize that __invoke has a special ABI
  3. A change to the binaryen/emscripten ABI to recognize this situation

This revision implements the middle option, teaching the backend to
treat __invoke_ functions specially in sret lowering. This is achieved

  1. Introducing a new CallingConv ID for invoke functions
  2. When this CallingConv ID is seen in the backend and the first argument is marked as sret (a function pointer would never be marked as sret), swapping the first two arguments.

Reviewed By: tlively, aheejin
Differential Revision: https://reviews.llvm.org/D65463

llvm-svn: 367935