Clang performs various recursive operations (such as template instantiation),
and may use non-trivial amounts of stack space in each recursive step (for
instance, due to recursive AST walks). While we try to keep the stack space
used by such steps to a minimum and we have explicit limits on the number of
such steps we perform, it's impractical to guarantee that we won't blow out the
stack on deeply recursive template instantiations on complex ASTs, even with
only a moderately high instantiation depth limit.
The user experience in these cases is generally terrible: we crash with
no hint of what went wrong. Under this patch, we attempt to do better:
- Detect when the stack is nearly exhausted, and produce a warning with a nice template instantiation backtrace, telling the user that we might run slowly or crash.
- For cases where we're forced to trigger recursive template instantiation in arbitrarily-deeply-nested contexts, check whether we're nearly out of stack space and allocate a new stack (by spawning a new thread) after producing the warning.