Stack allocate at most two ScalarAssign elements. Using the default number of inlined elements triggered a static assert in some setups (https://reviews.llvm.org/D102075).
On some systems (windows debug) ScalarAssign is > 256 bytes. On Linux we are slightly below. For example, since the nested std::vector carries more debug info. In case the struct continuous to grow we probably want to switch to std::vector.