After some discussion and experimentation, we have seen that changing
the default number of vector register bits to LMUL=2 has struck a sweet
spot.
Whilst we could be clever here and make the vectorizer smarter about
dynamically selecting an LMUL that
a) Doesn't affect register pressure
b) Suitable for the microarchitecture
we would need to teach its heuristics about RISC-V register grouping
specifics.
Instead this just does the easy, pragmatic thing by changing the default
to a safe value that doesn't affect register pressure signifcantly[1],
but should increase throughput and unlock more interleaving.
[1] Register spilling when compiling sqlite at various levels of
-riscv-v-register-bit-width-lmul:
LMUL=1 2573 spills
LMUL=2 2583 spills
LMUL=4 2819 spills
LMUL=8 3256 spills