This code was copied from SVE and modified for RVV. For SVE, there
is only one size for builtin types so they didn't need to check
the size. For RVV, due to LMUL there are 7 different sizes of builtin
types so we do need to check the size.
I'm not sure we should have lax vector conversions at all for RVV.
That appears to be contributing to https://github.com/llvm/llvm-project/issues/64404
This patch at least fixes the obvious correctness issue.
This should be backported to LLVM 17.