In D154576#4476300, @4vtomat wrote:

In D154576#4476291, @craig.topper wrote:

Does this only change the calling convention when __attribute__((riscv_vector_cc)) is used? The attribute should be mentioned in the patch description.

Yes it only changes when __attribute__((riscv_vector_cc)) is used, thanks for reminding, I will mention in the description.

Rebase and thank @craig.topper and @kito-cheng for your comments.

Update code based on review.

update as suggested by @craig.topper

Update as suggested by @kito-cheng

add getMaxVLen and encodeSEW helper function

Updated based on reviews

updating based on the latest main branch

I fixed the extra bug found by the review and made some optimizations for the case where AVL is constant.
This is my first patch to the llvm project. Please feel free to point out any irregularities. Thanks.

In D120899#3358636, @craig.topper wrote:

This isn't the only problem with this code. It's broken for CPUs that implement this sentence from the spec. "this permits an implementation to set vl = ceil(AVL / 2) for VLMAX < AVL < 2*VLMAX".

As a concrete example:
If SEW=64 VLMAX is 8 and AVL is 9. The implementation is allowed to return 5 for the VL.
If we multiply the 9 by 2 we create an AVL of 18 for the SEW=32. The implementation is would be allowed to return 9 for the VL. We need it to return 10 to be 2x5 to match the SEW=64 VL.

The only way I see out of this is to insert a SEW=64 vsetvli to explicitly create a VL less than or equal to the SEW=64 VLMAX, then multiply that by 2. This will fix both the overflow you identified and the case I just described.