This is an archive of the discontinued LLVM Phabricator instance.

[X86] Don't allow illegal vector types to return by direct value on x86-64.
Needs ReviewPublic

Authored by craig.topper on Oct 30 2018, 9:52 PM.

Details

Summary

The backend can't lower this correctly and will try to split the return value into multiple registers.

This patches forces it to return via memory similar to what was already done for arguments.

Fixes PR39501.

Diff Detail

Event Timeline

craig.topper created this revision.Oct 30 2018, 9:52 PM
hjl.tools added inline comments.
lib/CodeGen/TargetInfo.cpp
2862

Doesn't -m32 have the same issue?

craig.topper retitled this revision from [X86] Don't allow illegal vector types to return by direct value. to [X86] Don't allow illegal vector types to return by direct value on x86-64..Oct 31 2018, 10:22 AM

Retitling to just the x86-64 case. 32-bit mode has issues on arguments too I think and will need more work. The IsIllegalVectorType function is a member of the X86_64ABIInfo so we need to refactor or add a new one for 32-bit.

As far as I know, according to the ABI docs, it's impossible to return a 256-bit vector from a function if AVX is disabled.

Given that we aren't rejecting the testcase, we're effectively implementing a new "no-AVX" ABI variant. That variant is not defined anywhere, so we might as well pick the fastest convention, returning the value in registers. I'm not sure why you would want to return the vector in memory instead.

As far as I know, according to the ABI docs, it's impossible to return a 256-bit vector from a function if AVX is disabled.

Given that we aren't rejecting the testcase, we're effectively implementing a new "no-AVX" ABI variant. That variant is not defined anywhere, so we might as well pick the fastest convention, returning the value in registers. I'm not sure why you would want to return the vector in memory instead.

Not a new "no-AVX" ABI variant. It should be pre-AVX ABI.

Whatever you call it, the question remains; what specification and/or compiler are we trying to be compatible with?

Whatever you call it, the question remains; what specification and/or compiler are we trying to be compatible with?

The ABI before AVX was introduced.

The ABI document before AVX was introduced didn't say anything at all about 256-bit vectors. So we're talking about compatibility with some other compiler. Which compiler?

The ABI document before AVX was introduced didn't say anything at all about 256-bit vectors. So we're talking about compatibility with some other compiler. Which compiler?

X86-64 compilers created before AVX was introduced. Say clang 1.0 doesn't know AVX and returns it in memory. So should clang 7.0 with -mno-avx.

I just tried clang 3.3, and as far as I can tell it behaves the same way as trunk.

If the argument is that we should be compatible with gcc on Linux, that's fine, but we should explicitly state that in a comment.

I just tried clang 3.3, and as far as I can tell it behaves the same way as trunk.

If the argument is that we should be compatible with gcc on Linux, that's fine, but we should explicitly state that in a comment.

Does clang 3.3 support AVX512?

No, AVX512 wasn't even announced when clang 3.3 came out.

No, AVX512 wasn't even announced when clang 3.3 came out.

Try this with clang 3.3 and clang 7.0.

[hjl@gnu-cfl-1 tmp]$ cat z.c
typedef int attribute((mode(SI))) si;
typedef si attribute((vector_size (64))) v;

v
foo (void)
{
v y = {'0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f' };
return y;
}
[hjl@gnu-cfl-1 tmp]$ gcc -S -O2 z.c
z.c: In function ‘foo’:
z.c:6:1: warning: AVX512F vector return without AVX512F enabled changes the ABI [-Wpsabi]
{
^
[hjl@gnu-cfl-1 tmp]$

With both 3.3 and trunk (I don't have a 7.0 handy; I can build it if it would be helpful):

movaps  .LCPI0_0(%rip), %xmm0   # xmm0 = [48,49,50,51]
movaps  .LCPI0_1(%rip), %xmm1   # xmm1 = [52,53,54,55]
movaps  .LCPI0_2(%rip), %xmm2   # xmm2 = [56,57,97,98]
movaps  .LCPI0_3(%rip), %xmm3   # xmm3 = [99,100,101,102]
retq

With both 3.3 and trunk (I don't have a 7.0 handy; I can build it if it would be helpful):

Please try clang 2.6 on both testcases.

With both 3.3 and trunk (I don't have a 7.0 handy; I can build it if it would be helpful):

Please try clang 2.6 on both testcases.

From the releases:

23 Oct 2009 2.6

... why would you care about a 9 year old version of clang?

With both 3.3 and trunk (I don't have a 7.0 handy; I can build it if it would be helpful):

Please try clang 2.6 on both testcases.

From the releases:

23 Oct 2009 2.6

... why would you care about a 9 year old version of clang?

It is about ABI consistency. When AVX isn't enabled, the old and compilers should
use the same calling convention.

Herald added a project: Restricted Project. · View Herald TranscriptAug 4 2022, 10:24 PM
Herald added a subscriber: StephenFan. · View Herald Transcript