This is an archive of the discontinued LLVM Phabricator instance.

[X86] Always extend the integer parameters in callee
Needs ReviewPublic

Authored by LiuChen3 on Apr 26 2022, 12:08 AM.

Details

Summary

For now clang will assume the integer parameters have been sign/zero extended in the caller, which will cause some ABI compatibility issues.
This patch will add -mextend-small-integer which can specify how to handle small integer arguments.

  1. Adds one new ConservativeExtend Kind, which means we shouldn't make any assumptions about the caller and callee. In this case, we must do zero/sign extension for integer parameters in caller and callee.
  2. Adds -mextend-small-integer=<arg> option:
    • none : Pass the small integer parameter directly in caller.
    • conserative: Always extend small integer parameter in the caller but do not assume in the callee that the caller actually did the extension. This is the default value.
    • assumed: Assume the small integer parameter has been extened in the caller.
    • default: Use the default rule for the target.

Diff Detail

Event Timeline

LiuChen3 created this revision.Apr 26 2022, 12:08 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 26 2022, 12:08 AM
LiuChen3 requested review of this revision.Apr 26 2022, 12:08 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 26 2022, 12:08 AM
skan added a comment.Apr 26 2022, 2:53 AM

Should we update the clang/docs/ReleaseNotes.rst for this?

Should we update the clang/docs/ReleaseNotes.rst for this?

Maybe? I will update it in the next patch.

pengfei added inline comments.Apr 26 2022, 3:12 AM
clang/docs/ClangCommandLineReference.rst
3120–3124

Combine like others?

clang/include/clang/CodeGen/CGFunctionInfo.h
203–206

AI.setSignExt(Ty->hasSignedIntegerRepresentation()) for short?
Or we can remove the else block since SignExt is initialized to false?

331–332

Can we move it to isExtend? e.g. TheKind == Expand | TheKind == ConservativeExtend

clang/lib/CodeGen/CGCall.cpp
2483

Does the change affect Windows? Seems Win64 doesn't extend on caller. https://godbolt.org/z/c95hvvsWf

clang/test/CodeGen/X86/integer_argument_passing.c
3

Maybe we can remove the tests for i386 given it's only for 64 bits ABI?

LiuChen3 added inline comments.Apr 26 2022, 3:34 AM
clang/lib/CodeGen/CGCall.cpp
2483

No. This patch didn't nothing for Win64 ABI.

clang/test/CodeGen/X86/integer_argument_passing.c
3

According to the meaning of ConservativeExtend, I think the 32bit ABI needs to be modified as well:
https://godbolt.org/z/W1Ma1T3f3
The dump of currently clang-cl:

_square:
        movb    4(%esp), %al
        mulb    %al
        mulb    8(%esp)
        retl

        .def    _baz;
        .scl    2;
        .type   32;
        .endef
        .section        .text,"xr",one_only,_baz
        .globl  _baz
        .p2align        4, 0x90
_baz:
        movswl  4(%esp), %eax
        pushl   %eax
        calll   _bar
        addl    $4, %esp
        retl

Of course with this patch the behavior of clang-cl is still different from cl.exe, but I think it fits the meaning of ConservativeExtend.

pengfei added inline comments.Apr 26 2022, 5:16 AM
clang/lib/CodeGen/CGCall.cpp
2483

But I found some Windows tests are affected?

clang/test/CodeGen/X86/integer_argument_passing.c
3

My point was, i386 is passing arguments by stack. The extensions don't make sense under the circumstances. That's what I understood the comments in above test 2007-06-18-SextAttrAggregate.c

pengfei added inline comments.Apr 26 2022, 5:21 AM
clang/test/CodeGen/X86/integer_argument_passing.c
3

Oh, seems I misunderstood it. The stack still needs extensions since it's aligned to 4 bytes. But from the above output, the clang-cl is wrong, because it extends on caller which MSVC extends on callee. So back the another question, we should change for Windows too, right?

LiuChen3 added inline comments.Apr 26 2022, 6:19 AM
clang/test/CodeGen/X86/integer_argument_passing.c
3

I just change the behavior of win32. Windows 64 will always do the extensions in callee. I didn't support ConservativeExtend for win64. The dump IR of currently clang-cl is:

define dso_local i8 @square(i8 noundef %a, i8 noundef %b) local_unnamed_addr #0 {
...
}

  %call = tail call i32 @bar(i16 noundef %conv) #3
...
}

define dso_local i32 @baz(i32 noundef %num) local_unnamed_addr #1 {
...
  %call = tail call i32 @bar(i16 noundef %conv) #3
...
}

I think maybe we don't need to do the extension both in caller and callee for WIN64?

rnk added inline comments.Apr 26 2022, 3:17 PM
clang/include/clang/CodeGen/CGFunctionInfo.h
121

Instead of introducing a new ABIInfo::Kind value, should this be a boolean flag that is part of ABIInfo::Extend? That seems like it would be simpler.

clang/test/CodeGen/X86/integer_argument_passing.c
3

I agree, I think there is only one use of getExtend in the win64 argument code, and it is for bool / i1:
https://github.com/llvm/llvm-project/blob/main/clang/lib/CodeGen/TargetInfo.cpp#L4359

Basically, on Windows, we never extend. That seems to correspond to the behavior we observed in MSVC.

Regarding Windows x86_32, I personally think your behavior change is correct. We shouldn't rely on MSVC to extend its arguments for us. In any case, it's conservatively correct, and hopefully nobody cares about minor efficiency issues for x86_32.

rjmccall added inline comments.Apr 26 2022, 5:05 PM
clang/docs/ClangCommandLineReference.rst
3120–3124

How about:

In the past, Clang passed small integer arguments on certain targets using a
parameter convention in which the caller was assumed to have sign-extended
or zero-extended the argument to a certain width.  This convention was not
conformant with the documented ABI on these platforms, which does not
require the caller to perform this extension.  Clang no longer assumes that
callers perform this extension, but for compatibility with code compiled by
previous releases of Clang, Clang defaults to still extending the argument in the
caller.  `-mno-conservative-extend` disables this, which may improve
performance and code size if compatibility with old versions of Clang is not
required.

This affects most 32-bit and 64-bit x86 targets, except:
- Windows, which Clang has never assumed extension on
- Apple platforms, which use a non-standard ABI that unconditionally assumes extension

Note that I need to check that that's what Apple actually wants to do. You should also reach out to the Sony folks to see what they want to do, but I expect that it's to assume extension unconditionally.

clang/include/clang/Basic/CodeGenOptions.def
485

"Whether callers should extend small integer parameters even though callees are not supposed to expect extension."

clang/include/clang/CodeGen/CGFunctionInfo.h
48

"...but do not assume in the callee that the caller actually did the extension."

121

I think adding a new Kind is better; it's a significantly different ABI that clients should be forced to handle correctly, which they will not if it's just a flag.

clang/include/clang/Driver/Options.td
3430

The callee isn't really extending anything; please the wording I suggest above.

clang/lib/CodeGen/CGCall.cpp
1625

Please write the code to work correctly for return types even if we aren't using it anywhere yet.

2482

"Under ConservativeExtend, add sext/zext only on the call site; the callee does not get to assume extension."

3686

Please make this correct when used on return types.

5632

Please make this correct when used on return types.

clang/lib/CodeGen/TargetInfo.cpp
1944

This looks wrong. In non-ConservativeExtend mode, we don't get to assume extension at all and should use Direct, right?

3832–3833

Same comment: this should use Direct when we're not in ConservativeExtend mode, right?

LiuChen3 added inline comments.Apr 26 2022, 11:29 PM
clang/docs/ClangCommandLineReference.rst
3120–3124

Thanks a lot! It's much clearer.
Just a small correction for windows: only windows64 is not affected.

clang/include/clang/CodeGen/CGFunctionInfo.h
331–332

I prefer to set it alone as it is a different Kind from Extend.

clang/lib/CodeGen/CGCall.cpp
2483

I checked it again and it should be that only win32 is affected.

clang/lib/CodeGen/TargetInfo.cpp
1944

As I understand it, Direct means do nothing with the parameters. Caller won't do the extension and callee can't assume the parameter is correct. This makes new clang behave in the opposite way to currently clang behavior, which will cause incompatibility issue. e.g:
https://godbolt.org/z/d3Peq4nsG

3832–3833

Same as above.

LiuChen3 updated this revision to Diff 425427.Apr 27 2022, 12:07 AM

Address comments

LiuChen3 marked 9 inline comments as done.Apr 27 2022, 12:14 AM
LiuChen3 added inline comments.
clang/lib/CodeGen/CGCall.cpp
2342

At present, ConservativeExtend has no specific definition for return value, so I just make it follow the behavior of Extend .

LiuChen3 updated this revision to Diff 425428.Apr 27 2022, 12:19 AM

add one missing comment

Should we update the clang/docs/ReleaseNotes.rst for this?

The ReleaseNotes says "written by LLVM Team". So I am not sure if I can update this.

LiuChen3 added inline comments.Apr 27 2022, 12:28 AM
clang/docs/ClangCommandLineReference.rst
3120–3124

Hi, @RKSimon , @probinson. This patch will extend the integer parameters in the caller. Is there any concern about this for Sony?

rjmccall added inline comments.Apr 27 2022, 2:21 AM
clang/docs/ClangCommandLineReference.rst
3120–3124

Slight revision:

In the past, on certain targets, Clang passed promotable integer arguments
(arguments with types smaller than int) using a parameter convention in
which the caller was assumed to have sign-extended or zero-extended the
argument to the width of an int. This convention did not conform to the
documented ABI for these targets, which does not require the caller to
perform this extension.

clang/lib/CodeGen/TargetInfo.cpp
1944

Oh, I see, you're thinking that -mno-conservative-extend means "do what old versions of clang did" rather than "break compatibility with old versions of clang and just follow the x86_64 ABI". That's definitely different from the documentation I suggested, so something's got to change.

I think these are the possibilities here:

  1. Some platforms, like Apple's, are probably going to define Clang's current behavior as the platform ABI. So those platforms need to continue to use Extend. My previous comment about using Direct wasn't paying due attention to this case.
  1. On other platforms, we need to maintain compatibility by default with both the platform ABI and old Clang behavior. Those platforms will need to use ConservativeExtend.
  1. Some people may want to opt out of (2) and just be compatible with the platform ABI, which has minor code-size and performance wins. If we support that with an option, I believe it should cause us to emit Direct. This is what I was thinking -mno-conservative-extend would mean.
  1. Some people may want to force the use of (1) even on platforms where that isn't the platform ABI. I don't know if this is really something we should support in the long term, but it might be valuable for people staging this change in. This is what you seem to be thinking -mno-conservative-extend would mean.

I would suggest these spellings for the argument, instead of making it boolean:

-mextend-small-integers=none    // Force the use of Direct
-mextend-small-integers=conservative // Force the use of ConservativeExtend
-mextend-small-integers=assumed // Force the use of Extend
-mextend-small-integers=default // Use the default rule for the target
RKSimon edited the summary of this revision. (Show Details)Apr 27 2022, 3:06 AM
LiuChen3 added inline comments.Apr 27 2022, 3:23 AM
clang/lib/CodeGen/TargetInfo.cpp
1944

Yes. That's what I mean. I totally misunderstood your meaning before.
I agree with your method. I will work on that. Just one concern about the 'default': the default behavior is 'conservative' instead of 'default'. Wouldn't that be a little weird? But with my limited English I can't think of a better word. 'default' is ok for me. :-)

rjmccall added inline comments.Apr 27 2022, 11:02 AM
clang/lib/CodeGen/TargetInfo.cpp
1944

The default will be platform-specific. For example, the de facto macOS x86_64 ABI is to extend in the caller, because clang is the system compiler, and that's what clang does. Apple just needs to update its documentation.

ychen added a subscriber: ychen.Apr 27 2022, 2:53 PM
LiuChen3 updated this revision to Diff 427551.May 6 2022, 1:22 AM

Use -mextend-small-integers=<arg> instead of boolean option

LiuChen3 edited the summary of this revision. (Show Details)May 6 2022, 1:53 AM
rjmccall added inline comments.May 9 2022, 4:33 PM
clang/docs/ClangCommandLineReference.rst
3164

Specifies how to handle small integer arguments on i386 and x86_64
targets that generally follow the System V ABI. The value must be `none`,
`conservative, assumed, or default`. The default behavior
depends on the target and can be explicitly requested with `default`.

In the past, on System V i386 and x86_64 targets, Clang passed promotable
integer arguments (non-variadic arguments with types smaller than 'int')
using a parameter convention in which the caller was assumed to have
sign-extended or zero-extended the argument to the width of an 'int'. This
convention did not conform to the documented ABI for these targets, which
does not require the caller to perform this extension. This can introduce
incompatibilities with code generated by other compilers on these targets.

On most targets using the System V ABI, Clang no longer assumes that
callers perform this extension. In order to maintain compatibility with
code compiled by old versions of Clang, Clang will still perform the extension
in the caller. This is the `conservative` behavior. If compatibility with old
Clang code is not required, extension in the caller can be disabled using
`none`, which may have minor performance and code-size benefits.

On certain targets which use Clang as the system compiler, Clang continues
to perform extension in the caller and assume it in in the callee. This is the
behavior of `assumed`. These platforms therefore diverge from the
standard System V ABI. If compatibility with other compilers which do not
recognize this divergence is required, `conservative` may be used.
Using `none` on these targets is allowed but may lead to ABI mismatches.
These targets include Apple and PlayStation platforms.

clang/include/clang/Driver/Options.td
3433
clang/lib/CodeGen/TargetInfo.cpp
1944

Please go ahead and implement the target-specific logic. Since Apple (and probably Sony) is opting out of it, it's incorrect to change the ABI there, even briefly.

I would suggest folding away the Default case so that you just deal with one of assumed/conservative/direct here, either when you construct the ABIInfo or maybe even when you parse the frontend options in CompilerInvocation.

Thanks, @rjmccall . I'm sorry I don't have much time on this patch recently. I will update it later.

I find the option names you have a bit confusing. I'd like to suggest calling them, instead:

caller: Extend a small integer parameter in the caller; callee will assume it has already been extended.
callee : Pass a small integer parameter directly in caller, extend in callee when converting to full-width.
both: Extend a small integer parameter in the caller; callee ALSO extends when converting to full-width.
default: Use the default rule for the target.

I think that gets more to the point of what's going on here, even though it's not exactly the case that "callee" always extends.

I'm fine with alternatives on the names. I just don't think we want names that imply that the caller and callee are parallel here, as if there were some sort of requirement that the callee always extends. In those conventions, the callee gets passed 8 or 16 meaningful bits; if it wants an extended value, it's responsible for doing that extension, just like it would be responsible for doing the opposite extension if it wanted that.

LiuChen3 updated this revision to Diff 449230.Aug 2 2022, 2:09 AM

rebase and address rjmccall's comments

Sorry for the late update. I hope you can take the time to continue reviewing, @rjmccall .
I'm not sure if I understand you correctly. Now if the -mextend-small-integers=default is used, compiler will report error.

I find the option names you have a bit confusing. I'd like to suggest calling them, instead:

caller: Extend a small integer parameter in the caller; callee will assume it has already been extended.
callee : Pass a small integer parameter directly in caller, extend in callee when converting to full-width.
both: Extend a small integer parameter in the caller; callee ALSO extends when converting to full-width.
default: Use the default rule for the target.

I think that gets more to the point of what's going on here, even though it's not exactly the case that "callee" always extends.

rjmccall's comment convinced me.

It looks like you haven't implemented the target-specific logic for this yet. I cannot let you commit until you do that, because you will be breaking the ABI on Apple platforms.

clang/include/clang/Basic/CodeGenOptions.h
150

This comment is incorrect.

clang/include/clang/Driver/Options.td
3435

We're not usually this verbose in the inline help text; this is basically an attempt to document the whole feature, which is excessive, We should do like we do for -mthread-model or similar enum options and just quickly describe the option and list the possible values.

clang/lib/CodeGen/TargetInfo.cpp
1948
3828

Thinking more about the naming issue, I think the real issue is that the proposed options are an implementation detail, not what we should expose to users. I realize that the non-boolean option was proposed by rjmmccall before, but I'd suggest that we go back to something more like your original boolean option. I believe that the discussion which resulted in the new option, was due to confusion around what "not conservative" should mean -- and that's just because the implementation did not honor fclang-abi-compat.

The main problem I have with the current proposal is that it's both confusing to describe (I still can't really understand the docs), and it invites trouble: if a user is trying to improve performance and specifies -mextend-small-integers=none, that will help on Linux but break the binary on Apple. There's really no reason to even offer "none" on apple.

So, let's provide the ability to adjust behavior according to the goals a user may have. The behaviors I think we need to expose are:

  1. (Default) Be compatible both with previous Clang ABI, and with GCC/ICC/new-Clang ABI.
  2. Be compatible with previous Clang ABI, don't care about compatibility with standard ABI.
  3. Be compatible with standard ABI, don't care about compatibility with previous Clang ABI.

So, for that purpose, I suggest we should use the existing -fclang-abi-compat option, plus a single on/off switch for whether to be conservatively-compatible (which I've renamed -mconservative-small-integer-abi but I don't feel strongly about that if you wanted to go back to -mconservative-extend)

Then, to achieve the 3 goals above, you can specify:

  1. No extra args (which implies the defaults of -fclang-abi-compat=16 -mconservative-small-integer-abi)
  2. -fclang-abi-compat=15 -mno-conservative-small-integer-abi
  3. -mno-conservative-small-integer-abi

On most SysV platforms, with -mno-conservative-small-integer-abi, abi-compat=15 will use Direct, and abi-compat=16 will use Extend. With -mconservative-small-integer-abi, both will use ConservativeExtend mode. On Apple/Playstation platforms, this ABI doesn't change between 15/16, so it should always use Extend regardless of either of the options. (That is, -mconservative-small-integer-abi is a no-op on those platforms.)

Then we can go back to the previous description from rjmmcall, with some minor adjustments to describe the interaction with -fclang-abi-compat:

-mconservative-small-integer-abi

In the past, on certain targets, Clang passed promotable integer arguments
(arguments with types smaller than int) using a parameter convention in which
the caller was assumed to have sign-extended or zero-extended the argument to
the width of an int. This convention did not conform to the documented ABI for
these targets, which does not require the caller to perform this
extension. Under `-fclang-abi-compat=16`, Clang no longer assumes that callers
perform this extension, but for compatibility with code compiled by previous
releases of Clang, Clang defaults to still extending the argument in the
caller.  `-mno-conservative-small-integer-abi` disables this, which may
improve performance and code size if compatibility across Clang versions is
not required.

This option is a no-op for targets which did not have an ABI change (Apple,
Windows, and PlayStation).

That is an interesting idea. I like that it integrates this into -fclang-abi-compat. The way that -mno-conservative-small-integer-abi ends up meaning opposite things based on the abi-compat setting worries me a lot, though.

That is an interesting idea. I like that it integrates this into -fclang-abi-compat. The way that -mno-conservative-small-integer-abi ends up meaning opposite things based on the abi-compat setting worries me a lot, though.

The flag means the same thing in both cases, from the user's POV: "Be compatible only with the specified -fclang-abi-compat behavior, not an earlier/later ABI version's behavior."

I know what you're saying, but I don't think it matches any model of how programmers use command line flags. You're imagining that a programmer sits down and considers all of their flags deeply and holistically before touching any of them, and that's just not how these things go in actual build systems. Flags have a tendency to be set in separate places and therefore to drift. Someone setting this option is doing it because they have a concrete compatibility need, and the flag going from meaning "be compatible with GCC" to meaning "be compatible with old Clangs" based on the value of a separate flag is surprising and likely to cause bugs.

I know what you're saying, but I don't think it matches any model of how programmers use command line flags. You're imagining that a programmer sits down and considers all of their flags deeply and holistically before touching any of them, and that's just not how these things go in actual build systems. Flags have a tendency to be set in separate places and therefore to drift. Someone setting this option is doing it because they have a concrete compatibility need, and the flag going from meaning "be compatible with GCC" to meaning "be compatible with old Clangs" based on the value of a separate flag is surprising and likely to cause bugs.

Funny, my argument is based precisely on the fact that people _don't_ understand deeply what things are doing, and that the original proposal is likely to trigger bugs via mistakes in its usage!

The default (-mconservative-small-integer-abi) is "be compatible with all Clang and GCC versions" -- it "just works". (I think everyone agrees upon that being the default.)

So, the use-case for -mconservative-small-integer-abi is not a need for compatibility but rather a need for performance/code-size overriding cross-compiler compatibility concerns. In that case, the user better be able to answer "what other compiler/version do you need to be compatible with, if any?" -- and set -fclang-abi-compat flag appropriately based on that answer.

That seems much easier thing to explain to a user than the details of where exactly extension may/must happen in the call ABI.

I know what you're saying, but I don't think it matches any model of how programmers use command line flags. You're imagining that a programmer sits down and considers all of their flags deeply and holistically before touching any of them, and that's just not how these things go in actual build systems. Flags have a tendency to be set in separate places and therefore to drift. Someone setting this option is doing it because they have a concrete compatibility need, and the flag going from meaning "be compatible with GCC" to meaning "be compatible with old Clangs" based on the value of a separate flag is surprising and likely to cause bugs.

Funny, my argument is based precisely on the fact that people _don't_ understand deeply what things are doing, and that the original proposal is likely to trigger bugs via mistakes in its usage!

The default (-mconservative-small-integer-abi) is "be compatible with all Clang and GCC versions" -- it "just works". (I think everyone agrees upon that being the default.)

So, the use-case for -mconservative-small-integer-abi is not a need for compatibility but rather a need for performance/code-size overriding cross-compiler compatibility concerns. In that case, the user better be able to answer "what other compiler/version do you need to be compatible with, if any?" -- and set -fclang-abi-compat flag appropriately based on that answer.

That seems much easier thing to explain to a user than the details of where exactly extension may/must happen in the call ABI.

I'm not arguing that we shouldn't work -fclang-abi-compat into this; I think that is a very good idea. And for many people, that'll be all they need. My concern is about the people who do add -mno-conservative-small-integer-abi to their builds.