This is an archive of the discontinued LLVM Phabricator instance.

Do not apply calling conventions to MSVC entry points
ClosedPublic

Authored by eandrews on Sep 15 2020, 8:24 AM.

Details

Summary

Fix link error for MSVC entry points when calling conventions are specified. MSVC entry points should have default calling convention.

Diff Detail

Event Timeline

eandrews requested review of this revision.Sep 15 2020, 8:24 AM
eandrews created this revision.
rnk accepted this revision.Sep 15 2020, 11:23 AM

lgtm, thanks.

This revision is now accepted and ready to land.Sep 15 2020, 11:23 AM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptSep 16 2020, 9:42 AM
In D87701#2274860, @rnk wrote:

lgtm, thanks.

Thanks for taking a look!

This is causing a link error in the windows chromium build:

lld-link: error: undefined symbol: _WinMain@16

referenced by D:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:102

msvcrtd.lib(exe_winmain.obj):(int __cdecl invoke_main(void))

whoops, sorry for weird formatting in the previous comment.

I'll revert for now.

dmajor added a subscriber: dmajor.Sep 17 2020, 1:41 PM

This broke Firefox builds too, in one of our helper binaries that uses a wWinMain, although I'm having trouble writing a minimal reproducer for it. Simply making a barebones wWinMain program doesn't hit the error.

If the patch re-lands, please cc me and I'll re-test.

This broke Firefox builds too, in one of our helper binaries that uses a wWinMain, although I'm having trouble writing a minimal reproducer for it. Simply making a barebones wWinMain program doesn't hit the error.

If the patch re-lands, please cc me and I'll re-test.

Will do. I think MSVC applies calling conventions to entry points when it is specified in function signature.

int WinMain(int argc) {return 1;}

Compiling this with /Gr generates symbol _WinMain@4 , meaning /Gr was ignored and fastcall was not applied to WinMain. But,

int __fastcall WinMain(int argc) {return 1;}

generates symbol @WinMain@4, meaning __fastcall calling convention was applied to WinMain.

It also looks like stdcall is default calling convention for WinMain.

Anyway, I'll upload a new patch for review once I understand the required behavior better.