This is an archive of the discontinued LLVM Phabricator instance.

Support [[no_unique_address]] for all targets.
Needs RevisionPublic

Authored by expnkx on Sep 25 2021, 11:10 AM.

Details

Summary

I know msvc does not support [[no_unique_address]], but it supports [[msvc::no_unique_address]]. There is no attribute that matches this behavior for msvc, which leads to silent abi break.

clang 14 is probably going to release when the MSVC breaks abi. Yes we can support msvc::no_unique_address too, but it adds headache for future maintanence.

Also, x86_64-windows-gnu target and GCC targeting windows does support [[no_unique_address]] too, the abi is already "broken" in some form.

I propose to just support [[no_unique_address]] for all targets to prevent future problems.

Diff Detail

Event Timeline

expnkx created this revision.Sep 25 2021, 11:10 AM
expnkx requested review of this revision.Sep 25 2021, 11:10 AM
expnkx updated this revision to Diff 375057.Sep 25 2021, 11:37 AM

remove test for msvc

aaron.ballman edited reviewers, added: erichkeane, rsmith, rjmccall, jyknight; removed: Restricted Project.Sep 27 2021, 3:11 PM

I've not given this enough thought, but I wanted to add some more reviewers for greater visibility.

Meanwhile, the premerge CI failures look related, so it looks like there are more tests to update.

expnkx updated this revision to Diff 375431.Sep 27 2021, 4:20 PM

do not know why it cannot be built. upload and then verify again

ninja: error: unknown target 'check-libc', did you mean 'check-lit'?

what's wrong with this?

expnkx updated this revision to Diff 375461.Sep 27 2021, 8:12 PM

I just find that Preprocessor test has a windows (msvc not gnu) specific test that tests no_unique_address.

add that to patch and see whether it is mergable.

rjmccall added a comment.EditedSep 27 2021, 8:23 PM

We should not add a standard feature with ABI impact if we aren't certain we're going to match MSVC's ABI on it. I'm sorry, but this needs to wait until there's an MSVC release that officially supports this.

expnkx updated this revision to Diff 375466.Sep 27 2021, 9:16 PM

itanium + microsoft test both

We should not add a standard feature with ABI impact if we aren't certain we're going to match MSVC's ABI on it. I'm sorry, but this needs to wait until there's an MSVC release that officially supports this.

but how are you going to deal with [[msvc::no_unique_address]] that works?

expnkx added a comment.EditedSep 27 2021, 9:25 PM

We should not add a standard feature with ABI impact if we aren't certain we're going to match MSVC's ABI on it. I'm sorry, but this needs to wait until there's an MSVC release that officially supports this.

You are not actually matching msvc abi's layout because msvc supports [[msvc::no_unique_address]]. That is an MSVC release that officially supports.

https://github.com/tearosccebe/fast_io/blob/fa355154463704cde13cc9917e315e73281b03e9/include/fast_io_freestanding_impl/io_buffer/main.h#L160

Only clang --target=x86_64-windows-msvc does not support no_unique_address. If they are using my library they already get a break since clang does not support [[msvc::no_unique_address]] and it does not support [[no_unique_address]] either.

If you really have concerns about it, has_cpp_attribute(no_unique_address) always works.
has_cpp_attribute(no_unique_address)

Also what happens if I am using --target=x86_64-windows-gnu (that is actually my default target on windows since I use Canadian compiler). You still get abi break.

Just break ABI. This is a good start to fix the language.

expnkx updated this revision to Diff 375468.Sep 27 2021, 9:42 PM

should be diff3

We should not add a standard feature with ABI impact if we aren't certain we're going to match MSVC's ABI on it. I'm sorry, but this needs to wait until there's an MSVC release that officially supports this.

This issue on STL's repo suggests that they're waiting Clang get support for no_unique_address on Windows to start using it. Stephan said here MSVC has implemented this, but with a different name.

rjmccall requested changes to this revision.Sep 27 2021, 9:59 PM

MSVC gets to chose the ABI rules for their platform. It is not Clang policy to pick ABI rules and then break them later.

If you'd like to contribute an implementation of [[msvc::no_unique_address]] that matches MSVC's ABI, that would be welcome. I don't know, maybe that's as simple as making it use the existing implementation; @zygoloid might know better. But "add an ABI feature ahead of the platform owner and then fix the ABI problems later" is not how we do things.

This revision now requires changes to proceed.Sep 27 2021, 9:59 PM
expnkx updated this revision to Diff 375690.Sep 28 2021, 1:50 PM

I cannot find out where to add attribute that starts with msvc::xxxxx.

I am just applying this "potential" patch to the clang upstream. In the future we can just remove TargetItaniumCXXABI attr in the clang table gen file without worrying about too much. So it is a "noop".

So this patch changes nothing but it makes us be convenient to fix it in the future.

expnkx updated this revision to Diff 375720.Sep 28 2021, 3:21 PM

it should first update the field and then test no_unique_address

expnkx updated this revision to Diff 375758.Sep 28 2021, 6:33 PM

fixing clang format

expnkx updated this revision to Diff 375775.Sep 28 2021, 8:37 PM

clang format + clang tidy

all clang tests pass locally.

flang is bugged. Do not know why flang cannot find llvm-jxx

aaron.ballman requested changes to this revision.Sep 29 2021, 4:28 AM

MSVC gets to chose the ABI rules for their platform. It is not Clang policy to pick ABI rules and then break them later.

If you'd like to contribute an implementation of [[msvc::no_unique_address]] that matches MSVC's ABI, that would be welcome. I don't know, maybe that's as simple as making it use the existing implementation; @zygoloid might know better. But "add an ABI feature ahead of the platform owner and then fix the ABI problems later" is not how we do things.

I concur with John.

I cannot find out where to add attribute that starts with msvc::xxxxx.

To Attr.td; you'd give a new spelling, like: CXX11<"msvc", "no_unique_address">

I am just applying this "potential" patch to the clang upstream. In the future we can just remove TargetItaniumCXXABI attr in the clang table gen file without worrying about too much. So it is a "noop".

So this patch changes nothing but it makes us be convenient to fix it in the future.

This patch is changing behavior and cannot be landed as-is. I don't think we should do anything in this area unless it's an implementation of [[msvc::no_unique_address]] that is ABI compatible with the Microsoft implementation.

This revision now requires changes to proceed.Sep 29 2021, 4:28 AM
erichkeane added inline comments.Sep 29 2021, 5:53 AM
clang/include/clang/Basic/Attr.td
1690

The way to do a [[msvc::no_unique_address]] is likely to create a new attribute here in Attr.td, then make the spelling be something like:

let Spellings = [CXX11<"msvc", "no_unique_address", N>];

Where N is a number that is the value of __has_attribute (which I note that MSVC has yet to implement?).

clang simply just does not truly support everything msvc supports. There are no things that deals [[msvc::]] attributes at all in the clang. In fact the point of using clang on windows is that we do not want to do the same as Microsoft does.

Support [[no_unique_address]] is nothing wrong tbh and that is what ISO C++ standard requires.

If we do not change, the Microsoft team won't wanna change either because they are afraid of breaking clang too.

I never understand why [[no_unique_address]] is a problem.

clang simply just does not truly support everything msvc supports. There are no things that deals [[msvc::]] attributes at all in the clang. In fact the point of using clang on windows is that we do not want to do the same as Microsoft does.

Support [[no_unique_address]] is nothing wrong tbh and that is what ISO C++ standard requires.

If we do not change, the Microsoft team won't wanna change either because they are afraid of breaking clang too.

I never understand why [[no_unique_address]] is a problem.

A requirement of Clang on Windows is that we are ABI compatible (so that we can link between them!), the same way that Linux must match the Itanium ABI. Allowing no_unique_address on Windows is an ABI compatibility issue. In this case, we have to pay attention to what the platform's ABI OWNERS define as the ABI, which in this case is Microsoft.

THAT is why this is a problem, and THAT is what is wrong with this change.

If Microsoft defines an ABI for what this does, we can follow them and just do that.

expnkx added a comment.EditedSep 30 2021, 9:41 AM

clang simply just does not truly support everything msvc supports. There are no things that deals [[msvc::]] attributes at all in the clang. In fact the point of using clang on windows is that we do not want to do the same as Microsoft does.

Support [[no_unique_address]] is nothing wrong tbh and that is what ISO C++ standard requires.

If we do not change, the Microsoft team won't wanna change either because they are afraid of breaking clang too.

I never understand why [[no_unique_address]] is a problem.

A requirement of Clang on Windows is that we are ABI compatible (so that we can link between them!), the same way that Linux must match the Itanium ABI. Allowing no_unique_address on Windows is an ABI compatibility issue. In this case, we have to pay attention to what the platform's ABI OWNERS define as the ABI, which in this case is Microsoft.

THAT is why this is a problem, and THAT is what is wrong with this change.

If Microsoft defines an ABI for what this does, we can follow them and just do that.

https://github.com/microsoft/STL/issues/1364

As Microsoft said, the reason why things like msvc::no_unique_address exist are because clang does not support it. In fact, it has become a vicious cycle. msvc waits clang, clang waits msvc.

It is a vicious cycle. I would argue it is LLVM's fault, not Microsoft's fault.

Sure you can add another attribute, i tried that, but adding another attribute for doing exactly the same thing through entire code base is just annoying and headache.

If clang does not support [[no_unique_address]], Microsoft is not going to support it either. Microsoft's own words.

[[no_unique_address]] is ISO C++ standard. Why not support it? I am sorry the entire LLVM did a crappy job on support C++20 standard in general.

clang simply just does not truly support everything msvc supports. There are no things that deals [[msvc::]] attributes at all in the clang. In fact the point of using clang on windows is that we do not want to do the same as Microsoft does.

Support [[no_unique_address]] is nothing wrong tbh and that is what ISO C++ standard requires.

If we do not change, the Microsoft team won't wanna change either because they are afraid of breaking clang too.

I never understand why [[no_unique_address]] is a problem.

A requirement of Clang on Windows is that we are ABI compatible (so that we can link between them!), the same way that Linux must match the Itanium ABI. Allowing no_unique_address on Windows is an ABI compatibility issue. In this case, we have to pay attention to what the platform's ABI OWNERS define as the ABI, which in this case is Microsoft.

THAT is why this is a problem, and THAT is what is wrong with this change.

If Microsoft defines an ABI for what this does, we can follow them and just do that.

https://github.com/microsoft/STL/issues/1364

As Microsoft said, the reason why things like msvc::no_unique_address exist are because clang does not support it. In fact, it has become a vicious cycle. msvc waits clang, clang waits msvc.

It is a vicious cycle. I would argue it is LLVM's fault, not Microsoft's fault.

Sure you can add another attribute, i tried that, but adding another attribute for doing exactly the same thing through entire code base is just annoying and headache.

If clang does not support [[no_unique_address]], Microsoft is not going to support it either. Microsoft's own words.

[[no_unique_address]] is ISO C++ standard. Why not support it? I am sorry the entire LLVM did a crappy job on support C++20 standard in general.

My understanding is Microsoft DOES have [[no_unique_address]], it just doesn't do anything, right? So it seems they have decided the ABI is 'pretend this attribute doesn't exist'. Implementing this as anything else is an ABI break, which we don't do.

https://github.com/microsoft/STL/issues/1364

As Microsoft said, the reason why things like msvc::no_unique_address exist are because clang does not support it. In fact, it has become a vicious cycle. msvc waits clang, clang waits msvc.

It is a vicious cycle. I would argue it is LLVM's fault, not Microsoft's fault.

I don't know why anyone would assume that Clang would be more responsible for defining the behavior of a feature on Windows than Microsoft. Microsoft is the owner for their ABI, not Clang. When this feature was introduced into Clang, we explicitly did not allow it on Windows until Microsoft defined what the ABI is for the feature. Now that Microsoft has decided to define [[msvc::no_unique_address]] with a particular ABI, it would be reasonable for us to support *that* spelling. However, it would be unacceptable for us to define the behavior for [[no_unique_address]] in a way that's different from the ABI Microsoft picks for it.

Note: it *might* be reasonable for us to define [[no_unique_address]] as being ignored on Windows targets as that seems to be the direction Microsoft wants to go. However, I personally view the Microsoft behavior here to be user-hostile and so I'd like to see whether they're planning to change directions before we decide to follow along. However, if Microsoft's position is that they're not going to implement the semantic effects for [[no_unique_address]], Clang will follow suit on the Windows target.

Sure you can add another attribute, i tried that, but adding another attribute for doing exactly the same thing through entire code base is just annoying and headache.

If clang does not support [[no_unique_address]], Microsoft is not going to support it either. Microsoft's own words.

And Microsoft's own choice.

[[no_unique_address]] is ISO C++ standard. Why not support it? I am sorry the entire LLVM did a crappy job on support C++20 standard in general.

In keeping with our code of conduct (https://llvm.org/docs/CodeOfConduct.html), please keep your comments respectful.

To pick up the thread here again, [[no_unique_address]] is done and settled in MSVC, with the slightly surprising semantics: [[no_unique_address]] is accepted, without any warning (in C++20 mode), but it has no effect. (This, not related to LLVM, but because they had shipped it in earlier versions without having an effect, and changing that later would break things.) [[msvc::no_unique_address]] does have an effect though. See https://github.com/microsoft/STL/issues/1364#issuecomment-1034167093 for a more authoritative source on that.

So, separately from implementing [[msvc::no_unique_address]], I think we also should also silence the current warning about unknown attribute for the standard [[no_unique_address]], to match MSVC.

To pick up the thread here again, [[no_unique_address]] is done and settled in MSVC, with the slightly surprising semantics: [[no_unique_address]] is accepted, without any warning (in C++20 mode), but it has no effect. (This, not related to LLVM, but because they had shipped it in earlier versions without having an effect, and changing that later would break things.) [[msvc::no_unique_address]] does have an effect though. See https://github.com/microsoft/STL/issues/1364#issuecomment-1034167093 for a more authoritative source on that.

So, separately from implementing [[msvc::no_unique_address]], I think we also should also silence the current warning about unknown attribute for the standard [[no_unique_address]], to match MSVC.

Oh, also, according to https://devblogs.microsoft.com/cppblog/msvc-cpp20-and-the-std-cpp20-switch/, the plan is to change [[no_unique_address]] to actually have an effect the next time the compiler breaks its C++ ABI at an unknown point in the future. (This shouldn't be an issue for Clang, as we'd have to make a conscious effort to implement the new ABI whenever that happens anyway.)

To pick up the thread here again, [[no_unique_address]] is done and settled in MSVC, with the slightly surprising semantics: [[no_unique_address]] is accepted, without any warning (in C++20 mode), but it has no effect. (This, not related to LLVM, but because they had shipped it in earlier versions without having an effect, and changing that later would break things.) [[msvc::no_unique_address]] does have an effect though. See https://github.com/microsoft/STL/issues/1364#issuecomment-1034167093 for a more authoritative source on that.

So, separately from implementing [[msvc::no_unique_address]], I think we also should also silence the current warning about unknown attribute for the standard [[no_unique_address]], to match MSVC.

Oh, also, according to https://devblogs.microsoft.com/cppblog/msvc-cpp20-and-the-std-cpp20-switch/, the plan is to change [[no_unique_address]] to actually have an effect the next time the compiler breaks its C++ ABI at an unknown point in the future. (This shouldn't be an issue for Clang, as we'd have to make a conscious effort to implement the new ABI whenever that happens anyway.)

Thanks for the update! I'll have to think about the appropriate way forward here a bit more. We have a rule with attributes to issue an "attribute ignored" warning for any attribute that isn't accepted by Clang, and we typically extend that to attributes which are accepted by Clang but have no impact in a way the user may be surprised by (e.g., if they write it on the wrong construct, we don't silently eat the attribute, we warn the user that the attribute is being ignored). So my initial thinking is, have an on-by-default diagnostic (grouped separately under the ignored attributes warning flag) about a standard attribute being accepted and purposefully ignored that's only used for [[no_unique_address]] in MS compatibility mode (for now, we may get more such attributes in the future). Then users get a different warning than they do today, and they can control it separately from -Wignored-attributes. I think still having a diagnostic is useful for some class of users (but not all). People who only care about compatibility with MSVC likely want the warning off by default, while people who care about compatibility outside of MSVC likely want the warning on by default. Given the security implications of ignoring the attribute, I think erring on the side of caution and warning by default is appropriate (this is a standard attribute, not a vendor one, so users are understandably going to expect the attribute to do something if they've used it correctly according to the standard; silencing it is a minor burden that will not impact other kinds of ignored attributes). The fact that MS intends to support the attribute in the future (which breaks ABI) is all the more reason why I think we should warn by default; Clang users shouldn't be caught off guard when MSVC eventually breaks their ABI.