This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Simplify the tuple constructor overload set
ClosedPublic

Authored by philnik on Apr 15 2023, 8:25 AM.

Details

Summary

This uses conditional explicit to avoid having two overloads for implicit/explicit conversions.

Diff Detail

Event Timeline

philnik created this revision.Apr 15 2023, 8:25 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 15 2023, 8:25 AM
philnik updated this revision to Diff 520235.May 7 2023, 6:46 PM

Address comments

philnik updated this revision to Diff 520238.May 7 2023, 7:17 PM

Disable GCC warnings

ldionne published this revision for review.May 8 2023, 8:08 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 8 2023, 8:08 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript
ldionne accepted this revision as: ldionne.May 8 2023, 8:19 AM
ldionne added a subscriber: aaron.ballman.

I really like this. It's a huge simplification. If it really can't be noticed by users (and it can't AFAICT), we're literally not losing anything and we're gaining a ton of readability and maintainability.

Nikolas also mentions:

  1. Diagnostics are much better
  2. Fewer notes are generated when an error occurs
  3. The overload set is much smaller

Overall, I think this is great. Like I mentioned on https://reviews.llvm.org/D148431#4323187, I think we should be careful with enabling those extensions widely just in case we notice an issue with doing that. We wouldn't want to be stuck in a place where we use a bunch of extensions that are really hard to revert but we realize there's a problem with using those.

Hence, I'd suggest that we only make this change before the LLVM 17 release. Then we can release with this extension (and only this one) and gain additional information. I think we will most likely not learn anything new, in which case we would conclude that using those extensions truly is not noticeable and doesn't hurt anyone, and we could apply them more widely. But if we learn something new, at least we'll have a single patch to revert instead of potentially many patches in various parts of the code.

CCing libc++ vendors for awareness and also @aaron.ballman since we had a discussion adjacent to this topic on Discord. If nobody raises a substantiated objection by the end of this week, I'd ship this.

ldionne added subscribers: Restricted Project, Restricted Project.May 8 2023, 8:20 AM

This looks like a really nice simplification! Do you happen to have compile time performance numbers as to how much overhead we save?

This looks like a really nice simplification! Do you happen to have compile time performance numbers as to how much overhead we save?

At least I don't have any. It could be quite interesting though.

ldionne accepted this revision.May 12 2023, 11:24 AM

CI is green (it reports as failed but I checked the job and it's all good).

Let's do this and revisit after LLVM 17 whether we want to expand. @philnik Please keep an eye out in case someone comments on this review saying that it broke something somehow. I think this should be transparent to users, but let's be careful.

This revision is now accepted and ready to land.May 12 2023, 11:24 AM

Actually, I'd also like to make sure that @Mordante is fine with this direction here.

I haven't looked at the patch itself in detail, but the stats look nice.

However I'm very concerned with the approach of this patch and the decision making in libc++.
As mentioned on Discord I think we open ourselves to Hyrum's law by depending on undocumented compiler features.
AFAIK no compiler documents these extensions and the guarantees they offer for these features. I really have
no idea what GCC guarantees or wants to promise. Since we don't test ToT GCC we will only discover breakage
after the fact.

As mentioned before, discussions on Discord often don't end in any conclusion/consensus before moving ahead.
I really would like to see libc++ improve in this area. I've talked @ldionne privately about this in the past.

I think it would be very good to discuss some of these policies, the same like considering the removal of
C++03 support. I agree these kind of changes make maintenance for us, the libc++ developers, a lot easier.
However, what is the size of our community that are negatively affected by these changes?

Since it's only discussed on Discord interested/affected parties may have missed the entire discussion.
I think it's important to communicate these kind of changes upfront. We don't have a
_LIBCPP_DISABLE_FEATURE_CONDITIONAL_EXPLICIT_CONSTRUCTOR like we do when we do in our normal deprecation.
So if this breaks somebody after LLVM 17 is released they have no easy way to opt-out. IMO we should add
that and keep it for (at least) two releases, just like we do with deprecation.

If we want to go in this direction we should document which extensions are allowed and which are not.
Preferably after making sure what the policy of these undocumented compiler features for different vendors
mean. Note that libc++ has removed backported features when they resulted in a maintenance burden.
I don't expect that be as troublesome in compilers, but I'm not sure.

I haven't looked at the patch itself in detail, but the stats look nice.

However I'm very concerned with the approach of this patch and the decision making in libc++.
As mentioned on Discord I think we open ourselves to Hyrum's law by depending on undocumented compiler features.
AFAIK no compiler documents these extensions and the guarantees they offer for these features. I really have
no idea what GCC guarantees or wants to promise. Since we don't test ToT GCC we will only discover breakage
after the fact.

Clang has actually started documenting these kinds of extensions: https://clang.llvm.org/docs/LanguageExtensions.html#language-extensions-back-ported-to-previous-standards. We already only discover breakages after the fact, so I don't see how that's relevant.

As mentioned before, discussions on Discord often don't end in any conclusion/consensus before moving ahead.
I really would like to see libc++ improve in this area. I've talked @ldionne privately about this in the past.

You were the only person to object to this. Everybody else who commented (Costa, Louis, Eric) was in favor of it.

I think it would be very good to discuss some of these policies, the same like considering the removal of
C++03 support. I agree these kind of changes make maintenance for us, the libc++ developers, a lot easier.
However, what is the size of our community that are negatively affected by these changes?
Since it's only discussed on Discord interested/affected parties may have missed the entire discussion.

We obviously don't know whether people are affected by this, but I would be very surprised if anybody was. We already assume lots of extensions specific to clang and GCC, and no vendor has expressed any concerns so far (also not with this patch).

I think it's important to communicate these kind of changes upfront. We don't have a
_LIBCPP_DISABLE_FEATURE_CONDITIONAL_EXPLICIT_CONSTRUCTOR like we do when we do in our normal deprecation.
So if this breaks somebody after LLVM 17 is released they have no easy way to opt-out. IMO we should add
that and keep it for (at least) two releases, just like we do with deprecation.

IMO that doesn't make sense. We only support Clang and GCC, which support these extensions, and vendors test everything pre-release. If anybody finds a problem with it, we can still revert. Even if that lets things fall through the cracks, we can still revert in 17.0.whatever, and the far majority of users won't have a problem.

If we want to go in this direction we should document which extensions are allowed and which are not.
Preferably after making sure what the policy of these undocumented compiler features for different vendors
mean. Note that libc++ has removed backported features when they resulted in a maintenance burden.
I don't expect that be as troublesome in compilers, but I'm not sure.

If I understand @aaron.ballman's comments correctly, they consider all documented extensions to be very hard to impossible to remove, and AFAICT quite easy to keep support for (at least for the from-later-standards ones). If you want, we can reach out to GCC folks to make sure they don't see things drastically different (which I don't expect given the amount of, even non-conforming, GNU extensions).

If we want to go in this direction we should document which extensions are allowed and which are not.
Preferably after making sure what the policy of these undocumented compiler features for different vendors
mean. Note that libc++ has removed backported features when they resulted in a maintenance burden.
I don't expect that be as troublesome in compilers, but I'm not sure.

If I understand @aaron.ballman's comments correctly, they consider all documented extensions to be very hard to impossible to remove, and AFAICT quite easy to keep support for (at least for the from-later-standards ones). If you want, we can reach out to GCC folks to make sure they don't see things drastically different (which I don't expect given the amount of, even non-conforming, GNU extensions).

Correct -- we'll remove an extension if there's significant evidence that it isn't being used in the wild and it is causing serious problems (security, maintenance, is fundamentally broken, etc). However, if the extension is being used by libc++, then it's definitely getting used in the wild and unless there's literally no other option but to tear it out, we'll basically have to find a way to make it work. Generally speaking though, documented extensions don't get removed.

[...]
I really have no idea what GCC guarantees or wants to promise. Since we don't test ToT GCC we will only discover breakage
after the fact.

That's a fair point, I think we can and should reach out to the GCC folks to clarify the guarantees with respect to compiler extensions.

As mentioned before, discussions on Discord often don't end in any conclusion/consensus before moving ahead.
I really would like to see libc++ improve in this area. I've talked @ldionne privately about this in the past.

I am very happy to work on improving this area of the project, but I'd like to understand better what you want. Would you prefer if this were discussed on Discourse instead of Discord? I'm fine with that although I don't believe it would meaningfully change the amount of consensus we'd get versus doing the same on Discord -- for example if you look at most RFCs on Discourse there is no clear consensus at the end of the discussion, and someone usually steps up and determines what consensus means in the context and decides to move forward or not. We've also discussed setting up a recurring VC meeting with the libc++ developers -- do you feel like it would have made a difference in this case? We could do it ~monthly and accumulate an agenda of items to discuss "in person".

I think it would be very good to discuss some of these policies, the same like considering the removal of
C++03 support. I agree these kind of changes make maintenance for us, the libc++ developers, a lot easier.
However, what is the size of our community that are negatively affected by these changes?

I agree, and in fact these policies are always discussed somewhere, usually on Discord. Perhaps that's the main sticking point and we should have something more formal.

Since it's only discussed on Discord interested/affected parties may have missed the entire discussion.
I think it's important to communicate these kind of changes upfront. We don't have a
_LIBCPP_DISABLE_FEATURE_CONDITIONAL_EXPLICIT_CONSTRUCTOR like we do when we do in our normal deprecation.
So if this breaks somebody after LLVM 17 is released they have no easy way to opt-out. IMO we should add
that and keep it for (at least) two releases, just like we do with deprecation.

I think the context of this change is extremely relevant in this case. This change should have no impact whatsoever on users. If it did, then I would immediately argue for us *not* to make that change. In fact, I asked for the cautious approach taken by this patch where we only use the compiler extension in one place and don't start using any new extensions until LLVM 17 is released, so as to make it as easy as possible to revert this decision if we find that this is user-affecting. Regarding deprecation/communication: since this is not user-impacting, I don't think it makes sense to have a deprecation or a macro to opt into the old way to write the same constructors. Basically, this patch is either "it works and we do it, or we realize that it has user impact and we revert the patch".

Because this is not user-affecting, I think the set of interested parties is basically just the libc++ developers themselves. So while we may want to use a more formal process to discuss these changes, I don't think anyone was left out of the discussion we had on Discord. Let's figure out what concrete steps we want to take to improve the decision making process and then take this change through it. How does that sound?

Summarizing the discussion I had yesterday with @Mordante:

I think some of the problems here come from the fact that communication in libc++ is pretty siloed. I meet with most of the core contributors on a weekly basis (and sometimes more), but those are one-on-one meetings. As a result, I know what's going on and what discussion happened with who, but not everyone has the same context. So for example, Nikolas and I actually discussed this change in depth and we considered potential user implications carefully, but that isn't necessarily clear from this review alone.

To try to alleviate this, we agreed to:

  1. Create a recurring monthly meeting for libc++ developers, similar to the regular meetings that happen for Clang. We can discuss general libc++ topics "in person" during those meetings and let everyone sync up with what's going on in the project.
  2. We will document on our Contributing page that significant changes or changes that could meaningfully affect users should usually go through RFCs. I want to be careful about not mandating a RFC for changes too systematically, since almost all libc++ changes can affect users in one way or another (I added a const somewhere and broke some code inside Google last week!). That would just make the process heavier than it needs to be. However, going through a RFC on Discourse for significant changes (and this is a significant change in the sense that it sets the precedent for using extensions in the codebase) is reasonable, so we'll document that.

In terms of this change specifically: Let's wait for the above things to be implemented, then take this proposal through a RFC on Discourse and come back to it with the information we'll have gained.

  1. Create a recurring monthly meeting for libc++ developers, similar to the regular meetings that happen for Clang. We can discuss general libc++ topics "in person" during those meetings and let everyone sync up with what's going on in the project.

If it's not overly burdensome, it might be great if some notes from these meetings were posted somewhere for history/visibility/engagement of folks who can't make the call?

[Github PR transition cleanup]

We've had numerous discussions on Discord about this since the discussion on this patch happened. We also asked the GCC folks what was their stance on GCC extensions in earlier language modes, and their reply basically matched what @aaron.ballman told us for Clang: extensions won't be removed unless they are broken in some significant way. At this point, I believe everyone's concerns have been addressed and I don't see what we could do to gain more consensus on this specific patch.

@Mordante Can you please let us know if you still have concerns with this patch? Otherwise, we would aim to land it by Friday October 13th (a bit more than a week away).

  1. Create a recurring monthly meeting for libc++ developers, similar to the regular meetings that happen for Clang. We can discuss general libc++ topics "in person" during those meetings and let everyone sync up with what's going on in the project.

Just to be clear, this is still on my TODO list, but I don't think it is reasonable to block this patch on setting that up, since any concerns with this patch can also be discussed right here on Phab or on Discord (like we've done).

jrtc27 added a subscriber: jrtc27.Oct 5 2023, 9:20 AM

To clarify: what versions of GCC and Clang are required to compile this code? Whilst we can assume GCC and Clang, if this is in code paths used for old language versions, we need to be sure compilers that are supported today for those language versions continue to be able to #include this header without issue (unlike files used solely during the build of libc++ itself, where we can more aggressively require newer compilers).

To clarify: what versions of GCC and Clang are required to compile this code? Whilst we can assume GCC and Clang, if this is in code paths used for old language versions, we need to be sure compilers that are supported today for those language versions continue to be able to #include this header without issue (unlike files used solely during the build of libc++ itself, where we can more aggressively require newer compilers).

libc++ currently requires at least Clang 16 or GCC 13, so we should be good on that front. We will probably add extra coverage to the CI once we enable extensions more generally.

jrtc27 added a comment.Oct 5 2023, 9:35 AM

To clarify: what versions of GCC and Clang are required to compile this code? Whilst we can assume GCC and Clang, if this is in code paths used for old language versions, we need to be sure compilers that are supported today for those language versions continue to be able to #include this header without issue (unlike files used solely during the build of libc++ itself, where we can more aggressively require newer compilers).

libc++ currently requires at least Clang 16 or GCC 13, so we should be good on that front. We will probably add extra coverage to the CI once we enable extensions more generally.

To build, or for anyone wishing to compile code that uses it as the standard library implementation? I would be very surprised if it's the latter. What about distributions that need to compile code with older compilers?

To clarify: what versions of GCC and Clang are required to compile this code? Whilst we can assume GCC and Clang, if this is in code paths used for old language versions, we need to be sure compilers that are supported today for those language versions continue to be able to #include this header without issue (unlike files used solely during the build of libc++ itself, where we can more aggressively require newer compilers).

libc++ currently requires at least Clang 16 or GCC 13, so we should be good on that front. We will probably add extra coverage to the CI once we enable extensions more generally.

To build, or for anyone wishing to compile code that uses it as the standard library implementation? I would be very surprised if it's the latter. What about distributions that need to compile code with older compilers?

For anyone that wants to use it. IDK about distributions that need to use old compilers, but how often do you install a custom standard library without also installing a custom compiler? Probably just about never.

To clarify: what versions of GCC and Clang are required to compile this code? Whilst we can assume GCC and Clang, if this is in code paths used for old language versions, we need to be sure compilers that are supported today for those language versions continue to be able to #include this header without issue (unlike files used solely during the build of libc++ itself, where we can more aggressively require newer compilers).

libc++ currently requires at least Clang 16 or GCC 13, so we should be good on that front. We will probably add extra coverage to the CI once we enable extensions more generally.

To build, or for anyone wishing to compile code that uses it as the standard library implementation? I would be very surprised if it's the latter. What about distributions that need to compile code with older compilers?

For anyone that wants to use it. IDK about distributions that need to use old compilers, but how often do you install a custom standard library without also installing a custom compiler? Probably just about never.

libc++ isn't always a custom standard library. It's the system standard library for FreeBSD (and Android, but there's probably less concern about weird old compiler versions there). Every 6 months a new libc++ comes out, so around every 6 months (always between a bit and a lot behind) FreeBSD updates to a new version of libc++, but the freebsd-ports collection is *huge* and has many old compiler versions, both GCC and Clang, available and used. We try to avoid them where possible, but sometimes it's what has to happen. GCC in particular; code that compiles with Clang tends to be more modern and less picky about compilers.

To clarify: what versions of GCC and Clang are required to compile this code? Whilst we can assume GCC and Clang, if this is in code paths used for old language versions, we need to be sure compilers that are supported today for those language versions continue to be able to #include this header without issue (unlike files used solely during the build of libc++ itself, where we can more aggressively require newer compilers).

libc++ currently requires at least Clang 16 or GCC 13, so we should be good on that front. We will probably add extra coverage to the CI once we enable extensions more generally.

To build, or for anyone wishing to compile code that uses it as the standard library implementation? I would be very surprised if it's the latter. What about distributions that need to compile code with older compilers?

For anyone that wants to use it. IDK about distributions that need to use old compilers, but how often do you install a custom standard library without also installing a custom compiler? Probably just about never.

libc++ isn't always a custom standard library. It's the system standard library for FreeBSD (and Android, but there's probably less concern about weird old compiler versions there). Every 6 months a new libc++ comes out, so around every 6 months (always between a bit and a lot behind) FreeBSD updates to a new version of libc++, but the freebsd-ports collection is *huge* and has many old compiler versions, both GCC and Clang, available and used. We try to avoid them where possible, but sometimes it's what has to happen. GCC in particular; code that compiles with Clang tends to be more modern and less picky about compilers.

I am aware that it's not always a custom library. May point is that it's always a custom library if you don't have a corresponding compiler, since distributions don't only ship a new libc++, at least AFAIK. This also doesn't seem to be much of a problem in reality, since we already assume these compilers in code. Anyways, I don't think our compiler support policy is relevant to whether this should land. If the support policy should change, that is a separate conversation.

[Github PR transition cleanup]

@Mordante Can you please let us know if you still have concerns with this patch? Otherwise, we would aim to land it by Friday October 13th (a bit more than a week away).

  1. Create a recurring monthly meeting for libc++ developers, similar to the regular meetings that happen for Clang. We can discuss general libc++ topics "in person" during those meetings and let everyone sync up with what's going on in the project.

Just to be clear, this is still on my TODO list, but I don't think it is reasonable to block this patch on setting that up, since any concerns with this patch can also be discussed right here on Phab or on Discord (like we've done).

I haven't gotten around looking at all the responses on this patch to form a good opinion. I'm aware that answers have been posted a while ago, however I've been extremely busy, both privately and other higher priority LLVM related things.

So I have still concerns since I didn't have time to get a good look.

This revision was landed with ongoing or failed builds.Oct 16 2023, 10:49 AM
This revision was automatically updated to reflect the committed changes.