This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Rewrite the tuple constructors to be strictly Standards conforming
ClosedPublic

Authored by ldionne on Feb 11 2021, 9:40 AM.

Details

Summary

This nasty patch rewrites the tuple constructors to match those defined
by the Standard. We were previously providing several extensions in those
constructors - those extensions are removed by this patch.

The issue with those extensions is that we've had numerous bugs filed
against us over the years for problems essentially caused by them. As a
result, people are unable to use tuple in ways that are blessed by the
Standard, all that for the perceived benefit of providing them extensions
that they never asked for.

Since this is an API break, I communicated it in the release notes.
I do not foresee major issues with this break because I don't think the
extensions are too widely relied upon, but we can ship it and see if we
get complaints before the next LLVM release - that will give us some
amount of information regarding how much use these extensions have.

Diff Detail

Event Timeline

ldionne created this revision.Feb 11 2021, 9:40 AM
ldionne requested review of this revision.Feb 11 2021, 9:40 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 11 2021, 9:40 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript

I would love to have some eyes on this. I know it's a nasty patch with a lot of metaprogramming, but I'd like to get some opinions on:

  1. The correctness of the change
  2. What people think about removing those extensions
  3. Anything else reviewers want to talk about :-)

Pinging a few people that might be interested by this: @CaseyCarter @arthur.j.odwyer @curdeius @zoecarver @EricWF (not exhaustive)

Oh, and some other levels of testing we can do:

  1. Build our internal code bases with the change (I'm thinking Apple and Google but others are welcome to do it too)
  2. Maybe we can have some open-source libraries try out the change - for example I'd love to see if Range-v3 sees issues with this

More comments later :)

libcxx/include/tuple
461

The standard has about 100 ways of abbreviating the word "constructor".

464

I assume the _Dep is just a "dummy type" for the _EnableIf? If so, maybe we could keep calling it _Dummy, because we use that name in a lot of places in the codebase to indicate that the type doesn't matter, or is just used for a following _EnableIf. Right now, when I look at this, I'm wondering if it was changed to add a "dependency" to this constructor (which I don't think is the case).

Additionally, if you change this to say class _T2 = _Tp or class _Dummy = _Tp you can omit the _And, I think. (Note: here it doesn't really matter because you'd go back to using __all but below I think you can eliminate some extra template logic, which is good for both readability and compile time.)

468

Nit: space.

(Same comment on a few of these ctors.)

492

This constructor could be _NOEXCEPT_(_And<is_nothrow_default_constructible<_Tp>...>::value), right?

I think there are a couple of missing noexcepts here, but I think that's better done as a follow-up patch, to keep things separated. I am happy to work on this, after this patch lands. (Also, I won't point it out again, because it's not really relevant to this patch.)

zoecarver added inline comments.Feb 11 2021, 10:26 AM
libcxx/include/tuple
481

When were these conditionally explicit overloads added? I think C++17.

This is a bit annoying because in C++20 we get the explicit(bool) operator which would make this implementation about 50% smaller.

It looks like that was added to clang in V10. I wonder if we could get away with a macro to do this. There would be a small overlap of users on the V6-V10 clang version using C++17 who might be affected.

zoecarver added inline comments.Feb 11 2021, 11:05 AM
libcxx/include/tuple
590

Nit: it might be simpler to just say sizeof...(_Up) != 1 || is_same<_Up, tuple>... (or even __is_same(_Up..., tuple)). Then you could get rid of _IsThisTuple.

938

If you're using _Up1 here, do you need the _Dep template type param?

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/alloc_copy.pass.cpp
57

Is this not an extension? If it is an extension, don't we want to keep this behind a condition so that the test suite is more portable?

ldionne updated this revision to Diff 323162.Feb 11 2021, 2:40 PM
ldionne marked 7 inline comments as done.

Address Zoe's comments.

Also, I didn't mention it but I didn't add new tests. This is technically not increasing the API surface (in fact decreasing it), and the existing tests seemed to have the right coverage already.

libcxx/include/tuple
461

Ahah, indeed.

464

_Dep stands for _Dependent. IOW, I'm making the _And<...> instantiation dependent on a template argument to avoid it being evaluated eagerly. I can use _Dummy, but I went for the shortest thing that made sense since it's used to make _FirstType<...> and _SecondType<...> dependent below, where having a long name hurts readability. I think _Dummy is fine, though.

Additionally, if you change this to say class _T2 = _Tp

I can't do that, because _Tp is a parameter pack. Otherwise, I would replace all these occurrences by something notionally equivalent to <class ..._DependentTp = _Tp>, and I could get rid of much of the complexity. Unless I've missed something major, that isn't valid C++ right now though.

481

When were these conditionally explicit overloads added? I think C++17.

Hmm, I'm not 100% sure, but that sounds about right. Perhaps we've been supporting those in older Standards as an extension. While I generally dislike extensions, I think that not enabling those implicit/explicit constructors pre-C++17 would be a pretty big cost in complexity for little benefit, so I'd rather implement them even pre-C++17. Open to discussion, of course.

This is a bit annoying because in C++20 we get the explicit(bool) operator

Yes, I agree, those are a huge pain to write without explicit(bool), but I don't think there's a way around that TBH. To be fair, there is quite a bit of code duplication, but it's simple code. I was also planning on simplifying the calls to the __base_ constructor in a follow-up patch to reduce the magnitude of the duplication. Let me know if you have a blocking comment or if you want me to take an action here.

492

I tried keeping the same noexcepts that we had before this patch. None of the allocator_arg_t constructors had noexcept before the patch, so their don't either after the patch.

I don't really like those constructors :-).

590

I agree, I'd like to get rid of _IsThisTuple too. I tried the following three alternatives:

_Not<is_same<__uncvref_t<_Up>, tuple>...>

This doesn't work because we're expanding a pack into _Not, which takes a single template parameter.

_Not<is_same<__uncvref_t<_Up>..., tuple> >

This doesn't work because we're expanding a pack into is_same, which takes two parameters exactly.

_BoolConstant<!__is_same(__uncvref_t<_Up>..., tuple)>

This doesn't work but I don't know why.

Let me know if you can think of another way to get rid of _IsThisTuple.

938

Ok, so that's an excellent question, and I would have expected the answer to be "No". In fact, I initially wrote all those without these _Lazy calls whenever some arguments were dependent, but I was getting errors saying (basically): "you're expanding parameter packs of different lengths in the definition of the _EnableMoveFromPair alias".

I thought that sort of failure within the definition of an alias template would be considered part of the immediate context for SFINAE and it would result in the overload not being selected. But I was getting a hard error instead. I'm not sure whether I'm doing something subtly wrong, or my expectation about substitution failures in alias templates is wrong, or if it's a compiler bug. I decided not to investigate further because this worked, but I can if you request it.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/alloc_copy.pass.cpp
57

Actually, I couldn't figure out what about the test case was an extension, so I thought it might have been marked as an extension incorrectly.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/alloc_move.pass.cpp
57

Same here, is this really testing an extension? I can't see what's an extension about it.

I haven't really tried to understand this code, but if it passes tests, I certainly don't object to it.

libcxx/docs/ReleaseNotes.rst
57

IMHO it would be nice to provide code snippets (Godbolt links?) to things that have changed.
IIUC the removed signatures are https://godbolt.org/z/v49qKe
but I don't know what signatures (if any) are being added.

libcxx/include/tuple
590

FWIW, I don't understand what the "extension" is here, nor what "mis-behaved user constructors" we're worried about. But I haven't cared enough to remove that line and re-run the unit tests to find out.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR27684_contains_ref_to_incomplete_type.pass.cpp
17

// See https://llvm.org/PR27684 — as long as you're normalizing comments here.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/UTypes.pass.cpp
40

Nit: when there are fewer initializers than elements or fewer constructor arguments than tuple elements.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/alloc.pass.cpp
97

I would like to see tests here for

std::tuple<int, int> t2(derived, A1<int>(), 42, 42);
std::tuple<int, int> t3(derived, A1<int>(), std::tuple<int, int>{42, 42});

just to prove that these work for non-empty tuples too.

libcxx/include/tuple
938

I think Zoe is right that you could remove _Dummy. I have no opinion on the rest of the simplification you tried. But just this should work, right?:

template <class _Alloc, class _Up1, class _Up2, _EnableIf<
    _And<
        _Lazy<_EnableMoveFromPair, _Up1, _Up2>,
        _Not<_Lazy<_And, // explicit check
            is_convertible<_Up1, _FirstType<_Tp..., void> >,
            is_convertible<_Up2, _SecondType<_Tp..., void> >
        > >
    >::value
, int> = 0>

(replacing _Dummy with void in the places IIUC we don't care what it is)
(or maybe you need SecondType<_Tp..., void, void>?)

What would you think about extending the _v type traits to always be available (no matter the standard version)? Then we could remove a lot of the _Ands and similar types which would mean we would instantiate a lot few types, therefore, having much better compile time. Not a big deal, but it might be nice.

libcxx/include/tuple
464

I suppose _Dep makes sense now that I understand what it means. But I still think _Dummy is better because that's what we use everywhere else, so thanks for changing it.

I can't do that, because _Tp is a parameter pack.

Ah, yes, you're right. For some reason, I thought you could, but it looks like that's not a feature.

481

While I generally dislike extensions, I think that not enabling those implicit/explicit constructors pre-C++17 would be a pretty big cost in complexity for little benefit, so I'd rather implement them even pre-C++17.

I think if we removed the extension, we'd actually reduce the complexity significantly. Here's what I had in mind:

#if STD > 14 && CLANG >=10
#define _EXPLICIT_COND(...) explicit(...)
#else
#define _EXPLICIT_COND
#endif

Then we could just annotate all these overloads with _EXPLICIT_COND(...). The problem is we'd be non-conforming for those using C++17 on clang V9 or lower, and this would potentially break people's code if they were on C++11 or 14. WDYT? Could we get away with it? 😉

This is a non-blocking comment. I just thought it might be a way to reduce a bunch of code (but it looks it probably won't work for now, but maybe we can come back to it in a few years).

590

This works for me:

template<class ...U>
    static const size_t u_types_constraints =
        sizeof...(U) == sizeof...(T) &&
        sizeof...(U) >= 1 &&
        !is_same<Tuple, U...>::value;

template<class ...U, class = _EnableIf<u_types_constraints<U...> > >
    Tuple(U&&...) { }

FWIW, I don't understand what the "extension" is here, nor what "mis-behaved user constructors" we're worried about. But I haven't cared enough to remove that line and re-run the unit tests to find out.

I also am curious about what this means. I thought it was just to ensure that we picked the right overload (so that this doesn't look like a move constructor when _Up = tuple).

803

Wait... why do we have _Dummy types here? Why can't this just be:

template <class U1, class U2>
static const size_t __enable_copy_from_pair = sizeof...(_Tp) == 2 && 
    is_constructible<_FirstType <_Tp...>, const U1&>::value &&
    is_constructible<_SecondType<_Tp...>, const U2&>::value;
zoecarver added inline comments.Feb 11 2021, 10:26 PM
libcxx/include/tuple
938

First, why do we need the void types? We've already checked that there are at least two elements, and this should be a short-circuiting operator.

Second, I think type alias types should be counted as dependent types as long as they're substituted with template type arguments. So, what Arthur proposed should work. But I can play around with this patch a bit more locally tomorrow and try to get it working.

zoecarver added inline comments.Feb 12 2021, 11:36 AM
libcxx/include/tuple
590

Follow up: I got this working here. I like this approach much better because we instantiate less than half the types, so it should be much better for compile time.

Side note: I spent way too long banging my head against the wall trying to figure out why we can only zip parameter packs in type aliases (and not static vars). It turns out there's an issue for this open: 1844. Basically, various compilers read "immediate context" differently, so sometimes we get a hard error and sometimes we get a substitution failure. Here's a Godbolt to demonstrate the problem.

libcxx/include/tuple
590

@zoecarver: Your Godbolt (https://godbolt.org/z/erqqGT ) makes perfect sense to me. Here's a "reduced" (but still icky) version: https://godbolt.org/z/x6c67o
If you have a type alias that says "Yy is the type you get by zipping these packs," and the packs can't be zipped, well, that's SFINAE-friendly. If you have a static constexpr bool whose initializer says "Xx is true if zipping these packs gives you foo, or false if zipping these packs gives you bar," and the packs can't be zipped, well, that's not SFINAE-friendly (not an "immediate context," if I've got my standardese right).

938

why do we need the void types?

Because when sizeof...(_Tp) == 1, then _SecondType<_Tp...> is ill-formed — which would make the entire type-expression

_EnableIf<
    _And<
        _Lazy<_EnableMoveFromPair, _Up1, _Up2>,
        _Not<_Lazy<_And, // explicit check
            is_convertible<_Up1, _FirstType<_Tp..., void> >,
            is_convertible<_Up2, _SecondType<_Tp..., void> >
        > >
    >::value
, int>

immediately ill-formed — which would make this constructor template drop out of overload resolution whenever sizeof...(_Tp) == 1. Hmm, I guess that's totally OK in this case!

The key thing to watch out for in all this metaprogramming ickiness is that regardless of short-circuiting evaluation of truth values, the compiler is still going to be instantiating the types that we name in the expression, such as _FirstType<_Tp..., void>. I guess that's exactly what _Lazy is for, though; and so if _SecondType<_Tp...> didn't suffice, the next thing we should try is not _SecondType<_Tp..., void> but rather _Lazy<_SecondType, _Tp...>.

ldionne updated this revision to Diff 323801.Feb 15 2021, 11:01 AM
ldionne marked 15 inline comments as done.Feb 15 2021, 11:01 AM

Address review comments, fix compilation on GCC and other minor simplifications to the SFINAE.

I addressed all comments with a direct suggestion. For other comments on using various patterns for doing SFINAE, I actually reworked everything to make GCC happy.

You'll notice that we now never call a metafunction with the incorrect number of arguments. GCC does not seem to handle SFINAE failures for mismatched parameter pack lengths correctly, so I had to use this approach.

I still need to investigate a test failure in C++11 and C++14 mode - the behavior after this patch matches the libstdc++ behavior, so I think the test was wrong, but I need to understand it better.

libcxx/include/tuple
481

Does Clang trunk provide explicit(bool) as an extension in C++ < 20 modes? If so, then I suggest we come back once we drop support for older Clangs and remove the duplication.

Otherwise, if Clang doesn't support explicit(bool) in pre-C++20 mode, we will never be able to remove this duplication because we still have to implement the explicit/implicit constructors for tuple pre-C++20 (it was applied as a LWG issue IIRC).

590

FWIW, I don't understand what the "extension" is here, nor what "mis-behaved user constructors" we're worried about.

It refers to the test in libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR23256_constrain_UTypes_ctor.pass.cpp. That test fails if you remove that line.

We're on the same page regarding the SFINAE behavior for variable templates vs template aliases, but I don't think it's relevant to the issue directly at hand.

The issue here is that we want the overload to be enabled when sizeof...(_Up) > 1. But when that's the case, is_same will cause a SFINAE error because we'll try to instantiate it with too many template arguments. Basically, in order to get rid of _IsThisTuple, we'd have to write something like this instead:

_And<_BoolConstant<sizeof...(_Up) == 1>, _Lazy<is_same, uncvref_t<_Up>..., tuple> >

I don't think that's better, but WDYT?

803

Because when sizeof...(_Tp) < 2, the "expression" _FirstType<_Tp...> is a hard error. That "expression" is "evaluated" when the tuple template is instantiated, not when we instantiate the template alias.

938

the next thing we should try is not _SecondType<_Tp..., void> but rather _Lazy<_SecondType, _Tp...>.

That doesn't work, because _SecondType is an alias that expands to the type itself. So _Lazy would end up inheriting from that type, which isn't what we want.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/UTypes.pass.cpp
40

Thanks, much better indeed!

Also, I just realized that we can use fold expressions here (on all supported compilers even in C++03 mode). I bet that would be faster/cleaner than instantiating all the _Ands.

libcxx/include/tuple
590

But when that's the case, is_same will cause a SFINAE error because we'll try to instantiate it with too many template arguments.

I think this is the point of confusion. This is also why we are able to use a static variable. Because tuple is not a parameter pack, so there are never too many template arguments.

However, in the diff I liked to, Arthur made a really good point, which is that using a static variable might not be faster here if we have to uncvref_t all of the Us.

So, I'm happy to ship this version. (Though, there is a small part of my brain saying, "I wonder if it would be faster to use a static variable and _IsThisTuple?")

758

I think there should be an (is_constructible_v<T, U&&> && ...) check.

zoecarver added inline comments.Feb 16 2021, 1:26 PM
libcxx/include/tuple
481

Yes. If there were a world where libc++ only supported Clang V10 and GCC V9, I think this could work really well.

Also, I just realized that we can use fold expressions here (on all supported compilers even in C++03 mode). I bet that would be faster/cleaner than instantiating all the _Ands.

So we can't actually use fold expressions because they don't short-circuit instantiations. A fold expression like trait<args>::value && ... gets expanded to trait<arg0>::value && trait<arg1>::value && trait<arg2>::value && <etc>. While the evaluation of the boolean is short-circuited, the instantiation of trait<argn>::value is not, which is what we're trying to do here. As a result, if for example trait<arg2>::value causes a hard error, it will be instantiated even if trait<arg0>::value instantiates fine and evaluates to false. Does that make sense?

Also, I just realized that we can use fold expressions here (on all supported compilers even in C++03 mode). I bet that would be faster/cleaner than instantiating all the _Ands.

So we can't actually use fold expressions because they don't short-circuit instantiations. A fold expression like trait<args>::value && ... gets expanded to trait<arg0>::value && trait<arg1>::value && trait<arg2>::value && <etc>. While the evaluation of the boolean is short-circuited, the instantiation of trait<argn>::value is not, which is what we're trying to do here. As a result, if for example trait<arg2>::value causes a hard error, it will be instantiated even if trait<arg0>::value instantiates fine and evaluates to false. Does that make sense?

Hmm. I understand what you're saying, but I think that's only the case if it's a static var. If it's actually part of the type (i.e., it's used to specialize a _BoolConstant), I think it works the same way. Here's an example.

[...]

Hmm. I understand what you're saying, but I think that's only the case if it's a static var. If it's actually part of the type (i.e., it's used to specialize a _BoolConstant), I think it works the same way. Here's an example.

Your example is broken - it SFINAEs out because you're zipping parameter packs of different lengths. See https://godbolt.org/z/qna759.

Nope, but that's fixed by the parent patch https://reviews.llvm.org/D50106.

libcxx/include/tuple
590

I would like to continue down this rabbit-hole, but I think we're all talking past each other at this point.

  • There's the question of pack expansions, like where it's legit to put is_same<tuple, Up...> and where not.
  • There's the question of eager vs lazy instantiation, like where it's legit to put (is_convertible<Tp, Up>::value && ...) and where _And<is_convertible<Tp, Up>...> (this is https://bugs.llvm.org/show_bug.cgi?id=23256 as I understand it).
  • There's some desire to minimize instantiations, for speed. But there's also an explicit goal to avoid certain completions of is_convertible<T,U> because PR23256.
  • There's maybe other questions.

(Perhaps _And<_Not<x>...> could be replaced by _NoneOf<x...>, if our toolbox were expanded to include an implementation of _NoneOf.)

I'd like to see either of these next steps:

  • @ldionne to just land this thing already, or
  • @ldionne to identify a single measurable goal, e.g. "let's figure out how to remove the identifier _IsThisTuple while still passing the tests in this patch," which we can then resume rabbit-holing on (but via suggesting actual patches that pass all the tests locally, not just brainstorming untested ideas).
libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/deduct.pass.cpp
16

What about gcc-8?
And does this patch change something that breaks gcc-9 and gcc-10, or is it more like "we never tested it and I just now noticed that it's broken"? Maybe this belongs in a severable commit.

Your example is broken - it SFINAEs out because you're zipping parameter packs of different lengths. See https://godbolt.org/z/qna759.

Oh, I thought you were saying that it wouldn't work because zipping parameter packs of different sizes would be a hard error. Anyway, I think I see what you're saying now. Thanks for the Godbolt link.


I think @Quuxplusone makes a good point. And for what it's worth, I'd be fine with this patch landing in its current state. I think we should attempt to remove extra template instantiations, but this can also be done after the fact and AFAICT this is already better than main (or at least easier to grok).

I think I'd land this patch as-is and then try to identify some instantiations we want to remove and do that. Arthur, thanks for taking us back on track :-).

I'll need to figure out what the issue is with the remaining tests that fail in C++11 and C++14. Then I'll land this patch internally at Apple in a test build and see if there's any breakage. If that all works, I'll land this here.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/deduct.pass.cpp
16

I forgot to add gcc-8, I'll do that.

This patch causes GCC 10 to segfault when CTAD is used.

ldionne updated this revision to Diff 333379.Mar 25 2021, 11:46 AM

Rebase onto main

I was trying to close the loop on this and got down to the following behaving differently in C++11/14 and C++17/20: https://godbolt.org/z/6x75njn7v

#include <cstdio>

template <class ...T>
struct tuple {
  tuple() { }
  template <class ...U>
  tuple(tuple<U...>&&) { std::printf("construction\n"); }
};

struct Foo { };

struct Derived : tuple<int> {
  template<class U>
  operator tuple<U>() && { std::printf("conversion\n"); return {}; }
};

int main(int, char**) {
  tuple<Foo> bar(Derived{});
}

It looks like in C++11 and 14, we prefer the constructor, whereas in C++17 and above, we prefer the conversion. Both GCC and Clang have the same behavior. So I'm going to tweak the tests accordingly.

ldionne updated this revision to Diff 334794.Apr 1 2021, 12:26 PM

Fix tests in C++11 and C++14

ldionne accepted this revision as: Restricted Project.Apr 1 2021, 12:31 PM

I will ship this if CI passes, as discussed with the reviewers.

zoecarver accepted this revision.Apr 1 2021, 12:33 PM

⛵️it!

This revision is now accepted and ready to land.Apr 1 2021, 12:33 PM
EricWF added a comment.Apr 1 2021, 1:08 PM

I'm a bit worried about some of the tests that were deleted. Not the ones for extensions, but some of the ones added to prevent regressions with SFINAE.

I would like to look into why they now fail.

EricWF requested changes to this revision.Apr 1 2021, 1:08 PM
This revision now requires changes to proceed.Apr 1 2021, 1:08 PM

Here is one regression I've found so far:

https://godbolt.org/z/sahqsqarb

ldionne updated this revision to Diff 335636.Apr 6 2021, 12:53 PM

Fix bug with eager default constructor

Here is one regression I've found so far:

https://godbolt.org/z/sahqsqarb

Thanks for the find. Did you find anything else?

I'm a bit worried about some of the tests that were deleted. Not the ones for extensions, but some of the ones added to prevent regressions with SFINAE.

Which specific tests are you worried about? I looked and I think the only ones I removed were related to libc++ extensions which do not exist after this patch.

There's another bug with SFINAE depth increases that's blowing up at Google.
I suspect we can get that back.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/test_lazy_sfinae.pass.cpp
22

Why were these tests deleted?

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/tuple_array_template_depth.pass.cpp
1

Why was this file deleted?

ldionne marked 2 inline comments as done.Apr 7 2021, 10:23 AM

There's another bug with SFINAE depth increases that's blowing up at Google.
I suspect we can get that back.

Can you post a reproducer? I'll be happy to fix it.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/test_lazy_sfinae.pass.cpp
39

That class was never used AFAICT.

57–66

This test requires libc++'s reduced arity initialization, as the comment says. That extension was removed.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/tuple_array_template_depth.pass.cpp
1

Because the extension that allows constructing a tuple from an array was removed (as noted in the release notes).

ldionne marked an inline comment as done.Apr 12 2021, 2:49 PM

There's another bug with SFINAE depth increases that's blowing up at Google.
I suspect we can get that back.

Can you post a reproducer? I'll be happy to fix it.

I'll commit this on Wednesday unless I have more information. I don't want to break your internal users, but I'd like to get moving with this patch since it has good chances of being difficult to rebase, so keeping it around for a while is quite risky.

ldionne accepted this revision as: Restricted Project.Apr 23 2021, 9:45 AM
This revision was not accepted when it landed; it landed in state Needs Review.Apr 23 2021, 9:46 AM
This revision was automatically updated to reflect the committed changes.
miyuki added a subscriber: miyuki.Apr 28 2021, 6:36 AM

Hi. It looks like this change broke the following code:

#include <tuple>

void test() {
    using pair_t = std::pair<int, char>;
    constexpr std::tuple<long, long> t(pair_t(0, 'a'));
}
<source>:5:38: error: constexpr variable 't' must be initialized by a constant expression
    constexpr std::tuple<long, long> t(pair_t(0, 'a'));
                                     ^~~~~~~~~~~~~~~~~
<source>:5:38: note: non-constexpr constructor 'tuple<int, char, _And, 0>' cannot be used in a constant expression
/opt/compiler-explorer/clang-trunk-20210428/bin/../include/c++/v1/tuple:900:5: note: declared here
    tuple(pair<_Up1, _Up2>&& __p)
    ^
1 error generated.
Compiler returned: 1

https://godbolt.org/z/6sd9TTMrq

Could you please have a look into it?

jgorbe added a subscriber: jgorbe.Apr 29 2021, 11:09 AM

We're seeing some issues with this patch manifested as complicated tuple expressions that hit compiler limits with template depth. A bit of a synthetic example, but here it is: https://godbolt.org/z/TzvzWhYYs

template <size_t... I>
void MakeFoo(std::index_sequence<I...>) {
  auto tup = std::make_tuple(I...);
  (void)tup;
}

void Broken() {
  // At trunk, when this is >= 508, we see:
  // fatal error: recursive template instantiation exceeded maximum depth of
  // 1024

  auto seq = std::make_index_sequence<508>{};
  MakeFoo(seq);
}

Prior to this patch, the breaking point is at 1021. Interestingly though, this didn't reproduce in the latest clang/llvm 12 release on godbolt. I did a second bisection to D50106 as the point where std::make_index_sequence<1021>{} newly fails.

In general this removes a bunch of carefully constructed SFINAE meant to improve compile times.
I'll look into getting that back.

libcxx/include/tuple
878

Why can't we use alias templates rather than instantiating a big type here?

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR31384.pass.cpp
52

What's going on here? Why are there different answers depending on the dialect?

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/tuple_array_template_depth.pass.cpp
1

This tested template depth. And this patch broke large tuples. It should have been rewritten in some other form.

We need to reallow big tuples and add another test.

1

It's worth noting that this was a conforming extension. Because array supports the tuple-like protocol. I'm disappointed this was removed.

I'm working to see if I can fix forward the template depth and constexpr
issues, but I would appreciate some help from @ldionne as the original
author.

I'm working to see if I can fix forward the template depth and constexpr
issues, but I would appreciate some help from @ldionne as the original
author.

I'm working on the constexpr issue right now. I only just saw this.

In general this removes a bunch of carefully constructed SFINAE meant to improve compile times.

The SFINAE in this patch is also meant to instantiate as few templates as possible when it can. After looking into it, I think the problem is that our implementation of _And is a bit naive, it shouldn't recurse as much. I'm pretty sure that'll solve the problem.

libcxx/include/tuple
878

Cause we're hiding the instantiation of the _And behind the struct instantiation, such that when e.g. _BoolConstant<sizeof...(_Tp) == 2> is false below, we don't instantiate anything. Mentioning a template specialization is cheap when you don't actually instantiate it.

This is the fix for the constexpr issue:

commit 3b8ab30ed526b15c3f82c61bb9476606db6f1630
Author: Louis Dionne <ldionne.2@gmail.com>
Date:   Fri Apr 30 15:52:26 2021 -0400

    [libc++] Fix constexpr-ness of std::tuple's constructor

    Mentioned in https://reviews.llvm.org/D96523.

I'm also going to overhaul all the tests as a follow-up, but I wanted to get the fix out the door ASAP.

Now I'll look into the recursion depth issue.

Another issue we're seeing is interactions between std::any and gmock -- I'll see if I can reduce it to something w/o the gmock dependency.

class X {
  // D96523 fixes this:
  // MOCK_METHOD(void, Func1, (const std::any&));
  // D96523 breaks this:
  MOCK_METHOD(void, Func2, (const std::any&, int));
};

(or: https://www.godbolt.org/z/454e8Mznx)

I don't know yet if this is an issue with libc++ or a misuse of libc++ by gtest.

ldionne added a comment.EditedApr 30 2021, 1:30 PM

This should fix the instantiation depth issue: https://reviews.llvm.org/D101661

@rupprecht Please ping me once you've reduced to something that doesn't involve Gmock and I'll look into it.

This should fix the instantiation depth issue: https://reviews.llvm.org/D101661

@rupprecht Please ping me once you've reduced to something that doesn't involve Gmock and I'll look into it.

#include <any>
#include <tuple>

template <typename a>
class b {
  b(a);
};

template <typename>
struct c;
template <typename m, typename... d>
struct c<m(d...)> {
  using e = std::tuple<b<d>...>;
};

template <typename>
class f;

template <typename g>
class h {
  typename c<g>::e i;
};

template <typename m, typename... d>
struct f<m(d...)> {
  h<m(d...)> n();
};

class j {
  void k() { l.n(); }
  // Note: D96523 breaks this:
  f<void(std::any, int)> l;
  // Note: D96523 fixes this:
  // f<void(std::any)> l;
};

https://www.godbolt.org/z/4W8b1G7Yd

Thanks a lot for the reduction. Here it is further reduced without libc++'s std::tuple or std::any: https://www.godbolt.org/z/5r7schPMK

If you add the static_assert, the error goes away. That is very suspicious to me. I know how to "fix" it from the standard library side, but it seems like we should take that to Clang folks too. WDYT?

The bug with std::any should be addressed by https://reviews.llvm.org/D101770. As I said, I think it's a compiler bug, but https://reviews.llvm.org/D101770 should work around it.

I think we're all good now. The template depth fix in 84f0bb619507caf7c4809400138dae1da578282a actually exposed a long standing clang bug which needed another fix in 6bbfa0fd408e81055c360c2e059554dd76fd7f09. Thanks for the quick followups!

ychen added a subscriber: ychen.Nov 9 2021, 4:12 PM
ychen added inline comments.
libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR31384.pass.cpp
52

Yeah, I'm curious too. GCC/libstdc++ seems to have the same behavior.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR31384.pass.cpp
52

https://godbolt.org/z/vGPYqEnfo
C++14 sets bar to tuple<Explicit>{ Explicit(42) } by converting d's int(42) using Explicit(int) and then building a tuple out of that.
C++17 sets bar to tuple<Explicit>{} by invoking d.operator tuple<Explicit>().
Really, this test should be verifying std::get<0>(bar) == [42 or 0] as well as count == [1 or 2].

I don't know exactly why C++14 hadn't been considering the conversion operator, or why C++17 started considering it, but I would assume that the reason is buried somewhere in the changes due to LWG2312, LWG2419, LWG2549. (Actually I'm very puzzled as to why the conversion operator wouldn't have been the obviously best match in C++14. Did C++17 change something in the core language about conversion operators, completely independent of tuple?)

Note that in October 2021, tuple's constructor constraints changed again due to LWG3121, LWG3155. I assume libc++ has not yet implemented those changes; I don't know if they'll affect this test case again at all.

ychen added inline comments.Nov 10 2021, 1:08 PM
libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR31384.pass.cpp
52

After some digging, this seems from rG67ef14fe486e169f937737672d68. But still no idea what is changed in the standard text.

libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR31384.pass.cpp
52

Perhaps this is relevant:
https://godbolt.org/z/hKbncT4zf
GCC 7+ and Clang 6+ agree that in C++17 this is legal, but in C++14 it's not.

struct Widget { Widget(const Widget&) = delete; };
struct From { operator Widget() const; };
~~~
From f;
(void)Widget(f);

C++14 seems to treat this as a call to some constructor of Widget (namely, the copy constructor, with an argument of f.operator Widget()); C++17 "more rightly" treats it as a request to produce a Widget by whatever means, such as just calling operator Widget alone. ...Or actually, instead, maybe C++17 is still treating this as Widget(f.operator Widget()); it's just that now Widget(some-prvalue-widget) is a no-op instead of trying to call the copy constructor. I wonder if there's any way to observe that difference, or if it's just philosophical at this point.

ychen added inline comments.Dec 2 2021, 1:02 PM
libcxx/test/std/utilities/tuple/tuple.tuple/tuple.cnstr/PR31384.pass.cpp
52
libcxx/include/type_traits