This Implements P2266 Simpler implicit move.
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential D99005
[clang] Implement P2266 Simpler implicit move mizvekov on Mar 19 2021, 8:24 PM. Authored by
Details
Diff Detail
Event TimelineComment Actions For coroutines I have D68845. The current code is wrong, and not trivial to fix. We have never properly implemented P1825 and will probably just go with P2266 unconditionally, because that two-phase lookup is very cumbersome to imlement. Comment Actions Okay, once this is properly tested, we can spin a separate patch from this, just taking the coroutine bits unconditionally, and propose a new DR. Comment Actions
Comment Actions Conspicuously missing: tests for decltype(auto) return types, tests for co_return. Personally I'd rather see these new p2266-related tests go in a separate test file, not appended to the existing file, so as to keep each test file relatively simpler and shorter. (But I don't know that those-in-charge would agree.)
Comment Actions With my previous comment I meant that it's better if you leave out the co_return bits for now because it's wrong anyway. We can't use PerformMoveOrCopyInitialization. It would just generate merge conflicts. I'll probably just update my change once the committee made up its mind. I can rebase it on top of this if it needs to be dependent on C++20 vs C++2b. Comment Actions Though the first comment warns you that this is a work in progress and I just didn't get around to doing it yet ;) Though decltype I had tested manually, co_return I have just begun testing at all. I believe the decltype tests should go into the other file relevant to the pertinent section. One problem with leaving them here is that it would be extra noise for the tests for the older standards which don't even support the keyword at all.
They go into files named after section / paragraph, and I haven't come across examples where they were split yet, but I haven't looked specifically for this. I am using PerformMoveOrCopyInitialization with AllowNRVO always false for co_return, which effectively makes it a PerformCopyInitialization (just replacing them instead was one of the WIP cleanups I mentioned).
Comment Actions Okay, I think I see now what you mean. struct task { struct promise_type { ... void return_value(T &&value) {} }; }; task<NoCopyNoMove> local2val() { NoCopyNoMove value; co_return value; } We should expect the test above to work, by binding value to the rvalue reference in task's promise, right? Comment Actions Precisely, we don't want to do any initialization at all. (There will be an initialization for the parameters in BuildCallExpr.) Implicit move just means that we're converting the lvalue to an xvalue. On second thought, since @Quuxplusone's proposal makes things so simple you can just do this right yourself: instead of the initialization just do the cast, as D68845 does it. That's it. Then let's see what we can get for C++20. If the proposal is accepted and gets DR status for co_return we might not need my change at all. Comment Actions Changed:
Still missing:
Comment Actions Data points: I've run this new Clang in -std=c++2b mode over my employer's (medium-sized, not too crazy) C++17 codebase, and also built Clang with Clang. Neither one required any source-code changes to deal with p2266. (The C++17 codebase did need about 10 identical changes to fix up ambiguous operator== overloads for C++20.) Could we get someone to check out the Bloomberg codebase? Comment Actions So it turns out there was a pre-existing bug where ObjC++ blocks My latest refactoring accidentally fixed this in the return statement I am not sure there is an easy way to fix this, as BlockDecl This might require some surgery to fix, so for now I am just
Comment Actions
Comment Actions IMHO we should just land this already. (It's been sitting without major update since mid-April.) It is intended to affect only -std=c++2b mode, and implements a paper (full disclosure: my paper) targeting C++2b that EWG wants implementation experience with. So I have proposed to @mizvekov that he should land this on Monday, unless someone yells "stop" before then. Comment Actions Actually we can land it now. @rsmith gave me the go ahead in private. The windows pipeline passes. The latest debian build failure is some random fortran thing, so we are good. Comment Actions (In a build prior to https://reviews.llvm.org/rGc60dd3b2626a4d9eefd9f82f9a406b0d28d3fd72 "Revert '[clang] NRVO: Improvements and handling of more cases.'") I see the following (reduced from https://git.libreoffice.org/core/+/649313625b94e6b879848fc19b607b74375100bf/o3tl/qa/compile-temporary.cxx) started to fail under -std=c++2b with this change (and continues to compile fine with -std=c++20): $ cat test.cc template <typename T> constexpr T& temporary(T&& x) { return x; } template <typename T> constexpr T& temporary(T&) = delete; void f(int*); int g(); void h() { f(&temporary(int())); f(&temporary(g())); } $ clang++ -std=c++2b -fsyntax-only test.cc test.cc:1:62: error: non-const lvalue reference to type 'int' cannot bind to a temporary of type 'int' template <typename T> constexpr T& temporary(T&& x) { return x; } ^ test.cc:7:8: note: in instantiation of function template specialization 'temporary<int>' requested here f(&temporary(int())); ^ test.cc:8:8: error: no matching function for call to 'temporary' f(&temporary(g())); ^~~~~~~~~ test.cc:2:36: note: candidate function [with T = int] not viable: expects an lvalue for 1st argument template <typename T> constexpr T& temporary(T&) = delete; ^ test.cc:1:36: note: candidate template ignored: substitution failure [with T = int] template <typename T> constexpr T& temporary(T&& x) { return x; } ^ 2 errors generated. It is not clear to me whether that is an intended change in behavior according to http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2266r1.html "Simpler implicit move", or whether it is a bug in this implementation. Comment Actions Yeah that one is intentional, under P2266 as far as implicit move is concerned, that code behaves similarly to this code under C++20 (which is diagnosed for the same reason): template <typename T> constexpr T& temporary(T&& x) { return static_cast<T&&>(x); } If you want this semantics with P2266, you could write: template <typename T> constexpr T& temporary(T&& x) { return static_cast<T&>(x); } And the above also works on C++20. Thank you for reporting this, it's the kind of feedback we needed about real world usage of code that would break under these new rules. Comment Actions @sberg: Thanks for the example! @mizvekov's comments are correct, but I wanted to put some more comments here for posterity. (I'll also try to find a place for this real-world example in the next revision of p2266.) The code in question is o3tl::temporary, documented as "Cast an rvalue to an lvalue." and implemented as template<class T> T& temporary(T&& x) { return x; } Comment Actions For the record, the one other breakage of a LibreOffice build with this is the handful of places that needed https://git.libreoffice.org/core/+/433ab39b2175bdadb4916373cd2dc8e1aabc08a5%5E%21 "Adapt implicit OString return value construction to C++23 P2266R1": In a nutshell, LO has an OString class with (non-explicit) constructors from any kind of char pointer or array, where the "array of N const char" variant (no. 3 below) is special, as it determines the string length as N - 1 (whereas all the other variants search for the terminating NUL to determine the length). That requires another constructor overload (no. 2 below) for non-const arrays, taking a non-const lvalue reference, which now causes issues when that constructor shall implicitly be used in return statements: $ cat test.cc #include <cstddef> template<typename> struct CharPtrDetector {}; template<> struct CharPtrDetector<char *> { using type = int; }; template<> struct CharPtrDetector<char const *> { using type = int; }; template<typename> struct NonConstCharArrayDetector {}; template<std::size_t N> struct NonConstCharArrayDetector<char [N]> { using type = int; }; template<typename> struct ConstCharArrayDetector {}; template<std::size_t N> struct ConstCharArrayDetector<char const [N]> { using type = int; }; struct OString { template<typename T> OString(T const &, typename CharPtrDetector<T>::type = 0); // #1 template<typename T> OString(T &, typename NonConstCharArrayDetector<T>::type = 0); // #2 template<typename T> OString(T &, typename ConstCharArrayDetector<T>::type = 0); // #3 }; OString good1a() { char * p; //... return p; // use #1 }; OString good1b() { char const * p; //... return p; // use #1 }; OString bad2() { char buf[10]; //... return buf; // use #2 } OString good3() { return "foo"; // use #3 } $ clang++ -std=c++20 -fsyntax-only test.cc $ clang++ -std=c++2b -fsyntax-only test.cc test.cc:41:12: error: no viable conversion from returned value of type 'char [10]' to function return type 'OString' return buf; // use #2 ^~~ test.cc:17:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'char [10]' to 'const OString &' for 1st argument struct OString { ^ test.cc:17:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'char [10]' to 'OString &&' for 1st argument struct OString { ^ test.cc:21:5: note: candidate constructor [with T = char [10]] not viable: expects an lvalue for 1st argument OString(T &, typename NonConstCharArrayDetector<T>::type = 0); // #2 ^ test.cc:19:5: note: candidate template ignored: substitution failure [with T = char [10]]: no type named 'type' in 'CharPtrDetector<char [10]>' OString(T const &, typename CharPtrDetector<T>::type = 0); // #1 ^ ~~~~ test.cc:23:5: note: candidate template ignored: substitution failure [with T = char [10]]: no type named 'type' in 'ConstCharArrayDetector<char [10]>' OString(T &, typename ConstCharArrayDetector<T>::type = 0); // #3 ^ ~~~~ 1 error generated. Comment Actions
Comment Actions And, again for the record, with a build of LibreOffice with clang-cl (and -Xclang -std=c++2b) on Windows, at least against the C++ standard library from Visual Studio 2019 version 16.20.2, I ran into two issues in the standard library itself, when using std::getline and std::istream::operator>>: In file included from C:/lo-clang/core/setup_native/source/win32/customactions/reg_dlls/reg_dlls.cxx:12: C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\string(66,12): error: non-const lvalue reference to type 'basic_istream<...>' cannot bind to a temporary of type 'basic_istream<...>' return _Istr; ^~~~~ C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\string(78,12): note: in instantiation of function template specialization 'std::getline<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t>>' requested here return getline(_STD move(_Istr), _Str, _Delim); ^ C:/lo-clang/core/setup_native/source/win32/customactions/reg_dlls/reg_dlls.cxx(197,17): note: in instantiation of function template specialization 'std::getline<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t>>' requested here while (std::getline(ss, sToken, L'|')) ^ 1 error generated. make[1]: *** [C:/lo-clang/core/solenv/gbuild/LinkTarget.mk:298: C:/lo-clang/core/workdir/CxxObject/setup_native/source/win32/customactions/reg_dlls/reg_dlls.o] Error 1 c:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30037/include/string and [build CXX] l10ntools/source/merge.cxx In file included from C:/lo-clang/core/l10ntools/source/merge.cxx:21: In file included from C:/lo-clang/core/include\sal/log.hxx:20: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\sstream:11: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\istream:11: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\ostream:11: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\ios:11: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\xlocnum:16: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\streambuf:11: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\xiosbase:12: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\system_error:14: In file included from C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\stdexcept:12: C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\xstring(4971,12): error: non-const lvalue reference to type 'basic_istream<...>' cannot bind to a temporary of type 'basic_istream<...>' return _Istr; ^~~~~ C:/PROGRA~2/MIB055~1/2019/COMMUN~1/VC/Tools/MSVC/1429~1.300/Include\xstring(4977,29): note: in instantiation of function template specialization 'std::operator>><char, std::char_traits<char>, std::allocator<char>>' requested here return _STD move(_Istr) >> _Str; ^ C:/lo-clang/core/l10ntools/source/merge.cxx(125,18): note: in instantiation of function template specialization 'std::operator>><char, std::char_traits<char>, std::allocator<char>>' requested here aInputStream >> sPoFile; ^ 1 error generated. make[1]: *** [C:/lo-clang/core/solenv/gbuild/LinkTarget.mk:301: C:/lo-clang/core/workdir/CxxObject/l10ntools/source/merge.o] Error 1 In both cases, the reason is that library happens to implement the overload taking basic_istream& by forwarding to the overload taking basic_istream&& (and not the other way around, as libc++ and libstdc++ happen to do), and the fix is to replace return _Istr; in the overloads taking basic_istream&& _Istr with return static_cast<basic_istream<_Elem, _Traits>&>(_Istr); (No idea if any MSVC standard library developers are around here, who might want to be CC'ed on this.) Comment Actions Thank you again @sberg. I have talked to the MSVC STL maintainers. They would be open to a pull request for fixing this. Comment Actions Thanks a lot for the link. Filed https://github.com/microsoft/STL/pull/2025 "Adapt some return statements to P2266R1 'Simpler implicit move'" now. Comment Actions This change has made this snippet fail. Comment Actions Hello! But is this example a reduction from a real world code base? Comment Actions This is an example that also appears to be impacted by this change: struct C { C(); C(C &c1); }; void foo() { try { C c1; throw c1; } catch (C c2) { throw; } } https://godbolt.org/z/dvEbv7GKo I'm not certain if this is as expected of an issue, though. In the original example, C carried state that was set up after initialization but was relying on the fallback to the non-idiomatic copy constructor when doing the throw. WDYT? Comment Actions I'm not certain it's reasonable to wait for MSVC STL to change as that leaves every existing user of older MSVC STLs out in the cold. This is causing some significant regressions at Intel, to the extent that I wonder if this should be temporarily reverted until the MSVC STL headers can be compiled again. Comment Actions Yeah that is the equivalent scenario for throw, we treat c1 as a temporary there. The same workaround applies, you can static cast to non-const lvalue reference. As far as the implementation is concerned, it is following the proposal to the letter here. That does interfere with the wishes of the committee to get implementation experience with this. Comment Actions Thanks for verifying!
I'm less convinced than the committee; I put a high value on existing, working, conforming code continuing to work in newer language modes. However, this is what the standard requires for that language mode, so I can live with it unless the code pattern shows up in some common system header that users can't change themselves, and I haven't run into evidence of that for this particular case yet.
I would argue it gives the committee valuable implementation experience feedback: the change breaks at least one popular system header.
I don't think it's reasonable to leave this on by default for MSVC compatibility; it's not like std::ifstream is an obscure part of the STL or there's only one version of the STL that's impacted. The suggestions I have for how to move forward are:
I'd prefer to keep existing code working without requiring users to enable a flag, if at all possible (esp because I don't expect the flag to be needed forever). That said, I still think it makes sense to temporarily revert this while doing that work. Comment Actions But we already knew that :-)
I find interesting the idea of combining these two options. We can suppress the effect when compiling with fms-compatibility but only on the STL headers themselves.
I'd like to give some time for other stakeholders to give their opinion if this is not too urgent, specially @Quuxplusone and @rsmith. Comment Actions
I have no strong/well-informed opinions here. I have an idea to offer, if there is precedent for it. My idea is, keep it as an error if the rvalue resolution finds nothing, but (1) fall back to an lvalue resolution for error-recovery purposes, and (2) suppress the error in system headers. Then, the usage in MSVC STL will keep working but only because it's in a system header. I seem to recall that there is precedent for diagnostics that are "non-downgradable error in user code, yet still suppressed in system headers." If I'm imagining things, then please do not invent such a facility purely for this feature. My second choice is to do whatever minimalistic hack suffices to satisfy the customer (Aaron etc. in this case), but only with the understanding that we plan to eliminate that hack as soon as MSVC STL gets updated. My third choice is to revert the whole thing and then land a new version of it that always falls back to lvalue resolution, but with an on-by-default warning (not error) (so this would probably affect a ton of your test cases); then the user can downgrade that warning as a workaround. Comment Actions That would still leave the effect of changing the deduced type of a decltype(auto) returning function. But I am on the same boat here in that I am not sure if you are imagining things as well :)
Again, the change in flow where we make the expression an xvalue early and definitively before return type deduction occurs makes things more complicated. My vote is the second choice, as I think making a workaround where we temporarily do the C++20 thing only in some contexts is far simpler, specially if we don't have to update a ton of test cases to cover it :) Comment Actions @aaron.ballman I created a DR for the option of temporarily disabling the effects of this change under "-fms-compatibility". Comment Actions This makes clang-cl basically unusable for a significant portion of people (enough so that I'm asking for a temporary revert) and I don't think we should continue to leave trunk in a broken state while deciding what to do. Our usual community approach is to revert out changes that cause significant breakage because we always want people (such as downstreams) to be able to track against trunk. That said, based on below, hold off on reversion as I may have misunderstood the fix patch (see below) and that may be the quicker and less disruptive approach.
Ah! I missed that context in the other patch, thank you for this! I'll go comment on the other review so we keep the discussion in one place, but that makes me more comfortable with the approach from that review. |
These labels seem to be getting unwieldy. I propose that you rename cxx11_14_17 to cxx17 and cxx11_14_17_20 to cxx17_20.
(There are no cases where C++11, '14, '17's behavior differ.)
So then we'll have cxx17, cxx17_20, cxx20, cxx20_2b, and cxx2b.
IMHO it might even be nice to eliminate the overlapping labels; just, when a message is expected in two out of three modes, write two expectations.
What do you think? (This could also be severed into its own earlier commit.)