[libc++] add declval failure assertion for instantiation

[ldionne]

Summary

Fixes https://github.com/llvm/llvm-project/issues/61202

[fsb4000]

inline variables is a c++17 feature

[fsb4000]

add a test

[ldionne]

Can you please add a test? This could be a .verify.cpp test with code similar to #61202

[fsb4000]

change a compile.fail test to a verify test

[shafik]

Thank you for fixing this so quickly.

[fsb4000]

add _LIBCPP_HIDE_FROM_ABI

[fsb4000]

fix a test

[frederick-vs-ja]

libcxx/include/__utility/declval.h

30–34

I think it's better not to modify the template head.
Doesn't libc++ have a concentrated __dependent_false? If so, I think we can still use !__is_same(_Tp, _Tp) in static_assert.

31

The return type is... cursed for cv void.

It seems that decltype(std::declval<const void>) should be const void() (until C++17) / const void() noexcept (since C++17) (same for volatile void and const volatile void), but the cv-qualifiers are dropped in decltype(std::__declval<_Tp>(0)).

[fsb4000]

libcxx/include/__utility/declval.h

30–34

I found that pattern there: https://github.com/llvm/llvm-project/blob/90f6a4cc73d5c40add913e6ed5d34a7a9db58c79/libcxx/include/__compare/common_comparison_category.h#L63-L79

[fsb4000]

libcxx/include/__utility/declval.h

31

I found that Eric Niebler added the patch: https://bugs.llvm.org/show_bug.cgi?id=27798

It was typename add_rvalue_reference<_Tp>::type before.

[fsb4000]

libcxx/include/__utility/declval.h

31

I've created: https://github.com/llvm/llvm-project/issues/61232

[philnik]

libcxx/include/__utility/declval.h

31

I think the performance improvement might not apply anymore, since we have a builtin now. Using __add_rvalue_reference_t<_Tp> should fix the bug and probably not result in any performance regressions. I'll try to make a small benchmark.

[EricWF]

This change is not correct and cannot proceed.

It changes the meaning of this code:

template <class T, class U, class = U[std::declval<T>()]>
constexpr bool xxx(int) { return true; }

template <class T, class U>
constexpr bool xxx(long) { return false; }

constexpr bool y = xxx<long long, int>(0);

declval has been constructed very carefully to performant, and correct in all the weird cases where Clang will try to eagerly instantiate something (Ex. when it tries to check for C++11 narrowing when evaluating constant expressions).
Please be cautious when modifying weird things like declval.

[fsb4000]

@EricWF

What do you mean?

The change is correct.

You mean gcc and clang compiler frontends have bugs?

By the way libstdc++ has the static_assert since Nov 13, 2009 (https://github.com/gcc-mirror/gcc/commit/7274deff738213914a31413a9d1a671020694175)

and GCC failed "error: static assertion failed: declval() must not be used!"

https://gcc.godbolt.org/z/MssGMMqsd

I see that MSVC works with that example: https://gcc.godbolt.org/z/cjE8dard7

However it has the same library implementation...

[fsb4000]

I've created https://github.com/llvm/llvm-project/issues/61254

[shafik]

In D145376#4176228, @EricWF wrote:

This change is not correct and cannot proceed.

It changes the meaning of this code:

template <class T, class U, class = U[std::declval<T>()]>
constexpr bool xxx(int) { return true; }

template <class T, class U>
constexpr bool xxx(long) { return false; }

constexpr bool y = xxx<long long, int>(0);

declval has been constructed very carefully to performant, and correct in all the weird cases where Clang will try to eagerly instantiate something (Ex. when it tries to check for C++11 narrowing when evaluating constant expressions).
Please be cautious when modifying weird things like declval.

Thank you for catching this, I am hoping that we have test for these edge cases, if not do you think you can enumerate more of them so that we can ensure that we are doing the right thing.

[EricWF]

Why?

We went out of our way to make declval as low cost as we possibly could. Adding a body for the compiler to see is just work.

In D145376#4176608, @fsb4000 wrote:

@EricWF

What do you mean?

The change is correct.

You mean gcc and clang compiler frontends have bugs?

By the way libstdc++ has the static_assert since Nov 13, 2009 (https://github.com/gcc-mirror/gcc/commit/7274deff738213914a31413a9d1a671020694175)

and GCC failed "error: static assertion failed: declval() must not be used!"

https://gcc.godbolt.org/z/MssGMMqsd

I see that MSVC works with that example: https://gcc.godbolt.org/z/cjE8dard7

However it has the same library implementation...

Cool. Thanks for the archaeology. But even if they're bugs, it means we break code. So this change isn't going anywhere as-is until
the different compilers converge on a behavior.

In the meantime, If you want to produce a better diagnostics in cases where the thing is ODR used, you can use the __attribute__((unavailable)) to do that.

[EricWF]

Also, I'm not sure a static assert pointing to the stable name in the
standard is helpful to our users. Will they even know what it is?

[ldionne]

In D145376#4179024, @EricWF wrote:

Why?

We went out of our way to make declval as low cost as we possibly could. Adding a body for the compiler to see is just work.

Well, while this is technically a QOI issue (since ODR-using declval makes the program ill-formed no-diagnostic-required), we usually strive to provide nice error messages for users, when we can. So this seems desirable to me (assuming that doesn't make our implementation incorrect, obviously).

Summarizing the situation, it looks like Clang (with libstdc++) and GCC (with libstdc++) both reject valid code right now: https://gcc.godbolt.org/z/4Kb9qG5e5
However, what we're trying to achieve is that Clang (with libc++) does not accept this invalid code: https://gcc.godbolt.org/z/aPKjb7odo

I think we should just file a bug report against Clang and GCC and get those fixed, and then we can proceed with this change. It seems weird to me that the body of the function is instantiated when trying to check for narrowing, since we are providing the return type of declval explicitly. That just seems like a bug that could bite in other scenarios to me. And once Clang and GCC don't try to instantiate the body of declval, I don't think there would be performance concerns anymore (@EricWF do you disagree?).

libcxx/test/std/input.output/iostream.format/output.streams/ostream/deleted_output_functions.verify.cpp

21

This is a nice catch! If this patch gets stuck, I would consider committing this separately since it's just a NFC fix to a test that is clearly incorrect.

libcxx/test/std/utilities/utility/declval/declval.verify.cpp

16–24

I think you could just do this:

int x = std::declval<int>(); // expected-error-re@*:* {{{{(static_assert|static assertion)}} failed{{.*}}Calling declval is ill-formed, see [declval]/2.}}

There is no need to have a main function in a .verify.cpp test, since it only compiles the code but doesn't try to link or run it!

[frederick-vs-ja]

In D145376#4179548, @ldionne wrote:

In D145376#4179024, @EricWF wrote:

Why?

We went out of our way to make declval as low cost as we possibly could. Adding a body for the compiler to see is just work.

Well, while this is technically a QOI issue (since ODR-using declval makes the program ill-formed no-diagnostic-required), we usually strive to provide nice error messages for users, when we can. So this seems desirable to me (assuming that doesn't make our implementation incorrect, obviously).

Summarizing the situation, it looks like Clang (with libstdc++) and GCC (with libstdc++) both reject valid code right now: https://gcc.godbolt.org/z/4Kb9qG5e5
However, what we're trying to achieve is that Clang (with libc++) does not accept this invalid code: https://gcc.godbolt.org/z/aPKjb7odo

I think we should just file a bug report against Clang and GCC and get those fixed, and then we can proceed with this change. It seems weird to me that the body of the function is instantiated when trying to check for narrowing, since we are providing the return type of declval explicitly. That just seems like a bug that could bite in other scenarios to me. And once Clang and GCC don't try to instantiate the body of declval, I don't think there would be performance concerns anymore (@EricWF do you disagree?).

I do believe diagnostic is required (otherwise, implementations are allowed to ignore "Mandates:").

1. [declval]/2 uses "Mandates:";
2. [structure.specifications]/3.2 says that violation of "Mandates:" make the program ill-formed;
3. [intro.compliance.general] implies that unless "no diagnostic required" is explicitly mentioned, diagnostic is required for ill-formedness.

[frederick-vs-ja]

In D145376#4176228, @EricWF wrote:

This change is not correct and cannot proceed.

It changes the meaning of this code:

template <class T, class U, class = U[std::declval<T>()]>
constexpr bool xxx(int) { return true; }

template <class T, class U>
constexpr bool xxx(long) { return false; }

constexpr bool y = xxx<long long, int>(0);

declval has been constructed very carefully to performant, and correct in all the weird cases where Clang will try to eagerly instantiate something (Ex. when it tries to check for C++11 narrowing when evaluating constant expressions).
Please be cautious when modifying weird things like declval.

I think this is a bug of the Clang frontend.
P0859R0 clarified when a function template or a defaulted function should be instantiated due to constant evaluation. std::declval is never a constexpr function, so it is never needed for constant evaluation, and the (ill-formed) type-id U[std::declval<T>()] should result in substitution failure without instantiating std::declval<T>.

[EricWF]

Just use __attribute__((unavailable("declval cannot be called")) or some such.

[cjdb]

libcxx/include/__utility/declval.h

31

Can this please be a more user-friendly diagnostic? Saying "This is ill-formed. Read this [standarese]" doesn't help the user understand what is going on or how to fix it. Something along the lines of this will be more in line with the Clang effort to improve diagnostics (something that hasn't been broached with libc++ yet, but here's a good place to try a new diagnostic).

static_assert(
  always_false<T>,
  "`std::declval` can only be used in an unevaluated context. It's likely that our current usage is trying to extract a value from the function.");

[EricWF]

Disregard my suggestion about unavailable. It seems to always fire, including in unevaluated contexts. I feel silly.

[EricWF]

Is this code ODR used? I suspect it is, and so it's not valid, but I'm not sure and I ran into a bunch of different instances of this pattern while testing this change.

template <class T>
decltype(auto) compute_return_type() {
    // ...
    // stuff that makes this function look reasonable...
    // ...
    return std::declval<T>();
}

using MyT = decltype(compute_return_type<int>());

[EricWF]

OK, so I've come around on this change. But please give me 2 weeks to cleanup the breakages it's going to cause inside Google.
I'll come back and let you know how many breakages were bugs vs how many were just weird meta-programming.

[ldionne]

In D145376#4203303, @EricWF wrote:

Is this code ODR used? I suspect it is, and so it's not valid, but I'm not sure and I ran into a bunch of different instances of this pattern while testing this change.

template <class T>
decltype(auto) compute_return_type() {
    // ...
    // stuff that makes this function look reasonable...
    // ...
    return std::declval<T>();
}

using MyT = decltype(compute_return_type<int>());

Yeah, that code would be invalid because declval() is ODR used IIUC.

Commandeering to finish as part of the GH PR Transition.

[ldionne]

Rebase and apply comments. I will merge this if CI is green. Per the update in https://github.com/llvm/llvm-project/issues/61254, I don't think this causes any well-formed code to be rejected, and it fixes a real bug (https://github.com/llvm/llvm-project/issues/61202).

[ldionne]

Format added test.

[EricWF]

I'll unblock this. But I don't love it.

[ldionne]

Merging -- the CI was green. The benchmark issue was something transient that has been fixed now.

libcxx/include/__utility/declval.h

 _LIBCPP_SUPPRESS_DEPRECATED_PUSH
 template <class _Tp>
 _Tp&& __declval(int);
 template <class _Tp>
 _Tp __declval(long);
 _LIBCPP_SUPPRESS_DEPRECATED_POP
 
 template <class _Tp>
-decltype(std::__declval<_Tp>(0)) declval() _NOEXCEPT;
+_LIBCPP_HIDE_FROM_ABI decltype(std::__declval<_Tp>(0)) declval() _NOEXCEPT {
+  static_assert(!__is_same(_Tp, _Tp),
+                "std::declval can only be used in an unevaluated context. "
+                "It's likely that your current usage is trying to extract a value from the function.");
+}
 
 _LIBCPP_END_NAMESPACE_STD
 
 #endif // _LIBCPP___UTILITY_DECLVAL_H

libcxx/test/std/input.output/iostream.format/output.streams/ostream/deleted_output_functions.verify.cpp

 #include <ostream>
 #include <sstream>
 #include <utility>
 
 #include "test_macros.h"
 
 void f() {
   std::ostringstream s;
+#ifndef TEST_HAS_NO_CHAR8_T
+  char8_t c8_s[]       = u8"test";
+  const char8_t* c8_cs = u8"test";
+#endif
+  char16_t c16_s[]       = u"test";
+  const char16_t* c16_cs = u"test";
+  char32_t c32_s[]       = U"test";
+  const char32_t* c32_cs = U"test";
 
 #ifndef TEST_HAS_NO_WIDE_CHARACTERS
-
-  s << wchar_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<wchar_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<const wchar_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  wchar_t w_s[]       = L"test";
+  const wchar_t* w_cs = L"test";
+  s << wchar_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << w_s;       // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << w_cs;      // expected-error {{overload resolution selected deleted operator '<<'}}
 
   std::wostringstream sw;
 #  ifndef TEST_HAS_NO_CHAR8_T
-  sw << char8_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << std::declval<char8_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << std::declval<const char8_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << char8_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << c8_s;      // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << c8_cs;     // expected-error {{overload resolution selected deleted operator '<<'}}
 #  endif
 
-  sw << char16_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << std::declval<char16_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << std::declval<const char16_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << char32_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << std::declval<char32_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  sw << std::declval<const char32_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << char16_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << c16_s;      // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << c16_cs;     // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << char32_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << c32_s;      // expected-error {{overload resolution selected deleted operator '<<'}}
+  sw << c32_cs;     // expected-error {{overload resolution selected deleted operator '<<'}}
 
 #endif // TEST_HAS_NO_WIDE_CHARACTERS
 
 #ifndef TEST_HAS_NO_CHAR8_T
-  s << char8_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<char8_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<const char8_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << char8_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << c8_s;      // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << c8_cs;     // expected-error {{overload resolution selected deleted operator '<<'}}
 #endif
-  s << char16_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<char16_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<const char16_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << char32_t();                      // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<char32_t*>();       // expected-error {{overload resolution selected deleted operator '<<'}}
-  s << std::declval<const char32_t*>(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << char16_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << c16_s;      // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << c16_cs;     // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << char32_t(); // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << c32_s;      // expected-error {{overload resolution selected deleted operator '<<'}}
+  s << c32_cs;     // expected-error {{overload resolution selected deleted operator '<<'}}
 }

libcxx/test/std/utilities/utility/declval/declval.verify.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// <utility>
+
+// template <class T> typename add_rvalue_reference<T>::type declval() noexcept;
+
+#include <utility>
+
+int x = std::declval<
+    int>(); // expected-error-re@*:* {{static assertion failed{{.*}}std::declval can only be used in an unevaluated context.}}