[libc++][ABI BREAK] Do not use the C++03 emulation for std::nullptr_t by default

[ldionne]

Summary

We only support Clangs that implement nullptr as an extension in C++03 mode,
and we don't support GCC in C++03 mode. Hence, this patch disables the
use of the std::nullptr_t emulation in C++03 mode by default. Doing that
is technically an ABI break since it changes the mangling for std::nullptr_t.
However:

(1) The only affected users are those compiling in C++03 mode that have

std::nullptr_t as part of their ABI, which should be reasonably rare.

(2) Those users already have a lingering problem in that their code will

be incompatible in C++03 and C++11 modes because of that very ABI break.
Hence, the only users that could really be inconvenienced about this
change is those that planned on compiling in C++03 mode forever - for
other users, we're just breaking them now instead of letting them break
themselves later on when they try to upgrade to C++11.

(3) The ABI break will cause a linker error since the mangling changed,

and will not result in an obscure runtime error.

Furthermore, if anyone is broken by this, they can define the
_LIBCPP_ABI_USE_CXX03_NULLPTR_EMULATION macro to return to the
previous behavior. We will then remove that macro after shipping
this for one release if we haven't seen widespread issues.

Concretely, the motivation for making this change is to make our own ABI
consistent in C++03 and C++11 modes and to remove complexity around the
definition of nullptr.

Furthermore, we could investigate making nullptr a keyword in C++03 mode
as a Clang extension -- I don't think that would break anyone, since
libc++ already defines nullptr as a macro to something else. Only users
that do not use libc++ and compile in C++03 mode could potentially be
broken by that.

[ldionne]

I'm putting this change up both for discussion and also to give people the chance to run test builds on internal code bases using this change. I think this change hinges quite a bit on the assumption that people almost never rely on std::nullptr_t in their ABI in C++03 mode. If that assumption turns out to be false, we may need to revisit the feasibility of this change.

Pinging some vendors to see what they think about this. @danalbert @phosek @dim @emaste

Perhaps @cjdb could run a build at Google? I'll do it at Apple.

[• Quuxplusone]

FWLIW, I'm generally in favor of this. Although I think it's ironic that we already had an ABI flag for this, such that we'd have gotten this change for free if we ever managed to reach ABIv2. Feels like this PR is the most public admission yet that we'll never actually be able to reach std::__2.

libcxx/include/cstddef

46 ↗

(On Diff #371414)

Surely this should just unconditionally say typedef decltype(nullptr) nullptr_t; — why would we ever not want std::nullptr_t to exist?
(Vice versa, personally I don't understand why we define a type named ::nullptr_t; but I assume there's some good reason not to remove it now.)

[ldionne]

In D109459#2990186, @Quuxplusone wrote:

FWLIW, I'm generally in favor of this. Although I think it's ironic that we already had an ABI flag for this, such that we'd have gotten this change for free if we ever managed to reach ABIv2. Feels like this PR is the most public admission yet that we'll never actually be able to reach std::__2.

I don't think that's accurate. IMO, there are things we need to guard behind an ABI flag because they would actually break people and cause problems if we did not. Then there are things (like this nullptr_t emulation) that wouldn't break anyone anyways if we turned it on, so we might as well do it in the ABI v1 (proof is needed regarding whether I'm right about that, and I'm running builds to check that out right now).

IMO, the idea of an ABI v2 is a bit funky, I think we need to define it better. I think something more useful would be to formalize how different platforms get to select different ABI flags, since "breaking" the ABI is easiest done when one introduces a new platform. That doesn't happen super often, but it's better than nothing. Anyway, this is a discussion for a different forum.

libcxx/include/cstddef

46 ↗

(On Diff #371414)

Sure, I can drop _LIBCPP_USING_IF_EXISTS here since it's always defined (by us).

Regarding ::nullptr_t, I was surprised to read that <stddef.h> is supposed to provide such a type 🤷🏼‍♂️.

[ldionne]

Try to fix build on GCC and address Arthur's comment about using_if_exists

[ldionne]

Fix the modules build.

[danalbert]

I think this is probably fine from the NDK perspective. Even before Clang defaulted to C++11 or newer, our build system did that when using libc++. Other build systems may not have, but unfortunately I'm not currently able to get at the data to get a concrete answer.

[ldionne]

I did an internal Apple build and didn't see any issues. So this is OK from Apple's internal code base perspective.

However, it doesn't mean that this will be okay for users of libc++, whose ABIs could conceivably leak the mangling of std::nullptr_t. I'd still like to get input from other vendors before making this move. Accepting as myself only to record that this is OK for Apple.

[ldionne]

@dexonsmith has had useful views on ABI changes in the past, I'm curious to have his opinion on this change, since I'm still ambivalent.

TL;DR: Today, std::nullptr_t is a different type in C++03 and C++11, which means that the ABI breaks between C++03 and C++11. This patch breaks the ABI in C++03 mode once, right now, to reconcile the C++03 and C++11 ABIs and remove some convolution from libc++. My hunch is that this will not affect too many people and hence it's reasonable. Remember that nullptr is a C++11 construct, so it would have to be someone compiling their code in C++03 mode *and* using our nullptr extension, *and* leaking nullptr_t in their ABI. Do you have an opinion?

[ldionne]

Rebase onto main

[ldionne]

Rebase onto main. I'm still on the fence about this.

[dexonsmith]

In D109459#3043589, @ldionne wrote:

@dexonsmith has had useful views on ABI changes in the past, I'm curious to have his opinion on this change, since I'm still ambivalent.

I missed this before!

TL;DR: Today, std::nullptr_t is a different type in C++03 and C++11, which means that the ABI breaks between C++03 and C++11. This patch breaks the ABI in C++03 mode once, right now, to reconcile the C++03 and C++11 ABIs and remove some convolution from libc++. My hunch is that this will not affect too many people and hence it's reasonable. Remember that nullptr is a C++11 construct, so it would have to be someone compiling their code in C++03 mode *and* using our nullptr extension, *and* leaking nullptr_t in their ABI. Do you have an opinion?

This feels relatively safe to me for the reasons given, but hard to be sure. One option would be to change behaviour behind a vendor+user macro combo for a couple of releases to test (vendors and/or users could revert to behaviour as-is today if/when necessary), then drop the configurations once there’s more confidence. I’m not sure it’s worth the trouble though.

@rjmccall, any thoughts from you?

Re clang/libc++ behaviour, if you wanted to continue providing the extension but take an ABI break, I think you could have clang expose a __nullptr_type__ in C++03 that it mangled like the C++11 nullptr_t and then add a typedef for it in libc++ to expose it.

[ldionne]

In D109459#3159309, @dexonsmith wrote:

In D109459#3043589, @ldionne wrote:

@dexonsmith has had useful views on ABI changes in the past, I'm curious to have his opinion on this change, since I'm still ambivalent.

I missed this before!

TL;DR: Today, std::nullptr_t is a different type in C++03 and C++11, which means that the ABI breaks between C++03 and C++11. This patch breaks the ABI in C++03 mode once, right now, to reconcile the C++03 and C++11 ABIs and remove some convolution from libc++. My hunch is that this will not affect too many people and hence it's reasonable. Remember that nullptr is a C++11 construct, so it would have to be someone compiling their code in C++03 mode *and* using our nullptr extension, *and* leaking nullptr_t in their ABI. Do you have an opinion?

This feels relatively safe to me for the reasons given, but hard to be sure. One option would be to change behaviour behind a vendor+user macro combo for a couple of releases to test (vendors and/or users could revert to behaviour as-is today if/when necessary), then drop the configurations once there’s more confidence. I’m not sure it’s worth the trouble though.

Thanks for chiming in. I like this a lot. I will repurpose this patch to enable the ABI break by default while still letting users return to the previous behavior by defining a macro. After shipping that for one release, we can rip it out entirely if there aren't widespread issues reported.

[ldionne]

Turn on ABI break by default, but let users turn it off.

[• Quuxplusone]

FWIW, I like this idea.

libcxx/docs/ReleaseNotes.rst

102–111

s/favour/favor/
s/Standard modes/standard modes/ ?
s/these user's/these users'/
s/linking/linker/ ?

[ldionne]

Address wording issues in the release note. Thanks Arthur!

I'm going to ship this now since I think this is pretty uncontroversial.

[ldionne]

Thanks a lot @dexonsmith, you had the magic suggestion that is simple, elegant and lets us move forward.

libcxx/docs/ReleaseNotes.rst

- The declarations of functions ``declare_reachable``, ``undeclare_reachable``, ``declare_no_pointers``,
   existing compiled code. All of these functions have always behaved as no-ops.
 
 ABI Changes
 -----------
 
 - The C++17 variable templates ``is_error_code_enum_v`` and
   ``is_error_condition_enum_v`` are now of type ``bool`` instead of ``size_t``.
 
+- The C++03 emulation type for ``std::nullptr_t`` has been removed in favor of
+  using ``decltype(nullptr)`` in all standard modes. This is an ABI break for
+  anyone compiling in C++03 mode and who has ``std::nullptr_t`` as part of their
+  ABI. However, previously, these users' ABI would be incompatible with any other
+  binary or static archive compiled with C++11 or later. If you start seeing linker
+  errors involving ``std::nullptr_t`` against previously compiled binaries, this may
+  be the cause. You can define the ``_LIBCPP_ABI_USE_CXX03_NULLPTR_EMULATION`` macro
+  to return to the previous behavior. That macro will be removed in LLVM 15. Please
+  comment `here <https://reviews.llvm.org/D109459>`_ if you are broken by this change
+  and need to define the macro.
+
 Build System Changes
 --------------------
 
 - Building the libc++ shared or static library requires a C++ 20 capable compiler.
   Consider using a Bootstrapping build to build libc++ with a fresh Clang if you
   can't use the system compiler to build libc++ anymore.
 
 - Historically, there has been numerous ways of building libc++ and libc++abi. This has

libcxx/include/__config

 // Fix undefined behavior in how std::list stores its linked nodes.
 #  define _LIBCPP_ABI_LIST_REMOVE_NODE_POINTER_UB
 // Fix undefined behavior in  how __tree stores its end and parent nodes.
 #  define _LIBCPP_ABI_TREE_REMOVE_NODE_POINTER_UB
 // Fix undefined behavior in how __hash_table stores its pointer types.
 #  define _LIBCPP_ABI_FIX_UNORDERED_NODE_POINTER_UB
 #  define _LIBCPP_ABI_FORWARD_LIST_REMOVE_NODE_POINTER_UB
 #  define _LIBCPP_ABI_FIX_UNORDERED_CONTAINER_SIZE_TYPE
-// Don't use a nullptr_t simulation type in C++03 instead using C++11 nullptr
-// provided under the alternate keyword __nullptr, which changes the mangling
-// of nullptr_t. This option is ABI incompatible with GCC in C++03 mode.
-#  define _LIBCPP_ABI_ALWAYS_USE_CXX11_NULLPTR
 // Define a key function for `bad_function_call` in the library, to centralize
 // its vtable and typeinfo to libc++ rather than having all other libraries
 // using that class define their own copies.
 #  define _LIBCPP_ABI_BAD_FUNCTION_CALL_KEY_FUNCTION
 // Override the default return value of exception::what() for
 // bad_function_call::what() with a string that is specific to
 // bad_function_call (see http://wg21.link/LWG2233). This is an ABI break
 // because it changes the vtable layout of bad_function_call.

 #  endif
 // Feature macros for disabling pre ABI v1 features. All of these options
 // are deprecated.
 #  if defined(__FreeBSD__)
 #    define _LIBCPP_DEPRECATED_ABI_DISABLE_PAIR_TRIVIAL_COPY_CTOR
 #  endif
 #endif
 
+// By default, don't use a nullptr_t emulation type in C++03.
+//
+// This is technically an ABI break from previous releases, however it is
+// very unlikely to impact anyone. If a user is impacted by this break,
+// they can return to using the C++03 nullptr emulation by defining
+// _LIBCPP_ABI_USE_CXX03_NULLPTR_EMULATION.
+//
+// This switch will be removed entirely in favour of never providing a
+// C++03 emulation after one release.
+//
+// IMPORTANT: IF YOU ARE READING THIS AND YOU TURN THIS MACRO ON, PLEASE LEAVE
+//            A COMMENT ON https://reviews.llvm.org/D109459 OR YOU WILL BE BROKEN
+//            IN THE FUTURE WHEN WE REMOVE THE ABILITY TO USE THE C++03 EMULATION.
+#ifndef _LIBCPP_ABI_USE_CXX03_NULLPTR_EMULATION
+# define _LIBCPP_ABI_ALWAYS_USE_CXX11_NULLPTR
+#endif
+
 #if defined(_LIBCPP_BUILDING_LIBRARY) || defined(_LIBCPP_ABI_UNSTABLE) || _LIBCPP_ABI_VERSION >= 2
 // Enable additional explicit instantiations of iostreams components. This
 // reduces the number of weak definitions generated in programs that use
 // iostreams by providing a single strong definition in the shared library.
 # define _LIBCPP_ABI_ENABLE_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1
 
 // Define a key function for `bad_function_call` in the library, to centralize
 // its vtable and typeinfo to libc++ rather than having all other libraries