This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Fix the behavior of throwing `operator new` under -fno-exceptions
AbandonedPublic

Authored by ldionne on May 15 2023, 1:52 PM.

Details

Reviewers
None
Group Reviewers
Restricted Project
Restricted Project
Summary

In D144319, Clang tried to land a change that would cause some functions
that are not supposed to return nullptr to optimize better. As reported
in https://reviews.llvm.org/D144319#4203982, libc++ started seeing failures
in its CI shortly after this change was landed.

As explained in D146379, the reason for these failures is that libc++'s
throwing operator new can in fact return nullptr when compiled with
exceptions disabled. However, this contradicts the Standard, which
clearly says that the throwing version of operator new(size_t)
should never return nullptr. This is actually a long standing issue.
I've previously seen a case where LTO would optimize incorrectly based
on the assumption that operator new doesn't return nullptr, an
assumption that was violated in that case because libc++.dylib was
compiled with -fno-exceptions.

Unfortunately, fixing this is kind of tricky. The Standard has a few
requirements for the allocation functions, some of which are impossible
to satisfy under -fno-exceptions:

  1. operator new(size_t) must never return nullptr
  2. operator new(size_t, nothrow_t) must call the throwing version and return nullptr on failure to allocate
  3. We can't throw exceptions when compiled with -fno-exceptions

In the case where exceptions are enabled, things work nicely. new(size_t)
throws and new(size_t, nothrow_t) uses a try-catch to return nullptr.
However, when compiling the library with -fno-exceptions, we can't throw
an exception from new(size_t), and we can't catch anything from
new(size_t, nothrow_t). The only thing we can do from new(size_t)
is actually abort the program, which does not make it possible for
new(size_t, nothrow_t) to catch something and return nullptr.

This patch makes the following changes:

  1. When compiled with -fno-exceptions, the throwing version of operator new will now abort on failure instead of returning nullptr on failure. This resolves the issue that the compiler could mis-compile based on the assumption that nullptr is never returned. This constitutes an API and ABI breaking change for folks compiling the library with -fno-exceptions (which is not the general public, who merely uses libc++ headers but use a shared library that has already been compiled). This should mostly impact vendors and other folks who compile libc++.dylib themselves.
  1. When the library is compiled with -fexceptions, the nothrow version of operator new has no change. When the library is compiled with -fno-exceptions, the nothrow version of operator new will now check whether the throwing version of operator new has been overridden. If it has not been overridden, then it will use an implementation equivalent to that of the throwing operator new, except it will return nullptr on failure to allocate (instead of terminating). However, if the throwing operator new has been overridden, it is now an error NOT to also override the nothrow operator new. Indeed, there is no way for us to implement a valid nothrow operator new without knowing the exact implementation of the throwing version.

TODO: Missing test cases

  • Add an assertion test to ensure that we catch the case where operator new has been overridden but operator new(nothrow) has not and the library is compiled with -fno-exceptions. We should fail with an assertion.
  • Add a test to ensure that operator new(nothrow) returns nullptr on failure to allocate even when the library was compiled with -fno-exceptions.
  • Add tests for the __is_function_overridden machinery.

rdar://103958777

Diff Detail

Event Timeline

ldionne created this revision.May 15 2023, 1:52 PM
Herald added a project: Restricted Project. · View Herald TranscriptMay 15 2023, 1:52 PM
ldionne requested review of this revision.May 15 2023, 1:52 PM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptMay 15 2023, 1:52 PM
Herald added a reviewer: Restricted Project. · View Herald Transcript
Herald added a reviewer: Restricted Project. · View Herald Transcript
ldionne added subscribers: jwakely, EricWF, Restricted Project.May 15 2023, 2:06 PM

Pinging vendors for discussion.

In particular, @EricWF it would be nice to know how Google feels about this, given that it's a large code base built using -fno-exceptions (presumably the dylib too?).

Also, @jwakely I'd like to know how you feel about the fact that new(size_t, nothrow_t) aborts instead of returning nullptr when the shared library is built using -fno-exceptions. Is that a concious choice you made in libstdc++, and if so, what's the rationale? Naively, I would say this surely seems a bit surprising, but it seems to be the only way to satisfy the Standard's requirements under -fno-exceptions (since the nothrow_t version must call the throwing version).

DianQK added a subscriber: DianQK.May 15 2023, 6:14 PM

Also, @jwakely I'd like to know how you feel about the fact that new(size_t, nothrow_t) aborts instead of returning nullptr when the shared library is built using -fno-exceptions. Is that a concious choice you made in libstdc++, and if so, what's the rationale?

I consider it to be a bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106477 (which I first noted at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68210#c2 in 2016, but still haven't fixed).
It's not a conscious choice, just how it works because nobody has implemented anything better yet.

Naively, I would say this surely seems a bit surprising, but it seems to be the only way to satisfy the Standard's requirements under -fno-exceptions (since the nothrow_t version must call the throwing version).

See the bug report above for another approach, which I've not had time to finish working on (and which won't work on all targets). It detects whether the program has replaced operator new(size_t) and if it _hasn't_, then we can make the nothrow_t version just return a null pointer directly. If the user has replaced operator new(size_t) then they'd better replace the nothrow_t one as well if they want to use -fno-exceptions.

Pinging vendors for discussion.

In particular, @EricWF it would be nice to know how Google feels about this, given that it's a large code base built using -fno-exceptions (presumably the dylib too?).

Let me test it and find out.

Also, @jwakely I'd like to know how you feel about the fact that new(size_t, nothrow_t) aborts instead of returning nullptr when the shared library is built using -fno-exceptions. Is that a concious choice you made in libstdc++, and if so, what's the rationale? Naively, I would say this surely seems a bit surprising, but it seems to be the only way to satisfy the Standard's requirements under -fno-exceptions (since the nothrow_t version must call the throwing version).

Yeah, I'm curious about this too.

I consider it to be a bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106477 (which I first noted at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68210#c2 in 2016, but still haven't fixed).
It's not a conscious choice, just how it works because nobody has implemented anything better yet.

Thanks for the additional context!

Naively, I would say this surely seems a bit surprising, but it seems to be the only way to satisfy the Standard's requirements under -fno-exceptions (since the nothrow_t version must call the throwing version).

See the bug report above for another approach, which I've not had time to finish working on (and which won't work on all targets). It detects whether the program has replaced operator new(size_t) and if it _hasn't_, then we can make the nothrow_t version just return a null pointer directly. If the user has replaced operator new(size_t) then they'd better replace the nothrow_t one as well if they want to use -fno-exceptions.

That's interesting. I think we could do something similar. However, let's say we could do whatever we wanted and design from scratch, would you agree that it might make sense for operator new(size_t) to be equivalent to operator new(size_t, nothrow_t) when compiled with -fno-exceptions? I know the Standard doesn't formally acknowledge the existence of -fno-exceptions, but if we agreed that this is the desired behavior, we could perhaps relax the wording of operator new to make that implementation valid. WDYT?

I've spoken to some users in the embedded world and the consensus seems to be that having operator new(size_t) not return nullptr when it fails to allocate is problematic. Some have quite a bit of code already written where operator new(size_t) may return nullptr if it fails to allocate (which is a violation of the current spec technically).

The Standard has a few requirements for the allocation functions, some of which are impossible to satisfy under -fno-exceptions:

  1. operator new(size_t) must never return nullptr
  2. operator new(size_t, nothrow_t) must call the throwing version and return nullptr on failure to allocate
  3. We can't throw exceptions when compiled with -fno-exceptions

Feels like building the library with -fno-exceptions is just plain non-conforming?

Requirement 2 seems odd, the only justification I can come up with is to allow replacing _only_ the throwing version, and having the nothrow version still Just Work in a compatible way. Naively I'd have thought the other way around would make more sense (throwing version calls nothrow version and turns nullptr into throw/abort) but that's not what got standardized.

The Standard has a few requirements for the allocation functions, some of which are impossible to satisfy under -fno-exceptions:

  1. operator new(size_t) must never return nullptr
  2. operator new(size_t, nothrow_t) must call the throwing version and return nullptr on failure to allocate
  3. We can't throw exceptions when compiled with -fno-exceptions

Feels like building the library with -fno-exceptions is just plain non-conforming?

Yes, I think that's accurate. We basically don't have a way to be conforming when we build the library itself with -fno-exceptions. @jwakely's suggestion makes us very close to that, in the sense that operator new(size_t) will behave as expected, and operator new(size_t, nothrow_t) will give the impression that we're calling operator new(size_t). The only remaining problem is what happens if the library is built with -fno-exceptions and the user overrides operator new(size_t) *but not* operator new(size_t, nothrow_t). In that case, our operator new(size_t, nothrow_t) will call the user's operator new(size_t) (which must terminate-on-failure in order to be conforming), and we'll still be non-conforming the same way we are today.

I think what I would really like is for a relaxation that does not force operator new(size_t, nothrow_t) to call operator new(size_t), since implementations effectively cannot do that. That would weaken a useful guarantee provided by the standard for the benefit of platforms that use -fno-exceptions -- I'm not sure that's the right tradeoff and even less sure that's something WG21 would ever consider.

Requirement 2 seems odd, the only justification I can come up with is to allow replacing _only_ the throwing version, and having the nothrow version still Just Work in a compatible way. Naively I'd have thought the other way around would make more sense (throwing version calls nothrow version and turns nullptr into throw/abort) but that's not what got standardized.

Yes and yes, I agree and I think it would have made more sense. However, I think changing that is impossible at this point.

ldionne updated this revision to Diff 533245.Jun 21 2023, 7:01 AM
ldionne retitled this revision from [libc++] Make sure `operator new` never returns nullptr, even under -fno-exceptions to [libc++] Fix the behavior of throwing `operator new` under -fno-exceptions.
ldionne edited the summary of this revision. (Show Details)

Update with new approach that detects whether operator new has been overridden or not. This is a WIP for comments only.

Update with new approach that detects whether operator new has been overridden or not. This is a WIP for comments only.

Cool! Nice to see this idea put into practice. Now I should do the same for libstdc++ :-)

ychen added a subscriber: ychen.Aug 7 2023, 12:24 PM
ldionne abandoned this revision.Oct 18 2023, 11:57 AM

I am going to abandon this change on Phabricator since I've moved it to https://github.com/llvm/llvm-project/pull/69498 instead.