This is an archive of the discontinued LLVM Phabricator instance.

[Preprocessor] Allow libc++ to detect when aligned allocation is unavailable.
ClosedPublic

Authored by EricWF on Mar 28 2018, 8:06 PM.

Details

Summary

Libc++ needs to know when aligned allocation is supported by clang, but is otherwise unavailable at link time. Otherwise, libc++ will incorrectly end up generating calls to __builtin_operator_new/__builtin_operator_delete which alignment arguments.

This patch implements the following changes:

  • The __cpp_aligned_new feature test macro to no longer be defined when aligned allocation is otherwise enabled but unavailable.
  • The Darwin driver no longer passes -faligned-alloc-unavailable when the user manually specifies any of -faligned-allocation, -fno-aligned-allocation, or -nostdinc++. (Though I'm not sure we should this for -nostdinc++ since it makes this behavior to test within libc++, since the libc++ tests must always pass -nostdinc++).
  • Instead of a warning Clang now generates a hard error when an aligned allocation or deallocation function is referenced but unavailable.

Diff Detail

Repository
rL LLVM

Event Timeline

EricWF created this revision.Mar 28 2018, 8:06 PM
EricWF added a comment.Apr 3 2018, 5:10 PM

Ping. I would like to get a way for libc++ to detect this case before the next release.

Another approach is __has_feature but I don't think it is applicable in this case.

Is there a way right now to detect that aligned allocation is supported by clang, regardless of link time? Asking to make sure we are consistent.

Another approach is __has_feature but I don't think it is applicable in this case.

Is there a way right now to detect that aligned allocation is supported by clang, regardless of link time? Asking to make sure we are consistent.

There is a C++ feature test macro, as specified by the standard, __cpp_aligned_new. But I'm not sure how to be consistent with that.

Another approach is __has_feature but I don't think it is applicable in this case.

Is there a way right now to detect that aligned allocation is supported by clang, regardless of link time? Asking to make sure we are consistent.

There is a C++ feature test macro, as specified by the standard, __cpp_aligned_new. But I'm not sure how to be consistent with that.

After more thinking I believe a predefined macro is the best option. And in spirit it is consistent with SD-6: SG10 Feature Test Recommendations.

As for the naming, I see it is consistent with -faligned-alloc-unavailable but for me it is hard to tell immediately if "unavailable" means "unsupported at all" or "supported but unavailable on the target". Maybe it is just me and others don't have such problem but probably including "runtime" in the macro name would help. Something like __ALIGNED_ALLOCATION_UNAVAILABLE_RUNTIME__.

For the overall aligned allocation plan overall. We also need to keep in mind users who provide their own aligned allocation / deallocation functions, so they can still do that. So far everything seems to be OK but we need to be careful.

lib/Frontend/InitPreprocessor.cpp
1059–1060 ↗(On Diff #140181)

Don't know what the macro will be in the end, please consider adding a comment clarifying runtime availability.

test/Preprocessor/predefined-macros.c
288 ↗(On Diff #140181)

Would it be useful to test -fno-aligned-allocation too? Extensive testing in different combinations doesn't add much value but -std=c++17 -fno-aligned-allocation can be useful.

Eric, do you have more thoughts on this issue?

Hmm, perhaps our strategy for handling aligned allocation on Darwin should be revisited. We shouldn't be defining __cpp_aligned_allocation if we believe it doesn't work -- that will break code that uses aligned allocation where available and falls back to something else where it's unavailable.

@ahatanak: how about this:

  • Change the driver to not pass -faligned-alloc-unavailable if an explicit -faligned-allocation or -fno-aligned-allocation flag is given; update Clang's note to suggest explicitly passing -faligned-allocation rather than -Wno-aligned-alloc-unavailable if the user provides their own aligned allocation function.
  • Change -faligned-alloc-unavailable so that it does not define __cpp_aligned_allocation.
  • Change Sema's handling of the "aligned allocation unavailable" case so that, after warning on selecting an aligned allocation function, it then removes those functions from the candidate set and tries again.

That is: on old Darwin, we should not define __cpp_aligned_allocation (even in C++17), produce the "no aligned allocation support" warning in C++17 mode, and then not try to call the aligned allocation function. But if -faligned-allocation or -fno-aligned-allocation is specified explicitly, then the user knows what they're doing and they get no warning.

That is: on old Darwin, we should not define __cpp_aligned_allocation (even in C++17), produce the "no aligned allocation support" warning in C++17 mode, and then not try to call the aligned allocation function. But if -faligned-allocation or -fno-aligned-allocation is specified explicitly, then the user knows what they're doing and they get no warning.

What when compiler has __builtin_operator_new, __builtin_operator_delete? If I build libc++ tests with recent Clang which has these builtins and run tests with libc++.dylib from old Darwin, there are no linkage errors. Should we define __cpp_aligned_allocation in this case?

What when compiler has __builtin_operator_new, __builtin_operator_delete? If I build libc++ tests with recent Clang which has these builtins and run tests with libc++.dylib from old Darwin, there are no linkage errors. Should we define __cpp_aligned_allocation in this case?

I don't see why that should make any difference -- those builtins call the same functions that the new and delete operator call. Perhaps libc++ isn't calling the forms of those builtins that take an alignment argument yet?

Hmm, perhaps our strategy for handling aligned allocation on Darwin should be revisited. We shouldn't be defining __cpp_aligned_allocation if we believe it doesn't work -- that will break code that uses aligned allocation where available and falls back to something else where it's unavailable.

@ahatanak: how about this:

  • Change the driver to not pass -faligned-alloc-unavailable if an explicit -faligned-allocation or -fno-aligned-allocation flag is given; update Clang's note to suggest explicitly passing -faligned-allocation rather than -Wno-aligned-alloc-unavailable if the user provides their own aligned allocation function.
  • Change -faligned-alloc-unavailable so that it does not define __cpp_aligned_allocation.
  • Change Sema's handling of the "aligned allocation unavailable" case so that, after warning on selecting an aligned allocation function, it then removes those functions from the candidate set and tries again.

That is: on old Darwin, we should not define __cpp_aligned_allocation (even in C++17), produce the "no aligned allocation support" warning in C++17 mode, and then not try to call the aligned allocation function. But if -faligned-allocation or -fno-aligned-allocation is specified explicitly, then the user knows what they're doing and they get no warning.

I talked with @vsapsai today and we think this looks like a better idea.

A couple of questions:

  • Currently clang errors out when aligned operator new is selected but the OS's version is too old to support it. What's the reason we want to change this now to be a warning rather than an error?
  • So clang no longer needs to define macro __ALIGNED_ALLOCATION_UNAVAILABLE__ and libc++ will use __cpp_aligned_new (I think you meant __cpp_aligned_new, not __cpp_aligned_allocation?) to determine whether aligned allocation functions should be defined or made available in the header?

Somewhat tangential, in discussion with Duncan he mentioned that -nostdinc++ should turn off assumptions about old Darwin. So if you build libc++ yourself, you don't care what does the system stdlib support. I agree with that and think it doesn't interfere with the latest proposal but adds more to it.

  • Currently clang errors out when aligned operator new is selected but the OS's version is too old to support it. What's the reason we want to change this now to be a warning rather than an error?

I think it's fine to leave it as a DefaultError warning. I think it'd also be fine to change it to simply be an error (rather than an error-with-warning-flag) and remove the error recovery, and that actually seems a bit better: then our behavior under -faligned-alloc-unavailable would be equivalent to treating the implicit aligned forms of operator new/delete as if they had an availability attribute on them by default, which I think seems very reasonable.

  • So clang no longer needs to define macro __ALIGNED_ALLOCATION_UNAVAILABLE__ and libc++ will use __cpp_aligned_new (I think you meant __cpp_aligned_new, not __cpp_aligned_allocation?) to determine whether aligned allocation functions should be defined or made available in the header?

Yes, that's the idea. __cpp_aligned_new exists to allow code to conditionally use aligned new if it's available, and we should do our best to honor that.

What when compiler has __builtin_operator_new, __builtin_operator_delete? If I build libc++ tests with recent Clang which has these builtins and run tests with libc++.dylib from old Darwin, there are no linkage errors. Should we define __cpp_aligned_allocation in this case?

I don't see why that should make any difference -- those builtins call the same functions that the new and delete operator call. Perhaps libc++ isn't calling the forms of those builtins that take an alignment argument yet?

It looks like clang currently doesn't issue a warning when a call to builtin_operator_new or builtin_operator_delete calls an aligned allocation function that is not support by the OS version. I suppose we should fix this?

// no warning issued when triple is "thumbv7-apple-ios5.0.0" even though aligned allocation is unavailable.
void *p = __builtin_operator_new(100, std::align_val_t(32));

I think clang should error out or warn when aligned operator or builtin operator new/delete functions are used when they are not available (r306722 should have handled them).

I'm also not sure not defining __cpp_aligned_new is sufficient. My understanding is that if __cpp_aligned_new is not defined, libc++ will define _LIBCPP_HAS_NO_ALIGNED_ALLOCATION, which will cause the aligned new/delete functions declared in <new> to be removed. That is fine if the new/delete function that is used in a program is one of the function implicitly declared by the compiler (e.g., 'void operator delete ( void*, std::align_val_t)'), but if it isn't one of those functions (e.g., 'void operator delete (void *, std::align_val_t, const std::nothrow_t&)'), clang will produce diagnostics that are not very informative to the user (error: no matching function for call to 'operator delete') instead of the one added in r306722 (aligned allocation function of type is only available on ...).

For example:

#include <new>
  
void foo1(void *p, std::align_val_t a, std::nothrow_t t) {
  operator delete(p, a, t);
}
EricWF added a comment.Jun 1 2018, 4:19 PM

The new path forward sounds good to me. I'll work on implementing it.

EricWF added a comment.Jun 1 2018, 4:30 PM

What when compiler has __builtin_operator_new, __builtin_operator_delete? If I build libc++ tests with recent Clang which has these builtins and run tests with libc++.dylib from old Darwin, there are no linkage errors. Should we define __cpp_aligned_allocation in this case?

I don't see why that should make any difference -- those builtins call the same functions that the new and delete operator call. Perhaps libc++ isn't calling the forms of those builtins that take an alignment argument yet?

It looks like clang currently doesn't issue a warning when a call to builtin_operator_new or builtin_operator_delete calls an aligned allocation function that is not support by the OS version. I suppose we should fix this?

// no warning issued when triple is "thumbv7-apple-ios5.0.0" even though aligned allocation is unavailable.
void *p = __builtin_operator_new(100, std::align_val_t(32));

If we switch to no longer defining __cpp_aligned_new when it's unavailable that means libc++ shouldn't generate calls to an aligned allocation or deallocation function. Do we need the compiler to protect libc++ from itself here?

EricWF updated this revision to Diff 149596.Jun 1 2018, 6:51 PM
EricWF edited the summary of this revision. (Show Details)

Update the patch with the form suggested in the previous conversation. See the updated summary for a description of the behavior.

The Darwin driver no longer passes -faligned-alloc-unavailable when -nostdinc++ is specified, but I'm not sure this is the behavior
we want. Having -nostdinc++ have side-effects which affect the well-formedness of new/delete expressions seems non-obvious and potentially problematic.

Could you elaborate on what kind of changes you are planning to make in libc++ after committing this patch?

EricWF added a comment.Jun 4 2018, 4:33 PM

Could you elaborate on what kind of changes you are planning to make in libc++ after committing this patch?

Libc++ shouldn't actually need any changes if this current patch lands. Currently libc++ is in a "incorrect" state where
it generates calls to __builtin_operator_new(size_t, align_val_t) when __cpp_aligned_new is defined but when aligned new/delete
are actually unavailable.

If we change __cpp_aligned_new to no longer be defined when aligned new is unavailable, then libc++ will start doing the right thing.
See r328180 for the relevent commits which made these libc++ changes.

I see, thank you.

clang front-end currently fails to issue a warning or error when an aligned allocation/deallocation functions are required but not available in a few cases (e.g., delete called from a deleting destructor, calls to operator or builtin operator new/delete). I suppose those bugs should be fixed in separate patches.

EricWF added a comment.Jun 5 2018, 5:19 PM

I see, thank you.

clang front-end currently fails to issue a warning or error when an aligned allocation/deallocation functions are required but not available in a few cases (e.g., delete called from a deleting destructor, calls to operator or builtin operator new/delete). I suppose those bugs should be fixed in separate patches.

I don't think we need to emit warnings from __builtin_operator_new/__builtin_operator_delete. Libc++ is the only consumer, and I think we can trust it to know what it's doing.

I see, thank you.

clang front-end currently fails to issue a warning or error when an aligned allocation/deallocation functions are required but not available in a few cases (e.g., delete called from a deleting destructor, calls to operator or builtin operator new/delete). I suppose those bugs should be fixed in separate patches.

I don't think we need to emit warnings from __builtin_operator_new/__builtin_operator_delete. Libc++ is the only consumer, and I think we can trust it to know what it's doing.

Shouldn't clang warn when users explicitly call an aligned builtin operator new or delete in their code and the OS is too old to support the operator?

For example:

typedef __SIZE_TYPE__ size_t;
namespace std {
enum class align_val_t : size_t {};
}

int main() {
  void *p = __builtin_operator_new(100, std::align_val_t(32));
  return 0;
}
EricWF added a comment.Jun 5 2018, 9:29 PM

I see, thank you.

clang front-end currently fails to issue a warning or error when an aligned allocation/deallocation functions are required but not available in a few cases (e.g., delete called from a deleting destructor, calls to operator or builtin operator new/delete). I suppose those bugs should be fixed in separate patches.

I don't think we need to emit warnings from __builtin_operator_new/__builtin_operator_delete. Libc++ is the only consumer, and I think we can trust it to know what it's doing.

Shouldn't clang warn when users explicitly call an aligned builtin operator new or delete in their code and the OS is too old to support the operator?

For example:

typedef __SIZE_TYPE__ size_t;
namespace std {
enum class align_val_t : size_t {};
}

int main() {
  void *p = __builtin_operator_new(100, std::align_val_t(32));
  return 0;
}

Yeah, I think you're right. Initially I thought __builtin_operator_new was documented as being intended for usage only in the STL, but it doesn't quite say that.
I have a patch which moves the checks into DiagnoseUseOfDecl, which should catch all of the cases we haven't already handled.

Sorry for the churn but can you please take out -nostdinc++ part out of this change? After more thinking and discussion we think there is a chance developers can use -nostdinc++ not only for building the standard library. -nostdinc++ is a signal of building the standard library but it's not strong enough. Most likely developers will be unaware how -nostdinc++ affects aligned allocations and that can lead to linker failures at runtime. So if you deploy on older macOS versions it is safer to assume aligned allocation isn't available. But if you provide your own libc++ with aligned allocations, you can still use -faligned-allocation to make that work. For now we prefer to use that approach and if it proves to be too onerous for developers, we can reintroduce -nostdinc++ logic.

For building libc++ we rely on _LIBCPP_BUILDING_NEW to have support for aligned allocations even though the build host doesn't support them. So we don't need to change libc++ build.

Could you elaborate on what kind of changes you are planning to make in libc++ after committing this patch?

Libc++ shouldn't actually need any changes if this current patch lands. Currently libc++ is in a "incorrect" state where
it generates calls to __builtin_operator_new(size_t, align_val_t) when __cpp_aligned_new is defined but when aligned new/delete
are actually unavailable.

If we change __cpp_aligned_new to no longer be defined when aligned new is unavailable, then libc++ will start doing the right thing.
See r328180 for the relevent commits which made these libc++ changes.

Looks like in libc++ we need to remove _LIBCPP_STD_VER check for aligned allocations, something like

 #if !defined(_LIBCPP_HAS_NO_ALIGNED_ALLOCATION) && \
-    (!(defined(_LIBCPP_BUILDING_NEW) || _LIBCPP_STD_VER > 14 || \
+    (!(defined(_LIBCPP_BUILDING_NEW) || \
     (defined(__cpp_aligned_new) && __cpp_aligned_new >= 201606)))
 # define _LIBCPP_HAS_NO_ALIGNED_ALLOCATION
 #endif

Is that correct? I didn't check the rest of the code, probably TEST_HAS_NO_ALIGNED_ALLOCATION needs some clean up too.

EricWF added a comment.Jun 9 2018, 1:12 PM

Could you elaborate on what kind of changes you are planning to make in libc++ after committing this patch?

Libc++ shouldn't actually need any changes if this current patch lands. Currently libc++ is in a "incorrect" state where
it generates calls to __builtin_operator_new(size_t, align_val_t) when __cpp_aligned_new is defined but when aligned new/delete
are actually unavailable.

If we change __cpp_aligned_new to no longer be defined when aligned new is unavailable, then libc++ will start doing the right thing.
See r328180 for the relevent commits which made these libc++ changes.

Looks like in libc++ we need to remove _LIBCPP_STD_VER check for aligned allocations, something like

 #if !defined(_LIBCPP_HAS_NO_ALIGNED_ALLOCATION) && \
-    (!(defined(_LIBCPP_BUILDING_NEW) || _LIBCPP_STD_VER > 14 || \
+    (!(defined(_LIBCPP_BUILDING_NEW) || \
     (defined(__cpp_aligned_new) && __cpp_aligned_new >= 201606)))
 # define _LIBCPP_HAS_NO_ALIGNED_ALLOCATION
 #endif

Is that correct? I didn't check the rest of the code, probably TEST_HAS_NO_ALIGNED_ALLOCATION needs some clean up too.

Huh, that original formulation is correct-ish, and so is your proposed changes. There are two problems here:

  1. We want to expose the aligned allocation declarations to users in C++17, even if the compiler will never call them because

the feature has been disabled.

  1. We don't want to generate calls to them via __builtin_operator_new when we've only enabled the declarations because of C++17.

I'll make some changes to libc++ to address this. Thanks for pointing it out.

EricWF updated this revision to Diff 150635.Jun 9 2018, 1:51 PM

Remove -nostdinc++ check as requested.

With this change and the mentioned libc++ change the tests with old libc++ dylib are passing (didn't test all possible configurations though). Would like to get more feedback from other reviewers on this matter.

EricWF added a comment.Jul 4 2018, 2:49 PM

@rsmith Ping. This needs to land before the next branch for release.

Ping.

Are there any more reviewers I should add to this?

Quuxplusone added inline comments.
include/clang/Basic/DiagnosticSemaKinds.td
6425 ↗(On Diff #150635)

I observe that this mnemonic is misspelled; but that's not this patch's fault.
The traditional spelling would probably be just note_aligned_allocation_unavailable.

lib/Driver/ToolChains/Darwin.cpp
2027 ↗(On Diff #150635)

Peanut gallery asks: Why is the cc1 option spelled differently from the clang driver option? Don't they do the same thing?

rsmith accepted this revision.Jul 25 2018, 3:27 PM
rsmith added inline comments.
lib/Driver/ToolChains/Darwin.cpp
2027 ↗(On Diff #150635)

At the -cc1 level, there are three different levels:

  • no aligned allocation in the language
  • aligned allocation in the language but it doesn't work (eg, C++17 on old Darwin)
  • aligned allocation in the language and it works

The driver aligned allocation flags force the compilation into case 1 or 3. Cases 1 and 2 do not advertise __cpp_aligned_new. Case 2 provides an error if an aligned allocation is attempted, whereas case 1 provides an underaligned allocation with a warning. This flag puts us into case 2, so it isn't the same as any driver flag.

This revision is now accepted and ready to land.Jul 25 2018, 3:27 PM

Eric, will you have time to commit this patch? If you don't have time and don't have objections, I plan to land this change on your behalf.

EricWF added a comment.Aug 3 2018, 3:45 PM

Hi, I'm in the process of moving cities. please take over.

Thanks, and sorry

This revision was automatically updated to reflect the committed changes.

Hi, I'm in the process of moving cities. please take over.

Thanks, and sorry

No worries. Thanks for spending time on this issue and making the fix, committing is the easy part. Good luck with the move.