Support arrays in make_shared and allocate_shared (P0674R1)

[zoecarver]

Summary

This patch implements P0674R1. It might be a bit rough around the edges, but I wanted to get it out so it can be reviewed and I can make changes to it as I straighten out the last few parts.

The main changes are how __shared_ptr_pointer deallocates itself and (obviously) the added overloads.

• Relevant section of the standard.
• Paper.
• Based on D62259.

Thanks for the help @mclow.lists and @EricWF

[ldionne]

libcxx/include/memory

2624

Well, one place where we might want to use it is in the statically-sized array version of the block (see below why I think we should have one).

2628

I don't understand the benefit of passing false_type here when we want to default construct, instead of just relying on variadics and always passing down some args... to __construct_elements (and hence to the object's constructors themselves). I think it would be simpler to do that.

Also, I think it makes sense to mark this constructor as explicit, and it needs to be marked with _LIBCPP_HIDE_FROM_ABI.

2648

I see you're using an int here, but what if construction fails at a very large index, the size_t to int narrowing could be problematic. Is there a reason why you're using int?

2691–2696

The way I understand this is that we should actually always call __destroy_elements(__first, __i + 1); to destroy the half-open range [__first, __first + __i + 1) (which includes the last object that was fully constructed, but not the object that was partially constructed). However, when the current element (__first + __i) is an array, it has already been destroyed before we caught the exception, so we can skip it and just destroy [__first, __first + __i).

This implementation makes that intent clearest IMO:

// <explanation>
if constexpr (is_array_v<_ValueType>)
    __destroy_elements(__first, __i);
else
    __destroy_elements(__first, __i + 1);

Do you agree?

3501

Aha! I actually took this from your earlier patch: make_shared can be implemented in terms of allocate_shared, so it can just be one function template with no constraints that calls out to all the various allocate_shared overloads.

I don't think that works, actually. For example, std::make_shared<int[3]>(1, 2, 3) should SFINAE away, but with your implementation that would be a hard error. Can you add a test and see whether that's correct? If so, it means we need to conform to the declarations laid out in the standard (there's often reasons why the standard lists overloads explicitly).

3529

We save on storing the size itself in the control block, and the size is known statically, which could lead to different code being generated (e.g for small arrays). I think it's fairly important to do it, since it isn't very costly for us.

The representation for statically-sized arrays can be:

[[no_unique_address]] _Alloc __alloc_;
union { // make sure the default constructor doesn't initialize __data_
    _Tp __data_[_Np];
};

[zoecarver]

libcxx/include/memory

2648

I'm using int because __n will become -1 in the for loop. I could keep it unsigned and just compare __n < "size", but that seems a bit more confusing and would require a second variable. So, I'll make __n a long.

[zoecarver]

• Add static size control block.
• Fix based on review.

[zoecarver]

• Format

[zoecarver]

libcxx/include/memory

2732

Actually, because we're inheriting this base class, the [[no_unique_address]] isn't needed.

[zoecarver]

libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/create_array.pass.cpp

10

Oops, forgot the SFINAE test. I'll add that now.

[zoecarver]

• Add SFINAE tests.
• Remove "no_unique_address".

libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/create_array.pass.cpp

10

I know this is complicated to test, but I think it is important. It may make sense to split tests into multiple files to make it easier to understand. Your call.

I personally don't mind the mono-test and it makes things easier to find, IMHO. But I also don't have a strong oppinion at all, would you rather me split this test up?

[zoecarver]

@ldionne this is ready for re-review when you're ready to re-review it :)

[zoecarver]

@ldionne friendly ping.

[ldionne]

I don't think we gain much from having a single __shared_ptr_array_base common base. Actually, this forces us to have additional virtual methods in both subclasses. Instead, we should just have two different holder classes, one for statically-sized arrays, and one for dynamic arrays.

libcxx/docs/Cxx2aStatusPaperStatus.csv

3

This should be 13.0 now

[zoecarver]

• Bump LLVM release version.
• Remove __shared_ptr_array_base.

[ldionne]

Thanks a lot for all your work on this patch. This is a tough one to get right. I know I'm requesting a lot, but I think it's important to get it right and we'll be even happier when we finally ship this :-).

libcxx/include/memory

2668

We could also use __first[i] here, I think it's clearer?

2690

I haven't looked for that specifically, but can you confirm that you are testing these two code paths?

2697

Can we name them xx_bounded and xx_unbounded instead? It seems to fit better?

2702–2705

Currently, this function is only used in a single place (which is inside the implementation of this class). Furthermore, we duplicate that logic when we compute the size to allocate down there in make_shared_xx.

One possibility would be to move this to a free function like

template <class T>
size_t __get_shared_ptr_unbounded_block_size(size_t __n) {
    // ...
}

Then we could reuse it from both places. Do you agree that would be an improvement?

2725

Don't we need to run the destructor of __alloc_ here?

2731

Is this legal C++? I thought that was a compiler extension and arrays had to have at least one element?

2741

Since this is only used in a single place in the body of __shared_ptr_array_bound, I think we could just inline the function at its single point of use and remove it. WDYT?

2760

Same comment regarding destruction of __alloc_.

3518

I *think* this can cause us to under allocate, for example if there needs to be padding between __count_ and __data_:

[[no_unique_address]] _Alloc __alloc_;
size_t __count_;
// padding possible here - would that be counted in sizeof(_ControlBlock)?
_DataBlock __data_[0];

This concern is moot if I'm right about _DataBlock __data_[0] being an extension and if we change to _DataBlock __data_[1] instead.

[zoecarver]

libcxx/include/memory

2658

Why not pass by value?

2690

Yes, these two code paths are tested. See L324 where we use struct D to throw.

2725

Yep. Added a regression test, too.

2731

Yes, you're right. Fixed. (Supprising that we didn't get a warning, though?)

2741

Agreed. Same with __get_count.

3518

Moot because I fixed __data_[0 -> 1].

However, I thought this was the whole point of using a zero size array at the end of a struct? Otherwise, why not just make an empty struct. So, yes, that padding would be accounted for:

struct UnsizedTail {
  int size;
  alignas(8) char buf[0];
};

// UnsizedTail = {i32, [4 x i8 padding], [0 x i8]
static_assert(sizeof(UnsizedTail) == 8);

That being said... when would there be padding between size_t and an aligned storage member?

[zoecarver]

libcxx/include/memory

2658

Whoops. I thought I deleted this comment. This was just a note to myself.

Anyway, I am actually wondering about this. I see _Alloc const& all over the place, but allocators are generally supposed to be small (often empty) types, so why not pass them by value? We're copy constructing the allocator in this function, so it can't be to prevent that.

[zoecarver]

• Address review comments and add a few more tests.
• Format test file with clang-format.

[ldionne]

This basically LGTM except for the few nitpicks below. We already talked about most of that offline, just adding here so you have a written trace.

libcxx/include/memory

2658

Generally, we take by value (and then move into some internal storage) when we take ownership of an object -- or we add a version that takes that object by rvalue reference. You could do that here and move into __value_alloc if you wanted.

3512

We need to handle the case where __n == 0. Also, we should add a test for it since I'm not seeing anything that says it's not allowed by the API.

libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/create_array.pass.cpp

2

Can we add a test with overaligned types? (like we discussed offline)

10

You can remove c++98 from the Lit features, we don't use it anymore.

48

Can you add a comment mentioning that? It's clever.

[zoecarver]

libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/create_array.pass.cpp

48

Done. Thanks :)

[zoecarver]

• Address review comments.
• Re-generate feature test macros after rebase.
• Format changes with git-clang-format.

[zoecarver]

• Poke buildkit.

[zoecarver]

OK, that seems to have worked. @ldionne for future reference: the CI was failing to apply this patch because I had two parent revisions and a child revision which were abandoned/already landed. I think the abandoned one was causing problems because it had conflicts. Removing those patches fixed the issue, but it would be nice to automatically skip abandoned parent patches.

[zoecarver]

• Fix for -fno-exceptions.

[zoecarver]

• Destroy alloc before deallocating this.
• Cast char to size_t in assert.

[zoecarver]

• Fix UBSAN and GCC build bots.

[zoecarver]

• Fix ASAN bots.
• Format test files.

[zoecarver]

libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/create_array.pass.cpp

11

I really don't like disabling this whole test for sanitizer-new-delete. We need this because of the globalMemCounter tests. This is the same thing the unique_ptr tests use.

@ldionne do you know if there's a macro or some other way we can detect if we're running on sanitizer-new-delete? If so, we could just disable the two tests that actually cause this to fail.

[ldionne]

libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/create_array.pass.cpp

11

I don't think there's such a macro. You could split it off into its own test file if you wanted.

[zoecarver]

• Fix UBSAN bot.
• Fix macOS bot.