This is an archive of the discontinued LLVM Phabricator instance.

[libcxx][views] Add drop_view.
ClosedPublic

Authored by zoecarver on May 6 2021, 5:41 PM.

Details

Summary

The first view in the libc++ ranges library ๐Ÿš€

Diff Detail

Event Timeline

zoecarver created this revision.May 6 2021, 5:41 PM
zoecarver requested review of this revision.May 6 2021, 5:41 PM
Herald added a project: Restricted Project. ยท View Herald TranscriptMay 6 2021, 5:41 PM
Herald added a reviewer: Restricted Project. ยท View Herald Transcript
Herald added a subscriber: libcxx-commits. ยท View Herald Transcript
zoecarver added inline comments.May 6 2021, 5:42 PM
libcxx/test/std/ranges/range.adaptors/range.drop/range.drop.view.pass.cpp
179

I just had a thought: I should add some tests where views::all comes into play (so make sure we use the deduction guides to automatically create drop_view<subrange> and drop_view<ref_view>). I'll do this tomorrow morning.

cjdb requested changes to this revision.May 6 2021, 7:12 PM

Overall, this is great! Sadly, I can't LGTM till caching is implemented, but I think we'll be good to land this sometime next week :-)

libcxx/include/__iterator/primitives.h
98

If this is temporary, carry on. If there's a genuine concern, please highlight it in D101922.

libcxx/include/__ranges/drop_view.h
34โ€“38

I'd very much appreciate a few simple libcxx tests for this concept. It's so important that I want so much as a light breeze nearby to cause a build failure.

42

Nit 1: please suffix members with an underscore.
Nit 2: please put private stuff at the bottom unless it needs to be seen.

59โ€“61

Let's hold off on merging this with main till ranges::next is available (I'll throw up a patch as soon as D101922 is approved).

You also need to cache the result for forward ranges (see [range.semi.wrap]). This is observable behaviour, so please add a test to ensure the caching happens.

66โ€“68

I'd suggest wrapping this into a static __begin_impl so there's a shared impl. Same with size, but not end.

libcxx/test/std/ranges/range.adaptors/range.drop/range.drop.view.pass.cpp
24

Nit: I'd prefer if this were std::array<int, 8>.

28โ€“67

You have four views that satisfy range<T const>, if I'm not mistaken. Please ensure there's at least one non-const input view, and one non-const forward view too. Also, please check out test/support/test_range.h to see if some of these are already available (I don't think they are), and add some utilities there, as they'll be necessary in other range tests too.

62โ€“64

const-qualified access member functions should probably return const-iterators. Similarly elsewhere.

69

cpp17_ or cpp20_input_iterator

78โ€“80

Please also expand to check const qualification. This is very important!

81โ€“85

These should possibly go into their own ctad.compile.pass.cpp.

111โ€“136

I'd prefer it if these were in functions such as check_begin(). The more intrinsic documentation we have, the less likely comments can go out of date.

117โ€“118

Can we test with some odd numbers and even non-powers of 2 too please?

155โ€“172

We should have const-qualified tests for all these.

This revision now requires changes to proceed.May 6 2021, 7:12 PM
zoecarver updated this revision to Diff 343716.May 7 2021, 10:33 AM
  • Implement caching.
zoecarver updated this revision to Diff 343718.May 7 2021, 10:37 AM
  • Uncomment the static_assert.
cjdb requested changes to this revision.May 7 2021, 11:06 AM
cjdb added inline comments.
libcxx/include/__ranges/drop_view.h
45

Caching should only be available for forward ranges (it's a logic error to cache for input ranges, and also not to spec). Also, please make this [[no_unique_address]].

This revision now requires changes to proceed.May 7 2021, 11:06 AM
zoecarver updated this revision to Diff 343746.May 7 2021, 1:11 PM
  • Move size into a helper function.
ldionne requested changes to this revision.May 7 2021, 1:20 PM

Overall LGTM, left some comments and we just spoke offline too, so you know what else I want. Thanks a lot for working on this, I think this is a huge milestone!!

libcxx/include/__ranges/drop_view.h
36

This should eventually be lifted to a common header because I believe many views are going to depend on this exposition only concept.

48

The combination of = default on this constructor and default member initializers is IMO somewhat misleading. When I see = default, I generally think "oh, the constructor default constructs everything". But this is not the case here.

I think I would instead drop default-member-initializers and explicitly construct the members in the two constructors.

(Yes, I know default member initializers is the way the Standard exposts the class, but they don't have iterator caching).

Edit:

Actually coming back to this, I don't understand why we cache begin(__base) here in the constructor. Shouldn't we only cache it in begin()?

62

I think it would be more natural to use

if (!input_or_output_iterator<iterator_t<_View>>) {
 // cache here
}
65

Is this intended? else {}

If not... there's something funky here if your tests are passing! If so, I would reorganize to avoid this unusual construct.

libcxx/test/std/ranges/range.adaptors/range.drop/range.drop.view.pass.cpp
90

Add a comment // test CTAD

This revision now requires changes to proceed.May 7 2021, 1:20 PM
tcanens added a subscriber: tcanens.May 7 2021, 3:41 PM
tcanens added inline comments.
libcxx/include/__ranges/drop_view.h
45

There's no requirement that ranges::begin on a default-constructed view is even well-defined (see: ref_view).

Additionally, when a drop_view is copied, the cache must be invalidated in the copy.

tcanens added inline comments.May 7 2021, 4:06 PM
libcxx/include/__ranges/drop_view.h
45

...and when a drop_view is moved-from, the cache must be invalidated in the original.

zoecarver updated this revision to Diff 343778.May 7 2021, 4:11 PM
zoecarver marked 4 inline comments as done.May 7 2021, 4:11 PM

Apply some of the comments.

I still need to:
-> Add a test for move-only input-iterator.
-> Expand the const tests.
-> Add tests for all_t CTAD.

tcanens added inline comments.May 8 2021, 5:01 AM
libcxx/include/__ranges/drop_view.h
57

I'd put __count before __base - see https://lists.isocpp.org/lib/2020/09/17540.php.

cjdb added inline comments.May 8 2021, 12:10 PM
libcxx/include/__ranges/drop_view.h
42

__cache_base is a bit too general a name.

45

@tcanens is this a non-propagating cache?

tcanens added inline comments.May 8 2021, 12:32 PM
libcxx/include/__ranges/drop_view.h
45

Yep. This is the original use case of non-propagating-cache in range-v3.

zoecarver added inline comments.May 10 2021, 9:10 AM
libcxx/include/__ranges/drop_view.h
45

Where in the standard does it say this is a non-propagating cache? Or is that what your paper does?

62

Unfortunately, I just realized that all iterators model input or output iterator, so I think we'll have to stick with forward_range.

cjdb added inline comments.
libcxx/include/__ranges/drop_view.h
45

The standard is very underspecified in what kind of cache is necessary here. I just checked, and range-v3 and cmcstl2 disagree on what kind of cache is necessary: range-v3 uses a non-propagating cache; cmcstl2 switches between a non-propagating cache and a simple cache, depending on whether or not we have a forwarding-range (which I think is now borrowed_range).
I'm inclined to support Tim's opinion here, because it's simpler, and doesn't need to introduce a __cached_position type that's dependent on __non_propagating_cache. Any opinions @CaseyCarter?

tcanens added inline comments.May 10 2021, 9:46 AM
libcxx/include/__ranges/drop_view.h
45

Where in the standard does it say this is a non-propagating cache? Or is that what your paper does?

It falls out of the fact that nothing in the standard allows "take a drop_view, call begin() on it, copy it, and call begin() on the copy" to not work.

zoecarver updated this revision to Diff 344559.May 11 2021, 2:05 PM

Add tests for views::all ctad.

zoecarver planned changes to this revision.May 20 2021, 12:07 PM

I need to update this to use semiregular box.

cjdb requested changes to this revision.Jun 10 2021, 8:59 AM

Please update according to the recent changes made by P2325.

ldionne requested changes to this revision.Jun 21 2021, 9:36 AM
  • Some of my comments (especially about splitting tests) also apply to other in-flight reviews like transform_view.
  • Please don't forget to mark comments as done when they're not relevant anymore (whether they have been implemented or you've decided you wouldn't do it), so it's easy to keep track of things.

I'll wait for this to be rebased onto non_propagating_cache before I re-review the implementation.

libcxx/test/std/ranges/range.adaptors/range.drop/range.drop.view.pass.cpp
1

Please split this into more files as it makes sense. Each file tests one thing, and what's being tested is at the top of level in a comment, as we always do.

24

I'll actually disagree here - I think it's good to stay minimal when we don't lose anything doing so.

There was quite a bit of discussion offline about the need to invalidate the cached begin() or not. I think we now all agree that we must not propagate the cached value on copies or moves. Here's what I ended up with that is not dependent on non_propagating_cache (we can go back afterwards and perhaps simplify some of this if we have non_propagating_cache):

namespace ranges {
  template<class _Range>
  concept __simple_view =
    view<_Range> && range<const _Range> &&
    same_as<sentinel_t<_Range>, iterator_t<const _Range>> &&
    same_as<iterator_t<_Range>, sentinel_t<const _Range>>;

  template<view _View>
  class drop_view
    : public view_interface<drop_view<_View>>,
  {
    static constexpr _UseCache = forward_range<_View> && !random_access_range<_View>;
    using _Cache = optional<iterator_t<_View>>;
    struct _Empty { };
    [[no_unique_address]] conditional_t<_UseCache, _Cache, _Empty> __cached_begin_;

    _View __base_;
    range_difference_t<_View> __count_;

public:
    constexpr drop_view() requires default_ยญinitializable<_View> 
      : __cached_begin_(), __base_(), __count_(0) 
    { }

    constexpr drop_view(_View __base, range_difference_t<_View> __count)
      : __cached_begin_(), __base_(_VSTD::move(__base)), __count_(__count) {
      _LIBCPP_ASSERT(__count_ >= 0, "count must be greater than or equal to zero.");
    }

    constexpr drop_view(drop_view const& __other) 
      : __cached_begin_() // purposefully not propagating the cached value
      , __base_(__other.__base_)
      , __count_(__other.__count_)
    { }

    constexpr drop_view(drop_view&& __other) 
      : __cached_begin_() // purposefully not propagating the cached value
      , __base_(_VSTD::move(__other.__base_))
      , __count_(_VSTD::move(other.__count_))
    { }

    constexpr drop_view& operator=(drop_view const& __other) {
      if constexpr (_UseCache) {
        __cached_begin_.reset();
      }
      __base_ = __other.__base_;
      __count_ = __other.__count_;
      return *this;
    }

    constexpr drop_view& operator=(drop_view&& __other) {
      if constexpr (_UseCache) {
        __cached_begin_.reset();
        __other.__cached_begin_.reset();
      }
      __base_ = _VSTD::move(__other.__base_);
      __count_ = _VSTD::move(__other.__count_);
      return *this;
    }

    constexpr _View base() const& requires copy_constructible<_View> { return __base_; }
    constexpr _View base() && { return _VSTD::move(__base_); }

    constexpr auto begin()
      requires (!(__simple_view<_View> && random_access_range<const _View> && sized_range<const _View>))
    {
      if constexpr (_UseCache) {
        if (__cached_begin_.empty())
          __cached_begin_ = rangesโ€‹::โ€‹next(rangesโ€‹::โ€‹begin(__base_ยญ), __count_ยญ, rangesโ€‹::โ€‹end(__base_ยญ));
        return *__cached_begin_;

      } else {
        return rangesโ€‹::โ€‹next(rangesโ€‹::โ€‹begin(__base_ยญ), __count_ยญ, rangesโ€‹::โ€‹end(__base_ยญ));
      }
    }

    constexpr auto begin() const
      requires random_access_range<const _View> && sized_range<const _View>
    {
      return rangesโ€‹::โ€‹next(rangesโ€‹::โ€‹begin(__base_ยญ), __count_ยญ, rangesโ€‹::โ€‹end(__base_ยญ));
    }

    constexpr auto end()
      requires (!__simple_view<_View>)
    { return ranges::end(__base_); }

    constexpr auto end() const
      requires range<const _View>
    { return ranges::end(__base_); }

    static constexpr auto __size(auto& __self) {
      const auto __s = ranges::size(__self.__base_);
      const auto __c = static_cast<decltype(__s)>(__self.__count_);
      return __s < __c ? 0 : __s - __c;
    }

    constexpr auto size()
      requires sized_range<_View>
    { return drop_view::__size(*this); }

    constexpr auto size() const
      requires sized_range<const _View>
    { return drop_view::__size(*this); }
  };

  template<class _Range>
  drop_view(_Range&&, range_difference_t<_Range>)
    -> drop_view<views::all_t<_Range>>;
} // namespace ranges
zoecarver updated this revision to Diff 353505.Jun 21 2021, 3:43 PM
  • Split up tests.
  • Use std::optional as a non-propagating cache (and add a test).
zoecarver marked 13 inline comments as done.Jun 21 2021, 3:50 PM
zoecarver added inline comments.
libcxx/include/__ranges/drop_view.h
36

This was already done.

42

I'm going to leave it at the top. I don't really see a reason to put it at the bottom, especially when classes have private members by default.

45

For posterity: marking this whole chain as resolved after our discussion on discord.

libcxx/test/std/ranges/range.adaptors/range.drop/range.drop.view.pass.cpp
20

Remove this.

zoecarver marked 3 inline comments as done.Jun 21 2021, 3:53 PM
zoecarver updated this revision to Diff 353506.Jun 21 2021, 3:54 PM

Apply a few remaining review comments.

libcxx/include/__ranges/drop_view.h
42

I'm going to leave it at the top

Please do as @ldionne says, and follow libc++ style. Private/data members go at the bottom (and data members get a suffix-underscore; thanks for addressing that part already).
Also, please consider putting __cached_begin_ in front of __count_, for layout reasons. It's more likely that _View begins with an _Empty member than that _View::difference_type does.

@ldionne wrote:

Here's what I ended up with that is not dependent on non_propagating_cache (we can go back afterwards and perhaps simplify some of this if we have non_propagating_cache)

FYI, that looks pretty nice to me. And then non_propagating_cache would just be a thin wrapper around an optional โ€” all it needs to provide is .has_value(), .get() (maybe more than one ref-qualification of .get()?), .assign (i.e. the-equivalent-of-operator=, not the-equivalent-of-.emplace), and a default constructor. One might reasonably say "If Louis' code works, then why do we even want a factored-out non_propagating_cache type?" But, I answer, it's still useful to wrap optional, in order to reduce its surface area to just these four necessary bits.

cjdb accepted this revision.Jun 21 2021, 5:01 PM

Per offline discussion, I'm okay with this going ahead, provided we return to add __non_propagating_cache once I've landed it.

libcxx/include/__ranges/drop_view.h
134

Please line up requires-clauses with the start of the function declaration. Small clause (such as this) can go on the same line.

I've usually turned a blind eye to this because comments about formatting are noise that lead to unproductive debates, and my intention has been to get clang-format to take care of this inconsistency in one fell swoop. Having said that, since I don't know when I'll be able to apply the tool project-wide and not get something weird, it's probably worth adopting the "official" libc++ style for future contributors' sakes.

142โ€“145

Does this need to be fixed?

libcxx/test/std/ranges/range.adaptors/range.drop/general.pass.cpp
61 โ†—(On Diff #353506)

I like that you're using this!

tcanens added inline comments.Jun 21 2021, 5:13 PM
libcxx/include/__ranges/drop_view.h
38

The cache needs to be used for random access ranges too unless the range is also sized.

libcxx/include/__ranges/drop_view.h
134

FWIW, I agree about "unproductive debates" ;) but if there's an official libc++ style, it's always been "indent the subordinate clause by one level."

template<class T>
  requires foo<T>
auto bar()
  -> decltype(baz)
{
  return quux;
}

I don't think your insistence on following clang-format's (broken) formatting guidance here is helpful to the project. I predict that clang-format will eventually implement C++20 formatting, and then it'll be good that Zoe's been following the "indent requires-clauses" style.

cjdb added inline comments.Jun 21 2021, 6:42 PM
libcxx/include/__ranges/drop_view.h
134
ldionne requested changes to this revision.Jun 22 2021, 8:56 AM

I would like to see tests for the constructors. They're going to be simple tests, but we should be testing everything systematically. Elsewhere we have xxxx.ctor.pass.cpp tests. Also, can you please explicitly mark where you're testing each overload of begin() and end()?

I've got a few comments, but they are pretty small. We should be able to land this today.

libcxx/include/__ranges/drop_view.h
50

noexcept(is_nothrow_default_constructible_v<_View>)?

Also, this should be tested.

67

I think __cached_begin_(nullopt) might be clearer - it's a nice reminder that we're not default-constructing the iterator itself, but only an optional. (here and elsewhere)

134

I had not realized that the WP was indenting requires-clauses, but I think it might make sense to try to follow that. @cjdb is there a reason why you went for no indentation in the first place?

What I care most about is that we're consistent and that our code is readable and idiomatic as much as possible. Is there an established way of formatting those in the community?

libcxx/test/std/ranges/range.adaptors/range.drop/begin.pass.cpp
13 โ†—(On Diff #353506)

Those should describe what's being tested in the test. We've always done that in libc++, and frankly it's quite useful to know exactly what's being tested (especially since we love to have subtly different overloads of functions in the stdlib).

This revision now requires changes to proceed.Jun 22 2021, 8:56 AM
zoecarver added inline comments.Jun 22 2021, 9:14 AM
libcxx/include/__ranges/drop_view.h
38

Why does the range being sized matter? Doesn't a random access range provide amortized constant-time complexity for element lookup (which is the requirement for drop_view)?

42

I reordered these based on Tim's suggestion:

I'd put count before base - see https://lists.isocpp.org/lib/2020/09/17540.php.

But you're right, I think cached_begin should still come first (then count, then base).

42

Private/data members go at the bottom (and data members get a suffix-underscore; thanks for addressing that part already).

Where does it say that private data members go at the bottom? If it's not documented anywhere, I'll go off of what style we currently use. Doing a quick audit of the first three types that come to mind, I see that basic_string, shared_ptr (and all of its helper classes), and map all put their data members at the top. After I continued to look, the only class I could find that didn't do this is vector, but even its base/helper classes put their private members at the top.

I find these comments about formatting (here and elsewhere) counter productive. I try not to comment on formatting unless it's really egregious. Until we create some formatting rules (which I think would be a great idea, because then contributors would know how to format their code, and you wouldn't have to waste time in reviews asking for changes), let's not comment about formatting unless it makes the code harder to understand.

142โ€“145

At some point, yes. But we don't have all_t yet and this is not functionally different than what it will be.

tcanens added inline comments.Jun 22 2021, 10:18 AM
libcxx/include/__ranges/drop_view.h
38

You need the size of the underlying view to figure out how much to drop in constant time.

If you don't know that, then you have to advance the iterator one step at a time until you either reach the end of the range or N steps, and that's linear.

cjdb added inline comments.Jun 22 2021, 10:24 AM
libcxx/include/__ranges/drop_view.h
50

Why are we not just = defaulting this constructor?

134

is there a reason why you went for no indentation in the first place?

It's the style I adopted back in 2016 when working on cmcstl2; libc++ didn't have any established precedent for requires-clauses, so I paved a way forward. I also find it kind of arbitrary to indent it? We don't indent in the middle of a code block unless we open some sort of grouping tokens (usually parens or braces).
It also swallows up valuable horizontal width, which can impact readability.

Is there an established way of formatting those in the community?

  • cmcstl2 and cjdb-ranges line it up
  • Microsoft/STL and libstdc++ indent

All four use same-line requires-clauses for short requirements, which I think we should adopt in libc++.

I had not realized that the WP was indenting requires-clauses, but I think it might make sense to try to follow that.

Sure, if you want to rescind your offer from D99691, I'll play ball. I would like to know why you think it makes sense to follow the standard wording, but we can follow up with that offline.

zoecarver marked 3 inline comments as done.

Update based on review.

libcxx/include/__ranges/drop_view.h
38

Aha, that makes sense. Thanks for explaining! :)

libcxx/test/std/ranges/range.adaptors/range.drop/begin.pass.cpp
13 โ†—(On Diff #353506)

Good point. Fixing.

ldionne accepted this revision.Jun 22 2021, 12:02 PM

LGTM pending CI and applying the comments you noted when we spoke just now.

This is awesome! Let's move to transform_view now!

libcxx/include/__ranges/drop_view.h
42

@zoecarver is right that we sometimes have private data members at the top, and sometimes at the bottom. I think either way's fine.

Regarding formatting comments, I think what's hurting is not that we have them during reviews, but that not everybody agrees on how to format things. That creates back-and-forth and arguing over those details, and that is what's harmful. Taking 2 seconds to reformat something that isn't formatted properly isn't the issue IMO.

This revision is now accepted and ready to land.Jun 22 2021, 12:02 PM

Update based on the last few review comments.

When the CI is green I'll land it.

tcanens added inline comments.Jun 22 2021, 1:27 PM
libcxx/include/__ranges/drop_view.h
107

For the random-access-and-sized case, begin should not call ranges::next - that can be linear for sized-but-not-sized-sentinel ranges. It should just return ranges::begin(__base_) + std::min(__count_, ranges::distance(__base_)). Ditto below.

zoecarver updated this revision to Diff 353809.Jun 22 2021, 3:41 PM

Remove shadowing for GCC.

zoecarver updated this revision to Diff 353996.Jun 23 2021, 8:47 AM

Fix memset warning for GCC.

zoecarver added inline comments.Jun 23 2021, 8:51 AM
libcxx/include/__ranges/drop_view.h
107

I think it's OK to use next because we can ensure that next will always use something equivalent to ranges::begin(__base_) + std::min(__count_, ranges::distance(__base_)) (even if it's technically permitted to do something slower by the standard). I'd rather not add a special case here.

tcanens added inline comments.Jun 23 2021, 8:59 AM
libcxx/include/__ranges/drop_view.h
107

I think it's OK to use next because we can ensure that next will always use something equivalent to ranges::begin(__base_) + std::min(__count_, ranges::distance(__base_)) (even if it's technically permitted to do something slower by the standard).

The point is that you can't. For a sized-but-not-sized-sentinel range, the size information is only known from the range itself. Once you decompose the range into an iterator and a sentinel for ranges::next, that information is gone.

This revision was landed with ongoing or failed builds.Jun 23 2021, 10:11 AM
This revision was automatically updated to reflect the committed changes.
zoecarver added inline comments.Jun 23 2021, 10:16 AM
libcxx/include/__ranges/drop_view.h
107

I see what you're getting at. So we'd need to change the condition to be sized_sentinel_for (not sized_range) so that it matches what ranges::advance does (or, better yet, do what you suggested and take advantage of the sized range inside of this function).