[libcxx][ranges] Adds `common_iterator`.

[ldionne]

libcxx/include/__iterator/common_iterator.h

106

Maybe add _LIBCPP_ASSERT(_VSTD::holds_­alternative<_Iter>(__hold_))?

115

Could you break that { onto a different line? It's kind of hard to see where the requires end and the function body begins.

146

Can we hoist the value of xxxx.index() into local variables? (here and below)

157

Please make sure you test all the branches below (here and in the other functions below too).

203–205

This would fit nicely on a single line IMO.

libcxx/test/std/iterators/predef.iterators/iterators.common/iterator_traits.compile.pass.cpp

57

This doesn't look right. According to the spec (http://eel.is/c++draft/iterators.common#common.iter.types-1.3), this should either be the type of operator->() or void, but never Iter const&. This means you most likely have a bug in the implementation.

Also, please add a test where pointer ends up being void.

[zoecarver]

libcxx/include/__iterator/common_iterator.h

157

Added a few tests. Have verified all paths are taken.

libcxx/test/std/iterators/predef.iterators/iterators.common/iterator_traits.compile.pass.cpp

57

This doesn't look right. According to the spec (http://eel.is/c++draft/iterators.common#common.iter.types-1.3), this should either be the type of operator->() or void, but never Iter const&. This means you most likely have a bug in the implementation.

Turns out this is actually correct. This is supposed to be the return type of common_iterator's operator->(). And if _Iter has an operator->() then it's supposed to simply return get<Iter>(v). (However, I'd argue that should maybe return get<Iter>(v).operator->(), if you agree, maybe I'll file an LWG issue.)

Sound good now? This threw me off too :P

57

Also, please add a test where pointer ends up being void.

Done.

[zoecarver]

Apply Louis' comments.

libcxx/include/__iterator/common_iterator.h

221

@cjdb thanks for helping me figure this one out :)

[tcanens]

libcxx/test/std/iterators/predef.iterators/iterators.common/iterator_traits.compile.pass.cpp

57

This doesn't look right. According to the spec (http://eel.is/c++draft/iterators.common#common.iter.types-1.3), this should either be the type of operator->() or void, but never Iter const&. This means you most likely have a bug in the implementation.

Turns out this is actually correct. This is supposed to be the return type of common_iterator's operator->(). And if _Iter has an operator->() then it's supposed to simply return get<Iter>(v). (However, I'd argue that should maybe return get<Iter>(v).operator->(), if you agree, maybe I'll file an LWG issue.)

Producing decltype(a.operator->()) matches http://eel.is/c++draft/iterator.traits#1.sentence-3 but not the actual wording in http://eel.is/c++draft/iterators.common#common.iter.types-1.3 which says that it is the type of a.operator->(), aka remove_reference_t<decltype(a.operator->())>. What's the intent here @CaseyCarter?

[tcanens]

libcxx/include/__iterator/common_iterator.h

98

These should look at __other_idx - this assignment is valid even if *this is valueless.

273–275

This conditional_t seems unnecessary?

[ldionne]

libcxx/include/__iterator/common_iterator.h

79

This should be in the lambda above for it to trigger before we try to access __other.__hold_ unchecked.

93

I think you want _VSTD::__unchecked_get here and elsewhere.

98

Can you add a test where you assign to a common_iterator with a valueless_by_exception() variant?

108

As discussed live: we can use a better message here.

170

How is this behavior really what we want? IIUC, that means that if you compare two common_iterators that both contain different iterators (say It1 and It2) that are not comparable to each other, the common_iterators will compare equal to each other even if It1 and It2 are "pointing" to entirely different elements. Am I misunderstanding something here, or that's an incredibly subtle (and broken) behavior to have at runtime? @tcanens Can you shed some light on this?

In all cases, we need a test where we exercise that.

[zoecarver]

libcxx/include/__iterator/common_iterator.h

41

Add a comment: "iter_reference_t" is never a reference here because if it is we catch it in the if constexpr branch above where this one is called.

108

This should say something like "common_iterator not holding an iterator (holding a sentinel) must hold iterator."

170

We should assert these aren't both zero, even though it's non-conforming, people will thank us.

[tcanens]

libcxx/include/__iterator/common_iterator.h

170

This is meant for C++20 input iterators that aren't comparable with each other. Because incrementing an input iterator invalidates all copies, you can't have two valid iterators pointing to different elements in the same range.

We should assert these aren't both zero, even though it's non-conforming, people will thank us.

Please don't. i == i had better work.

[CaseyCarter]

libcxx/test/std/iterators/predef.iterators/iterators.common/iterator_traits.compile.pass.cpp

57

http://eel.is/c++draft/iterator.traits#1.sentence-3 is authoritative; http://eel.is/c++draft/iterators.common#common.iter.types-1.3 should be corrected to agree. Presumably the writer of the latter paragraph was under the impression that a.operator->() is always a prvalue.

[zoecarver]

libcxx/include/__iterator/common_iterator.h

170

This is meant for C++20 input iterators that aren't comparable with each other. Because incrementing an input iterator invalidates all copies, you can't have two valid iterators pointing to different elements in the same range.

Shouldn't this assert (or require) that these are input iterators then?

Please don't. i == i had better work.

But this is not i == i, it's x == y. What you're saying makes sense for input iterators, but what about forward iterators, or even contiguous iterators?

(Thanks for all the help with interpreting the standard's wording here, by the way.)

[tcanens]

libcxx/include/__iterator/common_iterator.h

170

common_iterator is a C++17 compatibility shim. I don't think this is worse than returning true when both are sentinels - two sentinels can mean completely different things, even if they have the same type.

[zoecarver]

libcxx/include/__iterator/common_iterator.h

170

common_iterator is a C++17 compatibility shim. I don't think this is worse than returning true when both are sentinels - two sentinels can mean completely different things, even if they have the same type.

The difference is sentinels must always be the end of the range. I can get behind saying "this must always be true because the sentinels must always be the end of the same range which must be the same element." The part I'm having trouble with is it's OK to have two different forward iterators that point to the same range in different places. Those should return false when compared (or somehow generate an error or UB).

[zoecarver]

Apply remaining few review comments :)

libcxx/include/__iterator/common_iterator.h

170

In all cases, we need a test where we exercise that.

Tested in both assign.pass.cpp (line 35, 50, 68, 74) and eq.pass.cpp (line 78, 88, 89).

[• Quuxplusone]

libcxx/include/__iterator/common_iterator.h

170

I'd prefer to see an explicit static_assert here:

static_assert(!equality_comparable_with<_Iter, _I2>);
return true;

I think this assertion would sufficiently explain why we aren't checking i1 == i2 here — it's because we physically can't. Also, the assertion serves as an important backstop against subsumption bugs. The absolute worst thing that could happen here is that the constraint on line 179 bit-rots under maintenance and we end up executing this code for iterators that are comparable.

174

_VSTD::__unchecked_get<0>(__x.__hold_) == _VSTD::__unchecked_get<1>(__y.__hold_)

I don't trust __unchecked_get<_Iter> to do the right thing when _Sent is cvref-qualified _Iter or vice versa (or they're in some other subtle corner-case way related to each other). Integer indices, on the other hand, are nice and solid and trustworthy. Also, cppreference documents that the code should use get<i> and get<j>, not get<I> and get<S2>; it's just barely possible that there might be a technical reason for that.

[tcanens]

libcxx/include/__iterator/common_iterator.h

170

forward iterators are required to be comparable, so it would go to the other overload.

[cjdb]

libcxx/include/__iterator/common_iterator.h

170

But this is not i == i, it's x == y. What you're saying makes sense for input iterators, but what about forward iterators, or even contiguous iterators?

This is the overload that's chosen when we evaluate i == i, so Tim's got a point. I'm also not sure what the assertion would achieve? Are you trying to prevent i == i?

[cjdb]

Thanks for working on such an ugly type. In addition to my inline comments, please add

• a module map entry
• iterator conformance tests

libcxx/include/__iterator/common_iterator.h

36

Can we define __proxy and __postfix_proxy out-of-line please?

64

This should be private. Also, hold is not a particularly descriptive name to me.

108

How about "Dereferencing a common_iterator is only possible if it is holding an iterator, but this one is holding a sentinel."?

123–125

Can we get this into a named object please? Perhaps __common_iterator_has_arrow?

154–156

Please name this.

273–275

Seconded. Please use _If, if you can.

libcxx/test/std/iterators/predef.iterators/iterators.common/plus_plus.pass.cpp

87

Please prefix "postfix" to this sentence.

libcxx/test/support/test_iterators.h

165

I think this functionality should instead be reflected in cpp20_input_iterator (and in a separate patch, if it breaks any code).

[zoecarver]

libcxx/include/__iterator/common_iterator.h

170

I'm talking about two completely unrelated iterators, for example, this:

auto iter1 = random_access_iterator<int*>(buffer + 1);
auto commonIter1 = std::common_iterator<decltype(iter1), sentinel_type<int*>>(iter1);

auto iter2 = forward_iterator<int*>(buffer + 4);
auto commonIter2 = std::common_iterator<decltype(iter2), sentinel_type<int*>>(iter2);

assert(commonIter1 == commonIter2);

(An actual test case ^)

If above is UB, it's not clear to me where that is stated. Even so, if it is UB for some reason, I think it would be great to add an assertion. And if it's well defined, well that seems terrible (and my preference would be to create an LWG issue and still add an assertion or something, because that's definitely a but, but maybe others disagree).

[tcanens]

libcxx/include/__iterator/common_iterator.h

170

So pathological mix-up scenarios where you compare completely unrelated things? It's hard to think of a non-contrived example where that matters.

I don't really care about this case either way; I'm just not convinced that it's worth the trouble (coming up with something that doesn't damage valid uses can be tricky).

@CaseyCarter thoughts?

[CaseyCarter]

libcxx/include/__iterator/common_iterator.h

170

It's hard to care about this. The domain of equality for forward_iterators is iterators obtained from the same range, so generic code can't perform this comparison. To do so in concrete code - when you know the result is useless - would be silly. In other words: if you think this is weird, then don't do it.

Conversely, if you want comparison of common_iterator<I1, S> and common_iterator<I2, S> to be sensible you can always make I1 and I2 model equality_comparable.

[zoecarver]

Apply Chris' review comments.

[zoecarver]

libcxx/include/__iterator/common_iterator.h

36

Why?

64

Because of the friend issues we discussed with transform_view on #libcxx we unfortunately can't have this be private.

123

@tcanens Another potential LWG issue: shouldn't this be is_pointer_v<_Iter> || requires(const _Iter& __i) { __i.operator->(); }? If not, do we need to have the is_pointer condition below? Can it ever be hit?

123–125

I don't want to add another "has arrow" concept that means something different from __has_arrow. Depending on my comment above, we might be able to use that here, though.

170

I guess it's pathological, but it would also be pathological to even compare any iterators (read: not sentinels) that use this overload. Basically, you're saying "you should only compare iterators here that are equal." In which case, why does this overload even need to work with iterators (read: not sentinels)?

We have two cases: 1) someone is using this correctly. They have two iterators that are equal. This returns true.

2. Someone is using this incorrectly. They have two iterators that are not equal.

You're saying the second case isn't worth worrying about because it would be pathological. This implies that no one should ever call this function without knowing if their iterators are equal beforehand. This begs the question: why do they even need to call this function in the first place? If they know their iterators are equal, it's pointless to compare them.

I think we should add a precondition or something that says i or j is 1. And I'd like to add that as an assertion here.

174

I'm not opposed to using indexes, but a concrete example would be nice, to show that there's actually a problem. I find using types much more readable, so I'd want a solid reason not to do it.

libcxx/test/support/test_iterators.h

165

Common iterator requires that we're copyable, so we really need this type. But I'm fully supportive of making cpp20_input_iterator non-default-constructible in a separate patch.

[tcanens]

libcxx/include/__iterator/common_iterator.h

123

is_reference_v<iter_reference_t<_I2>> should be enough to catch pointers.

170

Basically, you're saying "you should only compare iterators here that are equal." In which case, why does this overload even need to work with iterators (read: not sentinels)?

Because perfectly valid generic code written to C++17 iterator requirements may want to do this, perhaps because it makes the code simpler. Breaking valid code makes common_iterator unfit for its only purpose.

This is like "why does array<T, 0> has a front() when it's always UB to call it"? Well, because generic code may want to do so from a dynamically unreachable context.

I think we should add a precondition or something that says i or j is 1. And I'd like to add that as an assertion here.

That's a non-starter as far as I'm concerned.

[cjdb]

libcxx/include/__iterator/common_iterator.h

36

I find class-in-class definitions to clutter the outer-class definition. For the record, it's still common_iterator<_Iter, _Sent>::__proxy.

170

Basically, you're saying "you should only compare iterators here that are equal." In which case, why does this overload even need to work with iterators (read: not sentinels)?

Because perfectly valid generic code written to C++17 iterator requirements may want to do this, perhaps because it makes the code simpler. Breaking valid code makes common_iterator unfit for its only purpose.

This is like "why does array<T, 0> has a front() when it's always UB to call it"? Well, because generic code may want to do so from a dynamically unreachable context.

That it's UB means an implementation can assert that N != 0.

I think we should add a precondition or something that says i or j is 1. And I'd like to add that as an assertion here.

That's a non-starter as far as I'm concerned.

Could you please elaborate on why you think this is a non-starter? Having said that, I keep bouncing on my opinion for this matter whenever new evidence is put forward.

[ldionne]

libcxx/include/__iterator/common_iterator.h

170

@CaseyCarter said:

It's hard to care about this. The domain of equality for forward_iterators is iterators obtained from the same range, so generic code can't perform this comparison.

Ok. So basically, we never expect this overload to be called with It1 and It2 being forward_iterators, and this should only be called for input_iterators? In that case, I don't understand what's preventing us from adding a constraint !forward_iterator<It1> && !forward_iterator<It2> -- it's really cheap to do that, and we'd catch this pathological case at compile-time instead of having really strange behavior at runtime. And this will still work for all input_iterators, which is the only intended use case for this comparison operator if I followed correctly. Do you agree?

@zoecarver Whatever we end up doing here, I agree with Arthur's suggestion to add an assertion like static_assert(!equality_comparable_with<_Iter, _I2>); before returning true.

[tcanens]

libcxx/include/__iterator/common_iterator.h

170

Basically, you're saying "you should only compare iterators here that are equal." In which case, why does this overload even need to work with iterators (read: not sentinels)?

Because perfectly valid generic code written to C++17 iterator requirements may want to do this, perhaps because it makes the code simpler. Breaking valid code makes common_iterator unfit for its only purpose.

This is like "why does array<T, 0> has a front() when it's always UB to call it"? Well, because generic code may want to do so from a dynamically unreachable context.

That it's UB means an implementation can assert that N != 0.

At runtime, not at compile time. "It's always UB when called" doesn't justify = delete; or removal or static_assert in the body, because reasonable code can do if (not a.empty()) { do_something(a.front()); }.

Similarly here, just because the only meaningful comparison for iterators must return true doesn't justify making the comparison undefined.

I think we should add a precondition or something that says i or j is 1. And I'd like to add that as an assertion here.

That's a non-starter as far as I'm concerned.

Could you please elaborate on why you think this is a non-starter? Having said that, I keep bouncing on my opinion for this matter whenever new evidence is put forward.

Comparing a non-singular common_iterator with itself, or a copy of itself, must be valid. Making perfectly valid uses undefined at runtime just to catch pathological cases that are unlikely to ever arise in practice is totally backwards.

I'm open to something like what Louis suggested.

[zoecarver]

At runtime, not at compile time.
[...]
I'm open to something like what Louis suggested.

To clarify, I'm suggesting that we add a runtime assertion that does some combination of i == 1 || j == 1 || (!forward_iterator<It1> && !forward_iterator<It2>).

[ldionne]

I'm not seeing any blocking issues anymore, and it looks reasonable to me. I think we should file the LWG issue for operator== and ship it. Other reviewers, please chime in if you have feedback you really want to see addressed before this is shipped, otherwise @zoecarver feel free to ship this after the weekend or so (to give time for people to see this).

(needs CI)

libcxx/include/__iterator/common_iterator.h

170

@zoecarver Let's implement it per the spec for now and file a LWG issue with a proposed resolution like this:

template<class _I2, sentinel_for<_Iter> _S2>
    requires sentinel_for<_Sent, _I2> && 
             !equality_comparable_with<_Iter, _I2> && // I think this is required to avoid an ambiguity with the other overload
             !forward_iterator<_Iter> &&
             !forward_iterator<_I2>
friend bool operator==(const common_iterator& __x, const common_iterator<_I2, _S2>& __y);

Either this is accepted by LWG and we fix it before we've had time to ship it anyways, or it's not and in that case we'll be conforming (and hopefully LWG explains why they think this isn't a good resolution). Are you willing to file it?

[zoecarver]

Rebase.

[zoecarver]

Rebase (again).

[zoecarver]

Fix modules issues (hopefully).

[cjdb]

libcxx/include/variant

209–210

Please sort the list of includes.