Page MenuHomePhabricator

Make LLVM build in C++20 mode
ClosedPublic

Authored by BRevzin on Apr 27 2020, 9:14 AM.

Details

Summary

Part of the <=> changes in C++20 make certain patterns of writing equality operators ambiguous with themselves (sorry!). This review goes through and adjusts all the comparison operators such that they should work in both C++17 and C++20 modes. It also makes two other small C++20-specific changes (adding a constructor to a type that cases to be an aggregate, and adding casts from u8 literals which no longer have type const char*)

There were four categories of errors that this review fixes. Here are canonical examples of them, ordered from most to least common, which you can view on compiler-explorer:

#include <utility>

// 1) Missing const
namespace missing_const {
    struct A {
    #ifndef FIXED
        bool operator==(A const&);
    #else
        bool operator==(A const&) const;
    #endif
    };

    bool a = A{} == A{}; // error
}

// 2) Type mismatch on CRTP
namespace crtp_mismatch {
    template <typename Derived>
    struct Base {
    #ifndef FIXED
        bool operator==(Derived const&) const;
    #else
        // in one case changed to taking Base const&
        friend bool operator==(Derived const&, Derived const&);
    #endif
    };

    struct D : Base<D> { };

    bool b = D{} == D{}; // error
}

// 3) iterator/const_iterator with only mixed comparison
namespace iter_const_iter {
    template <bool Const>
    struct iterator {
        using const_iterator = iterator<true>;

        iterator();

        template <bool B, std::enable_if_t<(Const && !B), int> = 0>
        iterator(iterator<B> const&);

    #ifndef FIXED
        bool operator==(const_iterator const&) const;
    #else
        friend bool operator==(iterator const&, iterator const&);
    #endif
    };
    
    bool c = iterator<false>{} == iterator<false>{} // error
          || iterator<false>{} == iterator<true>{}
          || iterator<true>{} == iterator<false>{}
          || iterator<true>{} == iterator<true>{};
}

// 4) Same-type comparison but only have mixed-type operator
namespace ambiguous_choice {
    enum Color { Red };

    struct C {
        C();
        C(Color);
        operator Color() const;
        bool operator==(Color) const;
#ifdef FIXED
        friend bool operator==(C, C);
#endif
    };

    bool c = C{} == C{}; // error
    bool d = C{} == Red;
}

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Wtf, where'd my other changes go?

I've hit this before, use arc diff --update D78938 <base_branch>.

BRevzin updated this revision to Diff 260521.Apr 27 2020, 6:17 PM

Trying this again.

Wtf, where'd my other changes go?

I've hit this before, use arc diff --update D78938 <base_branch>.

Thanks Hubert!

(peanut gallery: I'd consider, while you're touching these all anyway, changing them all to non-member (friended where required) as I believe that's best practice - allows equal implicit conversions on either side, for instance (even if some types have no implicit conversions - it at least provides a nice consistency/examples that people are likely to copy from))

(peanut gallery: I'd consider, while you're touching these all anyway, changing them all to non-member (friended where required) as I believe that's best practice - allows equal implicit conversions on either side, for instance (even if some types have no implicit conversions - it at least provides a nice consistency/examples that people are likely to copy from))

Hidden friend is probably the best way to write comparisons in C++17 and earlier, but I'm not sure that will hold in C++20 (even if LLVM isn't on C++20 and won't be for I imagine quite some time). With reversed candidates, I think member functions might be the way to go there - you still get implicit conversions on either side (just not on both sides at the same time) and hidden friends... are kind of weird, to be honest.

Also, I didn't touch all of them - only the ones that break in C++20 (a lot of which just missing a const). A lot of comparison operators are already fine. I'm not sure it's worth changing them just to look the same.

(peanut gallery: I'd consider, while you're touching these all anyway, changing them all to non-member (friended where required) as I believe that's best practice - allows equal implicit conversions on either side, for instance (even if some types have no implicit conversions - it at least provides a nice consistency/examples that people are likely to copy from))

Hidden friend is probably the best way to write comparisons in C++17 and earlier, but I'm not sure that will hold in C++20 (even if LLVM isn't on C++20 and won't be for I imagine quite some time). With reversed candidates, I think member functions might be the way to go there - you still get implicit conversions on either side (just not on both sides at the same time) and hidden friends... are kind of weird, to be honest.

Yeah, probably just experience with things the way they've been, but the symmetry is kind of nice without relying on deeper aspects of the newer features (& the benefit of the code being more suitable for C++17, where LLVM is currently).

Also, I didn't touch all of them - only the ones that break in C++20 (a lot of which just missing a const). A lot of comparison operators are already fine. I'm not sure it's worth changing them just to look the same.

Yeah - just meant the ones you are touching, might be nice to move them in that direction.

Anyway, I'll leave it to you/other reviewers - no /super/ strong feelings here.

jdoerfert resigned from this revision.Apr 30 2020, 6:08 AM
jfb accepted this revision.May 18 2020, 8:28 AM
jfb added a subscriber: jfb.

This seems fine, assuming you've run usual tests?

This revision is now accepted and ready to land.May 18 2020, 8:28 AM
davidstone added inline comments.
llvm/include/llvm/ADT/DirectedGraph.h
99

Missing return

BRevzin updated this revision to Diff 265874.May 23 2020, 10:20 AM
  • A few more changes from tests.
BRevzin updated this revision to Diff 265875.May 23 2020, 10:25 AM
  • Adding missing return.
BRevzin marked 2 inline comments as done.May 23 2020, 10:27 AM

I hadn't build the tests before, updated with a few more changes. Some of the tests require u8 literals, whose type changes in C++20. I had no idea what to do with that, so I just #ifdef-ed out those tests with the appropriate feature test macro.

BRevzin updated this revision to Diff 265900.May 23 2020, 5:11 PM
  • Backing out changes that aren't strictly comparison-related.
jfb accepted this revision.May 24 2020, 2:56 PM

One suggestions, otherwise looks good. Thanks for doing this :)

llvm/include/llvm/ADT/DirectedGraph.h
40โ€“41

That comment, so informative! ๐Ÿ˜

99

๐Ÿ˜ฑ

Did this not trigger a diagnostic when building? I wonder if it's just not on?

llvm/unittests/ADT/STLExtrasTest.cpp
475

Can you add a comment above (with "fancy pointer") so mere mortals understand the parens?

I noticed the missing return because there is a warning (not as error) that caught it, I think the warning about falling off the end of a non-void-returning function.

BRevzin updated this revision to Diff 266071.May 25 2020, 1:27 PM
BRevzin marked 2 inline comments as done.
  • Explaining the cryptic parentheses.
llvm/include/llvm/ADT/DirectedGraph.h
99

Yeah I was surprised too. I'm compiling with -Wall -Wextra...

BRevzin updated this revision to Diff 290023.Sep 4 2020, 2:01 PM

Updating this review with some additional changes that need to be made since I last touched it, and some of the previous changes had inadvertently broken the C++14 build so fixing those as well.

Not that I have anything particularly against this, but won't this likely rot fairly rapidly? It's not like LLVM is even on C++17 let alone C++20 yet, so trying to make it work like the latter when it's just going to break again seems a bit like wasted effort to me.

llvm/tools/llvm-objdump/llvm-objdump.cpp
809โ€“821

This seems unrelated to comparison checking?

llvm/unittests/ADT/STLExtrasTest.cpp
474

Nit: trailing full stop.

Not that I have anything particularly against this, but won't this likely rot fairly rapidly? It's not like LLVM is even on C++17 let alone C++20 yet, so trying to make it work like the latter when it's just going to break again seems a bit like wasted effort to me.

People will want to write C++20 programs that use LLVM headers, so I think it's important to help let them do that. Sure, it may rot, but incremental fixes down the line will be smaller.

BRevzin added inline comments.Sep 7 2020, 8:17 AM
llvm/tools/llvm-objdump/llvm-objdump.cpp
809โ€“821

This seems unrelated to comparison checking?

It is unrelated. But In C++20, u8 literals become their own type so this no longer compiled and I wanted to ensure that I could actually run the tests.

Not that I have anything particularly against this, but won't this likely rot fairly rapidly? It's not like LLVM is even on C++17 let alone C++20 yet, so trying to make it work like the latter when it's just going to break again seems a bit like wasted effort to me.

People will want to write C++20 programs that use LLVM headers, so I think it's important to help let them do that. Sure, it may rot, but incremental fixes down the line will be smaller.

Makes sense, thanks.

llvm/tools/llvm-objdump/llvm-objdump.cpp
809โ€“821

Could it be a pre-requisite patch then?

martong removed a subscriber: martong.Sep 8 2020, 6:23 AM
jfb added a comment.Sep 8 2020, 10:26 AM

On C++20 mode rotting: it won't if someone sets up a bot. If it rots, then it's easier to un-rot with Barry's patch.

llvm/tools/llvm-objdump/llvm-objdump.cpp
809โ€“821

I'm fine with this if the patch title is changed to "make LLVM build in C++20 mode", and description edited accordingly. Basically, it makes it easy to figure out which changes were done for C++20.

BRevzin retitled this revision from Fixing all comparisons for C++20 compilation. to Make LLVM build in C++20 mode.Sep 8 2020, 11:17 AM
BRevzin edited the summary of this revision. (Show Details)
In D78938#2261411, @jfb wrote:

On C++20 mode rotting: it won't if someone sets up a bot. If it rots, then it's easier to un-rot with Barry's patch.

I assume this would be a private bot? It can't be a public bot, since LLVM isn't even on C++17, let alone C++20, and so it shouldn't be part of minimum requirements that somebody has a compiler that can build C++20. Whilst I personally am quite happy with moving LLVM forward, I develop on Windows primarily, so don't have the same need to support a long tail of old *nix versions etc.

@BRevzin, you should a) mention the u8/const char* issue in the description too, and also what compiler you used to build this with. I fully expect at this stage that there are some C++20 compilers that might have slightly different interpretations of things which this won't resolve, so knowing which one this is intended to work with could help with historical research.

@BRevzin, you should a) mention the u8/const char* issue in the description too, and also what compiler you used to build this with. I fully expect at this stage that there are some C++20 compilers that might have slightly different interpretations of things which this won't resolve, so knowing which one this is intended to work with could help with historical research.

It's mentioned in the description. I built with clang-10.

In D78938#2261411, @jfb wrote:

On C++20 mode rotting: it won't if someone sets up a bot. If it rots, then it's easier to un-rot with Barry's patch.

I assume this would be a private bot? It can't be a public bot, since LLVM isn't even on C++17, let alone C++20, and so it shouldn't be part of minimum requirements that somebody has a compiler that can build C++20. Whilst I personally am quite happy with moving LLVM forward, I develop on Windows primarily, so don't have the same need to support a long tail of old *nix versions etc.

I'd be fine with it being a public bot - it's not saying LLVM can only be compiled with C++20-supporting compilers, that'd be very different & that's the discussion we'll have when we want to start using C++20 in LLVM. But saying "LLVM is intended to be C++20 compatible" is something we can/shuold be saying much sooner than that. Like we say that LLVM's compatible with a certain variety of compilers in C++14 mode too - not everyone has or is testing on all those compilers every time they commit, but buildbots test a range of them (and test a range of hardware - again, hardware I don't have/don't intend to test with) & we clean things up that they report, ideally.

I'd think a C++20 buildbot could at least be relatively fast/easy (doesn't have to do multi-stage bootstraps (though it could - making sure the evolving C++20 support in Clang remains compatible with the LLVM project codebase itself)) - doesn't even need to run any tests, really, just compile.

llvm/include/llvm/DebugInfo/DWARF/DWARFExpression.h
170โ€“174

Why are some being removed? That seems harder to justify. Even if they're not called, it may be more valuable to have the symmetry to reduce friction if/when they are needed. (iterators seem pretty common to compare for inequality - such as in a loop condition testing I != E)

llvm/include/llvm/IR/BasicBlock.h
331โ€“332

What tripped over/required this SFINAE?

llvm/include/llvm/Support/BinaryStreamRef.h
124โ€“125

Be curious of the answer here - and, honestly, I'd be fine with changing them all to friends. It makes them more consistent - equal rank for implicit conversions on LHS and RHS, etc. (generally considered best practice basically to not define op overloads as members if they can be defined as non-members)

llvm/unittests/ADT/STLExtrasTest.cpp
474

Probably more suitable to use qualify the name rather than use parens (teh comment's still helpful to explain why either strategy is used) - that's what's done with llvm::make_unique, for instance.

BRevzin added inline comments.Sep 27 2020, 7:56 PM
llvm/include/llvm/DebugInfo/DWARF/DWARFExpression.h
170โ€“174

They're not being removed. These functions still exist - it's just that now they're being injected by the base class template with this exact signature (rather than before where they were slightly different), so that now these are redefinition issues.

There's no loss of functionality here.

llvm/include/llvm/IR/BasicBlock.h
331โ€“332

There's somewhere which compared a const iterator to a non-const iterator, that ends up doing conversions in both directions under C++20 rules, one direction of which is perfectly fine and the other was a hard error. Need to make the non-const iterator not constructible from a const iterator.

dblaikie added inline comments.Sep 28 2020, 1:59 PM
llvm/include/llvm/IR/BasicBlock.h
331โ€“332

Is this true for all iterators? Or some quirk of how this one is written/used (that could be fixed/changed there instead)?

Quuxplusone added inline comments.Sep 28 2020, 4:25 PM
llvm/include/llvm/IR/BasicBlock.h
331โ€“332

IMO there is a (much) bigger task hiding here, which is to audit every type in the codebase whose name contains the string "Iterator" and compare them to the C++20 Ranges std::forward_iterator concept. My impression is that the vast majority of real-world "iterator types" are not iterators according to C++20 Ranges, and that this can have arbitrarily weird effects when you mix them with the C++20 STL.

However, that is massive scope creep re this particular patch. I think the larger question of "do all our iterators need X / are all our iterators written wrong" should be scoped-outside-of this patch.

dblaikie added inline comments.Sep 28 2020, 4:40 PM
llvm/include/llvm/IR/BasicBlock.h
331โ€“332

Sorry, not suggesting that kind of scope creep - but do want to understand whether this is representative of the way code should generally be written, or whether this is working around some other issue/different fix.

BRevzin added inline comments.Sep 29 2020, 7:08 AM
llvm/include/llvm/IR/BasicBlock.h
331โ€“332

So I undid this change to copy the exact issue that I ran into. But it actually ended up still compiling anyway. Part of the issue might be that I keep futzing with the cmake configuration since it takes me more than an hour to compile, so maybe there's some target that needed this change that I no longer compile.

But the kind of problem I think this had was:

template <typename T>
struct iterator {
    T* p;
    
    template <typename U>
    iterator(iterator<U> rhs)
        : p(rhs.p)
    { } 

    bool operator==(iterator const& rhs);
};

bool check(iterator<int const> a, iterator<int> b) {
    return a == b;
}

which compiles fine in C++17 but is ambiguous in C++20 because b.operator==(a) is also a candidate (even though it's not _really_ a candidate, and would be a hard error if selected). the sfinae removes the bad candidate from the set.

It's true for all iterators in general in that you want const_iterator constructible from iterator but not the reverse (unless they're the same type).

lebedev.ri added inline comments.
llvm/include/llvm/DebugInfo/DWARF/DWARFExpression.h
170โ€“174

Does LLVM still build fine in C++14/C++17 modes afterwards?

BRevzin added inline comments.Sep 29 2020, 7:19 AM
llvm/include/llvm/DebugInfo/DWARF/DWARFExpression.h
170โ€“174

Yes.

dblaikie added inline comments.Sep 29 2020, 8:45 AM
llvm/include/llvm/IR/BasicBlock.h
331โ€“332

Fair enough - don't mind keeping it in then.

llvm/include/llvm/Support/BinaryStreamRef.h
124โ€“125

Ping on this (& I'd usually call the parameters LHS and RHS rather than Self and Other)

llvm/unittests/ADT/STLExtrasTest.cpp
474

Ping on this.

Thanks @lebedev.ri for the pointer!
I started working on exactly the same thing as I was trying to link a C++20 project with LLVM.
@BRevzin is there anything missing in this patch? Do you have commit access or do you need help to land this?

@BRevzin Please share your name and email if you want someone to commit it for you https://llvm.org/docs/DeveloperPolicy.html#commit-messages

Thanks @lebedev.ri for the pointer!
I started working on exactly the same thing as I was trying to link a C++20 project with LLVM.
@BRevzin is there anything missing in this patch? Do you have commit access or do you need help to land this?

There are two comments that @dblaikie made that need to still be addressed (one about renaming parameters of a function from Self and Other to LHS and RHS, and one about changing the ADL-inhibition strategy from (to_address)(x) to llvm::to_address(x)), both of which combined should take about a minute to do and then however long to compile.

I've forgotten how to push changes and am kind of confused at the state of my current branch anyway, and it takes so long to do anything on my laptop that I'm more than happy to let you take over. I do not have commit access anyway.

Per @MaskRay, I'm Barry Revzin <barry.revzin@gmail.com>.

This revision was landed with ongoing or failed builds.Dec 17 2020, 2:45 AM
Closed by commit rG92310454bf0f: Make LLVM build in C++20 mode (authored by BRevzin, committed by nlopes). ยท Explain Why
This revision was automatically updated to reflect the committed changes.

@BRevzin @nlopes This is causing MSVC build failure please can you take a look?

E:\llvm\llvm-project\llvm\include\llvm/DebugInfo/DWARF/DWARFDie.h(405): note: see declaration of 'std::reverse_iterator<llvm::DWARFDie::iterator>'
E:\llvm\llvm-project\llvm\lib\DWARFLinker\DWARFLinker.cpp(383): note: see reference to function template instantiation 'bool std::operator !=<llvm::DWARFDie::iterator,llvm::DWARFDie::iterator>(const std::reverse_iterator<llvm::DWARFDie::iterator> &,const std::reverse_iterator<llvm::DWARFDie::iterator> &)' being compiled
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.28.29333\include\xutility(2086): error C2039: '_Get_current': is not a member of 'std::reverse_iterator<llvm::DWARFDie::iterator>'
E:\llvm\llvm-project\llvm\include\llvm/DebugInfo/DWARF/DWARFDie.h(405): note: see declaration of 'std::reverse_iterator<llvm::DWARFDie::iterator>'

@BRevzin @nlopes This is causing MSVC build failure please can you take a look?

E:\llvm\llvm-project\llvm\include\llvm/DebugInfo/DWARF/DWARFDie.h(405): note: see declaration of 'std::reverse_iterator<llvm::DWARFDie::iterator>'
E:\llvm\llvm-project\llvm\lib\DWARFLinker\DWARFLinker.cpp(383): note: see reference to function template instantiation 'bool std::operator !=<llvm::DWARFDie::iterator,llvm::DWARFDie::iterator>(const std::reverse_iterator<llvm::DWARFDie::iterator> &,const std::reverse_iterator<llvm::DWARFDie::iterator> &)' being compiled
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.28.29333\include\xutility(2086): error C2039: '_Get_current': is not a member of 'std::reverse_iterator<llvm::DWARFDie::iterator>'
E:\llvm\llvm-project\llvm\include\llvm/DebugInfo/DWARF/DWARFDie.h(405): note: see declaration of 'std::reverse_iterator<llvm::DWARFDie::iterator>'

Just saw that you fixed it already. Thank you!

Just saw that you fixed it already. Thank you!

'Avoided' might be a better term than 'fixed' tbh - I didn't delve much into why it was breaking, or what effect it has on C++20 work.

bmahjour added inline comments.
llvm/include/llvm/ADT/DirectedGraph.h
40โ€“41

It would be better to make these non-member functions as well, to be consistent with the DGNode.

friend bool operator==(const EdgeType &Elhs, const EdgeType &Erhs) {
  return Elhs.isEqualTo(Erhs);
}
friend bool operator!=(const EdgeType &Elhs, const EdgeType &Erhs) {
  return !(Elhs == Erhs);
}
40โ€“41

That comment, so informative! ๐Ÿ˜

Yeah, it's not the best comment. It is trying to say that the isEqualTo function gets selected, based on the type of the derived class in the CRTP, and that the selection is done at compile-time using type information, rather than at runtime using dynamic polymorphism. Please feel free to update it to say that or offer any other suggestions you might have.