[libc++] Re-enable std::pair trivial copy constructor for FreeBSD >= 14

[ldionne]

Summary

After many years of using the really old std::pair ABI which did not yet
have a trivial copy constructor, FreeBSD 14 and later will finally get
rid of it. Only use the old ABI for FreeBSD 13 and earlier.

Note: on the FreeBSD side, we will bump our libc++.so version for this,
and keep an old compatibility library in a separate package.

[philnik]

LGTM. For how many versions do we support older targets?

[emaste]

For reference, we're going to support FreeBSD 13 until Jan 2026.
https://www.freebsd.org/security/

[emaste]

Sounds good to me. We can bump the .so version in the base system, but how do we treat it upstream (i.e., here in llvm) and in the ports tree?

[philnik]

In D126462#3539933, @emaste wrote:

Sounds good to me. We can bump the .so version in the base system, but how do we treat it upstream (i.e., here in llvm) and in the ports tree?

Would it make sense from your side to to declare ABIv2 stable? It sounds to me like you break the ABI between FreeBSD 13 and 14, right? That would remove the problem of differing versions between upstream and the system version. No guarantees that we actually do this, but I would be in favour of setting such a precedent. (And we would probably need a few months to get everything going.)

[dim]

In D126462#3540371, @philnik wrote:

In D126462#3539933, @emaste wrote:

Sounds good to me. We can bump the .so version in the base system, but how do we treat it upstream (i.e., here in llvm) and in the ports tree?

Would it make sense from your side to to declare ABIv2 stable?

I had been sort of waiting for this, but after so many years I failed to hold my breath. :-) Because that would've solved our problem with the old std::pair ABI, obviously.

It sounds to me like you break the ABI between FreeBSD 13 and 14, right? That would remove the problem of differing versions between upstream and the system version. No guarantees that we actually do this, but I would be in favour of setting such a precedent. (And we would probably need a few months to get everything going.)

Well, recently rG0d8cb8b399adcd17e8bf17be7814d030308c8b82 broke builds on FreeBSD, precisely because of this old std::pair ABI, so that is mostly what triggered this. I remember there were some other programs (chromium or googletest, probably) that also had build issues, because they were silently assuming std::pair was trivially copyable.

Since FreeBSD attempts to keep ABIs stable in released versions, for our 12.x and 13.x branches there isn't any good way forward, except keeping the old ABI forever, until the branches reach EOL. But for 14.0, which is our trunk/main/head, there is no such ABI promise, therefore we can simply bump libc++.so to .2 now and fix the issue that way. If the upstream libc++ ABI version gets bumped to 2 though, we'll have to bump our .so to .3 I'm afraid, since the ELF soversion doesn't really allow "half-sized" version numbers.

That said, I'm unsure how many other downstream consumers are waiting for the upstream libc++ ABI to change? I haven't been following exactly what new features or incompatibilities have been covered by ABIv2; are there a lot of those?

[philnik]

Since FreeBSD attempts to keep ABIs stable in released versions, for our 12.x and 13.x branches there isn't any good way forward, except keeping the old ABI forever, until the branches reach EOL.

That was my guess. We have to keep ABIv1 for obvious reasons.

But for 14.0, which is our trunk/main/head, there is no such ABI promise, therefore we can simply bump libc++.so to .2 now and fix the issue that way. If the upstream libc++ ABI version gets bumped to 2 though, we'll have to bump our .so to .3 I'm afraid, since the ELF soversion doesn't really allow "half-sized" version numbers.

Do you have to bump the version now or would it be possible for you to wait maybe one or two months so we can get things sorted out upstream?

That said, I'm unsure how many other downstream consumers are waiting for the upstream libc++ ABI to change?

I mean, we have to do it at some point or never and someone has to start if we want to do it at some point.

I haven't been following exactly what new features or incompatibilities have been covered by ABIv2; are there a lot of those?

https://github.com/llvm/llvm-project/blob/0be0a53df65cb402359c257922d80ab93d86fb40/libcxx/include/__config#L59-L118 is the list of current ABI-breaking changes.

@ldionne I think this requires your input. #libc_vendors is anybody else interested in a stable ABIv2?

[dim]

In D126462#3540480, @philnik wrote:

...

But for 14.0, which is our trunk/main/head, there is no such ABI promise, therefore we can simply bump libc++.so to .2 now and fix the issue that way. If the upstream libc++ ABI version gets bumped to 2 though, we'll have to bump our .so to .3 I'm afraid, since the ELF soversion doesn't really allow "half-sized" version numbers.

Do you have to bump the version now or would it be possible for you to wait maybe one or two months so we can get things sorted out upstream?

We've been carrying the old ABI for a very long time, so in my opinion it should not be too much of a problem to wait a little while longer. That would still entail some work on our side for providing the old .so.1 compat library and such, but bumping only once instead of twice is less disruptive, of course.

However, I am not completely sure how far off FreeBSD 14.0's "freeze" is, e.g. when a stable/14 branch is created, and the main branch is bumped to 15.0. Since FreeBSD 13.1-RELEASE is just out the door, there might still be enough time. @emaste you might be more aware of any proposed FreeBSD release schedule?

https://github.com/llvm/llvm-project/blob/0be0a53df65cb402359c257922d80ab93d86fb40/libcxx/include/__config#L59-L118 is the list of current ABI-breaking changes.

Well, I find that list quite long and impressive already! But whether it is enough for an ABI bump is up to the community of course.

[emaste]

However, I am not completely sure how far off FreeBSD 14.0's "freeze" is, e.g. when a stable/14 branch is created, and the main branch is bumped to 15.0.

There's no schedule set yet, but based on historical practice I would expect it at the end of 2023. I think it's no problem if ABIv2 happens within two or three months.

[dim]

In any case it would be nice if it was possible to decide whether ABIv2 will be pinned down (and ABIv3 is made into the 'new' unstable) before 15.0 is branched. :-)

[ldionne]

The change itself LGTM.

Stabilizing ABI v2 raises some interesting questions. For example, if we stabilize ABI v2, does it mean that we consider ABI v1 as "deprecated"? Do we start doing all development on ABI v2 by default and keep around one (or several) CI jobs to continue testing ABI v1? Do we start building ABI v2 by default when someone configures libc++? What ABI version do we ship in the LLVM release?

Concretely, if we deprecate ABI v1, we'll put everyone who require ABI stability in a situation where they are using a deprecated thing. Unless there is a rather serious effort from those folks (I include myself here) to move off of ABI v1, there is basically no timeline for moving off, which means that we'd have introduced a deprecation situation with no real horizon for getting out of it. So far, our ABI flags have been treated as just knobs -- basically you cherry-pick the ABI changes you want when you introduce a new target, and that's what defines your ABI on that target. You then keep the same ABI until that target is end-of-lifed, which may often be not-in-the-foreseeable-future.

So while I do support migrating to a newer ABI in principle, I think the discussion needs to happen a bit more widely in the LLVM community. I think it should be a common library + compiler effort, and we should design some sort of plan for how we're going to push vendors over to the new ABI over a course of X years, and then remove support for the old ABI. Note that X can be a large number of years, but it has to be finite, otherwise we'll just create problems for ourselves by adding one axis in the configuration space that we need to support in libc++.

I'm interested to hear what others think about this.

[Mordante]

In D126462#3555878, @ldionne wrote:

The change itself LGTM.

Stabilizing ABI v2 raises some interesting questions. For example, if we stabilize ABI v2, does it mean that we consider ABI v1 as "deprecated"? Do we start doing all development on ABI v2 by default and keep around one (or several) CI jobs to continue testing ABI v1? Do we start building ABI v2 by default when someone configures libc++? What ABI version do we ship in the LLVM release?

I think we can ship different dylibs using different ABI versions making the transition easier.

Concretely, if we deprecate ABI v1, we'll put everyone who require ABI stability in a situation where they are using a deprecated thing. Unless there is a rather serious effort from those folks (I include myself here) to move off of ABI v1, there is basically no timeline for moving off, which means that we'd have introduced a deprecation situation with no real horizon for getting out of it. So far, our ABI flags have been treated as just knobs -- basically you cherry-pick the ABI changes you want when you introduce a new target, and that's what defines your ABI on that target. You then keep the same ABI until that target is end-of-lifed, which may often be not-in-the-foreseeable-future.

So while I do support migrating to a newer ABI in principle, I think the discussion needs to happen a bit more widely in the LLVM community. I think it should be a common library + compiler effort, and we should design some sort of plan for how we're going to push vendors over to the new ABI over a course of X years, and then remove support for the old ABI. Note that X can be a large number of years, but it has to be finite, otherwise we'll just create problems for ourselves by adding one axis in the configuration space that we need to support in libc++.

I'm interested to hear what others think about this.

In general I like to get a stable ABI v2 since some bug-fixes require ABI v2, e.g _LIBCPP_ABI_REGEX_CONSTANTS_NONZERO.
Another reason to switch to ABI v2 would be to make some of our optimizations available for a larger audience.
However in my experience such an ABI change is quite disruptive, even on open source platforms. libstdc++5.0 changed
their ABI to avoid COW in std::string (which was needed to support C++11). I recall this gave quite some issues when it
happened. In fact I even had the joy to re-experience these issues last year. I expect this to be even worse for platforms
that use more close-source software; thus I expect this to be a larger issue for Apple.

I think it's good to start the discussion whether or not we should stabilize ABI v2, maybe discord is the better place for that discussion

[philnik]

I think it would be a good idea to couple the ABI version to the C++ version. i.e. if -std=c++20 use ABIv1 by default (and allow people via a compiler flag to use ABIv2) and if -std=c++2b use ABIv2 (and have no way to use ABIv1). That would force people to use the new ABI as they update to a newer standard. This would mean that an ABI is never really deprecated. Old C++ version would get deprecated and removed. I think a good number would be 5 C++ releases. That would mean that C++03 should have been deprecated with the release of C++20 and C++11 should be deprecated with the release of C++23. After deprecating the oldest supported version for one C++ release it should be removed (i.e. C++03 will be removed after C++23 has been released and so on).

Do we start doing all development on ABI v2 by default and keep around one (or several) CI jobs to continue testing ABI v1?

I think it would make sense to use the default ABI version. i.e. Use ABIv1 For C++03 to C++20 and use ABIv2 for C++23.

Do we start building ABI v2 by default when someone configures libc++? What ABI version do we ship in the LLVM release?

I don't know how complicated it is, but I would guess all currently supported ABI version should be built and shipped.

Concretely, if we deprecate ABI v1, we'll put everyone who require ABI stability in a situation where they are using a deprecated thing. Unless there is a rather serious effort from those folks (I include myself here) to move off of ABI v1, there is basically no timeline for moving off, which means that we'd have introduced a deprecation situation with no real horizon for getting out of it. So far, our ABI flags have been treated as just knobs -- basically you cherry-pick the ABI changes you want when you introduce a new target, and that's what defines your ABI on that target. You then keep the same ABI until that target is end-of-lifed, which may often be not-in-the-foreseeable-future.

With 4 C++ releases + 1 deprecated makes 15 years time for switching to the new ABI. I think that is enough time.

So while I do support migrating to a newer ABI in principle, I think the discussion needs to happen a bit more widely in the LLVM community.

Should that kind of discussion be on Discourse?

I think it should be a common library + compiler effort, and we should design some sort of plan for how we're going to push vendors over to the new ABI over a course of X years, and then remove support for the old ABI. Note that X can be a large number of years, but it has to be finite, otherwise we'll just create problems for ourselves by adding one axis in the configuration space that we need to support in libc++.

I'm interested to hear what others think about this.

[ldionne]

[Github PR transition cleanup]

It looks like this was never committed, but we agreed the change itself was good. I'll commandeer to land this now. Heads up @dim.

libcxx/include/__config

 // conflict with the dllexport-emitted copy, so we disable it. For XCOFF,
 // the linker will take issue with the symbols in the shared object if the
 // weak inline methods get visibility (such as from -fvisibility-inlines-hidden),
 // so disable it.
 #      define _LIBCPP_DEPRECATED_ABI_LEGACY_LIBRARY_DEFINITIONS_FOR_INLINE_FUNCTIONS
 #    endif
 // Feature macros for disabling pre ABI v1 features. All of these options
 // are deprecated.
-#    if defined(__FreeBSD__)
+#    if defined(__FreeBSD__) && __FreeBSD__ < 14
 #      define _LIBCPP_DEPRECATED_ABI_DISABLE_PAIR_TRIVIAL_COPY_CTOR
 #    endif
 // For XCOFF linkers, we have problems if we see a weak hidden version of a symbol
 // in user code (like you get with -fvisibility-inlines-hidden) and then a strong def
 // in the library, so we need to always rely on the library version.
 #    if defined(_AIX)
 #      define _LIBCPP_ABI_BAD_FUNCTION_CALL_KEY_FUNCTION
 #    endif