[libc++] Remove default definition of std::char_traits

[ldionne]

Summary

This patch removes the base template implementation for std::char_traits.
If my reading of http://eel.is/c++draft/char.traits is correct, the
Standard mandates that the library provides specializations for several
types like char and wchar_t, but not any implementation in the base
template. Indeed, such an implementation is bound to be incorrect for
most types anyways, since things like eof() and int_type will definitely
have to be customized.

Since the base template implementation should not have worked for anyone,
this shouldn't be a breaking change (I expect that anyone defining a
custom character type today will already have to provide their own
specialization of char_traits). However, since we're aware of some users
of char_traits for unsigned char and signed char, we're keeping those two
specializations around for two releases to give people some time to migrate.

[philnik]

libcxx/include/__string/char_traits.h

42–47

We probably shouldn't mark it with a visibility macro if we never define it. If people care about it they can add their own visibility macros to any specializations.

55

These don't have to be marked constexpr or noexcept for user-provided ones. It's probably also just noise for most people.

[ldionne]

Fix CI

[ldionne]

Fix issues with wchar_t

[philnik]

LGTM with green CI.

libcxx/test/std/strings/basic.string/string.modifiers/string_append/push_back.pass.cpp

27

I'd rather avoid opening the std namespace and instead use template<> struct std::char_traits<VeryLarge>.

[ldionne]

libcxx/include/__string/char_traits.h

55

I think you're right.

libcxx/test/std/strings/basic.string/string.modifiers/string_append/push_back.pass.cpp

27

I don't think that's valid in C++03. GCC gives an error and requires using -fpermissive.

[philnik]

libcxx/test/std/strings/basic.string/string.modifiers/string_append/push_back.pass.cpp

27

Huh, interesting. Weird that clang doesn't even give you a warning for that.

[tahonermann]

I'm concerned that this change will break "working" code. While the standard does not specify specializations for signed char and unsigned char, it isn't hard to find cases of people using basic_string<unsigned char>. I didn't look hard, but it looks like Emscripten would be broken by this change based on https://github.com/emscripten-core/emscripten/issues/7362.

All major compilers appear to "support" specializations for signed char and unsigned char: https://godbolt.org/z/h3xG7T48T. There is an old paper that argues for standardizing such specializations: https://wg21.link/n1985.

[ldionne]

In D138307#3941734, @tahonermann wrote:

I'm concerned that this change will break "working" code. While the standard does not specify specializations for signed char and unsigned char, it isn't hard to find cases of people using basic_string<unsigned char>. I didn't look hard, but it looks like Emscripten would be broken by this change based on https://github.com/emscripten-core/emscripten/issues/7362.

All major compilers appear to "support" specializations for signed char and unsigned char: https://godbolt.org/z/h3xG7T48T. There is an old paper that argues for standardizing such specializations: https://wg21.link/n1985.

I think a common ground here could be to support signed char and unsigned char explicitly as extensions. But do we agree that std::string<int> makes no sense and if it breaks some users that's fine?

Also, would you have an appetite for reviving https://wg21.link/n1985?

[tahonermann]

I think a common ground here could be to support signed char and unsigned char explicitly as extensions. But do we agree that std::string<int> makes no sense and if it breaks some users that's fine?

Yes, I'm not worried about other fundamental types; they are much more problematic than signed char and unsigned char for the reasons you indicated.

Also, would you have an appetite for reviving https://wg21.link/n1985?

Not really. I could bring it to SG16, but I don't expect it to garner much favor. It might get some points for standardizing existing practice, but I expect the SG16 response to be that we don't want to encourage use of such extensions.

[ldionne]

In D138307#3941791, @tahonermann wrote:

Also, would you have an appetite for reviving https://wg21.link/n1985?

Not really. I could bring it to SG16, but I don't expect it to garner much favor. It might get some points for standardizing existing practice, but I expect the SG16 response to be that we don't want to encourage use of such extensions.

In that case, how about this:

1. We rip out char_traits support for int and other nonsensical stuff
2. We keep supporting it for unsigned char and signed char, but we make it deprecated
3. We remove it in 1-2 releases

If there's no desire to standardize it because the Committee thinks it's not a good idea, we should get clients to move away from it. We don't want to make it painful just for the sake of it, but we should initiate the movement.

[ldionne]

Rebase onto main with some changes extracted into separate patches. Will update with signed char and unsigned char once we have agreed on a direction.

[tahonermann]

If there's no desire to standardize it because the Committee thinks it's not a good idea, we should get clients to move away from it. We don't want to make it painful just for the sake of it, but we should initiate the movement.

Agreed; the plan you proposed sounds good to me.

[ldionne]

Implement the agreed-upon direction

[tahonermann]

libcxx/test/std/ranges/range.adaptors/range.lazy.split/general.pass.cpp

36–43

Is the intent really to support conversions from ranges of different character types? That seems odd and doesn't seem to be required by the test.

[ldionne]

libcxx/test/std/ranges/range.adaptors/range.lazy.split/general.pass.cpp

36–43

No, you're right. Thanks for the catch, I'll fix that.

The intent is to be constructible from a string_view (specifically we need to be constructible from a null-terminated C-style string) whenever Char is one of these types, and my change here was a really sloppy way of doing that.

[ldionne]

Address comment.

[tahonermann]

Looks good!

[ldionne]

Fix CI and rebase

[jyknight]

This breaks Apache Arrow, which makes a basic_string_view<unsigned long>, here:
https://github.com/apache/arrow/blob/91ee6dad722ee154d63eea86ce5644e1e658b53b/cpp/src/arrow/util/bitmap.h#L53

/usr/include/c++/v1/string_view:279:45: error: implicit instantiation of undefined template 'std::char_traits<unsigned long>'
    static_assert((is_same<_CharT, typename traits_type::char_type>::value),
                                            ^
arrow/util/bitmap.h:153:16: note: in instantiation of template class 'std::basic_string_view<unsigned long>' requested here
    View<Word> words[N];
               ^
arrow/compute/kernels/scalar_if_else.cc:1595:15: note: in instantiation of function template specialization 'arrow::internal::Bitmap::VisitWords<3UL, (lambda at arrow/compute/kernels/scalar_if_else.cc:1595:35), unsigned long>' requested here
      Bitmap::VisitWords(bitmaps, [&](std::array<uint64_t, 3> words) {
              ^
arrow/compute/kernels/scalar_if_else.cc:1652:14: note: in instantiation of function template specialization 'arrow::compute::internal::(anonymous namespace)::ExecArrayCaseWhen<arrow::BooleanType>' requested here
      return ExecArrayCaseWhen<Type>(ctx, batch, out);
             ^
arrow/compute/kernels/codegen_internal.h:1173:47: note: in instantiation of member function 'arrow::compute::internal::(anonymous namespace)::CaseWhenFunctor<arrow::BooleanType>::Exec' requested here
      return Generator<BooleanType, Args...>::Exec;
                                              ^
arrow/compute/kernels/scalar_if_else.cc:2738:17: note: in instantiation of function template specialization 'arrow::compute::internal::GenerateTypeAgnosticPrimitive<arrow::compute::internal::(anonymous namespace)::CaseWhenFunctor>' requested here
    auto exec = GenerateTypeAgnosticPrimitive<CaseWhenFunctor>(*type);
                ^
/usr/include/c++/v1/__string/char_traits.h:42:8: note: template is declared here
struct char_traits;

Also breaks v8, which uses a std::basic_string<uint16_t> member, https://github.com/v8/v8/blob/0afe3f519821449f00c746df3747543d3458bb68/src/inspector/string-16.h#L113

[ldionne]

In D138307#3946938, @jyknight wrote:

This breaks Apache Arrow, which makes a basic_string_view<unsigned long>, here:
https://github.com/apache/arrow/blob/91ee6dad722ee154d63eea86ce5644e1e658b53b/cpp/src/arrow/util/bitmap.h#L53

[...]

Also breaks v8, which uses a std::basic_string<uint16_t> member, https://github.com/v8/v8/blob/0afe3f519821449f00c746df3747543d3458bb68/src/inspector/string-16.h#L113

That is intended, they should fix their usage of std::string. If they really want to use std::string for these "character types", then they can define struct MyChar { unsigned long c_; }; and then specialize std::char_traits<MyChar>.

[ldionne]

In D138307#3946939, @ldionne wrote:

In D138307#3946938, @jyknight wrote:

This breaks Apache Arrow, which makes a basic_string_view<unsigned long>, here:
https://github.com/apache/arrow/blob/91ee6dad722ee154d63eea86ce5644e1e658b53b/cpp/src/arrow/util/bitmap.h#L53

[...]

Also breaks v8, which uses a std::basic_string<uint16_t> member, https://github.com/v8/v8/blob/0afe3f519821449f00c746df3747543d3458bb68/src/inspector/string-16.h#L113

That is intended, they should fix their usage of std::string. If they really want to use std::string for these "character types", then they can define struct MyChar { unsigned long c_; }; and then specialize std::char_traits<MyChar>.

One thing we could do is instead deprecate the whole base template (but still provide an implementation) so as to give folks two releases to get off of it. I'm pretty skeptical about std::string working as intended for unsigned long, but whatever. @tahonermann What would you think about that?

[tahonermann]

This breaks Apache Arrow, which makes a basic_string_view<unsigned long>

Hmm, this usage is exposed in the interface (see the words() member function template of the Bitmap class template). Ick. Are there tests that demonstrate that the code actually works as intended when Word is not one of the standard character types?

Also breaks v8, which uses a std::basic_string<uint16_t> member

At least in this case, the use is an implementation detail. It looks to me like this class could be relatively easily changed to use std::vector instead. Additional code would be required to ensure null termination, but it at least looks like there are no obvious dependencies on char_traits.

One thing we could do is instead deprecate the whole base template (but still provide an implementation) so as to give folks two releases to get off of it. I'm pretty skeptical about std::string working as intended for unsigned long, but whatever.

That seems prudent. I think it is likely that we'll see additional reports of breakage.

This is something we should communicate to the community via the Announcements (https://discourse.llvm.org/c/announce/46) channel following the conclusion of discussion at https://discourse.llvm.org/t/rfc-add-new-discourse-channel-for-potentially-breaking-disruptive-changes-for-clang/65251.

[gulfem]

We also started seeing errors in our Fuchsia source code.

[91666/283659](64) CXX obj/src/lib/elfldltl/test/note-fuzzer.note-fuzzer.cc.o
FAILED: obj/src/lib/elfldltl/test/note-fuzzer.note-fuzzer.cc.o
../../../recipe_cleanup/clangyvv4ambu/bin/clang++ -MD -MF obj/src/lib/elfldltl/test/note-fuzzer.note-fuzzer.cc.o.d -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES -DND...
In file included from ../../src/lib/elfldltl/test/note-fuzzer.cc:5:
In file included from ../../src/lib/elfldltl/include/lib/elfldltl/fuzzer.h:12:
In file included from ../../src/lib/elfldltl/include/lib/elfldltl/layout.h:11:
../../../recipe_cleanup/clangyvv4ambu/bin/../include/c++/v1/string_view:279:45: error: implicit instantiation of undefined template 'std::char_traits<std::byte>'
    static_assert((is_same<_CharT, typename traits_type::char_type>::value),
                                            ^
../../src/lib/elfldltl/include/lib/elfldltl/note.h:97:9: note: in instantiation of template class 'std::basic_string_view<std::byte>' requested here
  Bytes desc;
        ^
../../../recipe_cleanup/clangyvv4ambu/bin/../include/c++/v1/__fwd/string.h:22:29: note: template is declared here
struct _LIBCPP_TEMPLATE_VIS char_traits;

https://luci-milo.appspot.com/ui/p/fuchsia/builders/ci/clang_toolchain.ci.core.x64-release/b8796686452214900177/overview

[rnk]

Would it be reasonable to retain the default implementation of char_traits under a __config_site macro? This will provide downstream projects with a migration path that is not tightly coupled to the libc++ version. They can update libc++ and complete the migration as separate steps. The migration path can be removed over a longer time horizon, like after the next release.

In our experience at Google, libc++ has recently made frequent breaking changes which prevent us from uprevving libc++, and oftentimes by extension, the entire LLVM toolchain. We are definitely committed and willing to rewrite code to conform to the standard, but the current pace of breakage and lack of migration paths is causing us problems, and I worry that we are just the canary hitting these issues first. I think other users will experience them later when it comes time to release.

I aspired to start a large discussion about appropriate project policies on breaking changes, but I never got around to it, and it looks like we've discovered the next breaking change before I was able to kick off an RFC. I will also be going on leave soon until May, so it's not likely that I'll be able to follow up, but I wanted to share my thoughts.

[ldionne]

In D138307#3947433, @rnk wrote:

Would it be reasonable to retain the default implementation of char_traits under a __config_site macro? This will provide downstream projects with a migration path that is not tightly coupled to the libc++ version. They can update libc++ and complete the migration as separate steps. The migration path can be removed over a longer time horizon, like after the next release.

I think the currently proposed approach in D138596 handles that -- you basically get a deprecation warning that can be turned off via the usual deprecation-removal mechanism, and then roughly one year from now, this becomes a hard error. I think this is very close to what you mention, unless I am misunderstanding something. Please let me know if D138596 doesn't solve your problem.

In our experience at Google, libc++ has recently made frequent breaking changes which prevent us from uprevving libc++, and oftentimes by extension, the entire LLVM toolchain. We are definitely committed and willing to rewrite code to conform to the standard, but the current pace of breakage and lack of migration paths is causing us problems, and I worry that we are just the canary hitting these issues first. I think other users will experience them later when it comes time to release.

FWIW, I completely acknowledge that we've been breaking more stuff recently. However, to provide some context, I would argue this is because we've never been making as much progress on our standards conformance, and also we've adopted a more aggressive stance with respect to catching invalid user code. Breakage induced by standards conformance is, I would argue, very difficult to avoid. Other breakage (such as this patch) is much easier to do something about.

I think another important thing to note here is that the project was not as active for some years, so folks perhaps got used to ultimate stability. Anyway, I'm all for providing reasonable migration paths when it's reasonable to do so, but the meaning of reasonable is what I expect we may have challenges agreeing on (not just you and me, but the overall LLVM/libc++ community and its downstream users).

[rnk]

In D138307#3947486, @ldionne wrote:

I think the currently proposed approach in D138596 handles that -- you basically get a deprecation warning that can be turned off via the usual deprecation-removal mechanism, and then roughly one year from now, this becomes a hard error. I think this is very close to what you mention, unless I am misunderstanding something. Please let me know if D138596 doesn't solve your problem.

Thanks, that sounds good to me.

I think another important thing to note here is that the project was not as active for some years, so folks perhaps got used to ultimate stability. Anyway, I'm all for providing reasonable migration paths when it's reasonable to do so, but the meaning of reasonable is what I expect we may have challenges agreeing on (not just you and me, but the overall LLVM/libc++ community and its downstream users).

Sounds like we're on the same page. I agree, the definition of "reasonable" is subjective and hard to know ahead of time, and I don't expect everyone to always agree. I'm satisfied as long as folks are willing to compromise from time to time and work together.

[mcgrathr]

This broke some existing cases using std::basic_string_view<std::byte> and the like.

[ldionne]

In D138307#3947514, @rnk wrote:

Sounds like we're on the same page. I agree, the definition of "reasonable" is subjective and hard to know ahead of time, and I don't expect everyone to always agree. I'm satisfied as long as folks are willing to compromise from time to time and work together.

Yes, absolutely. We really appreciate this kind of feedback from downstreams since it allows us to catch things that would be painful for people when migrating, and you can be certain that we are willing to compromise within reason to give folks a good user experience. Of course, we also have to take into account multiple factors, so sometimes it may not be possible to compromise as much as downstreams would like, but I think success lies in the existence of a dialogue like we're having here. For example, I think that the current migration path (where we keep all specializations of std::char_traits for some time) is a better choice than what we started with. It's simpler for the library and it provides a better user experience, so everyone wins from that.

In D138307#3947516, @mcgrathr wrote:

This broke some existing cases using std::basic_string_view<std::byte> and the like.

Yes, please see the above discussion. This will be restored for the time being with a deprecation warning and then removed in an upcoming release (currently planning for LLVM 18).

[jyknight]

Leaving an xref trail here, I filed issues against the two projects I reported above:

• Apache Arrow: https://github.com/apache/arrow/issues/15281
• v8: https://bugs.chromium.org/p/v8/issues/detail?id=13648

libcxx/docs/ReleaseNotes.rst

 
 - The contents of ``<experimental/memory_resource>`` are now deprecated since libc++ ships ``<memory_resource>`` now.
   Please migrate to ``<memory_resource>`` instead. Per libc++'s TS deprecation policy,
   ``<experimental/memory_resource>`` will be removed in LLVM 18.
 
 - The ``_LIBCPP_DEBUG`` macro is not honored anymore, and it is an error to try to use it. Please migrate to
   ``_LIBCPP_ENABLE_DEBUG_MODE`` instead.
 
+- A base template for ``std::char_traits`` is not provided anymore. The Standard mandates that the library
+  provides specializations for several types like ``char`` and ``wchar_t``, which libc++ does. However, we
+  also additionally provided a default implementation for ``std::char_traits<T>`` for arbitrary ``T``. Not
+  only does the Standard not mandate that one is provided, but such an implementation is bound to be incorrect
+  for some types.
+
 Upcoming Deprecations and Removals
 ----------------------------------
 
 API Changes
 -----------
 - The comparison operators on ``thread::id`` are now defined as free-standing
   functions instead of as hidden friends, in conformance with the C++ standard.
   Also see `issue 56187 <https://github.com/llvm/llvm-project/issues/56187>`_.

libcxx/include/__string/char_traits.h

 #endif
 
 _LIBCPP_PUSH_MACROS
 #include <__undef_macros>
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 template <class _CharT>
-struct _LIBCPP_TEMPLATE_VIS char_traits
+struct char_traits;
+/*
+The Standard does not define the base template for char_traits because it is impossible to provide
+a correct definition for arbitrary character types. Instead, it requires implementations to provide
+specializations for predefined character types like `char`, `wchar_t` and others. We provide this as
+exposition-only to document what members a char_traits specialization should provide:
 {
     using char_type  = _CharT;
-    using int_type   = int;
-    using off_type   = streamoff;
-    using pos_type   = streampos;
-    using state_type = mbstate_t;
-
-    static inline void _LIBCPP_CONSTEXPR_SINCE_CXX17
-        assign(char_type& __c1, const char_type& __c2) _NOEXCEPT {__c1 = __c2;}
-    static inline _LIBCPP_CONSTEXPR bool eq(char_type __c1, char_type __c2) _NOEXCEPT
-        {return __c1 == __c2;}
-    static inline _LIBCPP_CONSTEXPR bool lt(char_type __c1, char_type __c2) _NOEXCEPT
-        {return __c1 < __c2;}
-
-    static _LIBCPP_CONSTEXPR_SINCE_CXX17
-    int compare(const char_type* __s1, const char_type* __s2, size_t __n);
-    _LIBCPP_INLINE_VISIBILITY static _LIBCPP_CONSTEXPR_SINCE_CXX17
-    size_t length(const char_type* __s);
-    _LIBCPP_INLINE_VISIBILITY static _LIBCPP_CONSTEXPR_SINCE_CXX17
-    const char_type* find(const char_type* __s, size_t __n, const char_type& __a);
-    static _LIBCPP_CONSTEXPR_SINCE_CXX20
-    char_type*       move(char_type* __s1, const char_type* __s2, size_t __n);
-    _LIBCPP_INLINE_VISIBILITY
-    static _LIBCPP_CONSTEXPR_SINCE_CXX20
-    char_type*       copy(char_type* __s1, const char_type* __s2, size_t __n);
-    _LIBCPP_INLINE_VISIBILITY
-    static _LIBCPP_CONSTEXPR_SINCE_CXX20
-    char_type*       assign(char_type* __s, size_t __n, char_type __a);
-
-    static inline _LIBCPP_CONSTEXPR int_type  not_eof(int_type __c) _NOEXCEPT
-        {return eq_int_type(__c, eof()) ? ~eof() : __c;}
-    static inline _LIBCPP_CONSTEXPR char_type to_char_type(int_type __c) _NOEXCEPT
-        {return char_type(__c);}
-    static inline _LIBCPP_CONSTEXPR int_type  to_int_type(char_type __c) _NOEXCEPT
-        {return int_type(__c);}
-    static inline _LIBCPP_CONSTEXPR bool      eq_int_type(int_type __c1, int_type __c2) _NOEXCEPT
-        {return __c1 == __c2;}
-    static inline _LIBCPP_CONSTEXPR int_type  eof() _NOEXCEPT
-        {return int_type(EOF);}
+    using int_type   = ...;
+    using off_type   = ...;
+    using pos_type   = ...;
+    using state_type = ...;
+
+    static void assign(char_type&, const char_type&);
+    static bool eq(char_type, char_type);
+    static bool lt(char_type, char_type);
+
+    static int              compare(const char_type*, const char_type*, size_t);
+    static size_t           length(const char_type*);
+    static const char_type* find(const char_type*, size_t, const char_type&);
+    static char_type*       move(char_type*, const char_type*, size_t);
+    static char_type*       copy(char_type*, const char_type* __s2, size_t);
+    static char_type*       assign(char_type*, size_t, char_type);
+
+    static int_type  not_eof(int_type);
+    static char_type to_char_type(int_type);
+    static int_type  to_int_type(char_type);
+    static bool      eq_int_type(int_type, int_type);
+    static int_type  eof();
 };
-
-template <class _CharT>
-_LIBCPP_CONSTEXPR_SINCE_CXX17 int
-char_traits<_CharT>::compare(const char_type* __s1, const char_type* __s2, size_t __n)
-{
-    for (; __n; --__n, ++__s1, ++__s2)
-    {
-        if (lt(*__s1, *__s2))
-            return -1;
-        if (lt(*__s2, *__s1))
-            return 1;
-    }
-    return 0;
-}
-
-template <class _CharT>
-inline
-_LIBCPP_CONSTEXPR_SINCE_CXX17 size_t
-char_traits<_CharT>::length(const char_type* __s)
-{
-    size_t __len = 0;
-    for (; !eq(*__s, char_type(0)); ++__s)
-        ++__len;
-    return __len;
-}
-
-template <class _CharT>
-inline
-_LIBCPP_CONSTEXPR_SINCE_CXX17 const _CharT*
-char_traits<_CharT>::find(const char_type* __s, size_t __n, const char_type& __a)
-{
-    for (; __n; --__n)
-    {
-        if (eq(*__s, __a))
-            return __s;
-        ++__s;
-    }
-    return nullptr;
-}
-
-template <class _CharT>
-_LIBCPP_CONSTEXPR_SINCE_CXX20 _CharT*
-char_traits<_CharT>::move(char_type* __s1, const char_type* __s2, size_t __n)
-{
-    if (__n == 0) return __s1;
-    char_type* __r = __s1;
-    if (__s1 < __s2)
-    {
-        for (; __n; --__n, ++__s1, ++__s2)
-            assign(*__s1, *__s2);
-    }
-    else if (__s2 < __s1)
-    {
-        __s1 += __n;
-        __s2 += __n;
-        for (; __n; --__n)
-            assign(*--__s1, *--__s2);
-    }
-    return __r;
-}
-
-template <class _CharT>
-inline _LIBCPP_CONSTEXPR_SINCE_CXX20
-_CharT*
-char_traits<_CharT>::copy(char_type* __s1, const char_type* __s2, size_t __n)
-{
-    if (!__libcpp_is_constant_evaluated()) {
-        _LIBCPP_ASSERT(__s2 < __s1 || __s2 >= __s1+__n, "char_traits::copy overlapped range");
-    }
-    char_type* __r = __s1;
-    for (; __n; --__n, ++__s1, ++__s2)
-        assign(*__s1, *__s2);
-    return __r;
-}
-
-template <class _CharT>
-inline _LIBCPP_CONSTEXPR_SINCE_CXX20
-_CharT*
-char_traits<_CharT>::assign(char_type* __s, size_t __n, char_type __a)
-{
-    char_type* __r = __s;
-    for (; __n; --__n, ++__s)
-        assign(*__s, __a);
-    return __r;
-}
+*/
 
 template <class _CharT>
 _LIBCPP_HIDE_FROM_ABI static inline _LIBCPP_CONSTEXPR_SINCE_CXX20
 _CharT* __char_traits_move(_CharT* __dest, const _CharT* __source, size_t __n) _NOEXCEPT
 {
 #ifdef _LIBCPP_COMPILER_GCC
   if (__libcpp_is_constant_evaluated()) {
     if (__n == 0)

libcxx/test/std/ranges/range.adaptors/range.lazy.split/general.pass.cpp

 #include <cassert>
 #include <map>
 #include <string>
 #include <string_view>
 #include <utility>
 #include <vector>
 #include "types.h"
 
-// A constexpr-friendly lightweight string, primarily useful for comparisons.
-// Unlike `std::string_view`, it copies the given string into an
-// internal buffer and can work with non-contiguous inputs.
+// Basic utility to convert a range to a string-like type. This handles ranges
+// that do not contain character types and can work with non-contiguous inputs.
 template <class Char>
 class BasicSmallString {
-  std::basic_string<Char> buffer_{};
+  std::vector<Char> buffer_{};
 
 public:
-  constexpr BasicSmallString(std::basic_string_view<Char> v) : buffer_(v) {}
+  constexpr BasicSmallString(std::basic_string_view<char> v) : buffer_(v.begin(), v.end()) {}
+#ifndef TEST_HAS_NO_WIDE_CHARACTERS
+  constexpr BasicSmallString(std::basic_string_view<wchar_t> v) : buffer_(v.begin(), v.end()) {}
+#endif
+  constexpr BasicSmallString(std::basic_string_view<char8_t> v) : buffer_(v.begin(), v.end()) {}
+  constexpr BasicSmallString(std::basic_string_view<char16_t> v) : buffer_(v.begin(), v.end()) {}
+  constexpr BasicSmallString(std::basic_string_view<char32_t> v) : buffer_(v.begin(), v.end()) {}
 
   template <class I, class S>
   constexpr BasicSmallString(I b, const S& e) {
     for (; b != e; ++b) {
-      buffer_ += *b;
+      buffer_.push_back(*b);
     }
   }
 
   template <std::ranges::range R>
   constexpr BasicSmallString(R&& from) : BasicSmallString(from.begin(), from.end()) {}
 
   friend constexpr bool operator==(const BasicSmallString& lhs, const BasicSmallString& rhs) {
     return lhs.buffer_ == rhs.buffer_;

libcxx/test/std/strings/basic.string/string.modifiers/string_append/push_back.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <string>
 
 // void push_back(charT c) // constexpr since C++20
 
 #include <string>
+#include <algorithm>
 #include <cassert>
 
 #include "test_macros.h"
 #include "min_allocator.h"
 
-struct veryLarge
-{
+struct VeryLarge {
   long long a;
   char b;
 };
 
+namespace std {
+  template <>
+  struct char_traits<VeryLarge> {
+    using char_type           = VeryLarge;
+    using int_type            = int;
+    using off_type            = streamoff;
+    using pos_type            = streampos;
+    using state_type          = mbstate_t;
+
+    static TEST_CONSTEXPR_CXX20 void assign(char_type& c1, const char_type& c2) { c1 = c2; }
+    static bool eq(char_type c1, char_type c2);
+    static bool lt(char_type c1, char_type c2);
+
+    static int compare(const char_type* s1, const char_type* s2, size_t n);
+    static size_t length(const char_type* s);
+    static const char_type* find(const char_type* s, size_t n, const char_type& a);
+    static char_type* move(char_type* s1, const char_type* s2, size_t n);
+    static TEST_CONSTEXPR_CXX20 char_type* copy(char_type* s1, const char_type* s2, size_t n) {
+      std::copy_n(s2, n, s1);
+      return s1;
+    }
+    static TEST_CONSTEXPR_CXX20 char_type* assign(char_type* s, size_t n, char_type a) {
+      std::fill_n(s, n, a);
+      return s;
+    }
+
+    static int_type not_eof(int_type c);
+    static char_type to_char_type(int_type c);
+    static int_type to_int_type(char_type c);
+    static bool eq_int_type(int_type c1, int_type c2);
+    static int_type eof();
+  };
+} // end namespace std
+
 template <class S>
 TEST_CONSTEXPR_CXX20 void
 test(S s, typename S::value_type c, S expected)
 {
     s.push_back(c);
     LIBCPP_ASSERT(s.__invariants());
     assert(s == expected);
 }
#if TEST_STD_VER >= 11
     typedef std::basic_string<char, std::char_traits<char>, min_allocator<char>> S;
     test(S(), 'a', S(1, 'a'));
     test(S("12345"), 'a', S("12345a"));
     test(S("12345678901234567890"), 'a', S("12345678901234567890a"));
   }
 #endif
 
   {
     // https://llvm.org/PR31454
-    std::basic_string<veryLarge> s;
-    veryLarge vl = {};
+    std::basic_string<VeryLarge> s;
+    VeryLarge vl = {};
     s.push_back(vl);
     s.push_back(vl);
     s.push_back(vl);
   }
 
   return true;
 }