[libc++] Make to_chars base 10 header only.

[Mordante]

Summary

The functions to_chars and from_chars should offer 128-bit support. This
is the first step to implement 128-bit version of to_chars. Before
implementing 128-bit support the current code will be polished.

This moves the code from the dylib to the header in prepartion of

P2291 "Add Constexpr Modifiers to Functions to_chars and from_chars for
Integral Types in <charconv> Header"

Note some more cleanups will be done in follow-up commits

• Remove the _LIBCPP_AVAILABILITY_TO_CHARS from to_chars. With all code in the header the availablilty macro is no longer needed. This requires enabling the unit tests for additional platforms.
• The code in the dylib can switch to using the header implementation. This allows removing the code duplicated in the header and the dylib.

[philnik]

libcxx/include/charconv

108–111

Since we have to keep the functions in the dylib forever anyways, why not use them?

[Mordante]

libcxx/include/charconv

108–111

Since I want to remove the code from the dylib I thought it was easier to use a new name instead of using the gnu_inline attribute.

* The code in the dylib can switch to using the header implementation. This allows removing the code duplicated in the header and the dylib.

We can even removed the code in the dylib for ABI v2.

[Mordante]

Fixes CI errors.

[Mordante]

Keep exporting the ABI functions.

[ldionne]

LGTM!

libcxx/include/__charconv/to_chars_base_10.h

37

We could be more ADL-safe here and elsewhere in this file. I guess it's mostly a matter of style and consistency because the types we use these functions with are not going to do ADL hijacking.

libcxx/include/charconv

160

Pre-existing issue but we might as well fix it -- this should be ADL safe. Same elsewhere.

[ldionne]

libcxx/include/__charconv/tables.h

26

What we want is essentially inline constexpr char __digits_base_10[200], but we can't have that because we need to support C++11. IIRC, the canonical way to do this pre-C++17 was:

template <class = void>
struct __digits_base_10 {
  static constexpr char __value[200];
};

template <class _Tp>
char __digits_base_10<_Tp>::__value[200] = {
  // ...
};

Then, you access it as __digits_base_10<>::__value. It's not as pretty, but it gets you the linkonce_odr semantics that you are after even before C++17.

Note from live review: We may want to apply the same kind of transformation to the other places where we have this pattern in <charconv>. Using static constexpr might lead to code bloat.

38

Nit: it's a bit weird that this clang-format comment is not aligned with the previous one.

[Mordante]

Rebased and addresses review comments.

[Mordante]

libcxx/include/__charconv/tables.h

26

I did this place only for now. I'll have a look at the other places in a separated commit.

38

Agreed.

libcxx/include/CMakeLists.txt

set(files
   __bit/bit_cast.h
   __bit/byteswap.h
   __bit_reference
   __bits
   __bsd_locale_defaults.h
   __bsd_locale_fallbacks.h
   __charconv/chars_format.h
   __charconv/from_chars_result.h
+  __charconv/tables.h
+  __charconv/to_chars_base_10.h
   __charconv/to_chars_result.h
   __chrono/calendar.h
   __chrono/convert_to_timespec.h
   __chrono/day.h
   __chrono/duration.h
   __chrono/file_clock.h
   __chrono/hh_mm_ss.h
   __chrono/high_resolution_clock.h

libcxx/include/__charconv/tables.h

+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___CHARCONV_TABLES
+#define _LIBCPP___CHARCONV_TABLES
+
+#include <__config>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#ifndef _LIBCPP_CXX03_LANG
+
+namespace __itoa {
+
+template <class = void>
+struct __digits_base_10 {
+  static const char __value[200];
+};
+
+template <class _Tp>
+const char __digits_base_10<_Tp>::__value[200] = {
+    // clang-format off
+    '0', '0', '0', '1', '0', '2', '0', '3', '0', '4', '0', '5', '0', '6', '0', '7', '0', '8', '0', '9',
+    '1', '0', '1', '1', '1', '2', '1', '3', '1', '4', '1', '5', '1', '6', '1', '7', '1', '8', '1', '9',
+    '2', '0', '2', '1', '2', '2', '2', '3', '2', '4', '2', '5', '2', '6', '2', '7', '2', '8', '2', '9',
+    '3', '0', '3', '1', '3', '2', '3', '3', '3', '4', '3', '5', '3', '6', '3', '7', '3', '8', '3', '9',
+    '4', '0', '4', '1', '4', '2', '4', '3', '4', '4', '4', '5', '4', '6', '4', '7', '4', '8', '4', '9',
+    '5', '0', '5', '1', '5', '2', '5', '3', '5', '4', '5', '5', '5', '6', '5', '7', '5', '8', '5', '9',
+    '6', '0', '6', '1', '6', '2', '6', '3', '6', '4', '6', '5', '6', '6', '6', '7', '6', '8', '6', '9',
+    '7', '0', '7', '1', '7', '2', '7', '3', '7', '4', '7', '5', '7', '6', '7', '7', '7', '8', '7', '9',
+    '8', '0', '8', '1', '8', '2', '8', '3', '8', '4', '8', '5', '8', '6', '8', '7', '8', '8', '8', '9',
+    '9', '0', '9', '1', '9', '2', '9', '3', '9', '4', '9', '5', '9', '6', '9', '7', '9', '8', '9', '9'};
+// clang-format on
+
+} // namespace __itoa
+
+#endif // _LIBCPP_CXX03_LANG
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___CHARCONV_TABLES

libcxx/include/__charconv/to_chars_base_10.h

+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___CHARCONV_TO_CHARS_BASE_10_H
+#define _LIBCPP___CHARCONV_TO_CHARS_BASE_10_H
+
+#include <__charconv/tables.h>
+#include <__config>
+#include <cstdint>
+#include <cstring>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#ifndef _LIBCPP_CXX03_LANG
+
+namespace __itoa {
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append1(char* __buffer, _Tp __value) noexcept {
+  *__buffer = '0' + static_cast<char>(__value);
+  return __buffer + 1;
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append2(char* __buffer, _Tp __value) noexcept {
+  std::memcpy(__buffer, &__digits_base_10<>::__value[(__value)*2], 2);
+  return __buffer + 2;
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append3(char* __buffer, _Tp __value) noexcept {
+  return __itoa::__append2(__itoa::__append1(__buffer, (__value) / 100), (__value) % 100);
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append4(char* __buffer, _Tp __value) noexcept {
+  return __itoa::__append2(__itoa::__append2(__buffer, (__value) / 100), (__value) % 100);
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append2_no_zeros(char* __buffer, _Tp __value) noexcept {
+  if (__value < 10)
+    return __itoa::__append1(__buffer, __value);
+  else
+    return __itoa::__append2(__buffer, __value);
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append4_no_zeros(char* __buffer, _Tp __value) noexcept {
+  if (__value < 100)
+    return __itoa::__append2_no_zeros(__buffer, __value);
+  else if (__value < 1000)
+    return __itoa::__append3(__buffer, __value);
+  else
+    return __itoa::__append4(__buffer, __value);
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI char* __append8_no_zeros(char* __buffer, _Tp __value) noexcept {
+  if (__value < 10000)
+    __buffer = __itoa::__append4_no_zeros(__buffer, __value);
+  else {
+    __buffer = __itoa::__append4_no_zeros(__buffer, __value / 10000);
+    __buffer = __itoa::__append4(__buffer, __value % 10000);
+  }
+  return __buffer;
+}
+
+_LIBCPP_HIDE_FROM_ABI inline char* __base_10_u32(uint32_t __value, char* __buffer) noexcept {
+  if (__value < 100000000)
+    __buffer = __itoa::__append8_no_zeros(__buffer, __value);
+  else {
+    // __value = aabbbbcccc in decimal
+    const uint32_t __a = __value / 100000000; // 1 to 42
+    __value %= 100000000;
+
+    __buffer = __itoa::__append2_no_zeros(__buffer, __a);
+    __buffer = __itoa::__append4(__buffer, __value / 10000);
+    __buffer = __itoa::__append4(__buffer, __value % 10000);
+  }
+
+  return __buffer;
+}
+
+_LIBCPP_HIDE_FROM_ABI inline char* __base_10_u64(uint64_t __value, char* __buffer) noexcept {
+  if (__value < 100000000)
+    __buffer = __itoa::__append8_no_zeros(__buffer, static_cast<uint32_t>(__value));
+  else if (__value < 10000000000000000) {
+    const uint32_t __v0 = static_cast<uint32_t>(__value / 100000000);
+    const uint32_t __v1 = static_cast<uint32_t>(__value % 100000000);
+
+    __buffer = __itoa::__append8_no_zeros(__buffer, __v0);
+    __buffer = __itoa::__append4(__buffer, __v1 / 10000);
+    __buffer = __itoa::__append4(__buffer, __v1 % 10000);
+  } else {
+    const uint32_t __a = static_cast<uint32_t>(__value / 10000000000000000); // 1 to 1844
+    __value %= 10000000000000000;
+
+    __buffer = __itoa::__append4_no_zeros(__buffer, __a);
+
+    const uint32_t __v0 = static_cast<uint32_t>(__value / 100000000);
+    const uint32_t __v1 = static_cast<uint32_t>(__value % 100000000);
+    __buffer = __itoa::__append4(__buffer, __v0 / 10000);
+    __buffer = __itoa::__append4(__buffer, __v0 % 10000);
+    __buffer = __itoa::__append4(__buffer, __v1 / 10000);
+    __buffer = __itoa::__append4(__buffer, __v1 % 10000);
+  }
+
+  return __buffer;
+}
+
+} // namespace __itoa
+
+#endif // _LIBCPP_CXX03_LANG
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___CHARCONV_TO_CHARS_BASE_10_H

libcxx/include/charconv

 
 */
 
 #include <__assert> // all public C++ headers provide the assertion handler
 #include <__availability>
 #include <__bits>
 #include <__charconv/chars_format.h>
 #include <__charconv/from_chars_result.h>
+#include <__charconv/tables.h>
+#include <__charconv/to_chars_base_10.h>
 #include <__charconv/to_chars_result.h>
 #include <__config>
 #include <__debug>
 #include <__errc>
 #include <__utility/unreachable.h>
 #include <cmath> // for log2f
 #include <cstdint>
 #include <cstdlib>
 #include <cstring>
 #include <limits>
 #include <type_traits>
 
 #if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
 #  pragma GCC system_header
 #endif
 
 _LIBCPP_PUSH_MACROS
 #include <__undef_macros>
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 #ifndef _LIBCPP_CXX03_LANG
 
-namespace __itoa {
-_LIBCPP_AVAILABILITY_TO_CHARS _LIBCPP_FUNC_VIS char* __u64toa(uint64_t __value, char* __buffer) noexcept;
-_LIBCPP_AVAILABILITY_TO_CHARS _LIBCPP_FUNC_VIS char* __u32toa(uint32_t __value, char* __buffer) noexcept;
-} // namespace __itoa
-
 to_chars_result to_chars(char*, char*, bool, int = 10) = delete;
 from_chars_result from_chars(const char*, const char*, bool, int = 10) = delete;
 
 namespace __itoa
 {
 
 static constexpr uint64_t __pow10_64[] = {
     UINT64_C(0),
struct _LIBCPP_HIDDEN __traits_base
     {
         auto __t = (64 - std::__libcpp_clz(static_cast<type>(__v | 1))) * 1233 >> 12;
         return __t - (__v < __pow10_64[__t]) + 1;
     }
 
     _LIBCPP_AVAILABILITY_TO_CHARS
     static _LIBCPP_HIDE_FROM_ABI char* __convert(_Tp __v, char* __p)
     {
-        return __u64toa(__v, __p);
+        return __itoa::__base_10_u64(__v, __p);
     }
 
     static _LIBCPP_HIDE_FROM_ABI decltype(__pow10_64)& __pow() { return __pow10_64; }
 };
 
 template <typename _Tp>
 struct _LIBCPP_HIDDEN
     __traits_base<_Tp, decltype(void(uint32_t{declval<_Tp>()}))>
 {
     using type = uint32_t;
 
     static _LIBCPP_HIDE_FROM_ABI int __width(_Tp __v)
     {
         auto __t = (32 - std::__libcpp_clz(static_cast<type>(__v | 1))) * 1233 >> 12;
         return __t - (__v < __pow10_32[__t]) + 1;
     }
 
     _LIBCPP_AVAILABILITY_TO_CHARS
     static _LIBCPP_HIDE_FROM_ABI char* __convert(_Tp __v, char* __p)
     {
-        return __u32toa(__v, __p);
+        return __itoa::__base_10_u32(__v, __p);
     }
 
     static _LIBCPP_HIDE_FROM_ABI decltype(__pow10_32)& __pow() { return __pow10_32; }
 };
 
 template <typename _Tp>
 inline _LIBCPP_HIDE_FROM_ABI bool
 __mul_overflowed(unsigned char __a, _Tp __b, unsigned char& __r)

libcxx/include/module.modulemap

module std [system] {
   // No submodule for cassert. It fundamentally needs repeated, textual inclusion.
   module charconv {
     header "charconv"
     export *
 
     module __charconv {
       module chars_format      { private header "__charconv/chars_format.h" }
       module from_chars_result { private header "__charconv/from_chars_result.h" }
+      module tables            { private header "__charconv/tables.h" }
+      module to_chars_base_10  { private header "__charconv/to_chars_base_10.h" }
       module to_chars_result   { private header "__charconv/to_chars_result.h" }
     }
 
   }
   module chrono {
     header "chrono"
     export *
 

libcxx/src/charconv.cpp

append8_no_zeros(char* buffer, T v) noexcept
     else
     {
         buffer = append4_no_zeros(buffer, v / 10000);
         buffer = append4(buffer, v % 10000);
     }
     return buffer;
 }
 
-char*
+_LIBCPP_FUNC_VIS char*
 __u32toa(uint32_t value, char* buffer) noexcept
 {
     if (value < 100000000)
     {
         buffer = append8_no_zeros(buffer, value);
     }
     else
     {
         // value = aabbbbcccc in decimal
         const uint32_t a = value / 100000000;  // 1 to 42
         value %= 100000000;
 
         buffer = append2_no_zeros(buffer, a);
         buffer = append4(buffer, value / 10000);
         buffer = append4(buffer, value % 10000);
     }
 
     return buffer;
 }
 
-char*
+_LIBCPP_FUNC_VIS char*
 __u64toa(uint64_t value, char* buffer) noexcept
 {
     if (value < 100000000)
     {
         uint32_t v = static_cast<uint32_t>(value);
         buffer = append8_no_zeros(buffer, v);
     }
     else if (value < 10000000000000000)

libcxx/test/libcxx/private_headers.verify.cpp

 #include <__algorithm/upper_bound.h> // expected-error@*:* {{use of private header from outside its module: '__algorithm/upper_bound.h'}}
 #include <__availability> // expected-error@*:* {{use of private header from outside its module: '__availability'}}
 #include <__bit/bit_cast.h> // expected-error@*:* {{use of private header from outside its module: '__bit/bit_cast.h'}}
 #include <__bit/byteswap.h> // expected-error@*:* {{use of private header from outside its module: '__bit/byteswap.h'}}
 #include <__bit_reference> // expected-error@*:* {{use of private header from outside its module: '__bit_reference'}}
 #include <__bits> // expected-error@*:* {{use of private header from outside its module: '__bits'}}
 #include <__charconv/chars_format.h> // expected-error@*:* {{use of private header from outside its module: '__charconv/chars_format.h'}}
 #include <__charconv/from_chars_result.h> // expected-error@*:* {{use of private header from outside its module: '__charconv/from_chars_result.h'}}
+#include <__charconv/tables.h> // expected-error@*:* {{use of private header from outside its module: '__charconv/tables.h'}}
+#include <__charconv/to_chars_base_10.h> // expected-error@*:* {{use of private header from outside its module: '__charconv/to_chars_base_10.h'}}
 #include <__charconv/to_chars_result.h> // expected-error@*:* {{use of private header from outside its module: '__charconv/to_chars_result.h'}}
 #include <__chrono/calendar.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/calendar.h'}}
 #include <__chrono/convert_to_timespec.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/convert_to_timespec.h'}}
 #include <__chrono/day.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/day.h'}}
 #include <__chrono/duration.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/duration.h'}}
 #include <__chrono/file_clock.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/file_clock.h'}}
 #include <__chrono/hh_mm_ss.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/hh_mm_ss.h'}}
 #include <__chrono/high_resolution_clock.h> // expected-error@*:* {{use of private header from outside its module: '__chrono/high_resolution_clock.h'}}