[libc++][format] Adds a formattable concept.

[Mordante]

Summary

The concept is based on P2286R2 Formatting Ranges. It will be used to
optimise the storage of __format_arg_store as required by LWG-3473.

Depends on D120916

[Mordante]

Remove a part of a test intended for another commit.

[• Quuxplusone]

libcxx/include/__format/concepts.h

39–45

Let's add some linebreaks for readability after each requires-parameter:

template <class _Tp, class _CharT>
concept __formattable =
  semiregular<formatter<remove_cvref_t<_Tp>, _CharT>> &&
  requires(formatter<remove_cvref_t<_Tp>, _CharT> __f,
           formatter<remove_cvref_t<_Tp>, _CharT> __cf, // TODO: const
           _Tp __t,
           basic_format_context<_CharT*, _CharT> __fc,
           basic_format_parse_context<_CharT> __pc) {
    { __f.parse(__pc) } -> same_as<typename basic_format_parse_context<_CharT>::iterator>;
    { __cf.format(__t, __fc) } -> same_as<_CharT*>;
  };

[ldionne]

Please ping me on Discord if I should be reviewing this -- my understanding is that it's for a not-yet-approved paper? (if so, you can mark this as "changes planned" until you'd like a review)

libcxx/include/__format/concepts.h

33

This can be rebased out!

[Mordante]

Rebased and address review comments.

[Mordante]

In D120921#3387499, @ldionne wrote:

Please ping me on Discord if I should be reviewing this -- my understanding is that it's for a not-yet-approved paper? (if so, you can mark this as "changes planned" until you'd like a review)

Correct the paper isn't accepted yet and the paper targets C++23. However as mentioned in the description I need a formattable concept in C++20. I use the concept in D121514. So I either:

• Write my own concept, which would look slightly different.
• Use P2286's proposed concept.

Using an existing proposal allows me to reuse this "private" concept in C++23 avoiding code duplication.
So even when P2286 will never make it in the Standard this concept is still useful.
(I hope P2286 will make it in C++23 since I really like to proposal.)

libcxx/include/__format/concepts.h

39–45

I did part of the formatting.

[vitaut]

libcxx/include/__format/concepts.h

39

I think you should use format_context::iterator, not _CharT*.

[Mordante]

Thanks for the review!

libcxx/include/__format/concepts.h

39

That doesn't works since that only works for the char context. I'll change to code to use fmt-iter-for<charT> as used in the paper. I prefer to use the CharT* since it's the most simple output iterator. This is a valid choice according to the paper.

We don’t specify what the iterator type is of format_context or wformat_context, the expectation is that formatters accept any iterator. As such, it is unspecified in the concept which iterator will be checked - simply that it is some output_iterator<charT const&>. Implementations could use format_context::iterator and wformat_context::iterator, or they could have a bespoke minimal iterator dedicated for concept checking.

[Mordante]

Address review comments.

[vitaut]

libcxx/include/__format/concepts.h

39

Makes sense but maybe clarify it in a comment?

[Mordante]

Add comments as suggested.

[Mordante]

Rebased and updated to new test method.

[Mordante]

Rebased to trigger CI.

libcxx/include/CMakeLists.txt

set(files
   __filesystem/operations.h
   __filesystem/path.h
   __filesystem/path_iterator.h
   __filesystem/perm_options.h
   __filesystem/perms.h
   __filesystem/recursive_directory_iterator.h
   __filesystem/space_info.h
   __filesystem/u8path.h
+  __format/concepts.h
   __format/format_arg.h
   __format/format_args.h
   __format/format_context.h
   __format/format_error.h
   __format/format_fwd.h
   __format/format_parse_context.h
   __format/format_string.h
   __format/format_to_n_result.h

libcxx/include/__format/concepts.h

+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___FORMAT_CONCEPTS_H
+#define _LIBCPP___FORMAT_CONCEPTS_H
+
+#include <__concepts/same_as.h>
+#include <__concepts/semiregular.h>
+#include <__config>
+#include <__format/format_fwd.h>
+#include <__format/format_parse_context.h>
+#include <type_traits>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#if _LIBCPP_STD_VER > 17
+
+// The output iterator isn't specified. A formatter should accept any
+// output_iterator. This iterator is a minimal iterator to test the concept.
+// (Note testing for (w)format_context would be a valid choice, but requires
+// selecting the proper one depending on the type of _CharT.)
+template <class _CharT>
+using __fmt_iter_for = _CharT*;
+
+// The concept is based on P2286R6
+// It lacks the const of __cf as required by, the not yet accepted, LWG-3636.
+// The current formatters can't be easily adapted, but that is WIP.
+// TODO FMT properly implement this concepts once accepted.
+template <class _Tp, class _CharT>
+concept __formattable = (semiregular<formatter<remove_cvref_t<_Tp>, _CharT>>) &&
+                        requires(formatter<remove_cvref_t<_Tp>, _CharT> __f,
+                                 formatter<remove_cvref_t<_Tp>, _CharT> __cf, _Tp __t,
+                                 basic_format_context<__fmt_iter_for<_CharT>, _CharT> __fc,
+                                 basic_format_parse_context<_CharT> __pc) {
+                          { __f.parse(__pc) } -> same_as<typename basic_format_parse_context<_CharT>::iterator>;
+                          { __cf.format(__t, __fc) } -> same_as<__fmt_iter_for<_CharT>>;
+                        };
+
+#endif //_LIBCPP_STD_VER > 17
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___FORMAT_CONCEPTS_H

libcxx/include/__format/format_fwd.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef _LIBCPP___FORMAT_FORMAT_FWD_H
 #define _LIBCPP___FORMAT_FORMAT_FWD_H
 
 #include <__availability>
 #include <__config>
+#include <__iterator/concepts.h>
 
 #if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
 #  pragma GCC system_header
 #endif
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 #if _LIBCPP_STD_VER > 17
 
 template <class _Context>
 class _LIBCPP_TEMPLATE_VIS _LIBCPP_AVAILABILITY_FORMAT basic_format_arg;
 
 template <class _Context, class... _Args>
 struct _LIBCPP_TEMPLATE_VIS __format_arg_store;
 
+template <class _OutIt, class _CharT>
+  requires output_iterator<_OutIt, const _CharT&>
+class _LIBCPP_TEMPLATE_VIS _LIBCPP_AVAILABILITY_FORMAT basic_format_context;
+
 template <class _Tp, class _CharT = char>
 struct _LIBCPP_TEMPLATE_VIS _LIBCPP_AVAILABILITY_FORMAT formatter;
 
 #endif //_LIBCPP_STD_VER > 17
 
 _LIBCPP_END_NAMESPACE_STD
 
 #endif // _LIBCPP___FORMAT_FORMAT_FWD_H

libcxx/include/format

 // Make sure all feature-test macros are available.
 #include <version>
 // Enable the contents of the header only when libc++ was built with LIBCXX_ENABLE_INCOMPLETE_FEATURES.
 #if !defined(_LIBCPP_HAS_NO_INCOMPLETE_FORMAT)
 
 #include <__algorithm/clamp.h>
 #include <__config>
 #include <__debug>
+#include <__format/concepts.h>
 #include <__format/format_arg.h>
 #include <__format/format_args.h>
 #include <__format/format_context.h>
 #include <__format/format_error.h>
 #include <__format/format_fwd.h>
 #include <__format/format_parse_context.h>
 #include <__format/format_string.h>
 #include <__format/format_to_n_result.h>

libcxx/include/module.modulemap

module __filesystem {
       module u8path                       { private header "__filesystem/u8path.h" }
     }
   }
   module format {
     header "format"
     export *
 
     module __format {
+      module concepts                 { private header "__format/concepts.h" }
       module format_arg               { private header "__format/format_arg.h" }
       module format_args              { private header "__format/format_args.h" }
       module format_context {
         private header "__format/format_context.h"
         export optional
         export locale
       }
       module format_error             { private header "__format/format_error.h" }

libcxx/test/libcxx/diagnostics/detail.headers/format/concepts.module.verify.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// REQUIRES: modules-build
+
+// WARNING: This test was generated by 'generate_private_header_tests.py'
+// and should not be edited manually.
+
+// expected-error@*:* {{use of private header from outside its module: '__format/concepts.h'}}
+#include <__format/concepts.h>

libcxx/test/libcxx/utilities/format/format.formatter/format.formatter.spec/formattable.compile.pass.cpp

+//===----------------------------------------------------------------------===//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+// UNSUPPORTED: libcpp-has-no-incomplete-format
+
+// TODO FMT Move to std after P2286 has been accepted.
+
+// <format>
+
+// template<class T, class charT>
+// concept formattable = ...
+
+#include <array>
+#include <bitset>
+#include <bitset>
+#include <chrono>
+#include <complex>
+#include <concepts>
+#include <deque>
+#include <format>
+#include <forward_list>
+#include <list>
+#include <memory>
+#include <map>
+#include <optional>
+#include <queue>
+#include <set>
+#include <stack>
+#include <span>
+#include <tuple>
+#include <type_traits>
+#include <unordered_map>
+#include <unordered_set>
+#include <valarray>
+#include <variant>
+
+#include "test_macros.h"
+
+#ifndef TEST_HAS_NO_FILESYSTEM_LIBRARY
+#  include <filesystem>
+#endif
+#ifndef TEST_HAS_NO_LOCALIZATION
+#  include <regex>
+#endif
+#ifndef TEST_HAS_NO_THREADS
+#  include <thread>
+#endif
+
+template <class T, class CharT>
+void assert_is_not_formattable() {
+  static_assert(!std::__formattable<T, CharT>);
+}
+
+template <class T, class CharT>
+void assert_is_formattable() {
+  // Only formatters for CharT == char || CharT == wchar_t are enabled for the
+  // standard formatters. When CharT is a different type the formatter should
+  // be disabled.
+  if constexpr (std::same_as<CharT, char>
+#ifndef TEST_HAS_NO_WIDE_CHARACTERS
+                || std::same_as<CharT, wchar_t>
+#endif
+  )
+    static_assert(std::__formattable<T, CharT>);
+  else
+    assert_is_not_formattable<T, CharT>();
+}
+
+// Tests for P0645 Text Formatting
+template <class CharT>
+void test_P0645() {
+#ifndef TEST_HAS_NO_WIDE_CHARACTERS
+  // Tests the special formatter that converts a char to a wchar_t.
+  assert_is_formattable<char, wchar_t>();
+#endif
+  assert_is_formattable<CharT, CharT>();
+
+  assert_is_formattable<CharT*, CharT>();
+  assert_is_formattable<const CharT*, CharT>();
+  assert_is_formattable<std::basic_string<CharT>, CharT>();
+  assert_is_formattable<std::basic_string_view<CharT>, CharT>();
+
+  assert_is_formattable<bool, CharT>();
+
+  assert_is_formattable<signed char, CharT>();
+  assert_is_formattable<signed short, CharT>();
+  assert_is_formattable<signed int, CharT>();
+  assert_is_formattable<signed long, CharT>();
+  assert_is_formattable<signed long long, CharT>();
+#ifndef TEST_HAS_NO_INT128
+  assert_is_formattable<__int128_t, CharT>();
+#endif
+
+  assert_is_formattable<unsigned char, CharT>();
+  assert_is_formattable<unsigned short, CharT>();
+  assert_is_formattable<unsigned int, CharT>();
+  assert_is_formattable<unsigned long, CharT>();
+  assert_is_formattable<unsigned long long, CharT>();
+#ifndef TEST_HAS_NO_INT128
+  assert_is_formattable<__uint128_t, CharT>();
+#endif
+
+  assert_is_formattable<float, CharT>();
+  assert_is_formattable<double, CharT>();
+  assert_is_formattable<long double, CharT>();
+
+  assert_is_formattable<std::nullptr_t, CharT>();
+  assert_is_formattable<void*, CharT>();
+  assert_is_formattable<const void*, CharT>();
+}
+
+// Tests for P1361 Integration of chrono with text formatting
+//
+// Some tests are commented out since these types haven't been implemented in
+// chrono yet. After P1361 has been implemented these formatters should be all
+// enabled.
+template <class CharT>
+void test_P1361() {
+  assert_is_not_formattable<std::chrono::microseconds, CharT>();
+
+  assert_is_not_formattable<std::chrono::sys_time<std::chrono::microseconds>, CharT>();
+  //assert_is_formattable<std::chrono::utc_time<std::chrono::microseconds>, CharT>();
+  //assert_is_formattable<std::chrono::tai_time<std::chrono::microseconds>, CharT>();
+  //assert_is_formattable<std::chrono::gps_time<std::chrono::microseconds>, CharT>();
+  assert_is_not_formattable<std::chrono::file_time<std::chrono::microseconds>, CharT>();
+  assert_is_not_formattable<std::chrono::local_time<std::chrono::microseconds>, CharT>();
+
+  assert_is_not_formattable<std::chrono::day, CharT>();
+  assert_is_not_formattable<std::chrono::month, CharT>();
+  assert_is_not_formattable<std::chrono::year, CharT>();
+
+  assert_is_not_formattable<std::chrono::weekday, CharT>();
+  assert_is_not_formattable<std::chrono::weekday_indexed, CharT>();
+  assert_is_not_formattable<std::chrono::weekday_last, CharT>();
+
+  assert_is_not_formattable<std::chrono::month_day, CharT>();
+  assert_is_not_formattable<std::chrono::month_day_last, CharT>();
+  assert_is_not_formattable<std::chrono::month_weekday, CharT>();
+  assert_is_not_formattable<std::chrono::month_weekday_last, CharT>();
+
+  assert_is_not_formattable<std::chrono::year_month, CharT>();
+  assert_is_not_formattable<std::chrono::year_month_day, CharT>();
+  assert_is_not_formattable<std::chrono::year_month_day_last, CharT>();
+  assert_is_not_formattable<std::chrono::year_month_weekday, CharT>();
+  assert_is_not_formattable<std::chrono::year_month_weekday_last, CharT>();
+
+  assert_is_not_formattable<std::chrono::hh_mm_ss<std::chrono::microseconds>, CharT>();
+
+  //assert_is_formattable<std::chrono::sys_info, CharT>();
+  //assert_is_formattable<std::chrono::local_info, CharT>();
+
+  //assert_is_formattable<std::chrono::zoned_time, CharT>();
+}
+
+// Tests for P1636 Formatters for library types
+//
+// The paper hasn't been voted in so currently all formatters are disabled.
+// TODO validate whether the test is correct after the paper has been accepted.
+template <class CharT>
+void test_P1636() {
+  assert_is_not_formattable<std::basic_streambuf<CharT>, CharT>();
+  assert_is_not_formattable<std::bitset<42>, CharT>();
+  assert_is_not_formattable<std::complex<double>, CharT>();
+  assert_is_not_formattable<std::error_code, CharT>();
+#ifndef TEST_HAS_NO_FILESYSTEM_LIBRARY
+  assert_is_not_formattable<std::filesystem::path, CharT>();
+#endif
+  assert_is_not_formattable<std::shared_ptr<int>, CharT>();
+#ifndef TEST_HAS_NO_LOCALIZATION
+  assert_is_not_formattable<std::sub_match<CharT*>, CharT>();
+#endif
+#ifndef TEST_HAS_NO_THREADS
+  assert_is_not_formattable<std::thread::id, CharT>();
+#endif
+  assert_is_not_formattable<std::unique_ptr<int>, CharT>();
+}
+
+// Tests for P2286 Formatting ranges
+//
+// The paper hasn't been voted in so currently all formatters are disabled.
+// TODO validate whether the test is correct after the paper has been accepted.
+template <class CharT>
+void test_P2286() {
+  assert_is_not_formattable<std::array<int, 42>, CharT>();
+  assert_is_not_formattable<std::vector<int>, CharT>();
+  assert_is_not_formattable<std::deque<int>, CharT>();
+  assert_is_not_formattable<std::forward_list<int>, CharT>();
+  assert_is_not_formattable<std::list<int>, CharT>();
+
+  assert_is_not_formattable<std::set<int>, CharT>();
+  assert_is_not_formattable<std::map<int, int>, CharT>();
+  assert_is_not_formattable<std::multiset<int>, CharT>();
+  assert_is_not_formattable<std::multimap<int, int>, CharT>();
+
+  assert_is_not_formattable<std::unordered_set<int>, CharT>();
+  assert_is_not_formattable<std::unordered_map<int, int>, CharT>();
+  assert_is_not_formattable<std::unordered_multiset<int>, CharT>();
+  assert_is_not_formattable<std::unordered_multimap<int, int>, CharT>();
+
+  assert_is_not_formattable<std::stack<int>, CharT>();
+  assert_is_not_formattable<std::queue<int>, CharT>();
+  assert_is_not_formattable<std::priority_queue<int>, CharT>();
+
+  assert_is_not_formattable<std::span<int>, CharT>();
+
+  assert_is_not_formattable<std::valarray<int>, CharT>();
+
+  assert_is_not_formattable<std::pair<int, int>, CharT>();
+  assert_is_not_formattable<std::tuple<int>, CharT>();
+}
+
+class c {
+  void f();
+  void fc() const;
+  static void sf();
+};
+enum e { a };
+enum class ec { a };
+template <class CharT>
+void test_disabled() {
+#ifndef TEST_HAS_NO_WIDE_CHARACTERS
+  assert_is_not_formattable<const char*, wchar_t>();
+#endif
+  assert_is_not_formattable<const char*, char8_t>();
+  assert_is_not_formattable<const char*, char16_t>();
+  assert_is_not_formattable<const char*, char32_t>();
+
+  assert_is_not_formattable<c, CharT>();
+  assert_is_not_formattable<const c, CharT>();
+  assert_is_not_formattable<volatile c, CharT>();
+  assert_is_not_formattable<const volatile c, CharT>();
+
+  assert_is_not_formattable<e, CharT>();
+  assert_is_not_formattable<const e, CharT>();
+  assert_is_not_formattable<volatile e, CharT>();
+  assert_is_not_formattable<const volatile e, CharT>();
+
+  assert_is_not_formattable<ec, CharT>();
+  assert_is_not_formattable<const ec, CharT>();
+  assert_is_not_formattable<volatile ec, CharT>();
+  assert_is_not_formattable<const volatile ec, CharT>();
+
+  assert_is_not_formattable<int*, CharT>();
+  assert_is_not_formattable<const int*, CharT>();
+  assert_is_not_formattable<volatile int*, CharT>();
+  assert_is_not_formattable<const volatile int*, CharT>();
+
+  assert_is_not_formattable<c*, CharT>();
+  assert_is_not_formattable<const c*, CharT>();
+  assert_is_not_formattable<volatile c*, CharT>();
+  assert_is_not_formattable<const volatile c*, CharT>();
+
+  assert_is_not_formattable<e*, CharT>();
+  assert_is_not_formattable<const e*, CharT>();
+  assert_is_not_formattable<volatile e*, CharT>();
+  assert_is_not_formattable<const volatile e*, CharT>();
+
+  assert_is_not_formattable<ec*, CharT>();
+  assert_is_not_formattable<const ec*, CharT>();
+  assert_is_not_formattable<volatile ec*, CharT>();
+  assert_is_not_formattable<const volatile ec*, CharT>();
+
+  assert_is_not_formattable<void (*)(), CharT>();
+  assert_is_not_formattable<void (c::*)(), CharT>();
+  assert_is_not_formattable<void (c::*)() const, CharT>();
+
+  assert_is_not_formattable<std::optional<int>, CharT>();
+  assert_is_not_formattable<std::variant<int>, CharT>();
+
+  assert_is_not_formattable<std::shared_ptr<c>, CharT>();
+  assert_is_not_formattable<std::unique_ptr<c>, CharT>();
+
+  assert_is_not_formattable<std::array<c, 42>, CharT>();
+  assert_is_not_formattable<std::vector<c>, CharT>();
+  assert_is_not_formattable<std::deque<c>, CharT>();
+  assert_is_not_formattable<std::forward_list<c>, CharT>();
+  assert_is_not_formattable<std::list<c>, CharT>();
+
+  assert_is_not_formattable<std::set<c>, CharT>();
+  assert_is_not_formattable<std::map<c, int>, CharT>();
+  assert_is_not_formattable<std::multiset<c>, CharT>();
+  assert_is_not_formattable<std::multimap<c, int>, CharT>();
+
+  assert_is_not_formattable<std::unordered_set<c>, CharT>();
+  assert_is_not_formattable<std::unordered_map<c, int>, CharT>();
+  assert_is_not_formattable<std::unordered_multiset<c>, CharT>();
+  assert_is_not_formattable<std::unordered_multimap<c, int>, CharT>();
+
+  assert_is_not_formattable<std::stack<c>, CharT>();
+  assert_is_not_formattable<std::queue<c>, CharT>();
+  assert_is_not_formattable<std::priority_queue<c>, CharT>();
+
+  assert_is_not_formattable<std::span<c>, CharT>();
+
+  assert_is_not_formattable<std::valarray<c>, CharT>();
+
+  assert_is_not_formattable<std::pair<c, int>, CharT>();
+  assert_is_not_formattable<std::tuple<c>, CharT>();
+
+  assert_is_not_formattable<std::optional<c>, CharT>();
+  assert_is_not_formattable<std::variant<c>, CharT>();
+}
+
+template <class CharT>
+void test() {
+  test_P0645<CharT>();
+  test_P1361<CharT>();
+  test_P1636<CharT>();
+  test_P2286<CharT>();
+  test_disabled<CharT>();
+}
+
+void test() {
+  test<char>();
+#ifndef TEST_HAS_NO_WIDE_CHARACTERS
+  test<wchar_t>();
+#endif
+  test<char8_t>();
+  test<char16_t>();
+  test<char32_t>();
+
+  test<int>();
+}