[include-mapping] Introduce a human-edit CXXSymbolMapping file

[hokein]

Summary

A patch based on https://reviews.llvm.org/D143054

This file is allowed to be edit by human, and it overlays the generated symbol file. It contains a list of multiple-header symbols.
This patch introduces the file only. Usage will come afterwards.

[hokein]

update, separate out a new file.

[hokein]

upload the missing file.

[hokein]

refine the patch: include multiple-header symbols.

[kadircet]

clang/include/clang/Tooling/Inclusions/CXXSymbolMap.inc

12 ↗

(On Diff #494573)

i feel like cppreference is wrong here (at least per libstdc++-12, and https://eel.is/c++draft/locale.syn#header:%3clocale%3e)

#include <locale>
auto x = std::consume_header;

same for generate_header and little_endian.

it might be worth changing these into:

SYMBOL(consume_header, std::, <codecvt>)
// cppreference mentions this header as an alternative, but it isn't in practice.
// SYMBOL(consume_header, std::, <locale>)

19 ↗

(On Diff #494573)

can we actually use the ordering from standard: [cstddef.syn], [cstdlib.syn], [cstring.syn], [cwchar.syn], [cuchar.syn], [ctime.syn], [cstdio.syn] (1)

26 ↗

(On Diff #494573)

nit: i don't think we need the comments here, both headers are also mentioned in cppreference pages.
i am afraid these might get out of date (or only seldom mentioned) as future editors of this mapping might not be delicate enough.

[kadircet]

clang/include/clang/Tooling/Inclusions/CXXSymbolMap.inc

1 ↗

(On Diff #494573)

also maybe rename this to, AlternativeHeaderMap.inc ?

3 ↗

(On Diff #494573)

i'd rather say This is a hand-curated list for symbols provided by multiple headers, to address the short comings of cppreference or automated extraction logic. All entries for a symbol name provide the same declaration (hence these are not overloads/variants like std::remove from algorithm vs cstdio).

[hokein]

address review comments

clang/include/clang/Tooling/Inclusions/CXXSymbolMap.inc

1 ↗

(On Diff #494573)

Renamed it to StdAlternativeHeaderMap.inc, to align with the existing StdSymbolMap.inc name pattern for C++ symbols.

12 ↗

(On Diff #494573)

Right, thanks for checking it! You're right. I also have checked the C++11, C++17 standard, I don't find any relevant text about the header "<locale>" contains the these symbol definition.
I'd rather delete this entry, and only keep <codecvt>.

19 ↗

(On Diff #494573)

Sure.

26 ↗

(On Diff #494573)

Removed.

[kadircet]

clang/include/clang/Tooling/Inclusions/StdAlternativeHeaderMap.inc

3 ↗

(On Diff #495032)

This is a hand-curated list for C++ symbols reads like we're planning to put all special C++ symbols into this file, rather than just the ones that are provided by alternative headers. that's the reason why i mentioned it as This is a hand-curated list for symbols provided by multiple headers specifically.

[hokein]

clang/include/clang/Tooling/Inclusions/StdAlternativeHeaderMap.inc

3 ↗

(On Diff #495032)

Restricting the file only to multiple-header symbols seems a bit narrow (and the consume_header symbol only has one header which doesn't fit into this bucket nicely).

My take of this file is - we'll put all special C++ symbols that are not able to handle by the cppreference generator, multiple-header symbols are the most critical ones.

What do you think?

[kadircet]

clang/include/clang/Tooling/Inclusions/StdAlternativeHeaderMap.inc

3 ↗

(On Diff #495032)

(and the consume_header symbol only has one header which doesn't fit into this bucket nicely).

Well, i'd say they deserve their own list in that case.

My take of this file is - we'll put all special C++ symbols that are not able to handle by the cppreference generator, multiple-header symbols are the most critical ones.

I am afraid of that list getting too long and impossible to read manually any more.
but since these are going to be private files soon, we can always do that split once that's actually the case.

We need a different name for this file in that case though. As I thought we are only putting symbols with alternative headers into this file. What about StdSpecialSymbolMap.inc instead and update the file comment to:

This is a hand-curated list for C++ symbols that cannot be parsed/extracted via include-mapping tool.

and not talk about All headers for a symbol name provide the same declaration (hence these are not overloads/variants like std::remove from algorithm vs cstdio). as we're planning to add them to here as well. it's just we aren't adding them "right now".

[hokein]

Rename the file name for more general purposes.

clang/include/clang/Tooling/Inclusions/StdAlternativeHeaderMap.inc

3 ↗

(On Diff #495032)

Yeah, this sounds good to me.

Well, i'd say they deserve their own list in that case.

Yeah, this is an alternative (a mono file vs. muti files). I would start with a single file.
The list of these symbols is not that large now.

[kadircet]

thanks, lgtm!

clang/include/clang/Tooling/Inclusions/StdSpecialSymbolMap.inc

+//===-- StdSpecialSymbolMap.inc ---------------------------------*- C++ -*-===//
+//
+// This is a hand-curated list for C++ symbols that cannot be parsed/extracted
+// via the include-mapping tool (gen_std.py).
+//
+//===----------------------------------------------------------------------===//
+
+// Symbols that can be provided by any of the headers, ordered by the header
+// preference.
+// cppreference mentions the <locale> header is an alternative for these symbols,
+// but they are not per the standard.
+SYMBOL(consume_header, std::, <codecvt>)
+SYMBOL(generate_header, std::, <codecvt>)
+SYMBOL(little_endian, std::, <codecvt>)
+
+SYMBOL(mbstate_t, std::, <cwchar>)
+SYMBOL(mbstate_t, std::, <cuchar>)
+SYMBOL(size_t, std::, <cstddef>)
+SYMBOL(size_t, std::, <cstdlib>)
+SYMBOL(size_t, std::, <cstring>)
+SYMBOL(size_t, std::, <cwchar>)
+SYMBOL(size_t, std::, <cuchar>)
+SYMBOL(size_t, std::, <ctime>)
+SYMBOL(size_t, std::, <cstdio>)
+SYMBOL(unwrap_ref_decay, std::, <type_traits>)
+SYMBOL(unwrap_ref_decay, std::, <functional>)
+SYMBOL(unwrap_reference, std::, <type_traits>)
+SYMBOL(unwrap_reference, std::, <functional>)
+SYMBOL(unwrap_ref_decay_t, std::, <type_traits>)
+SYMBOL(unwrap_ref_decay_t, std::, <functional>)
+SYMBOL(wint_t, std::, <cwctype>)
+SYMBOL(wint_t, std::, <cwchar>)
+// C++ [iterator.range 25.7]: In addition to being available via inclusion of
+// the <iterator> header, the function templates in [iterator.range] are
+// available when any of the following headers are included: <array>, <deque>,
+// <forward_­list>, ... and <vector>.
+SYMBOL(begin, std::, <iterator>)
+SYMBOL(begin, std::, <array>)
+SYMBOL(begin, std::, <deque>)
+SYMBOL(begin, std::, <forward_list>)
+SYMBOL(begin, std::, <list>)
+SYMBOL(begin, std::, <map>)
+SYMBOL(begin, std::, <regex>)
+SYMBOL(begin, std::, <set>)
+SYMBOL(begin, std::, <span>)
+SYMBOL(begin, std::, <string>)
+SYMBOL(begin, std::, <string_view>)
+SYMBOL(begin, std::, <unordered_map>)
+SYMBOL(begin, std::, <unordered_set>)
+SYMBOL(begin, std::, <vector>)
+SYMBOL(cbegin, std::, <iterator>)
+SYMBOL(cbegin, std::, <array>)
+SYMBOL(cbegin, std::, <deque>)
+SYMBOL(cbegin, std::, <forward_list>)
+SYMBOL(cbegin, std::, <list>)
+SYMBOL(cbegin, std::, <map>)
+SYMBOL(cbegin, std::, <regex>)
+SYMBOL(cbegin, std::, <set>)
+SYMBOL(cbegin, std::, <span>)
+SYMBOL(cbegin, std::, <string>)
+SYMBOL(cbegin, std::, <string_view>)
+SYMBOL(cbegin, std::, <unordered_map>)
+SYMBOL(cbegin, std::, <unordered_set>)
+SYMBOL(cbegin, std::, <vector>)
+SYMBOL(cend, std::, <iterator>)
+SYMBOL(cend, std::, <array>)
+SYMBOL(cend, std::, <deque>)
+SYMBOL(cend, std::, <forward_list>)
+SYMBOL(cend, std::, <list>)
+SYMBOL(cend, std::, <map>)
+SYMBOL(cend, std::, <regex>)
+SYMBOL(cend, std::, <set>)
+SYMBOL(cend, std::, <span>)
+SYMBOL(cend, std::, <string>)
+SYMBOL(cend, std::, <string_view>)
+SYMBOL(cend, std::, <unordered_map>)
+SYMBOL(cend, std::, <unordered_set>)
+SYMBOL(cend, std::, <vector>)
+SYMBOL(crbegin, std::, <iterator>)
+SYMBOL(crbegin, std::, <array>)
+SYMBOL(crbegin, std::, <deque>)
+SYMBOL(crbegin, std::, <forward_list>)
+SYMBOL(crbegin, std::, <list>)
+SYMBOL(crbegin, std::, <map>)
+SYMBOL(crbegin, std::, <regex>)
+SYMBOL(crbegin, std::, <set>)
+SYMBOL(crbegin, std::, <span>)
+SYMBOL(crbegin, std::, <string>)
+SYMBOL(crbegin, std::, <string_view>)
+SYMBOL(crbegin, std::, <unordered_map>)
+SYMBOL(crbegin, std::, <unordered_set>)
+SYMBOL(crbegin, std::, <vector>)
+SYMBOL(crend, std::, <iterator>)
+SYMBOL(crend, std::, <array>)
+SYMBOL(crend, std::, <deque>)
+SYMBOL(crend, std::, <forward_list>)
+SYMBOL(crend, std::, <list>)
+SYMBOL(crend, std::, <map>)
+SYMBOL(crend, std::, <regex>)
+SYMBOL(crend, std::, <set>)
+SYMBOL(crend, std::, <span>)
+SYMBOL(crend, std::, <string>)
+SYMBOL(crend, std::, <string_view>)
+SYMBOL(crend, std::, <unordered_map>)
+SYMBOL(crend, std::, <unordered_set>)
+SYMBOL(crend, std::, <vector>)
+SYMBOL(data, std::, <iterator>)
+SYMBOL(data, std::, <array>)
+SYMBOL(data, std::, <deque>)
+SYMBOL(data, std::, <forward_list>)
+SYMBOL(data, std::, <list>)
+SYMBOL(data, std::, <map>)
+SYMBOL(data, std::, <regex>)
+SYMBOL(data, std::, <set>)
+SYMBOL(data, std::, <span>)
+SYMBOL(data, std::, <string>)
+SYMBOL(data, std::, <string_view>)
+SYMBOL(data, std::, <unordered_map>)
+SYMBOL(data, std::, <unordered_set>)
+SYMBOL(data, std::, <vector>)
+SYMBOL(empty, std::, <iterator>)
+SYMBOL(empty, std::, <array>)
+SYMBOL(empty, std::, <deque>)
+SYMBOL(empty, std::, <forward_list>)
+SYMBOL(empty, std::, <list>)
+SYMBOL(empty, std::, <map>)
+SYMBOL(empty, std::, <regex>)
+SYMBOL(empty, std::, <set>)
+SYMBOL(empty, std::, <span>)
+SYMBOL(empty, std::, <string>)
+SYMBOL(empty, std::, <string_view>)
+SYMBOL(empty, std::, <unordered_map>)
+SYMBOL(empty, std::, <unordered_set>)
+SYMBOL(empty, std::, <vector>)
+SYMBOL(end, std::, <iterator>)
+SYMBOL(end, std::, <array>)
+SYMBOL(end, std::, <deque>)
+SYMBOL(end, std::, <forward_list>)
+SYMBOL(end, std::, <list>)
+SYMBOL(end, std::, <map>)
+SYMBOL(end, std::, <regex>)
+SYMBOL(end, std::, <set>)
+SYMBOL(end, std::, <span>)
+SYMBOL(end, std::, <string>)
+SYMBOL(end, std::, <string_view>)
+SYMBOL(end, std::, <unordered_map>)
+SYMBOL(end, std::, <unordered_set>)
+SYMBOL(end, std::, <vector>)
+SYMBOL(rbegin, std::, <iterator>)
+SYMBOL(rbegin, std::, <array>)
+SYMBOL(rbegin, std::, <deque>)
+SYMBOL(rbegin, std::, <forward_list>)
+SYMBOL(rbegin, std::, <list>)
+SYMBOL(rbegin, std::, <map>)
+SYMBOL(rbegin, std::, <regex>)
+SYMBOL(rbegin, std::, <set>)
+SYMBOL(rbegin, std::, <span>)
+SYMBOL(rbegin, std::, <string>)
+SYMBOL(rbegin, std::, <string_view>)
+SYMBOL(rbegin, std::, <unordered_map>)
+SYMBOL(rbegin, std::, <unordered_set>)
+SYMBOL(rbegin, std::, <vector>)
+SYMBOL(rend, std::, <iterator>)
+SYMBOL(rend, std::, <array>)
+SYMBOL(rend, std::, <deque>)
+SYMBOL(rend, std::, <forward_list>)
+SYMBOL(rend, std::, <list>)
+SYMBOL(rend, std::, <map>)
+SYMBOL(rend, std::, <regex>)
+SYMBOL(rend, std::, <set>)
+SYMBOL(rend, std::, <span>)
+SYMBOL(rend, std::, <string>)
+SYMBOL(rend, std::, <string_view>)
+SYMBOL(rend, std::, <unordered_map>)
+SYMBOL(rend, std::, <unordered_set>)
+SYMBOL(rend, std::, <vector>)
+SYMBOL(size, std::, <iterator>)
+SYMBOL(size, std::, <array>)
+SYMBOL(size, std::, <deque>)
+SYMBOL(size, std::, <forward_list>)
+SYMBOL(size, std::, <list>)
+SYMBOL(size, std::, <map>)
+SYMBOL(size, std::, <regex>)
+SYMBOL(size, std::, <set>)
+SYMBOL(size, std::, <span>)
+SYMBOL(size, std::, <string>)
+SYMBOL(size, std::, <string_view>)
+SYMBOL(size, std::, <unordered_map>)
+SYMBOL(size, std::, <unordered_set>)
+SYMBOL(size, std::, <vector>)
+SYMBOL(ssize, std::, <iterator>)
+SYMBOL(ssize, std::, <array>)
+SYMBOL(ssize, std::, <deque>)
+SYMBOL(ssize, std::, <forward_list>)
+SYMBOL(ssize, std::, <list>)
+SYMBOL(ssize, std::, <map>)
+SYMBOL(ssize, std::, <regex>)
+SYMBOL(ssize, std::, <set>)
+SYMBOL(ssize, std::, <span>)
+SYMBOL(ssize, std::, <string>)
+SYMBOL(ssize, std::, <string_view>)
+SYMBOL(ssize, std::, <unordered_map>)
+SYMBOL(ssize, std::, <unordered_set>)
+SYMBOL(ssize, std::, <vector>)