Make libc++'s versioning namespace customizable

[EricWF]

Summary

This patch makes the versioning namespace libc++ uses customizable by the user using -DLIBCXX_ABI_NAMESPACE=__foo.

This allows users to build custom versions of libc++ which can be linked into binaries with other libc++ versions without causing symbol conflicts or ODR issues.

[smeenai]

Awesome, I don't have to maintain an internal patch for this anymore :)

[kristina]

LG, one suggestion, would it be possible to request the std:: namespace (disable inline namespaces altogether) by supplying a blank string?

Being able to avoid versioning is useful on embedded systems or any other systems that have a more controlled environment where only one libc++ is pretty much guaranteed? (this allows for compact mangling of certain types like std::string, which in turn saves a very significant amount of space as the full mangled name for it otherwise is fairly long and it appears in symbol tables and debug data for many kinds of template instantiations). Same applies to streams and some other types that have shorthand reserved mangled forms, at least with IA64 ABI.

If I understand correctly currently a blank string is simply ignored and __N form is used.

[ldionne]

CMakeLists.txt

685

__.+? Or do we want to allow __?

[EricWF]

In D53879#1280924, @kristina wrote:

LG, one suggestion, would it be possible to request the std:: namespace (disable inline namespaces altogether) by supplying a blank string?

Being able to avoid versioning is useful on embedded systems or any other systems that have a more controlled environment where only one libc++ is pretty much guaranteed? (this allows for compact mangling of certain types like std::string, which in turn saves a very significant amount of space as the full mangled name for it otherwise is fairly long and it appears in symbol tables and debug data for many kinds of template instantiations). Same applies to streams and some other types that have shorthand reserved mangled forms, at least with IA64 ABI.

If I understand correctly currently a blank string is simply ignored and __N form is used.

Yeah, that seems reasonable to me. I'll make the change.

CMakeLists.txt

685

In addition do we want _Foo. My thinking is we should probably allow users to foot-gun themselves as much as they want.

We should probably document that we make no promises that we don't introduce names which conflict with their custom namespace name.

[ldionne]

CMakeLists.txt

685

Actually, that's interesting. Another possibility would be to only allow users to use names that we can promise we won't reuse. For example we could say __x.+ is reserved for users, or something like that.

What we should really avoid is for someone to start using something like __3, which would conflict with something we want to use in the future. Even though it would be their problem in theory, in practice it might put pressure on us not to reuse the name (if it were an important vendor or something). Constraining what names they can use would clear this possibility out completely.

[smeenai]

CMakeLists.txt

685

I would strongly prefer the allowed names to not be that restricted, since existing custom inline namespaces might not confirm to it (our doesn't). Allowing __ followed by anything would work for us.

[kristina]

Maybe as a compromise restrict it to anything that isn't __[1-9] since I agree with the comments regarding arbitrary numbering making things difficult with regards to ABI evolution. For example a single letter would work and at the same time ensure it won't clash with future ABI versions.

[EricWF]

In D53879#1281139, @EricWF wrote:

In D53879#1280924, @kristina wrote:

LG, one suggestion, would it be possible to request the std:: namespace (disable inline namespaces altogether) by supplying a blank string?

Being able to avoid versioning is useful on embedded systems or any other systems that have a more controlled environment where only one libc++ is pretty much guaranteed? (this allows for compact mangling of certain types like std::string, which in turn saves a very significant amount of space as the full mangled name for it otherwise is fairly long and it appears in symbol tables and debug data for many kinds of template instantiations). Same applies to streams and some other types that have shorthand reserved mangled forms, at least with IA64 ABI.

If I understand correctly currently a blank string is simply ignored and __N form is used.

Yeah, that seems reasonable to me. I'll make the change.

Actually, I'll make the change as a follow up patch.

CMakeLists.txt

685

I've added a warning in the documentation that it's the responsibility of the user to choose a name that won't cause conflict, and that we make no promises. I think that's sufficient.

I've also weakened the error to a warning.

[kristina]

In D53879#1281207, @EricWF wrote:

Actually, I'll make the change as a follow up patch.

Sounds good, I think a blank string, as a special case, explicitly being passed down fairly clearly conveys the intentions to opt out of inline namespaces and versioning.

[ldionne]

CMakeLists.txt

685

I disagree that documentation is sufficient. I understand your point (in theory), but in practice the consequences for someone misusing our API can be large enough that we have to deal with it. So in practice, we need to harden a bit more against misuse, especially when it's that easy. Please at least do what @kristina suggests and make sure that people are not using __[0-9]+.

[ldionne]

docs/BuildingLibcxx.rst

373

The problem with that is that they can't know what names we will want to use. If someone used __5 today, they would be within their rights because they can't know that we'll want to use it in the future. We have to reserve something for our own usage.

[EricWF]

docs/BuildingLibcxx.rst

373

We can't know what names we'll use in general. But they can be smart about it and aware of the risks.

[EricWF]

• Enforce that __[0-9]+$ is reserved.

[kristina]

docs/BuildingLibcxx.rst

373

It's true but it's one of the things that one may not be aware of until suddenly a new ABI is rolled out that does cause a conflict. It makes sense for libc++ developers such as yourself to have a very small reserved namespace for "official" versioning changes. If people really want to risk shooting themselves in the foot they can edit the header, but something like __3 could be a ticking timebomb, and it's not always easy to understand why and it's better than breaking customer's code by accident as it's upstream after all, and it's much easier than write a long explanation on why __3 is a really bad idea but __a is okay.

[jfoidart]

Actually, I'll make the change as a follow up patch.

Hi @EricWF . I checked the latest source and did not see such a change. Apologies if I missed it, but if not, is there still the plan to make this change? Thanks!

CMakeLists.txt

 cmake_dependent_option(LIBCXX_INSTALL_EXPERIMENTAL_LIBRARY
         "Install libc++experimental.a" ON
         "LIBCXX_ENABLE_EXPERIMENTAL_LIBRARY;LIBCXX_INSTALL_LIBRARY" OFF)
 cmake_dependent_option(LIBCXX_INSTALL_FILESYSTEM_LIBRARY
         "Install libc++fs.a" ON
         "LIBCXX_ENABLE_FILESYSTEM;LIBCXX_INSTALL_LIBRARY" OFF)
 
 set(LIBCXX_ABI_VERSION "1" CACHE STRING "ABI version of libc++. Can be either 1 or 2, where 2 is currently not stable. Defaults to 1.")
+set(LIBCXX_ABI_NAMESPACE "" CACHE STRING "The inline ABI namespace used by libc++. It defaults to __n where `n` is the current ABI version.")
 option(LIBCXX_ABI_UNSTABLE "Unstable ABI of libc++." OFF)
 option(LIBCXX_ABI_FORCE_ITANIUM "Ignore auto-detection and force use of the Itanium ABI.")
 option(LIBCXX_ABI_FORCE_MICROSOFT "Ignore auto-detection and force use of the Microsoft ABI.")
 option(LIBCXX_HIDE_FROM_ABI_PER_TU_BY_DEFAULT "Enable per TU ABI insulation by default. To be used by vendors." OFF)
 set(LIBCXX_ABI_DEFINES "" CACHE STRING "A semicolon separated list of ABI macros to define in the site config header.")
 option(LIBCXX_USE_COMPILER_RT "Use compiler-rt instead of libgcc" OFF)
 
 if (NOT LIBCXX_ENABLE_SHARED AND NOT LIBCXX_ENABLE_STATIC)
elseif(LLVM_USE_SANITIZER AND MSVC)
     message(WARNING "LLVM_USE_SANITIZER is not supported on this platform.")
   endif()
 endif()
 
 # Configuration file flags =====================================================
 if (NOT LIBCXX_ABI_VERSION EQUAL 1)
   config_define(${LIBCXX_ABI_VERSION} _LIBCPP_ABI_VERSION)
 endif()
+if (NOT LIBCXX_ABI_NAMESPACE STREQUAL "")
+  if (NOT LIBCXX_ABI_NAMESPACE MATCHES "__.*")
+    message(WARNING "LIBCXX_ABI_NAMESPACE must be a reserved identifier.")
+  endif()
+  if (LIBCXX_ABI_NAMESPACE MATCHES "__[0-9]+$")
+    message(FATAL_ERROR "LIBCXX_ABI_NAMESPACE '${LIBCXX_ABI_NAMESPACE}' is reserved for use by libc++.")
+  endif()
+  config_define(${LIBCXX_ABI_NAMESPACE} _LIBCPP_ABI_NAMESPACE)
+endif()
 config_define_if(LIBCXX_ABI_UNSTABLE _LIBCPP_ABI_UNSTABLE)
 config_define_if(LIBCXX_ABI_FORCE_ITANIUM _LIBCPP_ABI_FORCE_ITANIUM)
 config_define_if(LIBCXX_ABI_FORCE_MICROSOFT _LIBCPP_ABI_FORCE_MICROSOFT)
 config_define_if(LIBCXX_HIDE_FROM_ABI_PER_TU_BY_DEFAULT _LIBCPP_HIDE_FROM_ABI_PER_TU_BY_DEFAULT)
 
 config_define_if_not(LIBCXX_ENABLE_GLOBAL_FILESYSTEM_NAMESPACE _LIBCPP_HAS_NO_GLOBAL_FILESYSTEM_NAMESPACE)
 config_define_if_not(LIBCXX_ENABLE_STDIN _LIBCPP_HAS_NO_STDIN)
 config_define_if_not(LIBCXX_ENABLE_STDOUT _LIBCPP_HAS_NO_STDOUT)

docs/BuildingLibcxx.rst

 
 .. option:: LIBCXX_ABI_UNSTABLE:BOOL
 
   **Default**: ``OFF``
 
   Build the "unstable" ABI version of libc++. Includes all ABI changing features
   on top of the current stable version.
 
+.. option:: LIBCXX_ABI_NAMESPACE:STRING
+
+  **Default**: ``__n`` where ``n`` is the current ABI version.
+
+  This option defines the name of the inline ABI versioning namespace. It can be used for building
+  custom versions of libc++ with unique symbol names in order to prevent conflicts or ODR issues
+  with other libc++ versions.
+
+  .. warning::
+    When providing a custom namespace, it's the users responsibility to ensure the name won't cause
+    conflicts with other names defined by libc++, both now and in the future.
+
 .. option:: LIBCXX_ABI_DEFINES:STRING
 
   **Default**: ``""``
 
   A semicolon-separated list of ABI macros to persist in the site config header.
   See ``include/__config`` for the list of ABI macros.
 
 .. _LLVM-specific variables:

include/__config

 #ifdef _LIBCPP_TRIVIAL_PAIR_COPY_CTOR
 #error "_LIBCPP_TRIVIAL_PAIR_COPY_CTOR" is no longer supported. \
        use _LIBCPP_DEPRECATED_ABI_DISABLE_PAIR_TRIVIAL_COPY_CTOR instead
 #endif
 
 #define _LIBCPP_CONCAT1(_LIBCPP_X,_LIBCPP_Y) _LIBCPP_X##_LIBCPP_Y
 #define _LIBCPP_CONCAT(_LIBCPP_X,_LIBCPP_Y) _LIBCPP_CONCAT1(_LIBCPP_X,_LIBCPP_Y)
 
-#define _LIBCPP_NAMESPACE _LIBCPP_CONCAT(__,_LIBCPP_ABI_VERSION)
+#ifndef _LIBCPP_ABI_NAMESPACE
+# define _LIBCPP_ABI_NAMESPACE _LIBCPP_CONCAT(__,_LIBCPP_ABI_VERSION)
+#endif
 
 #if __cplusplus < 201103L
 #define _LIBCPP_CXX03_LANG
 #endif
 
 #ifndef __has_attribute
 #define __has_attribute(__x) 0
 #endif

 #else
 #  define _LIBCPP_HIDE_FROM_ABI_AFTER_V1 _LIBCPP_HIDE_FROM_ABI
 #endif
 
 // Just so we can migrate to the new macros gradually.
 #define _LIBCPP_INLINE_VISIBILITY _LIBCPP_HIDE_FROM_ABI
 
 // Inline namespaces are available in Clang/GCC/MSVC regardless of C++ dialect.
-#define _LIBCPP_BEGIN_NAMESPACE_STD namespace std { inline namespace _LIBCPP_NAMESPACE {
+#define _LIBCPP_BEGIN_NAMESPACE_STD namespace std { inline namespace _LIBCPP_ABI_NAMESPACE {
 #define _LIBCPP_END_NAMESPACE_STD  } }
-#define _VSTD std::_LIBCPP_NAMESPACE
+#define _VSTD std::_LIBCPP_ABI_NAMESPACE
 _LIBCPP_BEGIN_NAMESPACE_STD _LIBCPP_END_NAMESPACE_STD
 
 #if _LIBCPP_STD_VER >= 17
 #define _LIBCPP_BEGIN_NAMESPACE_FILESYSTEM \
   _LIBCPP_BEGIN_NAMESPACE_STD inline namespace __fs { namespace filesystem {
 #else
 #define _LIBCPP_BEGIN_NAMESPACE_FILESYSTEM \
   _LIBCPP_BEGIN_NAMESPACE_STD namespace __fs { namespace filesystem {

include/__config_site.in

 #cmakedefine _LIBCPP_HAS_NO_THREAD_UNSAFE_C_FUNCTIONS
 #cmakedefine _LIBCPP_HAS_MUSL_LIBC
 #cmakedefine _LIBCPP_HAS_THREAD_API_PTHREAD
 #cmakedefine _LIBCPP_HAS_THREAD_API_EXTERNAL
 #cmakedefine _LIBCPP_HAS_THREAD_API_WIN32
 #cmakedefine _LIBCPP_HAS_THREAD_LIBRARY_EXTERNAL
 #cmakedefine _LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS
 #cmakedefine _LIBCPP_NO_VCRUNTIME
+#cmakedefine _LIBCPP_ABI_NAMESPACE @_LIBCPP_ABI_NAMESPACE@
 
 @_LIBCPP_ABI_DEFINES@
 
 #endif // _LIBCPP_CONFIG_SITE

src/iostream.cpp

 //===------------------------ iostream.cpp --------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "__std_stream"
 #include "string"
 #include "new"
 
 #define _str(s) #s
 #define str(s) _str(s)
-#define _LIBCPP_NAMESPACE_STR str(_LIBCPP_NAMESPACE)
+#define _LIBCPP_ABI_NAMESPACE_STR str(_LIBCPP_ABI_NAMESPACE)
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 #ifndef _LIBCPP_HAS_NO_STDIN
 _ALIGNAS_TYPE (istream) _LIBCPP_FUNC_VIS char cin[sizeof(istream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?cin@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_istream@DU?$char_traits@D@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?cin@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_istream@DU?$char_traits@D@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (__stdinbuf<char> ) static char __cin[sizeof(__stdinbuf <char>)];
 static mbstate_t mb_cin;
 _ALIGNAS_TYPE (wistream) _LIBCPP_FUNC_VIS char wcin[sizeof(wistream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?wcin@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_istream@_WU?$char_traits@_W@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?wcin@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_istream@_WU?$char_traits@_W@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (__stdinbuf<wchar_t> ) static char __wcin[sizeof(__stdinbuf <wchar_t>)];
 static mbstate_t mb_wcin;
 #endif
 
 #ifndef _LIBCPP_HAS_NO_STDOUT
 _ALIGNAS_TYPE (ostream) _LIBCPP_FUNC_VIS char cout[sizeof(ostream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?cout@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_ostream@DU?$char_traits@D@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?cout@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_ostream@DU?$char_traits@D@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (__stdoutbuf<char>) static char __cout[sizeof(__stdoutbuf<char>)];
 static mbstate_t mb_cout;
 _ALIGNAS_TYPE (wostream) _LIBCPP_FUNC_VIS char wcout[sizeof(wostream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?wcout@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_ostream@_WU?$char_traits@_W@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?wcout@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_ostream@_WU?$char_traits@_W@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (__stdoutbuf<wchar_t>) static char __wcout[sizeof(__stdoutbuf<wchar_t>)];
 static mbstate_t mb_wcout;
 #endif
 
 _ALIGNAS_TYPE (ostream) _LIBCPP_FUNC_VIS char cerr[sizeof(ostream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?cerr@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_ostream@DU?$char_traits@D@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?cerr@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_ostream@DU?$char_traits@D@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (__stdoutbuf<char>) static char __cerr[sizeof(__stdoutbuf<char>)];
 static mbstate_t mb_cerr;
 _ALIGNAS_TYPE (wostream) _LIBCPP_FUNC_VIS char wcerr[sizeof(wostream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?wcerr@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_ostream@_WU?$char_traits@_W@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?wcerr@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_ostream@_WU?$char_traits@_W@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (__stdoutbuf<wchar_t>) static char __wcerr[sizeof(__stdoutbuf<wchar_t>)];
 static mbstate_t mb_wcerr;
 
 _ALIGNAS_TYPE (ostream) _LIBCPP_FUNC_VIS char clog[sizeof(ostream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?clog@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_ostream@DU?$char_traits@D@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?clog@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_ostream@DU?$char_traits@D@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 _ALIGNAS_TYPE (wostream) _LIBCPP_FUNC_VIS char wclog[sizeof(wostream)]
 #if defined(_LIBCPP_ABI_MICROSOFT) && defined(__clang__)
-__asm__("?wclog@" _LIBCPP_NAMESPACE_STR "@std@@3V?$basic_ostream@_WU?$char_traits@_W@" _LIBCPP_NAMESPACE_STR "@std@@@12@A")
+__asm__("?wclog@" _LIBCPP_ABI_NAMESPACE_STR "@std@@3V?$basic_ostream@_WU?$char_traits@_W@" _LIBCPP_ABI_NAMESPACE_STR "@std@@@12@A")
 #endif
 ;
 
 _LIBCPP_HIDDEN ios_base::Init __start_std_streams;
 
 ios_base::Init::Init()
 {
 #ifndef _LIBCPP_HAS_NO_STDIN