[libcxxabi] Introduce an externally threaded libc++abi variant (take-2)

[rmaprath]

Summary

This is a replacement for D27204. Re-worked so that we use the threading API of libcxx instead of creating a whole new API for libcxxabi.

[EricWF]

This LGTM. I'll approve after the inline comments are addressed.

CMakeLists.txt

121 ↗

(On Diff #80763)

Maybe use a dependent option that sets it to the value of LIBCXX_ENABLE_THREADS when it is defined?

src/fallback_malloc.cpp

37 ↗

(On Diff #80763)

Can't we replace this class with std::mutex directly?

test/test_exception_storage.pass.cpp

42 ↗

(On Diff #80763)

What happened to the initializer?

[rmaprath]

CMakeLists.txt

121 ↗

(On Diff #80763)

Are you referring to CMAKE_DEPENDENT_OPTION macro? I don't see this macro used anywhere else in llvm, but I suppose that's not a concern?

Although, it would have to be something like:

CMAKE_DEPENDENT_OPTION(LIBCXXABI_HAS_EXTERNAL_THREAD_API "Externally Threaded libcxxabi" OFF  "LIBCXX_ENABLE_THREADS" OFF)

Because, LIBCXX_ENABLE_THREADS=ON should not mean LIBCXXABI_HAS_EXTERNAL_THREAD_API=ON.

src/fallback_malloc.cpp

37 ↗

(On Diff #80763)

Again, I should've included more context to the patch :(

The mutexor here is supposed to degrade to a nop when threading is disabled. I think we cannot use std::mutex without threading support.

Will update the patch with more context.

test/test_exception_storage.pass.cpp

42 ↗

(On Diff #80763)

I don't think it needs one here?

std::__licpp_thread_create() does not require std::__libcpp_thread_t argument to be statically initialized.

[rmaprath]

Patch updated with more context.

[rmaprath]

Gentle ping.

[rmaprath]

Ping.

[EricWF]

LGTM. I'll re-review the mutexor changes post-commit.

CMakeLists.txt

121 ↗

(On Diff #80763)

Ah OK. This looks good as is then.

Although it would be really nice if LIBCXX_HAS_EXTERNAL_THREAD_API=ON also enabled this option unless otherwise specified, but I don't see an easy way to do that.

src/fallback_malloc.cpp

37 ↗

(On Diff #80763)

We cannot use std::mutex without threading support but I would still rather use it.

I would typedef std::lock_guard<std::mutex> mutexor when threading is enabled and otherwise I would just #ifdef all usages away.

Also please add _LIBCPP_SAFE_STATIC to the heap_mutex declaration.

[rmaprath]

src/fallback_malloc.cpp

37 ↗

(On Diff #80763)

Hmmm, using std::mutex here breaks the shared library builds of libcxxabi. The problem is that methods like std::mutex::lock() and ~std::mutex() are only declared in the <mutex> header, with the implementation going into the dylib.

I was just about to update the patch because it worked for all my other configurations, but those configurations are either static builds or ones that suppress the -Wl, -z defs linker option passed to the shared library builds by default (this is an inherited option).

Perhaps it's time to get rid of -Wl, -z defs for all the libcxxabi configurations? That makes sense if libcxxabi is inherently dependent on libcxx.

[EricWF]

Why don't we commit this as is and we can deal with the mutex later?

libcxxabi/trunk/CMakeLists.txt

 option(LIBCXXABI_ENABLE_EXCEPTIONS "Use exceptions." ON)
 option(LIBCXXABI_ENABLE_ASSERTIONS "Enable assertions independent of build mode." ON)
 option(LIBCXXABI_ENABLE_PEDANTIC "Compile with pedantic enabled." ON)
 option(LIBCXXABI_ENABLE_WERROR "Fail and stop if a warning is triggered." OFF)
 option(LIBCXXABI_USE_LLVM_UNWINDER "Build and use the LLVM unwinder." OFF)
 option(LIBCXXABI_USE_COMPILER_RT "Use compiler-rt instead of libgcc" OFF)
 option(LIBCXXABI_ENABLE_THREADS "Build with threads enabled" ON)
 option(LIBCXXABI_HAS_PTHREAD_API "Ignore auto-detection and force use of pthread API" OFF)
+option(LIBCXXABI_HAS_EXTERNAL_THREAD_API
+  "Build libc++abi with an externalized threading API.
+  This option may only be set to ON when LIBCXXABI_ENABLE_THREADS=ON." OFF)
 option(LIBCXXABI_BUILD_32_BITS "Build 32 bit libc++abi." ${LLVM_BUILD_32_BITS})
 set(LIBCXXABI_TARGET_TRIPLE "" CACHE STRING "Target triple for cross compiling.")
 set(LIBCXXABI_GCC_TOOLCHAIN "" CACHE PATH "GCC toolchain for cross compiling.")
 set(LIBCXXABI_SYSROOT "" CACHE PATH "Sysroot for cross compiling.")
 set(LIBCXXABI_LIBCXX_LIBRARY_PATH "" CACHE PATH "The path to libc++ library.")
 
 # Default to building a shared library so that the default options still test
 # the libc++abi that is being built. There are two problems with testing a
if (NOT uppercase_CMAKE_BUILD_TYPE STREQUAL "RELEASE")
     list(APPEND LIBCXXABI_COMPILE_FLAGS -DNDEBUG)
   endif()
 endif()
 # Static library
 if (NOT LIBCXXABI_ENABLE_SHARED)
   list(APPEND LIBCXXABI_COMPILE_FLAGS -D_LIBCPP_BUILD_STATIC)
 endif()
 
+# Threading
 if (NOT LIBCXXABI_ENABLE_THREADS)
   if (LIBCXXABI_HAS_PTHREAD_API)
     message(FATAL_ERROR "LIBCXXABI_HAS_PTHREAD_API can only"
                         " be set to ON when LIBCXXABI_ENABLE_THREADS"
                         " is also set to ON.")
   endif()
+  if (LIBCXXABI_HAS_EXTERNAL_THREAD_API)
+    message(FATAL_ERROR "LIBCXXABI_HAS_EXTERNAL_THREAD_API can only"
+                        " be set to ON when LIBCXXABI_ENABLE_THREADS"
+                        " is also set to ON.")
+  endif()
   add_definitions(-D_LIBCXXABI_HAS_NO_THREADS)
 endif()
 
+if (LIBCXXABI_HAS_PTHREAD_API AND LIBCXXABI_HAS_EXTERNAL_THREAD_API)
+  message(FATAL_ERROR "The options LIBCXXABI_HAS_EXTERNAL_THREAD_API"
+                      "and LIBCXXABI_HAS_PTHREAD_API cannot be both"
+                      "set to ON at the same time.")
+endif()
+
+if (LIBCXXABI_HAS_EXTERNAL_THREAD_API AND LIBCXXABI_ENABLE_SHARED)
+  # Need to allow unresolved symbols if this is to work with shared library builds
+  if (APPLE)
+    add_link_flags("-undefined dynamic_lookup")
+  else()
+    # Relax this restriction from HandleLLVMOptions
+    string(REPLACE "-Wl,-z,defs" "" CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS}")
+  endif()
+endif()
+
 if (LIBCXXABI_HAS_PTHREAD_API)
   add_definitions(-D_LIBCPP_HAS_THREAD_API_PTHREAD)
   add_definitions(-D_LIBCXXABI_USE_THREAD_API_PTHREAD)
 endif()
 
+if (LIBCXXABI_HAS_EXTERNAL_THREAD_API)
+  add_definitions(-D_LIBCXXABI_HAS_THREAD_API_EXTERNAL)
+endif()
+
 if (MSVC)
   add_definitions(-D_CRT_SECURE_NO_WARNINGS)
 endif()
 
 # Define LIBCXXABI_USE_LLVM_UNWINDER for conditional compilation.
 if (LIBCXXABI_USE_LLVM_UNWINDER)
   add_definitions(-DLIBCXXABI_USE_LLVM_UNWINDER=1)
 endif()

libcxxabi/trunk/src/config.h

#if __has_attribute(__internal_linkage__)
   #define _LIBCXXABI_INLINE_VISIBILITY __attribute__ ((__internal_linkage__, __always_inline__))
  #else
   #define _LIBCXXABI_INLINE_VISIBILITY __attribute__ ((__visibility__("hidden"), __always_inline__))
  #endif
 #endif
 
 // Try and deduce a threading api if one has not been explicitly set.
 #if !defined(_LIBCXXABI_HAS_NO_THREADS) && \
+    !defined(_LIBCXXABI_HAS_THREAD_API_EXTERNAL) && \
     !defined(_LIBCXXABI_USE_THREAD_API_PTHREAD)
   #if defined(_POSIX_THREADS) && _POSIX_THREADS >= 0
     #define _LIBCXXABI_USE_THREAD_API_PTHREAD
   #else
     #error "No thread API"
   #endif
 #endif
 
 // Set this in the CXXFLAGS when you need it, because otherwise we'd have to
 // #if !defined(__linux__) && !defined(__APPLE__) && ...
 // and so-on for *every* platform.
 #ifndef LIBCXXABI_BAREMETAL
 #  define LIBCXXABI_BAREMETAL 0
 #endif
 
 #endif // LIBCXXABI_CONFIG_H

libcxxabi/trunk/src/cxa_exception.cpp

 //===------------------------- cxa_exception.cpp --------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //  
 //  This file implements the "Exception Handling APIs"
 //  http://mentorembedded.github.io/cxx-abi/abi-eh.html
 //  
 //===----------------------------------------------------------------------===//
 
 #include "config.h"
-#include "threading_support.h"
 #include "cxxabi.h"
 
 #include <exception>        // for std::terminate
 #include <cstring>          // for memset
 #include "cxa_exception.hpp"
 #include "cxa_handlers.hpp"
 #include "fallback_malloc.h"
 

libcxxabi/trunk/src/cxa_exception_storage.cpp

 //===--------------------- cxa_exception_storage.cpp ----------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //  
 //  This file implements the storage for the "Caught Exception Stack"
 //  http://mentorembedded.github.io/cxx-abi/abi-eh.html (section 2.2.2)
 //  
 //===----------------------------------------------------------------------===//
 
 #include "cxa_exception.hpp"
 
 #include "config.h"
-#include "threading_support.h"
+#include <__threading_support>
 
 #if defined(_LIBCXXABI_HAS_NO_THREADS)
 
 namespace __cxxabiv1 {
 extern "C" {
     static __cxa_eh_globals eh_globals;
     __cxa_eh_globals *__cxa_get_globals() { return &eh_globals; }
     __cxa_eh_globals *__cxa_get_globals_fast() { return &eh_globals; }

 #include "fallback_malloc.h"
 
 //  In general, we treat all threading errors as fatal.
 //  We cannot call std::terminate() because that will in turn
 //  call __cxa_get_globals() and cause infinite recursion.
 
 namespace __cxxabiv1 {
 namespace {
-    __libcxxabi_tls_key key_;
-    __libcxxabi_exec_once_flag flag_ = _LIBCXXABI_EXEC_ONCE_INITIALIZER;
+    std::__libcpp_tls_key key_;
+    std::__libcpp_exec_once_flag flag_ = _LIBCPP_EXEC_ONCE_INITIALIZER;
 
     void destruct_ (void *p) {
         __free_with_fallback ( p );
-        if ( 0 != __libcxxabi_tls_set ( key_, NULL ) )  
+        if ( 0 != std::__libcpp_tls_set ( key_, NULL ) )
             abort_message("cannot zero out thread value for __cxa_get_globals()");
         }
 
     void construct_ () {
-        if ( 0 != __libcxxabi_tls_create ( &key_, destruct_ ) )
+        if ( 0 != std::__libcpp_tls_create ( &key_, destruct_ ) )
             abort_message("cannot create thread specific key for __cxa_get_globals()");
         }
 }   
 
 extern "C" {
     __cxa_eh_globals * __cxa_get_globals () {
     //  Try to get the globals for this thread
         __cxa_eh_globals* retVal = __cxa_get_globals_fast ();
     
     //  If this is the first time we've been asked for these globals, create them
         if ( NULL == retVal ) {
             retVal = static_cast<__cxa_eh_globals*>
                         (__calloc_with_fallback (1, sizeof (__cxa_eh_globals)));
             if ( NULL == retVal )
                 abort_message("cannot allocate __cxa_eh_globals");
-            if ( 0 != __libcxxabi_tls_set ( key_, retVal ) )
-               abort_message("__libcxxabi_tls_set failure in __cxa_get_globals()");
+            if ( 0 != std::__libcpp_tls_set ( key_, retVal ) )
+               abort_message("std::__libcpp_tls_set failure in __cxa_get_globals()");
            }
         return retVal;
         }
 
     // Note that this implementation will reliably return NULL if not
     // preceded by a call to __cxa_get_globals().  This is an extension
     // to the Itanium ABI and is taken advantage of in several places in
     // libc++abi.
     __cxa_eh_globals * __cxa_get_globals_fast () {
     //  First time through, create the key.
-        if (0 != __libcxxabi_execute_once(&flag_, construct_))
+        if (0 != std::__libcpp_execute_once(&flag_, construct_))
             abort_message("execute once failure in __cxa_get_globals_fast()");
 //        static int init = construct_();
-        return static_cast<__cxa_eh_globals*>(__libcxxabi_tls_get(key_));
+        return static_cast<__cxa_eh_globals*>(std::__libcpp_tls_get(key_));
         }
     
 }
 }
 #endif

libcxxabi/trunk/src/cxa_guard.cpp

 //===---------------------------- cxa_guard.cpp ---------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "__cxxabi_config.h"
 
 #include "abort_message.h"
 #include "config.h"
-#include "threading_support.h"
+#include <__threading_support>
 
 #include <stdint.h>
 
 /*
     This implementation must be careful to not call code external to this file
     which will turn around and try to call __cxa_guard_acquire reentrantly.
     For this reason, the headers of this file are as restricted as possible.
     Previous implementations of this code for __APPLE__ have used
-    __libcxxabi_mutex_lock and the abort_message utility without problem.  This
-    implementation also uses __libcxxabi_condvar_wait which has tested
+    std::__libcpp_mutex_lock and the abort_message utility without problem. This
+    implementation also uses std::__libcpp_condvar_wait which has tested
     to not be a problem.
 */
 
 namespace __cxxabiv1
 {
 
 namespace
 {
bool is_initialized(guard_type* guard_object) {
     char* initialized = (char*)guard_object;
     return *initialized;
 }
 
 #endif
 #endif
 
 #ifndef _LIBCXXABI_HAS_NO_THREADS
-__libcxxabi_mutex_t guard_mut = _LIBCXXABI_MUTEX_INITIALIZER;
-__libcxxabi_condvar_t guard_cv = _LIBCXXABI_CONDVAR_INITIALIZER;
+std::__libcpp_mutex_t guard_mut = _LIBCPP_MUTEX_INITIALIZER;
+std::__libcpp_condvar_t guard_cv = _LIBCPP_CONDVAR_INITIALIZER;
 #endif
 
 #if defined(__APPLE__) && !defined(__arm__)
 
 typedef uint32_t lock_type;
 
 #if __LITTLE_ENDIAN__
 

 }  // unnamed namespace
 
 extern "C"
 {
 
 #ifndef _LIBCXXABI_HAS_NO_THREADS
 _LIBCXXABI_FUNC_VIS int __cxa_guard_acquire(guard_type *guard_object) {
     char* initialized = (char*)guard_object;
-    if (__libcxxabi_mutex_lock(&guard_mut))
+    if (std::__libcpp_mutex_lock(&guard_mut))
         abort_message("__cxa_guard_acquire failed to acquire mutex");
     int result = *initialized == 0;
     if (result)
     {
 #if defined(__APPLE__) && !defined(__arm__)
-        const lock_type id = __libcxxabi_thread_get_port();
+        const lock_type id = std::__libcpp_thread_get_port();
         lock_type lock = get_lock(*guard_object);
         if (lock)
         {
             // if this thread set lock for this same guard_object, abort
             if (lock == id)
                 abort_message("__cxa_guard_acquire detected deadlock");
             do
             {
-                if (__libcxxabi_condvar_wait(&guard_cv, &guard_mut))
+                if (std::__libcpp_condvar_wait(&guard_cv, &guard_mut))
                     abort_message("__cxa_guard_acquire condition variable wait failed");
                 lock = get_lock(*guard_object);
             } while (lock);
             result = !is_initialized(guard_object);
             if (result)
                 set_lock(*guard_object, id);
         }
         else
             set_lock(*guard_object, id);
 #else  // !__APPLE__ || __arm__
         while (get_lock(*guard_object))
-            if (__libcxxabi_condvar_wait(&guard_cv, &guard_mut))
+            if (std::__libcpp_condvar_wait(&guard_cv, &guard_mut))
                 abort_message("__cxa_guard_acquire condition variable wait failed");
         result = *initialized == 0;
         if (result)
             set_lock(*guard_object, true);
 #endif  // !__APPLE__ || __arm__
     }
-    if (__libcxxabi_mutex_unlock(&guard_mut))
+    if (std::__libcpp_mutex_unlock(&guard_mut))
         abort_message("__cxa_guard_acquire failed to release mutex");
     return result;
 }
 
 _LIBCXXABI_FUNC_VIS void __cxa_guard_release(guard_type *guard_object) {
-    if (__libcxxabi_mutex_lock(&guard_mut))
+    if (std::__libcpp_mutex_lock(&guard_mut))
         abort_message("__cxa_guard_release failed to acquire mutex");
     *guard_object = 0;
     set_initialized(guard_object);
-    if (__libcxxabi_mutex_unlock(&guard_mut))
+    if (std::__libcpp_mutex_unlock(&guard_mut))
         abort_message("__cxa_guard_release failed to release mutex");
-    if (__libcxxabi_condvar_broadcast(&guard_cv))
+    if (std::__libcpp_condvar_broadcast(&guard_cv))
         abort_message("__cxa_guard_release failed to broadcast condition variable");
 }
 
 _LIBCXXABI_FUNC_VIS void __cxa_guard_abort(guard_type *guard_object) {
-    if (__libcxxabi_mutex_lock(&guard_mut))
+    if (std::__libcpp_mutex_lock(&guard_mut))
         abort_message("__cxa_guard_abort failed to acquire mutex");
     *guard_object = 0;
-    if (__libcxxabi_mutex_unlock(&guard_mut))
+    if (std::__libcpp_mutex_unlock(&guard_mut))
         abort_message("__cxa_guard_abort failed to release mutex");
-    if (__libcxxabi_condvar_broadcast(&guard_cv))
+    if (std::__libcpp_condvar_broadcast(&guard_cv))
         abort_message("__cxa_guard_abort failed to broadcast condition variable");
 }
 
 #else // _LIBCXXABI_HAS_NO_THREADS
 
 _LIBCXXABI_FUNC_VIS int __cxa_guard_acquire(guard_type *guard_object) {
     return !is_initialized(guard_object);
 }

libcxxabi/trunk/src/cxa_thread_atexit.cpp

 //===----------------------- cxa_thread_atexit.cpp ------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "abort_message.h"
 #include "cxxabi.h"
-#include "threading_support.h"
+#include <__threading_support>
 #include <cstdlib>
 
 namespace __cxxabiv1 {
 
   using Dtor = void(*)(void*);
 
   extern "C"
 #ifndef HAVE___CXA_THREAD_ATEXIT_IMPL
namespace {
   // - dso_symbol is ignored.  This means that a shared library may be unloaded
   //   (via dlclose()) before its thread_local destructors have run.
   //
   // - thread_local destructors for the main thread are run by the destructor of
   //   a static object.  This is later than expected; they should run before the
   //   destructors of any objects with static storage duration.
   //
   // - thread_local destructors on non-main threads run on the first iteration
-  //   through the __libcxxabi_tls_key destructors.
+  //   through the __libccpp_tls_key destructors.
   //   std::notify_all_at_thread_exit() and similar functions must be careful to
   //   wait until the second iteration to provide their intended ordering
   //   guarantees.
   //
   // Another limitation, though one shared with ..._impl(), is that any
   // thread_locals that are first initialized after non-thread_local global
   // destructors begin to run will not be destroyed.  [basic.start.term] states
   // that all thread_local destructors are sequenced before the destruction of
struct DtorList {
     DtorList* next;
   };
 
   // The linked list of thread-local destructors to run
   __thread DtorList* dtors = nullptr;
   // True if the destructors are currently scheduled to run on this thread
   __thread bool dtors_alive = false;
   // Used to trigger destructors on thread exit; value is ignored
-  __libcxxabi_tls_key dtors_key;
+  std::__libcpp_tls_key dtors_key;
 
   void run_dtors(void*) {
     while (auto head = dtors) {
       dtors = head->next;
       head->dtor(head->obj);
       std::free(head);
     }
 
     dtors_alive = false;
   }
 
   struct DtorsManager {
     DtorsManager() {
-      // There is intentionally no matching __libcxxabi_tls_delete call, as
+      // There is intentionally no matching std::__libcpp_tls_delete call, as
       // __cxa_thread_atexit() may be called arbitrarily late (for example, from
       // global destructors or atexit() handlers).
-      if (__libcxxabi_tls_create(&dtors_key, run_dtors) != 0) {
-        abort_message("__libcxxabi_tls_create() failed in __cxa_thread_atexit()");
+      if (std::__libcpp_tls_create(&dtors_key, run_dtors) != 0) {
+        abort_message("std::__libcpp_tls_create() failed in __cxa_thread_atexit()");
       }
     }
 
     ~DtorsManager() {
-      // __libcxxabi_tls_key destructors do not run on threads that call exit()
+      // std::__libcpp_tls_key destructors do not run on threads that call exit()
       // (including when the main thread returns from main()), so we explicitly
       // call the destructor here.  This runs at exit time (potentially earlier
       // if libc++abi is dlclose()'d).  Any thread_locals initialized after this
       // point will not be destroyed.
       run_dtors(nullptr);
     }
   };
 } // namespace
 
 #endif // HAVE___CXA_THREAD_ATEXIT_IMPL
 
 extern "C" {
 
   _LIBCXXABI_FUNC_VIS int __cxa_thread_atexit(Dtor dtor, void* obj, void* dso_symbol) throw() {
 #ifdef HAVE___CXA_THREAD_ATEXIT_IMPL
     return __cxa_thread_atexit_impl(dtor, obj, dso_symbol);
 #else
     if (__cxa_thread_atexit_impl) {
       return __cxa_thread_atexit_impl(dtor, obj, dso_symbol);
     } else {
-      // Initialize the dtors __libcxxabi_tls_key (uses __cxa_guard_*() for
+      // Initialize the dtors std::__libcpp_tls_key (uses __cxa_guard_*() for
       // one-time initialization and __cxa_atexit() for destruction)
       static DtorsManager manager;
 
       if (!dtors_alive) {
-        if (__libcxxabi_tls_set(dtors_key, &dtors_key) != 0) {
+        if (std::__libcpp_tls_set(dtors_key, &dtors_key) != 0) {
           return -1;
         }
         dtors_alive = true;
       }
 
       auto head = static_cast<DtorList*>(std::malloc(sizeof(DtorList)));
       if (!head) {
         return -1;

libcxxabi/trunk/src/fallback_malloc.cpp

 //===------------------------ fallback_malloc.cpp -------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "fallback_malloc.h"
 
 #include "config.h"
-#include "threading_support.h"
+#include <__threading_support>
 
 #include <cstdlib> // for malloc, calloc, free
 #include <cstring> // for memset
 
 //  A small, simple heap manager based (loosely) on
 //  the startup heap manager from FreeBSD, optimized for space.
 //
 //  Manages a fixed-size memory pool, supports malloc and free only.
 //  No support for realloc.
 //
 //  Allocates chunks in multiples of four bytes, with a four byte header
 //  for each chunk. The overhead of each chunk is kept low by keeping pointers
 //  as two byte offsets within the heap, rather than (4 or 8 byte) pointers.
 
 namespace {
 
 // When POSIX threads are not available, make the mutex operations a nop
 #ifndef _LIBCXXABI_HAS_NO_THREADS
-static __libcxxabi_mutex_t heap_mutex = _LIBCXXABI_MUTEX_INITIALIZER;
+_LIBCPP_SAFE_STATIC
+static std::__libcpp_mutex_t heap_mutex = _LIBCPP_MUTEX_INITIALIZER;
 #else
 static void * heap_mutex = 0;
 #endif
 
 class mutexor {
 public:
 #ifndef _LIBCXXABI_HAS_NO_THREADS
-    mutexor ( __libcxxabi_mutex_t *m ) : mtx_(m) { __libcxxabi_mutex_lock ( mtx_ ); }
-    ~mutexor () { __libcxxabi_mutex_unlock ( mtx_ ); }
+    mutexor ( std::__libcpp_mutex_t *m ) : mtx_(m) {
+      std::__libcpp_mutex_lock ( mtx_ );
+    }
+    ~mutexor () { std::__libcpp_mutex_unlock ( mtx_ ); }
 #else
     mutexor ( void * ) {}
     ~mutexor () {}
 #endif
 private:
     mutexor ( const mutexor &rhs );
     mutexor & operator = ( const mutexor &rhs );
 #ifndef _LIBCXXABI_HAS_NO_THREADS
-    __libcxxabi_mutex_t *mtx_;
+    std::__libcpp_mutex_t *mtx_;
 #endif
-    };
 
 
 static const size_t HEAP_SIZE = 512;
 char heap [ HEAP_SIZE ] __attribute__((aligned));
 
 typedef unsigned short heap_offset;
 typedef unsigned short heap_size;
 

libcxxabi/trunk/src/threading_support.h

-//===------------------------ threading_support.h -------------------------===//
-//
-//                     The LLVM Compiler Infrastructure
-//
-// This file is dual licensed under the MIT and the University of Illinois Open
-// Source Licenses. See LICENSE.TXT for details.
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef _LIBCXXABI_THREADING_SUPPORT_H
-#define _LIBCXXABI_THREADING_SUPPORT_H
-
-#include "__cxxabi_config.h"
-#include "config.h"
-
-#ifndef _LIBCXXABI_HAS_NO_THREADS
-
-#if defined(_LIBCXXABI_USE_THREAD_API_PTHREAD)
-#include <pthread.h>
-
-#define _LIBCXXABI_THREAD_ABI_VISIBILITY inline _LIBCXXABI_INLINE_VISIBILITY
-
-// Mutex
-typedef pthread_mutex_t __libcxxabi_mutex_t;
-#define _LIBCXXABI_MUTEX_INITIALIZER PTHREAD_MUTEX_INITIALIZER
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_mutex_lock(__libcxxabi_mutex_t *mutex) {
-  return pthread_mutex_lock(mutex);
-}
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_mutex_unlock(__libcxxabi_mutex_t *mutex) {
-  return pthread_mutex_unlock(mutex);
-}
-
-// Condition variable
-typedef pthread_cond_t __libcxxabi_condvar_t;
-#define _LIBCXXABI_CONDVAR_INITIALIZER PTHREAD_COND_INITIALIZER
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_condvar_wait(__libcxxabi_condvar_t *cv,
-                             __libcxxabi_mutex_t *mutex) {
-  return pthread_cond_wait(cv, mutex);
-}
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_condvar_broadcast(__libcxxabi_condvar_t *cv) {
-  return pthread_cond_broadcast(cv);
-}
-
-// Execute once
-typedef pthread_once_t __libcxxabi_exec_once_flag;
-#define _LIBCXXABI_EXEC_ONCE_INITIALIZER PTHREAD_ONCE_INIT
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_execute_once(__libcxxabi_exec_once_flag *flag,
-                             void (*init_routine)(void)) {
-  return pthread_once(flag, init_routine);
-}
-
-// Thread id
-#if defined(__APPLE__) && !defined(__arm__)
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-mach_port_t __libcxxabi_thread_get_port()
-{
-    return pthread_mach_thread_np(pthread_self());
-}
-#endif
-
-// Thread
-typedef pthread_t __libcxxabi_thread_t;
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_thread_create(__libcxxabi_thread_t* __t,
-                           void* (*__func)(void*), void* __arg)
-{
-    return pthread_create(__t, 0, __func, __arg);
-}
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_thread_join(__libcxxabi_thread_t* __t)
-{
-    return pthread_join(*__t, 0);
-}
-
-// TLS
-typedef pthread_key_t __libcxxabi_tls_key;
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_tls_create(__libcxxabi_tls_key *key,
-                           void (*destructor)(void *)) {
-  return pthread_key_create(key, destructor);
-}
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-void *__libcxxabi_tls_get(__libcxxabi_tls_key key) {
-  return pthread_getspecific(key);
-}
-
-_LIBCXXABI_THREAD_ABI_VISIBILITY
-int __libcxxabi_tls_set(__libcxxabi_tls_key key, void *value) {
-  return pthread_setspecific(key, value);
-}
-#endif // _LIBCXXABI_USE_THREAD_API_PTHREAD
-#endif // !_LIBCXXABI_HAS_NO_THREADS
-#endif // _LIBCXXABI_THREADING_SUPPORT_H

libcxxabi/trunk/test/CMakeLists.txt

 endif()
 
 pythonize_bool(LIBCXXABI_BUILD_32_BITS)
 pythonize_bool(LIBCXX_ENABLE_SHARED)
 pythonize_bool(LIBCXXABI_ENABLE_SHARED)
 pythonize_bool(LIBCXXABI_ENABLE_THREADS)
 pythonize_bool(LIBCXXABI_ENABLE_EXCEPTIONS)
 pythonize_bool(LIBCXXABI_USE_LLVM_UNWINDER)
+pythonize_bool(LIBCXXABI_HAS_EXTERNAL_THREAD_API)
 set(LIBCXXABI_TARGET_INFO "libcxx.test.target_info.LocalTI" CACHE STRING
     "TargetInfo to use when setting up test environment.")
 set(LIBCXXABI_EXECUTOR "None" CACHE STRING
     "Executor to use when running tests.")
 
 set(AUTO_GEN_COMMENT "## Autogenerated by libcxxabi configuration.\n# Do not edit!")
 configure_file(
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
   ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg
   @ONLY)
 
 if (LIBCXXABI_ENABLE_SHARED)
   set(LIBCXXABI_TEST_DEPS cxxabi_shared)
 else()
   set(LIBCXXABI_TEST_DEPS cxxabi_static)
 endif()
 
+if (LIBCXXABI_HAS_EXTERNAL_THREAD_API)
+  list(APPEND LIBCXXABI_TEST_DEPS cxx_external_threads)
+endif()
+
 if (NOT LIBCXXABI_STANDALONE_BUILD)
   list(APPEND LIBCXXABI_TEST_DEPS cxx)
   if (LIBCXXABI_USE_LLVM_UNWINDER)
     list(APPEND LIBCXXABI_TEST_DEPS unwind)
   endif()
 endif()
 
 add_lit_testsuite(check-cxxabi "Running libcxxabi tests"
   ${CMAKE_CURRENT_BINARY_DIR}
   DEPENDS ${LIBCXXABI_TEST_DEPS}
   )
 
 # TODO: This is a legacy target name and should be removed at some point.
 add_custom_target(check-libcxxabi DEPENDS check-cxxabi)

libcxxabi/trunk/test/libcxxabi/test/config.py

def has_cpp_feature(self, feature, required_value):
         return int(self.cxx.dumpMacros().get('__cpp_' + feature, 0)) >= required_value
 
     def configure_features(self):
         super(Configuration, self).configure_features()
         if not self.get_lit_bool('enable_exceptions', True):
             self.config.available_features.add('libcxxabi-no-exceptions')
         if not self.has_cpp_feature('noexcept_function_type', 201510):
             self.config.available_features.add('libcxxabi-no-noexcept-function-type')
+        # test_exception_storage_nodynmem.pass.cpp fails under this specific configuration
+        if self.get_lit_bool('cxx_ext_threads', False) and self.get_lit_bool('libcxxabi_shared', False):
+            self.config.available_features.add('libcxxabi-shared-externally-threaded')
 
     def configure_compile_flags(self):
         self.cxx.compile_flags += ['-DLIBCXXABI_NO_TIMER']
         if self.get_lit_bool('enable_exceptions', True):
             self.cxx.compile_flags += ['-funwind-tables']
         else:
             self.cxx.compile_flags += ['-fno-exceptions', '-DLIBCXXABI_HAS_NO_EXCEPTIONS']
         if not self.get_lit_bool('enable_threads', True):

libcxxabi/trunk/test/lit.site.cfg.in

 config.target_info              = "@LIBCXXABI_TARGET_INFO@"
 config.executor                 = "@LIBCXXABI_EXECUTOR@"
 config.libcxxabi_shared         = "@LIBCXXABI_ENABLE_SHARED@"
 config.enable_shared            = "@LIBCXX_ENABLE_SHARED@"
 config.enable_exceptions        = "@LIBCXXABI_ENABLE_EXCEPTIONS@"
 config.host_triple              = "@LLVM_HOST_TRIPLE@"
 config.target_triple            = "@TARGET_TRIPLE@"
 config.use_target               = len("@LIBCXXABI_TARGET_TRIPLE@") > 0
+config.cxx_ext_threads          = "@LIBCXXABI_HAS_EXTERNAL_THREAD_API@"
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LIBCXXABI_SOURCE_DIR@/test/lit.cfg")

libcxxabi/trunk/test/test_exception_storage.pass.cpp

 //===-------------------- test_exception_storage.cpp ----------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "../src/config.h"
 
 #include <cstdlib>
 #include <algorithm>
 #include <iostream>
-#include "../src/threading_support.h"
+#include <__threading_support>
 #include <unistd.h>
 
 #include "../src/cxa_exception.hpp"
 
 typedef __cxxabiv1::__cxa_eh_globals globals_t ;
 
 void *thread_code (void *parm) {
     size_t *result = (size_t *) parm;
void *thread_code (void *parm) {
     *result = (size_t) glob1;
     sleep ( 1 );
     return parm;
     }
 
 #ifndef _LIBCXXABI_HAS_NO_THREADS
 #define NUMTHREADS  10
 size_t                 thread_globals [ NUMTHREADS ] = { 0 };
-__libcxxabi_thread_t   threads        [ NUMTHREADS ];
+std::__libcpp_thread_t   threads        [ NUMTHREADS ];
 #endif
 
 int main () {
     int retVal = 0;
 
 #ifndef _LIBCXXABI_HAS_NO_THREADS
 //  Make the threads, let them run, and wait for them to finish
     for ( int i = 0; i < NUMTHREADS; ++i )
-        __libcxxabi_thread_create ( threads + i, thread_code, (void *) (thread_globals + i));
+        std::__libcpp_thread_create ( threads + i, thread_code, (void *) (thread_globals + i));
     for ( int i = 0; i < NUMTHREADS; ++i )
-        __libcxxabi_thread_join ( &threads [ i ] );
+        std::__libcpp_thread_join ( &threads [ i ] );
 
     for ( int i = 0; i < NUMTHREADS; ++i )
         if ( 0 == thread_globals [ i ] ) {
             std::cerr << "Thread #" << i << " had a zero global" << std::endl;
             retVal = 1;
             }
         
     std::sort ( thread_globals, thread_globals + NUMTHREADS );

libcxxabi/trunk/test/test_fallback_malloc.pass.cpp

 //===--------------------- test_fallback_malloc.cpp -----------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include <iostream>
 #include <deque>
 
-#include "../src/threading_support.h"
+#include <__threading_support>
 
 typedef std::deque<void *> container;
 
 // #define  DEBUG_FALLBACK_MALLOC
 #define INSTRUMENT_FALLBACK_MALLOC
 #include "../src/fallback_malloc.cpp"
 
 container alloc_series ( size_t sz ) {