[libcxx] Add testing infrastructure for EXTERNAL_THREAD_API mode

[ldionne]

Summary

Add example library to be able to test libcxx with EXTERNAL_THREAD_API mode.
Add cmake variable -DLIBCXX_TESTING_EXTERNAL_THREAD_API, and modify
testing configuration and build files, to be able to build libcxx in such mode.
Add cmake cache file for -DLIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY.

Co-authored-by: Mikhail Maltsev <[e-mail]>

[ldionne]

I would like this and https://reviews.llvm.org/D86599 to be a single review, since I see those two as making the support for an external threading library official.

libcxx/CMakeLists.txt

262

I would like this test to be enabled unconditionally. We should always be testing it, and I don't want to add more logic to the CMake just to enable a test for another configuration.

Also, since this is just adding a test, we should be able to do so without touching the main CMakeLists.txt at all. The only code changes should be under libcxx/test, and the test should run through Lit like the other ones. I recognize this might be more work, but stringing the test of a given configuration by hacking around libc++'s build itself is just not a sustainable direction.

[ldionne]

However, if that wasn't clear, I'm really happy we're making support for the external threading API official by adding tests -- otherwise I would have suggested dropping support for it as being untested. Thanks for working on that, I just think we need to nail down the proper way of doing it.

[miyuki]

libcxx/CMakeLists.txt

262

Normally enabling LIBCXX_HAS_EXTERNAL_THREAD_API requires the vendor to provide a version of __external_threading (e.g. like we do in the Arm Compiler toolchain) in the include directory. It is intended to be vendor-specific and unfortunately, it is needed at build time (not just during the test run).

LIBCXX_TESTING_EXTERNAL_THREAD_API makes the library use an example version of __external_threading from test/support/external-thread-api which relies on example_thread_library.c and its symbols. The intent is to check that the LIBCXX_HAS_EXTERNAL_THREAD_API mode works in principle.

So unfortunately I don't see a way to test the LIBCXX_HAS_EXTERNAL_THREAD_API configuration without modifications in the build logic of the library itself. @ldionne, what do you think?

[Nicu]

Update commit to include changes from https://reviews.llvm.org/D86599.
Update Summary to reflect this change.

[Nicu]

In D86704#2242141, @ldionne wrote:

I would like this and https://reviews.llvm.org/D86599 to be a single review, since I see those two as making the support for an external threading library official.

Merging of the both changes has been done.

@ldionne do you still think the change should be only modifying libcxx/test? As @miyuki mentioned, __external_threading is needed at build time so the main CMakeList.txt needs modification. In this case, the target cxx-external-threads which was previously working with LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY, has been used for LIBCXX_EXTERNAL_THREAD_API so that the build logic is similar.

In the case you still want these modifications to be enclosed only under libcxx/test, do you have any suggestions or ideas for such an approach?

[miyuki]

@ldionne, do you have any thoughts on how we might progress this forward?

[ldionne]

In D86704#2301405, @miyuki wrote:

@ldionne, do you have any thoughts on how we might progress this forward?

Sorry for the radio silence -- I was working on something which should make this (and more) a lot easier to test.

I recently checked in CI configuration files that allow running tests under various configurations. I believe one way to test this external threading API would be to add a builder that tests this configuration. You can see the list of current configurations here: libcxx/utils/ci/run-buildbot.sh. Given this new setup where build bots are checked into the repo, I think adding a bot that test this configuration would be a good way of making sure this doesn't regress. What do you think?

Also, a meta question: Is there a reason why your platform can't simply provide the pthread or the C11 threading APIs, and libc++ has to provide this bridge instead? It would seem better for your platform to provide a standardized API instead, no? I'm trying to understand why the external threading API mode is useful (it feels like I've asked that before though, so just point me to the answer if that's the case).

[miyuki]

Given this new setup where build bots are checked into the repo, I think adding a bot that test this configuration would be a good way of making sure this doesn't regress. What do you think?

Yes, setting up a build bot is part of my original plan. I will be on holiday for the next two weeks, but I do plan to continue working on this after returning.

Is there a reason why your platform can't simply provide the pthread or the C11 threading APIs, and libc++ has to provide this bridge instead?

The original intent of the external threading API was to make it usable with bare-metal real-time OS-es which tend not to have POSIX or POSIX-like APIs. For example, __libcpp_mutex_t can include the OS structure representing a mutex, instead of allocating it elsewhere.

[ldionne]

In D86704#2303842, @miyuki wrote:

Given this new setup where build bots are checked into the repo, I think adding a bot that test this configuration would be a good way of making sure this doesn't regress. What do you think?

Yes, setting up a build bot is part of my original plan. I will be on holiday for the next two weeks, but I do plan to continue working on this after returning.

Is there a reason why your platform can't simply provide the pthread or the C11 threading APIs, and libc++ has to provide this bridge instead?

The original intent of the external threading API was to make it usable with bare-metal real-time OS-es which tend not to have POSIX or POSIX-like APIs. For example, __libcpp_mutex_t can include the OS structure representing a mutex, instead of allocating it elsewhere.

Right -- my question is why is it not possible for these bare-metal real-time OS-es to implement these APIs? Or, alternatively, for whoever's trying to build libc++ on top of these systems to provide the right API as-if the system provided it. Doesn't this seem like a better solution than defining customization points at the libc++ level?

I'm not trying to push back -- this has been supported and we can continue supporting it, and I'm grateful for your desire to improve the test coverage in that area. However, I'm genuinely interested to find out whether/why this is the best solution.

[miyuki]

Customization allows storing data of OS structures such as mutexes and threads in-place, thus avoiding the need for additional dynamic memory allocations, this is important in embedded environments.

[ldionne]

[Github PR transition cleanup]

I'll abandon this because this wouldn't apply cleanly anymore, but most importantly because I think the future of supporting external threading configurations is not by keeping the external threading API the way it is today. I think it needs to be reworked to reduce the complexity it creates inside libc++ and I don't think it makes sense to add a testing configuration for it. Basically the way I envision this is that we'd move the "external threading API" from the current mechanism to a "vendor friendly" way of supporting different threading backends, like we do to define e.g. the different clocks or the different sources of randomness.

libcxx/CMakeLists.txt

 option(LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY
     "Build libc++ with an externalized threading library.
      This option may only be set to ON when LIBCXX_ENABLE_THREADS=ON" OFF)
+option(LIBCXX_TESTING_EXTERNAL_THREAD_API
+  "Build libc++ with an example externalized threading API provided
+   in the testsuite." OFF)
 
 # Misc options ----------------------------------------------------------------
 # FIXME: Turn -pedantic back ON. It is currently off because it warns

     message(FATAL_ERROR "LIBCXX_HAS_PTHREAD_API can only be set to ON"
                         " when LIBCXX_ENABLE_THREADS is also set to ON.")
   endif()
+  if(LIBCXX_TESTING_EXTERNAL_THREAD_API)
+    message(FATAL_ERROR "LIBCXX_TESTING_EXTERNAL_THREAD_API can only be set to "
+                        "ON when LIBCXX_ENABLE_THREADS is also set to ON.")
+  endif()
   if(LIBCXX_HAS_EXTERNAL_THREAD_API)
     message(FATAL_ERROR "LIBCXX_HAS_EXTERNAL_THREAD_API can only be set to ON"
                         " when LIBCXX_ENABLE_THREADS is also set to ON.")

 
 endif()
 
+if (LIBCXX_TESTING_EXTERNAL_THREAD_API)
+  # LIBCXX_TESTING_EXTERNAL_THREAD_API implies LIBCXX_HAS_EXTERNAL_THREAD_API
+  set(LIBCXX_HAS_EXTERNAL_THREAD_API ON)
+  if (LIBCXX_ENABLE_SHARED)
+    message(FATAL_ERROR "LIBCXX_TESTING_EXTERNAL_THREAD_API is currently "
+                        " incompatible with LIBCXX_ENABLE_SHARED")
+  endif()
+endif()
+
 if (LIBCXX_HAS_EXTERNAL_THREAD_API)
   if (LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY)
     message(FATAL_ERROR "The options LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY and "

   endif()
 
   if (LIBCXX_HAS_COMMENT_LIB_PRAGMA)
-    if (LIBCXX_HAS_PTHREAD_LIB)
+    if (LIBCXX_HAS_PTHREAD_LIB AND NOT LIBCXX_HAS_EXTERNAL_THREAD_API)
       target_compile_definitions(${target} PRIVATE -D_LIBCPP_LINK_PTHREAD_LIB)
     endif()
     if (LIBCXX_HAS_RT_LIB)
       target_compile_definitions(${target} PRIVATE -D_LIBCPP_LINK_RT_LIB)
     endif()
   endif()
+
+  if (LIBCXX_TESTING_EXTERNAL_THREAD_API)
+    target_include_directories(${target} PRIVATE
+      "${CMAKE_CURRENT_SOURCE_DIR}/../test/support/external-thread-api")
+  endif()
 endfunction()
 
 # Warning flags ===============================================================

   list(APPEND LIBCXX_TEST_DEPS cxx_experimental)
 endif()
 
-if (LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY)
+if (LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY OR LIBCXX_TESTING_EXTERNAL_THREAD_API)
   list(APPEND LIBCXX_TEST_DEPS cxx_external_threads)
 endif()
 

libcxx/cmake/caches/Build-ext-thread-lib.cmake

+set(LIBCXX_CXX_ABI libcxxabi CACHE STRING "")
+set(LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY ON CACHE BOOL "")

libcxx/cmake/caches/Test-ext-thread-api.cmake

+set(LIBCXX_ENABLE_STATIC ON CACHE BOOL "")
+set(LIBCXX_ENABLE_SHARED OFF CACHE BOOL "")
+set(LIBCXX_HAS_EXTERNAL_THREAD_API ON CACHE BOOL "")
+set(LIBCXX_TESTING_EXTERNAL_THREAD_API ON CACHE BOOL "")
+set(LIBCXX_ENABLE_EXPERIMENTAL_LIBRARY OFF CACHE BOOL "")

libcxx/src/CMakeLists.txt

   endif()
 endif()
 
+if (LIBCXX_TESTING_EXTERNAL_THREAD_API)
+  set(EXTERNAL_THREADING_HEADER
+    "${CMAKE_CURRENT_SOURCE_DIR}/../test/support/external-thread-api/__external_threading")
+  list(APPEND LIBCXX_SOURCES "${EXTERNAL_THREADING_HEADER}")
+  list(APPEND LIBCXX_HEADERS "${EXTERNAL_THREADING_HEADER}")
+endif()
+
 # Add all the headers to the project for IDEs.
 if (LIBCXX_CONFIGURE_IDE)
   file(GLOB_RECURSE LIBCXX_HEADERS ${CMAKE_CURRENT_SOURCE_DIR}/../include/*)

   cxx_add_common_build_flags(cxx_experimental)
 endif()
 
+if (LIBCXX_TESTING_EXTERNAL_THREAD_API)
+  set(LIBCXX_EXAMPLE_THREAD_API_IMPL
+      "${CMAKE_CURRENT_SOURCE_DIR}/../test/support/external-thread-api/example_thread_library.c")
+    add_library(cxx_external_threads STATIC ${LIBCXX_EXAMPLE_THREAD_API_IMPL})
+  set_target_properties(cxx_external_threads
+    PROPERTIES
+      LINK_FLAGS    "${LIBCXX_LINK_FLAGS}"
+      COMPILE_FLAGS "${LIBCXX_COMPILE_FLAGS}"
+      OUTPUT_NAME   "c++external_threads"
+  )
+  target_link_libraries(cxx_external_threads PRIVATE pthread)
+endif()
 
 if (LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY)
   set(LIBCXX_EXTERNAL_THREADING_SUPPORT_SOURCES

libcxx/test/CMakeLists.txt

 pythonize_bool(LIBCXX_HAS_ATOMIC_LIB)
 pythonize_bool(LIBCXX_HAVE_CXX_ATOMICS_WITH_LIB)
 pythonize_bool(LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY)
+pythonize_bool(LIBCXX_TESTING_EXTERNAL_THREAD_API)
 pythonize_bool(LIBCXX_DEBUG_BUILD)
 pythonize_bool(LIBCXX_ENABLE_PARALLEL_ALGORITHMS)
 

libcxx/test/configs/legacy.cfg.in

 config.has_libatomic            = @LIBCXX_HAS_ATOMIC_LIB@
 config.debug_build              = @LIBCXX_DEBUG_BUILD@
 config.libcxxabi_shared         = @LIBCXX_LINK_TESTS_WITH_SHARED_LIBCXXABI@
-config.cxx_ext_threads          = @LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY@
+config.cxx_ext_thread_lib       = @LIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY@
+config.cxx_ext_thread_api       = @LIBCXX_TESTING_EXTERNAL_THREAD_API@
 config.pstl_src_root            = "@ParallelSTL_SOURCE_DIR@" if @LIBCXX_ENABLE_PARALLEL_ALGORITHMS@ else None
 config.pstl_obj_root            = "@ParallelSTL_BINARY_DIR@" if @LIBCXX_ENABLE_PARALLEL_ALGORITHMS@ else None
 config.libcxx_gdb               = "@LIBCXX_GDB@"

libcxx/test/support/external-thread-api/__external_threading

+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef __EXTERNAL_THREADING
+#define __EXTERNAL_THREADING
+
+#include <__config>
+
+#if defined(_LIBCPP_HAS_NO_THREADS)
+#error "__external_threading included when _LIBCPP_HAS_NO_THREADS is defined"
+#endif
+
+#if !defined(_LIBCPP_HAS_THREAD_API_EXTERNAL)
+#error "__external_threading included but _LIBCPP_HAS_THREAD_API_EXTERNAL not defined"
+#endif
+
+#if defined(_LIBCPP_HAS_THREAD_API_C11)
+#error "__external_threading is incompatible with _LIBCPP_HAS_THREAD_API_C11"
+#endif
+
+#if defined(_LIBCPP_HAS_THREAD_API_PTHREAD)
+#error "__external_threading is incompatible with _LIBCPP_HAS_THREAD_API_WIN32"
+#endif
+
+#if defined(_LIBCPP_LINK_PTHREAD_LIB)
+#error "libc++ must not be linked with pthread when external threading is used"
+#endif
+
+#if defined(_LIBCPP_HAS_THREAD_LIBRARY_EXTERNAL) || \
+    defined(_LIBCPP_BUILDING_THREAD_LIBRARY_EXTERNAL)
+#error "__external_threading is intended to be used with _LIBCPP_HAS_THREAD_API_EXTERNAL" \
+       " not _LIBCPP_HAS_THREAD_LIBRARY_EXTERNAL"
+#endif
+
+#include <chrono>
+#include <cerrno>
+#include <limits.h>
+#include "example_thread_library.h"
+
+#define _LIBCPP_NO_NATIVE_SEMAPHORES
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#define _LIBCPP_THREAD_ABI_VISIBILITY _LIBCPP_FUNC_VIS
+
+// Type definitions
+// Clocks
+typedef __example_timespec_t __libcpp_timespec_t;
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_clock_realtime(__libcpp_timespec_t* ts) {
+  return __example_clock_realtime(ts);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_clock_monotonic(__libcpp_timespec_t* ts) {
+  return __example_clock_monotonic(ts);
+}
+
+// Mutex
+typedef __example_mutex_t __libcpp_mutex_t;
+typedef __example_mutex_t __libcpp_recursive_mutex_t;
+
+#define _LIBCPP_MUTEX_INITIALIZER _EXAMPLE_MUTEX_INITIALIZER
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_recursive_mutex_init(__libcpp_recursive_mutex_t* __m) {
+  return __example_recursive_mutex_init(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_recursive_mutex_lock(__libcpp_recursive_mutex_t *__m) {
+  return __example_recursive_mutex_lock(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+bool __libcpp_recursive_mutex_trylock(__libcpp_recursive_mutex_t *__m) {
+  return __example_recursive_mutex_trylock(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_recursive_mutex_unlock(__libcpp_recursive_mutex_t *__m) {
+  return __example_recursive_mutex_unlock(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_recursive_mutex_destroy(__libcpp_recursive_mutex_t *__m) {
+  return __example_recursive_mutex_destroy(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_mutex_lock(__libcpp_mutex_t* __m) {
+  return __example_mutex_lock(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_mutex_trylock(__libcpp_mutex_t* __m) {
+  return __example_mutex_trylock(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_mutex_unlock(__libcpp_mutex_t* __m) {
+  return __example_mutex_unlock(__m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_mutex_destroy(__libcpp_mutex_t* __m) {
+  return __example_mutex_destroy(__m);
+}
+
+// Condition variable
+typedef __example_condvar_t __libcpp_condvar_t;
+
+#define _LIBCPP_CONDVAR_INITIALIZER _EXAMPLE_CONDVAR_INITIALIZER
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_condvar_signal(__libcpp_condvar_t* __cv) {
+  return __example_condvar_signal(__cv);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_condvar_broadcast(__libcpp_condvar_t* __cv) {
+  return __example_condvar_broadcast(__cv);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_condvar_wait(__libcpp_condvar_t* __cv, __libcpp_mutex_t* __m) {
+  return __example_condvar_wait(__cv, __m);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_condvar_timedwait(__libcpp_condvar_t* __cv, __libcpp_mutex_t* __m,
+    __libcpp_timespec_t* __ts) {
+  return __example_condvar_timedwait(__cv, __m, __ts);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_condvar_destroy(__libcpp_condvar_t* __cv) {
+  return __example_condvar_destroy(__cv);
+}
+
+// Execute once
+typedef __example_exec_once_flag __libcpp_exec_once_flag;
+#define _LIBCPP_EXEC_ONCE_INITIALIZER _EXAMPLE_EXEC_ONCE_INITIALIZER
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_execute_once(__libcpp_exec_once_flag *flag,
+    void (*init_routine)(void)) {
+  return __example_execute_once(flag, init_routine);
+}
+
+// Thread id
+typedef __example_thread_id __libcpp_thread_id;
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+bool __libcpp_thread_id_equal(__libcpp_thread_id t1, __libcpp_thread_id t2) {
+  return __example_thread_id_equal(t1, t2);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+bool __libcpp_thread_id_less(__libcpp_thread_id t1, __libcpp_thread_id t2) {
+  return __example_thread_id_less(t1, t2);
+}
+
+// Thread
+typedef __example_thread_t __libcpp_thread_t;
+
+#define _LIBCPP_NULL_THREAD _EXAMPLE_NULL_THREAD
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+bool __libcpp_thread_isnull(const __libcpp_thread_t* __t) {
+  return __example_thread_isnull(__t);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_thread_create(__libcpp_thread_t* __t, void* (*__func)(void*), void* __arg) {
+  return __example_thread_create(__t, __func, __arg);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+__libcpp_thread_id __libcpp_thread_get_current_id() {
+  return __example_thread_get_current_id();
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+__libcpp_thread_id __libcpp_thread_get_id(const __libcpp_thread_t* __t) {
+  return __example_thread_get_id(__t);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_thread_join(__libcpp_thread_t* __t) {
+  return __example_thread_join(__t);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_thread_detach(__libcpp_thread_t* __t) {
+  return __example_thread_detach(__t);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+void __libcpp_thread_yield() {
+  __example_thread_yield();
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+void __libcpp_thread_sleep_for(const chrono::nanoseconds& __ns)
+{
+  using namespace chrono;
+  seconds __s = duration_cast<seconds>(__ns);
+  __libcpp_timespec_t __ts;
+  typedef decltype(__ts.tv_sec) ts_sec;
+  _LIBCPP_CONSTEXPR ts_sec __ts_sec_max = LONG_MAX;
+
+  if (__s.count() < __ts_sec_max)
+  {
+    __ts.tv_sec = static_cast<ts_sec>(__s.count());
+    __ts.tv_nsec = static_cast<decltype(__ts.tv_nsec)>((__ns - __s).count());
+  }
+  else
+  {
+    __ts.tv_sec = __ts_sec_max;
+    __ts.tv_nsec = 999999999; // (10^9 - 1)
+  }
+
+  while (__example_thread_nanosleep(&__ts, &__ts) == -1 && errno == EINTR);
+}
+
+// Thread local storage
+typedef __example_tls_key __libcpp_tls_key;
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_tls_create(__libcpp_tls_key* __key, void (*__at_exit)(void*)) {
+  return __example_tls_create(__key, __at_exit);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+void* __libcpp_tls_get(__libcpp_tls_key __key) {
+  return __example_tls_get(__key);
+}
+
+inline _LIBCPP_THREAD_ABI_VISIBILITY
+int __libcpp_tls_set(__libcpp_tls_key __key, void* __p) {
+  return __example_tls_set(__key, __p);
+}
+
+#define _LIBCPP_TLS_DESTRUCTOR_CC
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // __EXTERNAL_THREADING

libcxx/test/support/external-thread-api/example_thread_library.h

+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef EXAMPLE_THREAD_LIBRARY_H
+#define EXAMPLE_THREAD_LIBRARY_H
+
+#include <stdint.h>
+#include <limits.h>
+#include <time.h>
+#include <pthread.h>
+#include <semaphore.h>
+
+#define _EXAMPLE_THREAD_ABI_VISIBILITY __attribute__((visibility("default")))
+
+#define _EXAMPLE_MUTEX_INITIALIZER PTHREAD_MUTEX_INITIALIZER
+#define _EXAMPLE_CONDVAR_INITIALIZER PTHREAD_COND_INITIALIZER
+#define _EXAMPLE_NULL_THREAD 0
+#define _EXAMPLE_EXEC_ONCE_INITIALIZER PTHREAD_ONCE_INIT
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef struct timespec __example_timespec_t;
+
+// Clocks
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_clock_realtime(__example_timespec_t* __ts);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_clock_monotonic(__example_timespec_t* __ts);
+
+// Mutex
+typedef pthread_mutex_t __example_mutex_t;
+typedef pthread_mutex_t __example_recursive_mutex_t;
+
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_recursive_mutex_init(__example_recursive_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_recursive_mutex_lock(__example_recursive_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_recursive_mutex_trylock(__example_recursive_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_recursive_mutex_unlock(__example_recursive_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_recursive_mutex_destroy(__example_recursive_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_mutex_lock(__example_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_mutex_trylock(__example_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_mutex_unlock(__example_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_mutex_destroy(__example_mutex_t* __m);
+
+// Condition variable
+typedef pthread_cond_t __example_condvar_t;
+
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_condvar_signal(__example_condvar_t* __cv);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_condvar_broadcast(__example_condvar_t* __cv);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_condvar_wait(__example_condvar_t* __cv, __example_mutex_t* __m);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_condvar_timedwait(__example_condvar_t* __cv,
+                                __example_mutex_t* __m,
+                                __example_timespec_t* __ts);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_condvar_destroy(__example_condvar_t* __cv);
+
+// Thread ids
+typedef uint32_t __example_thread_id;
+
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_id_equal(__example_thread_id __tid1,
+                              __example_thread_id __tid2);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_id_less(__example_thread_id __tid1,
+                             __example_thread_id __tid2);
+
+// Threads
+typedef pthread_t __example_thread_t;
+
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_isnull(const __example_thread_t* __t);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_create(__example_thread_t* __t, void* (*__func)(void*),
+                            void* __arg);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+__example_thread_id __example_thread_get_current_id();
+_EXAMPLE_THREAD_ABI_VISIBILITY
+__example_thread_id __example_thread_get_id(const __example_thread_t* __t);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_join(__example_thread_t* __t);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_detach(__example_thread_t* __t);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+void __example_thread_yield();
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_thread_nanosleep(const __example_timespec_t* req,
+                               __example_timespec_t* rem);
+
+// Exec once
+typedef pthread_once_t __example_exec_once_flag;
+
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_execute_once(__example_exec_once_flag* __flag,
+                           void (*__init_routine)(void));
+
+// Thread local storage
+typedef uint32_t __example_tls_key;
+
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_tls_create(__example_tls_key* __key, void (*__at_exit)(void*));
+_EXAMPLE_THREAD_ABI_VISIBILITY
+void* __example_tls_get(__example_tls_key __key);
+_EXAMPLE_THREAD_ABI_VISIBILITY
+int __example_tls_set(__example_tls_key __key, void* __p);
+
+#ifdef __cplusplus
+} // Extern C
+#endif
+
+#endif //EXAMPLE_THREAD_LIBRARY_H

libcxx/test/support/external-thread-api/example_thread_library.c

+// -*- C -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "example_thread_library.h"
+
+//Clocks
+
+int __example_clock_monotonic(__example_timespec_t* tp) {
+  return clock_gettime(CLOCK_MONOTONIC, tp);
+}
+
+int __example_clock_realtime(__example_timespec_t* tp) {
+  return clock_gettime(CLOCK_REALTIME, tp);
+}
+
+// Mutex
+int __example_recursive_mutex_init(__example_recursive_mutex_t* __m) {
+  pthread_mutexattr_t attr;
+  int __ec = pthread_mutexattr_init(&attr);
+  if (__ec)
+    return __ec;
+  __ec = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
+  if (__ec) {
+    pthread_mutexattr_destroy(&attr);
+    return __ec;
+  }
+  __ec = pthread_mutex_init(__m, &attr);
+  if (__ec) {
+    pthread_mutexattr_destroy(&attr);
+    return __ec;
+  }
+  __ec = pthread_mutexattr_destroy(&attr);
+  if (__ec) {
+    pthread_mutex_destroy(__m);
+    return __ec;
+  }
+  return 0;
+}
+
+int __example_recursive_mutex_lock(__example_recursive_mutex_t* __m) {
+  return pthread_mutex_lock(__m);
+}
+
+int __example_recursive_mutex_trylock(__example_recursive_mutex_t* __m) {
+  return pthread_mutex_trylock(__m) == 0;
+}
+
+int __example_recursive_mutex_unlock(__example_recursive_mutex_t* __m) {
+  return pthread_mutex_unlock(__m);
+}
+
+int __example_recursive_mutex_destroy(__example_recursive_mutex_t* __m) {
+  return pthread_mutex_destroy(__m);
+}
+
+int __example_mutex_lock(__example_mutex_t* __m) {
+  return pthread_mutex_lock(__m);
+}
+
+int __example_mutex_trylock(__example_mutex_t* __m) {
+  return pthread_mutex_trylock(__m) == 0;
+}
+
+int __example_mutex_unlock(__example_mutex_t* __m) {
+  return pthread_mutex_unlock(__m);
+}
+
+int __example_mutex_destroy(__example_mutex_t* __m) {
+  return pthread_mutex_destroy(__m);
+}
+
+// Condition variable
+int __example_condvar_signal(__example_condvar_t* __cv) {
+  return pthread_cond_signal(__cv);
+}
+
+int __example_condvar_broadcast(__example_condvar_t* __cv) {
+  return pthread_cond_broadcast(__cv);
+}
+
+int __example_condvar_wait(__example_condvar_t* __cv, __example_mutex_t* __m) {
+  return pthread_cond_wait(__cv, __m);
+}
+
+int __example_condvar_timedwait(__example_condvar_t* __cv,
+                                __example_mutex_t* __m,
+                                __example_timespec_t* __ts) {
+  return pthread_cond_timedwait(__cv, __m, __ts);
+}
+
+int __example_condvar_destroy(__example_condvar_t* __cv) {
+  return pthread_cond_destroy(__cv);
+}
+
+// Thread ids
+// Returns non-zero if the thread ids are equal, otherwise 0
+int __example_thread_id_equal(__example_thread_id t1, __example_thread_id t2) {
+  return pthread_equal(t1, t2) != 0;
+}
+
+// Returns non-zero if t1 < t2, otherwise 0
+int __example_thread_id_less(__example_thread_id t1, __example_thread_id t2) {
+  return t1 < t2;
+}
+
+// Thread
+int __example_thread_isnull(const __example_thread_t* __t) {
+  return *__t == _EXAMPLE_NULL_THREAD;
+}
+
+int __example_thread_create(__example_thread_t* __t, void* (*__func)(void*),
+                            void* __arg) {
+  return pthread_create(__t, 0, __func, __arg);
+}
+
+__example_thread_id __example_thread_get_current_id() { return pthread_self(); }
+
+__example_thread_id __example_thread_get_id(const __example_thread_t* __t) {
+  return *__t;
+}
+
+int __example_thread_join(__example_thread_t* __t) {
+  return pthread_join(*__t, 0);
+}
+
+int __example_thread_detach(__example_thread_t* __t) {
+  return pthread_detach(*__t);
+}
+
+void __example_thread_yield() { sched_yield(); }
+
+int __example_thread_nanosleep(const __example_timespec_t* __req,
+                               __example_timespec_t* __rem) {
+
+  return nanosleep(__req, __rem);
+}
+
+int __example_execute_once(__example_exec_once_flag* __flag,
+                           void (*__init_routine)(void)) {
+  return pthread_once(__flag, __init_routine);
+}
+
+// Thread local storage
+int __example_tls_create(__example_tls_key* __key, void (*__at_exit)(void*)) {
+  return pthread_key_create(__key, __at_exit);
+}
+
+void* __example_tls_get(__example_tls_key __key) {
+  return pthread_getspecific(__key);
+}
+
+int __example_tls_set(__example_tls_key __key, void* __p) {
+  return pthread_setspecific(__key, __p);
+}

libcxx/utils/libcxx/test/config.py

             self.cxx.compile_flags += ['-I' + os.path.join(pstl_src_root, 'test')]
             self.config.available_features.add('parallel-algorithms')
 
+        # Add external threading library
+        if self.get_lit_bool('cxx_ext_thread_api'):
+            ext_thread_path = os.path.join(self.libcxx_src_root, 'test/support/external-thread-api')
+            self.cxx.compile_flags += ['-I' + ext_thread_path]
+
     def configure_compile_flags_header_includes(self):
         support_path = os.path.join(self.libcxx_src_root, 'test', 'support')
         self.configure_config_site_header()

                 'C++ ABI setting %s unsupported for tests' % cxx_abi)
 
     def configure_extra_library_flags(self):
-        if self.get_lit_bool('cxx_ext_threads', default=False):
+        if (self.get_lit_bool('cxx_ext_thread_lib', default=False) or
+            self.get_lit_bool('cxx_ext_thread_api', default=False)):
             self.cxx.link_flags += ['-lc++external_threads']
         self.target_info.add_cxx_link_flags(self.cxx.link_flags)