[libc] Add simple implementations of mtx_lock and mtx_unlock.

[sivachandra]

Summary

These functions only support locking and unlocking of plain mutexes.
They will be extended in future changes to handled recursive and timed
mutexes.

Depends on D75380.

[sivachandra]

libc/src/threads/linux/mtx_lock.cpp

17

Though stdatomic.h is not a free standing header, we will use the one provided by the compilers. LLVM-libc will not provide it.

[MaskRay]

A mutex implementation will need congestion counts. If only a shim is needed, a bi-state mutex is sufficient.

I think pthread_mutex_timedlock and pthread_mutex_unlock can be implemented first, then rebase mtx_lock and mtx_unlock on pthread. A libc eventually has to implement pthread to be usable on a POSIX platform. It can be a shim, with just the basic functionality. Even if you don't want to think of pthread initially, mtx_lock should be based on mtx_timedlock.

We can assume Linux>=2.6.22 and use FUTEX_PRIVATE_FLAG.

[sivachandra]

In D74653#1877328, @MaskRay wrote:

A mutex implementation will need congestion counts. If only a shim is needed, a bi-state mutex is sufficient.

A tri-state mutex helps us avoid an unnecessary futext syscall.

I think pthread_mutex_timedlock and pthread_mutex_unlock can be implemented first, then rebase mtx_lock and mtx_unlock on pthread. A libc eventually has to implement pthread to be usable on a POSIX platform. It can be a shim, with just the basic functionality. Even if you don't want to think of pthread initially, mtx_lock should be based on mtx_timedlock.

In LLVM libc, we want to take the opposite approach of implementing pthread as an extension over standard C thread library.

Agree mtx_lock could be a specialization of mtx_timedlock. As I said in the description, this patch is only a simple implementation of a plain mutex. It allows me to avoid worrying about he timespec API for now. Having a plain mutex available paves way for adding and testing other parts of the libc system.

We can assume Linux>=2.6.22 and use FUTEX_PRIVATE_FLAG.

Agreed. I was not sure, but since you also bring it up, I can go ahead with that change. I will update in the next round.

[abrachet]

In D74653#1877381, @sivachandra wrote:

In D74653#1877328, @MaskRay wrote:

I think pthread_mutex_timedlock and pthread_mutex_unlock can be implemented first, then rebase mtx_lock and mtx_unlock on pthread. A libc eventually has to implement pthread to be usable on a POSIX platform. It can be a shim, with just the basic functionality. Even if you don't want to think of pthread initially, mtx_lock should be based on mtx_timedlock.

In LLVM libc, we want to take the opposite approach of implementing pthread as an extension over standard C thread library.

My intuition (I've never really used <threads.h>) is that it will be more difficult than the opposite because pthread provides many more options than C11's thread library. I think this patch is good and it'll be useful to have mutex's but my guess is when we implement pthread it will become untenable to use what we currently have.

libc/include/threads.h.def

10–11

LIBC_THREADS_H

libc/lib/CMakeLists.txt

15

You could add a comment here like the rest of them have, but I don't think its needed, its just more cohesive.

libc/src/threads/linux/mtx_lock.cpp

28

I think it would be more readable if this looked like

if (atomic_compare_exchange_strong(futex_data, &mutex_status, MS_Locked))
  return thrd_success;
switch(mutex_status) {
...
}

libc/src/threads/linux/mtx_unlock.cpp

28

The cast should no longer be necessary.

33

no body -> nobody

libc/src/threads/linux/mutex.h

18

We could make this enum class MutexStatus and drop the MS_ prefix and refer to these as MutexStatus::Free perhaps.

20

Prefer using.

[sivachandra]

Address comments.

[sivachandra]

libc/src/threads/linux/mutex.h

18

That would be nice, but then we will need casts to integer types as stdatomic.h API wants integer types.

[phosek]

Would it be possible to land the enumeration support separately? These are separate changes and I think it'd be cleaner to land them as two commits.

libc/config/linux/api.td

146

Do you plan to graduate these eventually to a common file (and layer Linux on top)? These API aren't Linux specific and we would like to provide these definitions on other platforms like Fuchsia.

libc/test/src/threads/linux/mtx_test.cpp

17

Can you make these constexpr?

[sivachandra]

• Seperate enum generation to a different patch
• Use threads from LLVM libc to construct mutex tests
• Remove use of sleep from tests

[sivachandra]

In D74653#1892046, @phosek wrote:

Would it be possible to land the enumeration support separately? These are separate changes and I think it'd be cleaner to land them as two commits.

Done now: https://reviews.llvm.org/D75379

libc/config/linux/api.td

146

Yes. When we have another platform file, we will put the common parts in a common file.

[phosek]

LGTM

libc/config/linux/api.td

let TypeDeclarations = [
     OffT,
   ];
 
   let Functions = [
     "mmap",
     "munmap",
   ];
 }
+
+def MtxT : TypeDecl<"mtx_t"> {
+  let Decl = [{
+    struct mtx_t {
+      unsigned char __internal_data[4];
+      int __mtx_type;
+    };
+  }];
+}
+
+def ThreadsAPI : PublicAPI<"threads.h"> {
+  let TypeDeclarations = [
+    MtxT,
+  ];
+
+  let Enumerations = [
+    "mtx_plain",
+    "mtx_recursive",
+    "mtx_timed",
+    "thrd_timedout",
+    "thrd_success",
+    "thrd_busy",
+    "thrd_error",
+    "thrd_nomem",
+  ];
+
+  let Functions = [
+    "mtx_init",
+    "mtx_lock",
+    "mtx_unlock",
+  ];
+}

libc/config/public_api.td

let Defn = !strconcat("#ifndef ", name, "\n",
                         "#define " , name, " ", value, "\n",
                         "#endif // ", name);
 }
 
 class PublicAPI<string name> {
   string HeaderName = name;
   list<MacroDef> Macros = [];
   list<TypeDecl> TypeDeclarations = [];
+  list<string> Enumerations = [];
   list<string> Structs = [];
   list<string> Functions = [];
 }

libc/include/CMakeLists.txt

add_gen_header(
   string_h
   DEF_FILE string.h.def
   GEN_HDR string.h
   DEPENDS
     llvm_libc_common_h
 )
 
 add_gen_header(
+  threads_h
+  DEF_FILE threads.h.def
+  GEN_HDR threads.h
+  DEPENDS
+    llvm_libc_common_h
+)
+
+add_gen_header(
   errno_h
   DEF_FILE errno.h.def
   PARAMS
     platform_errno=../config/${LIBC_TARGET_OS}/errno.h.in
   GEN_HDR errno.h
   DATA_FILES
     ../config/${LIBC_TARGET_OS}/errno.h.in
 )

libc/include/threads.h.def

+//===---------------- C standard library header threads.h -----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_THREADS_H
+#define LLVM_LIBC_THREADS_H
+
+#include <__llvm-libc-common.h>
+
+%%public_api()
+
+#endif // LLVM_LIBC_THREADS_H

libc/lib/CMakeLists.txt

 
 add_entrypoint_library(
   llvmlibc
   DEPENDS
     # errno.h entrypoints
     __errno_location
 
     # string.h entrypoints
     strcpy
     strcat
 
     # sys/mman.h entrypoints
     mmap
     munmap
+
+    # threads.h entrypoints.
+    mtx_init
+    mtx_lock
+    mtx_unlock
 )
 
 add_entrypoint_library(
   llvmlibm
   DEPENDS
     # math.h entrypoints
     round
 )
 
 add_redirector_library(
   llvmlibc_redirectors
   DEPENDS
     round_redirector
 )

libc/spec/linux.td

HeaderSpec Errno = HeaderSpec<
         Macro<"ESOCKTNOSUPPORT">,
         Macro<"ENOTNAM">,
         Macro<"ELNRNG">,
         Macro<"EL2NSYNC">,
         Macro<"EADV">,
         Macro<"ECOMM">,
       ],
       [], // Types
+      [], // Enumerations
       []  // Functions
   >;
 
   HeaderSpec SysMMan = HeaderSpec<
       "sys/mman.h",
       [Macro<"MAP_ANONYMOUS">],
       [], // Types
+      [], // Enumerations
       []  // Functions
   >;
 
   let Headers = [
     Errno,
     SysMMan,
   ];
 }

libc/spec/posix.td

HeaderSpec Errno = HeaderSpec<
         Macro<"ESTALE">,
         Macro<"ETIME">,
         Macro<"ETIMEDOUT">,
         Macro<"ETXTBSY">,
         Macro<"EWOULDBLOCK">,
         Macro<"EXDEV">,
       ],
       [], // Types
+      [], // Enumerations
       []  // Functions
   >;
 
   HeaderSpec SysMMan = HeaderSpec<
       "sys/mman.h",
       [
         // TODO: Add a facility to bunch macros into bitwise-or-able groups.
         // POSIX requires it, so such thing should be captured in this spec.
         Macro<"PROT_EXEC">,
         Macro<"PROT_NONE">,
         Macro<"PROT_READ">,
         Macro<"PROT_WRITE">,
 
         Macro<"MAP_FIXED">,
         Macro<"MAP_PRIVATE">,
         Macro<"MAP_SHARED">,
 
         Macro<"MAP_FAILED">,
       ],
       [
         SizeTType,
         OffTType,
       ],
+      [], // Enumerations
       [
         FunctionSpec<
             "mmap",
             RetValSpec<VoidPtr>,
             [ArgSpec<VoidPtr>,
              ArgSpec<SizeTType>,
              ArgSpec<IntType>,
              ArgSpec<IntType>,

libc/spec/spec.td

 class Type {}
 
 class NamedType<string name> : Type {
   string Name = name;
 }
 
 class Field<string name, Type type> {
   string Name = name;
   Type FieldType = type;
 }
 
 // Class to describe concrete structs specified by a standard.
 class Struct<string name> : NamedType<name> {
   list<Field> Fields;
 }
 
+class EnumNameValue<string name, string value = "__default_enum_value__"> {
+  string Name = name;
+  string Value = value;
+}
+
+class Enum<string name, list<EnumNameValue> enumerations> : NamedType<name> {
+  list<EnumNameValue> Enumerations = enumerations;
+}
+
 class PtrType<Type type> : Type {
   Type PointeeType = type;
 }
 
 class ConstType<Type type> : Type {
   Type UnqualifiedType = type;
 }
 

 // Common types
 def VoidPtr : PtrType<VoidType>;
 def SizeTType : NamedType<"size_t">;
 
 class Macro<string name> {
   string Name = name;
 }
 
+class EnumeratedNameValue<string name, string value = "__default__"> {
+  string Name = name;
+  string Value = value;
+}
+
 class Annotation {}
 
 class RetValSpec<Type type, list<Annotation> annotations = []> {
   Type ReturnType = type;
   list<Annotation> Annotations = annotations;
 }
 
 class ArgSpec<Type type, list<Annotation> annotations = [], string name = ""> {
   Type ArgType = type;
   list<Annotation> Annotations = annotations;
   string Name = name;
 }
 
 class FunctionSpec<string name, RetValSpec return, list<ArgSpec> args> {
   string Name = name;
   RetValSpec Return = return;
   list<ArgSpec> Args = args;
 }
 
 class HeaderSpec<string name,
                 list<Macro> macros,
                 list<Type> types,
+                list<EnumeratedNameValue> enumerations,
                 list<FunctionSpec> functions> {
   string Name = name;
   list<FunctionSpec> Functions = functions;
   list<Type> Types = types;
   list<Macro> Macros = macros;
+  list<EnumeratedNameValue> Enumerations = enumerations;
 }
 
 class StandardSpec<string name> {
   string Name = name;
   list<HeaderSpec> Headers;
 }

libc/spec/stdc.td

 def StdC : StandardSpec<"stdc"> {
   ConstType ConstVoidPtr = ConstType<VoidPtr>;
   RestrictedPtrType VoidRestrictedPtr = RestrictedPtrType<VoidType>;
   ConstType ConstVoidRestrictedPtr = ConstType<VoidRestrictedPtr>;
 
   PtrType CharPtr = PtrType<CharType>;
   ConstType ConstCharPtr = ConstType<CharPtr>;
   RestrictedPtrType CharRestrictedPtr = RestrictedPtrType<CharType>;
   ConstType ConstCharRestrictedPtr = ConstType<CharRestrictedPtr>;
 
+  NamedType MtxTType = NamedType<"mtx_t">;
+  PtrType MtxTTypePtr = PtrType<MtxTType>;
+
   HeaderSpec String = HeaderSpec<
       "string.h",
       [
           Macro<"NULL">,
       ],
       [
           SizeTType,
       ],
+      [], // Enumerations
       [
           FunctionSpec<
               "memcpy",
               RetValSpec<VoidPtr>, 
               [ArgSpec<VoidRestrictedPtr>,
                ArgSpec<ConstVoidRestrictedPtr>,
                ArgSpec<SizeTType>]
           >,
def StdC : StandardSpec<"stdc"> {
 
   HeaderSpec Math = HeaderSpec<
       "math.h",
       [], // Macros
       [
           NamedType<"float_t">,
           NamedType<"double_t">,
       ],
+      [], // Enumerations
       [
           FunctionSpec<"acos", RetValSpec<DoubleType>, [ArgSpec<DoubleType>]>,
           FunctionSpec<"acosl", RetValSpec<LongDoubleType>, [ArgSpec<LongDoubleType>]>,
       ]
   >;
 
   HeaderSpec StdIO = HeaderSpec<
       "stdio.h",
       [], // Macros
       [ // Types
           SizeTType,
       ],
+      [], // Enumerations
       [
           FunctionSpec<
               "snprintf",
               RetValSpec<IntType>,
               [ArgSpec<CharPtr>,
                ArgSpec<SizeTType>,
                ArgSpec<ConstCharRestrictedPtr>,
                ArgSpec<VarArgType>]
           >,
       ]
   >;
 
   HeaderSpec Errno = HeaderSpec<
       "errno.h",
       [
         Macro<"errno">,
         Macro<"EDOM">,
         Macro<"EILSEQ">,
         Macro<"ERANGE">,
       ],
       [], // Types
+      [], // Enumerations
       []  // Functions
   >;
 
+  HeaderSpec Threads = HeaderSpec<
+      "threads.h",
+      [], // Macros
+      [
+          MtxTType,
+      ],
+      [
+          EnumeratedNameValue<"mtx_plain">,
+          EnumeratedNameValue<"mtx_recursive">,
+          EnumeratedNameValue<"mtx_timed">,
+          EnumeratedNameValue<"thrd_timedout">,
+          EnumeratedNameValue<"thrd_success">,
+          EnumeratedNameValue<"thrd_busy">,
+          EnumeratedNameValue<"thrd_error">,
+          EnumeratedNameValue<"thrd_nomem">,
+      ],
+      [
+          FunctionSpec<
+              "mtx_init",
+              RetValSpec<IntType>,
+              [
+                  ArgSpec<MtxTTypePtr>,
+                  ArgSpec<IntType>,
+              ]
+          >,
+          FunctionSpec<
+              "mtx_lock",
+              RetValSpec<IntType>,
+              [
+                  ArgSpec<MtxTTypePtr>,
+              ]
+          >,
+          FunctionSpec<
+              "mtx_unlock",
+              RetValSpec<IntType>,
+              [
+                  ArgSpec<MtxTTypePtr>,
+              ]
+          >,
+      ]
+  >;
+
   let Headers = [
     Errno,
     Math,
     String,
     StdIO,
+    Threads,
   ];
 }

libc/src/CMakeLists.txt

 add_subdirectory(errno)
 add_subdirectory(math)
 add_subdirectory(string)
-# TODO: Add this target conditional to the target OS.
 add_subdirectory(sys)
+add_subdirectory(threads)
 
 add_subdirectory(__support)

libc/src/threads/CMakeLists.txt

+if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${LIBC_TARGET_OS})
+  add_subdirectory(${LIBC_TARGET_OS})
+endif()

libc/src/threads/linux/CMakeLists.txt

+add_header_library(
+  mutex
+  HDRS
+    mutex.h
+)
+
+add_entrypoint_object(
+  mtx_init
+  SRCS
+    mtx_init.cpp
+  HDRS
+    ../mtx_init.h
+  DEPENDS
+    mutex
+    support_common_h
+    threads_h
+)
+
+add_entrypoint_object(
+  mtx_lock
+  SRCS
+    mtx_lock.cpp
+  HDRS
+    ../mtx_lock.h
+  DEPENDS
+    linux_syscall_h
+    mutex
+    support_common_h
+    sys_syscall_h
+    threads_h
+)
+
+add_entrypoint_object(
+  mtx_unlock
+  SRCS
+    mtx_unlock.cpp
+  HDRS
+    ../mtx_unlock.h
+  DEPENDS
+    linux_syscall_h
+    mutex
+    support_common_h
+    sys_syscall_h
+    threads_h
+)

libc/src/threads/linux/mtx_init.cpp

+//===----------- Linux implementation of the mtx_init function ------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "include/threads.h" // For mtx_t definition.
+#include "src/__support/common.h"
+#include "src/threads/linux/mutex.h"
+
+namespace __llvm_libc {
+
+int LLVM_LIBC_ENTRYPOINT(mtx_init)(mtx_t *mutex, int type) {
+  *(reinterpret_cast<uint32_t *>(mutex->__internal_data)) = MS_Free;
+  mutex->__mtx_type = type;
+  return thrd_success;
+}
+
+} // namespace __llvm_libc

libc/src/threads/linux/mtx_lock.cpp

+//===----------- Linux implementation of the mtx_lock function ------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "config/linux/syscall.h" // For syscall functions.
+#include "include/sys/syscall.h"  // For syscall numbers.
+#include "include/threads.h"      // For mtx_t definition.
+#include "src/__support/common.h"
+#include "src/threads/linux/mutex.h"
+
+#include <linux/futex.h> // For futex operations.
+#include <stdatomic.h>   // For atomic_compare_exchange_strong.
+
+namespace __llvm_libc {
+
+// The implementation currently handles only plain mutexes.
+int LLVM_LIBC_ENTRYPOINT(mtx_lock)(mtx_t *mutex) {
+  FutexData *futex_data = reinterpret_cast<FutexData *>(mutex->__internal_data);
+  while (true) {
+    uint32_t mutex_status = MS_Free;
+    uint32_t locked_status = MS_Locked;
+
+    if (atomic_compare_exchange_strong(futex_data, &mutex_status, MS_Locked))
+      return thrd_success;
+
+    switch (mutex_status) {
+    case MS_Waiting:
+      // If other threads are waiting already, then join them. Note that the
+      // futex syscall will block if the futex data is still `MS_Waiting` (the
+      // 4th argument to the syscall function below.)
+      __llvm_libc::syscall(SYS_futex, futex_data, FUTEX_WAIT_PRIVATE,
+                           MS_Waiting, 0, 0, 0);
+      // Once woken up/unblocked, try everything all over.
+      continue;
+    case MS_Locked:
+      // Mutex has been locked by another thread so set the status to
+      // MS_Waiting.
+      if (atomic_compare_exchange_strong(futex_data, &locked_status,
+                                         MS_Waiting)) {
+        // If we are able to set the futex data to `MS_Waiting`, then we will
+        // wait for the futex to be woken up. Note again that the following
+        // syscall will block only if the futex data is still `MS_Waiting`.
+        __llvm_libc::syscall(SYS_futex, futex_data, FUTEX_WAIT_PRIVATE,
+                             MS_Waiting, 0, 0, 0);
+      }
+      continue;
+    case MS_Free:
+      // If it was MS_Free, we shouldn't be here at all.
+      [[clang::fallthrough]];
+    default:
+      // Mutex status cannot be anything else. So control should not reach
+      // here at all.
+      return thrd_error;
+    }
+  }
+}
+
+} // namespace __llvm_libc

libc/src/threads/linux/mtx_unlock.cpp

+//===---------- Linux implementation of the mtx_unlock function -----------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "config/linux/syscall.h" // For syscall functions.
+#include "include/sys/syscall.h"  // For syscall numbers.
+#include "include/threads.h"      // For mtx_t definition.
+#include "src/__support/common.h"
+#include "src/threads/linux/mutex.h"
+
+#include <linux/futex.h> // For futex operations.
+#include <stdatomic.h>   // for atomic_compare_exchange_strong.
+
+namespace __llvm_libc {
+
+// The implementation currently handles only plain mutexes.
+int LLVM_LIBC_ENTRYPOINT(mtx_unlock)(mtx_t *mutex) {
+  FutexData *futex_word = reinterpret_cast<FutexData *>(mutex->__internal_data);
+  while (true) {
+    uint32_t mutex_status = MS_Waiting;
+    if (atomic_compare_exchange_strong(futex_word, &mutex_status, MS_Free)) {
+      // If any thread is waiting to be woken up, then do it.
+      __llvm_libc::syscall(SYS_futex, futex_word, FUTEX_WAKE_PRIVATE, 1, 0, 0,
+                           0);
+      return thrd_success;
+    }
+
+    if (mutex_status == MS_Locked) {
+      // If nobody was waiting at this point, just free it.
+      if (atomic_compare_exchange_strong(futex_word, &mutex_status, MS_Free))
+        return thrd_success;
+    } else {
+      // This can happen, for example if some thread tries to unlock an already
+      // free mutex.
+      return thrd_error;
+    }
+  }
+}
+
+} // namespace __llvm_libc

libc/src/threads/linux/mutex.h

+//===--- Linux specific definitions to support mutex operations --*- C++ -*===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIBC_SRC_THREADS_LINUX_H
+#define LLVM_LIBC_SRC_THREADS_LINUX_H
+
+#include <stdint.h>
+
+// We use a tri-state mutex because we want to avoid making syscalls
+// as much as possible. In `mtx_unlock` a syscall to wake waiting threads is
+// made only if the mutex status is `MutexStatus::Waiting`.
+enum MutexStatus : uint32_t { MS_Free, MS_Locked, MS_Waiting };
+
+using FutexData = _Atomic uint32_t;
+
+#endif // LLVM_LIBC_SRC_THREADS_LINUX_H

libc/src/threads/mtx_init.h

+//===---------- Implementation header for mtx_init function ------ *-C++-* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "include/threads.h"
+
+namespace __llvm_libc {
+
+int mtx_int(mtx_t *mutex, int type);
+
+} // namespace __llvm_libc

libc/src/threads/mtx_lock.h

+//===---------- Implementation header for mtx_lock function ------ *-C++-* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "include/threads.h"
+
+namespace __llvm_libc {
+
+int mtx_lock(mtx_t *mutex);
+
+} // namespace __llvm_libc

libc/src/threads/mtx_unlock.h

+//===-------- Implementation header for mtx_unlock function ------ *-C++-* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "include/threads.h"
+
+namespace __llvm_libc {
+
+int mtx_unlock(mtx_t *mutex);
+
+} // namespace __llvm_libc

libc/test/src/CMakeLists.txt

 add_subdirectory(errno)
 add_subdirectory(string)
 add_subdirectory(sys)
+add_subdirectory(threads)

libc/test/src/threads/CMakeLists.txt

+add_subdirectory(linux)

libc/test/src/threads/linux/CMakeLists.txt

+add_libc_testsuite(libc_threads_unittests)
+
+add_libc_unittest(
+  mtx_test
+  SUITE
+    libc_threads_unittests
+  SRCS
+    mtx_test.cpp
+  DEPENDS
+    sys_syscall_h
+    threads_h
+    mtx_init
+    mtx_lock
+    mtx_unlock
+)
+
+target_link_libraries(mtx_test PRIVATE -lpthread)

libc/test/src/threads/linux/mtx_test.cpp

+//===---------------------- Unittests for mtx_t ---------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "include/threads.h"
+#include "src/threads/mtx_init.h"
+#include "src/threads/mtx_lock.h"
+#include "src/threads/mtx_unlock.h"
+#include "utils/UnitTest/Test.h"
+#include "utils/testutils/Support.h"
+#include "utils/testutils/Thread.h"
+
+int START = 0;
+int MAX = 10000;
+
+mtx_t mutex;
+static int shared_int = START;
+
+void counter(void) {
+  int last_count = START;
+  while (true) {
+    __llvm_libc::mtx_lock(&mutex);
+    if (shared_int == last_count + 1) {
+      shared_int++;
+      last_count = shared_int;
+    }
+    __llvm_libc::mtx_unlock(&mutex);
+    if (last_count >= MAX)
+      break;
+  }
+}
+
+TEST(MutexTest, RelayCounter) {
+  // The idea of this test is that two competing threads will update
+  // a counter only if the other thread has updated it.
+  __llvm_libc::testutils::Thread th(counter);
+
+  int last_count = START;
+  while (true) {
+    ASSERT_EQ(__llvm_libc::mtx_lock(&mutex), (int)thrd_success);
+    if (shared_int == START) {
+      ++shared_int;
+      last_count = shared_int;
+    } else if (shared_int != last_count) {
+      ASSERT_EQ(shared_int, last_count + 1);
+      ++shared_int;
+      last_count = shared_int;
+    }
+    ASSERT_EQ(__llvm_libc::mtx_unlock(&mutex), (int)thrd_success);
+    if (last_count > MAX)
+      break;
+  }
+
+  th.join();
+}
+
+mtx_t start_lock, step_lock;
+bool start, step;
+
+void stepper(void) {
+  __llvm_libc::mtx_lock(&start_lock);
+  start = true;
+  __llvm_libc::mtx_unlock(&start_lock);
+
+  __llvm_libc::mtx_lock(&step_lock);
+  step = true;
+  __llvm_libc::mtx_unlock(&step_lock);
+}
+
+TEST(MutexTest, WaitAndStep) {
+  // In this test, we start a new thread but block it before it can make a
+  // step. Once we ensure that the thread is blocked, we unblock it.
+  // After unblocking, we then verify that the thread was indeed unblocked.
+  step = false;
+  start = false;
+  ASSERT_EQ(__llvm_libc::mtx_lock(&step_lock), (int)thrd_success);
+
+  // The new thread will immediately start.
+  __llvm_libc::testutils::Thread th(stepper);
+
+  while (true) {
+    // Make sure the thread actually started.
+    ASSERT_EQ(__llvm_libc::mtx_lock(&start_lock), (int)thrd_success);
+    bool s = start;
+    ASSERT_EQ(__llvm_libc::mtx_unlock(&start_lock), (int)thrd_success);
+    if (s)
+      break;
+  }
+
+  // Sleep for 10 seconds and check that the step was not made.
+  // This could be flaky if the other thread is not scheduled for 10
+  // seconds, but in practice it never happens.
+  __llvm_libc::testutils::sleep(10);
+  ASSERT_FALSE(step);
+
+  // Unlock the step lock and wait for 10 seconds again. This should
+  // give enough time for the other thread to make the step.
+  ASSERT_EQ(__llvm_libc::mtx_unlock(&step_lock), (int)thrd_success);
+  __llvm_libc::testutils::sleep(10);
+
+  ASSERT_EQ(__llvm_libc::mtx_lock(&step_lock), (int)thrd_success);
+  ASSERT_TRUE(step);
+  ASSERT_EQ(__llvm_libc::mtx_unlock(&step_lock), (int)thrd_success);
+
+  th.join();
+}

libc/utils/HdrGen/PublicAPICommand.cpp

class APIGenerator {
   llvm::Record *PublicAPIClass;
 
   using NameToRecordMapping = std::unordered_map<std::string, llvm::Record *>;
   using NameSet = std::unordered_set<std::string>;
 
   // Mapping from names to records defining them.
   NameToRecordMapping MacroSpecMap;
   NameToRecordMapping TypeSpecMap;
+  NameToRecordMapping EnumerationSpecMap;
   NameToRecordMapping FunctionSpecMap;
   NameToRecordMapping MacroDefsMap;
   NameToRecordMapping TypeDeclsMap;
 
   NameSet Structs;
+  NameSet Enumerations;
   NameSet Functions;
 
   bool isaNamedType(llvm::Record *Def) { return isa(Def, NamedTypeClass); }
 
   bool isaStructType(llvm::Record *Def) { return isa(Def, StructClass); }
 
   bool isaPtrType(llvm::Record *Def) { return isa(Def, PtrTypeClass); }
 
for (llvm::Record *HeaderSpec : HeaderSpecList) {
           TypeSpecMap[std::string(TypeSpec->getValueAsString("Name"))] =
               TypeSpec;
 
         auto FunctionSpecList = HeaderSpec->getValueAsListOfDefs("Functions");
         for (llvm::Record *FunctionSpec : FunctionSpecList) {
           FunctionSpecMap[std::string(FunctionSpec->getValueAsString("Name"))] =
               FunctionSpec;
         }
+
+        auto EnumerationSpecList =
+            HeaderSpec->getValueAsListOfDefs("Enumerations");
+        for (llvm::Record *EnumerationSpec : EnumerationSpecList) {
+          EnumerationSpecMap[std::string(
+              EnumerationSpec->getValueAsString("Name"))] = EnumerationSpec;
+        }
       }
     }
   }
 
   void indexPublicAPIDef(llvm::Record *PublicAPI) {
     // While indexing the public API, we do not check if any of the entities
     // requested is from an included standard. Such a check is done while
     // generating the API.
     auto MacroDefList = PublicAPI->getValueAsListOfDefs("Macros");
     for (llvm::Record *MacroDef : MacroDefList)
       MacroDefsMap[std::string(MacroDef->getValueAsString("Name"))] = MacroDef;
 
     auto TypeDeclList = PublicAPI->getValueAsListOfDefs("TypeDeclarations");
     for (llvm::Record *TypeDecl : TypeDeclList)
       TypeDeclsMap[std::string(TypeDecl->getValueAsString("Name"))] = TypeDecl;
 
     auto StructList = PublicAPI->getValueAsListOfStrings("Structs");
     for (llvm::StringRef StructName : StructList)
       Structs.insert(std::string(StructName));
 
     auto FunctionList = PublicAPI->getValueAsListOfStrings("Functions");
     for (llvm::StringRef FunctionName : FunctionList)
       Functions.insert(std::string(FunctionName));
+
+    auto EnumerationList = PublicAPI->getValueAsListOfStrings("Enumerations");
+    for (llvm::StringRef EnumerationName : EnumerationList)
+      Enumerations.insert(std::string(EnumerationName));
   }
 
   void index(llvm::RecordKeeper &Records) {
     NamedTypeClass = Records.getClass(NamedTypeClassName);
     PtrTypeClass = Records.getClass(PtrTypeClassName);
     RestrictedPtrTypeClass = Records.getClass(RestrictedPtrTypeClassName);
     StructClass = Records.getClass(StructTypeClassName);
     ConstTypeClass = Records.getClass(ConstTypeClassName);
for (auto &Pair : TypeDeclsMap) {
         llvm::PrintFatalError(Name + " not found in any standard spec.\n");
 
       llvm::Record *TypeDecl = Pair.second;
       dedentAndWrite(TypeDecl->getValueAsString("Decl"), OS);
 
       OS << '\n';
     }
 
+    if (Enumerations.size() != 0)
+      OS << "enum {" << '\n';
+    for (auto &Name : Enumerations) {
+      if (EnumerationSpecMap.find(Name) == EnumerationSpecMap.end())
+        llvm::PrintFatalError(
+            Name + " is not listed as an enumeration in any standard spec.\n");
+
+      llvm::Record *EnumerationSpec = EnumerationSpecMap[Name];
+      OS << "  " << EnumerationSpec->getValueAsString("Name");
+      auto Value = EnumerationSpec->getValueAsString("Value");
+      if (Value == "__default__") {
+        OS << ",\n";
+      } else {
+        OS << " = " << Value << ",\n";
+      }
+    }
+    if (Enumerations.size() != 0)
+      OS << "};\n\n";
+
     OS << "__BEGIN_C_DECLS\n\n";
     for (auto &Name : Functions) {
       if (FunctionSpecMap.find(Name) == FunctionSpecMap.end())
         llvm::PrintFatalError(Name + " not found in any standard spec.\n");
 
       llvm::Record *FunctionSpec = FunctionSpecMap[Name];
       llvm::Record *RetValSpec = FunctionSpec->getValueAsDef("Return");
       llvm::Record *ReturnType = RetValSpec->getValueAsDef("ReturnType");