Support fiber API on macOS

[dvyukov]

Kuba, please take a look. This mostly touches Mac-specific files. I can't test this.

[kubamracek]

@yln Could you take a look? Do the tests actually pass on Darwin?

The _XOPEN_SOURCE stuff is ugly, but I guess it's necessary to use swapcontext on Darwin. For the pragma diagnostic push/ignore stuff, we might be better off just redeclaring the function ourselves without including ucontext.h. TBH, I'm not sure if there's a lot of value in supporting swapcontext on Darwin, as they are very likely broken when using many system libraries.

test/tsan/fiber_asm.cc

21–22 ↗

(On Diff #186428)

Can we use something like the ASM_SYMBOL macro from sanitizer_asm.h and put it into test.h please?

[yln]

ninja check-tsan reports no failing test on my machine.

[yuri]

Cleanup tests

[yuri]

In D58110#1396514, @kubamracek wrote:

For the pragma diagnostic push/ignore stuff, we might be better off just redeclaring the function ourselves without including ucontext.h.

For tsan_interceptors_mac.cc it can be done. For me such solution is more ugly than usage of #define and #pragma; but if you insist on it, I can do such change.

For tests it is bad idea because test should use API the same way as real-world clients. I moved declarations to the separate header. Now code looks better.

TBH, I'm not sure if there's a lot of value in supporting swapcontext on Darwin, as they are very likely broken when using many system libraries.

• QEMU is an important projected user of fiber API and it actually use swapcontext.
• We need to test fiber API somehow and swapcontext is the only portable way to do it.

PS. I want to explain why swapcontext interceptor is required on macOS and not on Linux.
__tsan_switch_to_fiber is called immediately before or after context switch.
Actual context switch code should not introduce memory dependancies; otherwise false positive races can be reported.
On Linux swapcontext implementation does not call any intercepted function, so condition is fulfilled.
On macOS we have to intercept swapcontext and disable interceptors inside it in order to achieve the same effect.

[kubamracek]

For the pragma diagnostic push/ignore stuff, we might be better off just redeclaring the function ourselves without including ucontext.h.

For tsan_interceptors_mac.cc it can be done. For me such solution is more ugly than usage of #define and #pragma; but if you insist on it, I can do such change.

I also think it's more ugly, but I'm afraid the code won't build against watchOS and tvOS, where swapcontext is not just deprecated, but it's completely unavailable. @yln Can you please try to build this for watchOS/tvOS?

[yuri]

In D58110#1398292, @kubamracek wrote:

@yln Can you please try to build this for watchOS/tvOS?

watchOS/tvOS build is broken right now in trunk:
https://pastebin.com/Y3gsxtA7

[yuri]

Added

#if !SANITIZER_WATCHOS && !SANITIZER_TVOS

around swapcontext interceptor

[yln]

I verified that the watchOS/tvOS runtimes (libclang_rt.tsan_*) are building with this patch applied to our internal master.

[kubamracek]

watchOS/tvOS build is broken right now in trunk:
https://pastebin.com/Y3gsxtA7

The watchOS/tvOS parts of ASan require an internal SDK that's not publicly available to build. I guess that counts as "broken", but it's an expected state.

I think the tests are still going to fail on watchOS and tvOS, they need to be marked as UNSUPPORTED. At this point, I'm inclined to prefer to declare swapcontext ourselves (as swapcontext *is* available at runtime on watchOS/tvOS, it's just not allowed in the SDK) to avoid this much ifdef'ing and platform-specific workarounds. But if you strongly disagree, I'm okay with this approach also. (The tests still need updating.)

[yuri]

Do not include <ucontext.h> and declare getcontext/setcontext manually

[yuri]

In D58110#1401436, @kubamracek wrote:

I think the tests are still going to fail on watchOS and tvOS, they need to be marked as UNSUPPORTED

I have not found any tests marked such way. What keywords to use for UNSUPPORTED?

[yln]

In D58110#1401839, @yuri wrote:

In D58110#1401436, @kubamracek wrote:

I think the tests are still going to fail on watchOS and tvOS, they need to be marked as UNSUPPORTED

I have not found any tests marked such way. What keywords to use for UNSUPPORTED?

I don't know exactly what/how should be done (Kuba, please advise), but the definition of the features is here:
https://github.com/llvm/llvm-project/blob/master/compiler-rt/test/lit.common.cfg#L130

[yuri]

Disable tests on tvOS and watchOS

[yuri]

If everything is OK, can you commit this patch?

[yuri]

In D58110#1405299, @yuri wrote:

If everything is OK, can you commit this patch?

Kuba?
Dmitry?

[dvyukov]

Kuba, please submit this. I can't test/build this.

[yuri]

In D58110#1408455, @dvyukov wrote:

Kuba, please submit this. I can't test/build this.

Kuba, please do not ignore

[kubamracek]

Sorry for the huge delay. @yln can you test this patch (that is passes all tests) and if yes, land it?

[yln]

Had to XFAIL a few tests for iOS (macOS works), see attached log.

ThreadSanitizer-arm64-ios_fiber-tests.log33 KBDownload

compiler-rt/trunk/lib/tsan/rtl/tsan_interceptors_mac.cc

 #if SANITIZER_MAC
 
 #include "interception/interception.h"
 #include "tsan_interceptors.h"
 #include "tsan_interface.h"
 #include "tsan_interface_ann.h"
 #include "sanitizer_common/sanitizer_addrhashmap.h"
 
+#include <errno.h>
 #include <libkern/OSAtomic.h>
 #include <objc/objc-sync.h>
+#include <sys/ucontext.h>
 
 #if defined(__has_include) && __has_include(<xpc/xpc.h>)
 #include <xpc/xpc.h>
 #endif  // #if defined(__has_include) && __has_include(<xpc/xpc.h>)
 
 typedef long long_t;  // NOLINT
 
+extern "C" {
+int getcontext(ucontext_t *ucp) __attribute__((returns_twice));
+int setcontext(const ucontext_t *ucp);
+}
+
 namespace __tsan {
 
 // The non-barrier versions of OSAtomic* functions are semantically mo_relaxed,
 // but the two variants (e.g. OSAtomicAdd32 and OSAtomicAdd32Barrier) are
 // actually aliases of each other, and we cannot have different interceptors for
 // them, because they're actually the same function.  Thus, we have to stay
 // conservative and treat the non-barrier versions as mo_acq_rel.
 static const morder kMacOrderBarrier = mo_acq_rel;
TSAN_INTERCEPTOR(int, objc_sync_exit, id obj) {
   if (!obj) return REAL(objc_sync_exit)(obj);
   uptr addr = SyncAddressForObjCObject(obj, thr, pc);
   MutexUnlock(thr, pc, addr);
   int result = REAL(objc_sync_exit)(obj);
   if (result != OBJC_SYNC_SUCCESS) MutexInvalidAccess(thr, pc, addr);
   return result;
 }
 
+TSAN_INTERCEPTOR(int, swapcontext, ucontext_t *oucp, const ucontext_t *ucp) {
+  {
+    SCOPED_INTERCEPTOR_RAW(swapcontext, oucp, ucp);
+  }
+  // Bacause of swapcontext() semantics we have no option but to copy its
+  // impementation here
+  if (!oucp || !ucp) {
+    errno = EINVAL;
+    return -1;
+  }
+  ThreadState *thr = cur_thread();
+  const int UCF_SWAPPED = 0x80000000;
+  oucp->uc_onstack &= ~UCF_SWAPPED;
+  thr->ignore_interceptors++;
+  int ret = getcontext(oucp);
+  if (!(oucp->uc_onstack & UCF_SWAPPED)) {
+    thr->ignore_interceptors--;
+    if (!ret) {
+      oucp->uc_onstack |= UCF_SWAPPED;
+      ret = setcontext(ucp);
+    }
+  }
+  return ret;
+}
+
 // On macOS, libc++ is always linked dynamically, so intercepting works the
 // usual way.
 #define STDCXX_INTERCEPTOR TSAN_INTERCEPTOR
 
 namespace {
 struct fake_shared_weak_count {
   volatile a64 shared_owners;
   volatile a64 shared_weak_owners;

compiler-rt/trunk/lib/tsan/rtl/tsan_interface.cc

 }
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __sanitizer_unaligned_store64(uu64 *addr, u64 v) {
   __tsan_unaligned_write8(addr);
   *addr = v;
 }
 
-#if !SANITIZER_MAC && !SANITIZER_ANDROID
 SANITIZER_INTERFACE_ATTRIBUTE
 void *__tsan_get_current_fiber() {
   return cur_thread();
 }
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void *__tsan_create_fiber(unsigned flags) {
   return FiberCreate(cur_thread(), CALLERPC, flags);
 }
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __tsan_destroy_fiber(void *fiber) {
   FiberDestroy(cur_thread(), CALLERPC, static_cast<ThreadState *>(fiber));
 }
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __tsan_switch_to_fiber(void *fiber, unsigned flags) {
   FiberSwitch(cur_thread(), CALLERPC, static_cast<ThreadState *>(fiber), flags);
 }
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __tsan_set_fiber_name(void *fiber, const char *name) {
   ThreadSetName(static_cast<ThreadState *>(fiber), name);
 }
-#endif // !SANITIZER_MAC && !SANITIZER_ANDROID
 }  // extern "C"
 
 void __tsan_acquire(void *addr) {
   Acquire(cur_thread(), CALLERPC, (uptr)addr);
 }
 
 void __tsan_release(void *addr) {
   Release(cur_thread(), CALLERPC, (uptr)addr);
 }

compiler-rt/trunk/lib/tsan/rtl/tsan_platform_linux.cc

if (thr == nullptr) {
                                       PROT_READ));
       }
     }
     CHECK_EQ(0, internal_sigprocmask(SIG_SETMASK, &oldset, nullptr));
   }
   return thr;
 }
 
+void set_cur_thread(ThreadState *thr) {
+  *get_android_tls_ptr() = reinterpret_cast<uptr>(thr);
+}
+
 void cur_thread_finalize() {
   __sanitizer_sigset_t emptyset;
   internal_sigfillset(&emptyset);
   __sanitizer_sigset_t oldset;
   CHECK_EQ(0, internal_sigprocmask(SIG_SETMASK, &emptyset, &oldset));
   ThreadState* thr = reinterpret_cast<ThreadState*>(*get_android_tls_ptr());
   if (thr != dead_thread_state) {
     *get_android_tls_ptr() = reinterpret_cast<uptr>(dead_thread_state);

compiler-rt/trunk/lib/tsan/rtl/tsan_platform_mac.cc

 // when TLVs are not accessible (early process startup, thread cleanup, ...).
 // The following provides a "poor man's TLV" implementation, where we use the
 // shadow memory of the pointer returned by pthread_self() to store a pointer to
 // the ThreadState object. The main thread's ThreadState is stored separately
 // in a static variable, because we need to access it even before the
 // shadow memory is set up.
 static uptr main_thread_identity = 0;
 ALIGNED(64) static char main_thread_state[sizeof(ThreadState)];
+static ThreadState *main_thread_state_loc = (ThreadState *)main_thread_state;
 
-ThreadState **cur_thread_location() {
-  ThreadState **thread_identity = (ThreadState **)pthread_self();
-  return ((uptr)thread_identity == main_thread_identity) ? nullptr
-                                                         : thread_identity;
+static ThreadState **cur_thread_location() {
+  uptr thread_identity = (uptr)pthread_self();
+  if (thread_identity == main_thread_identity || main_thread_identity == 0)
+    return &main_thread_state_loc;
+  return (ThreadState **)MemToShadow(thread_identity);
 }
 
 ThreadState *cur_thread() {
-  ThreadState **thr_state_loc = cur_thread_location();
-  if (thr_state_loc == nullptr || main_thread_identity == 0) {
-    return (ThreadState *)&main_thread_state;
-  }
-  ThreadState **fake_tls = (ThreadState **)MemToShadow((uptr)thr_state_loc);
-  ThreadState *thr = (ThreadState *)SignalSafeGetOrAllocate(
-      (uptr *)fake_tls, sizeof(ThreadState));
-  return thr;
+  return (ThreadState *)SignalSafeGetOrAllocate(
+      (uptr *)cur_thread_location(), sizeof(ThreadState));
+
+void set_cur_thread(ThreadState *thr) {
+  *cur_thread_location() = thr;
 }
 
 // TODO(kuba.brecka): This is not async-signal-safe. In particular, we call
 // munmap first and then clear `fake_tls`; if we receive a signal in between,
 // handler will try to access the unmapped ThreadState.
 void cur_thread_finalize() {
   ThreadState **thr_state_loc = cur_thread_location();
-  if (thr_state_loc == nullptr) {
+  if (thr_state_loc == &main_thread_state_loc) {
     // Calling dispatch_main() or xpc_main() actually invokes pthread_exit to
     // exit the main thread. Let's keep the main thread's ThreadState.
     return;
   }
-  ThreadState **fake_tls = (ThreadState **)MemToShadow((uptr)thr_state_loc);
-  internal_munmap(*fake_tls, sizeof(ThreadState));
-  *fake_tls = nullptr;
+  internal_munmap(*thr_state_loc, sizeof(ThreadState));
+  *thr_state_loc = nullptr;
 }
 #endif
 
 void FlushShadowMemory() {
 }
 
 static void RegionMemUsage(uptr start, uptr end, uptr *res, uptr *dirty) {
   vm_address_t address = start;
#endif
   }
 }
 
 #if !SANITIZER_GO
 void ImitateTlsWrite(ThreadState *thr, uptr tls_addr, uptr tls_size) {
   // The pointer to the ThreadState object is stored in the shadow memory
   // of the tls.
   uptr tls_end = tls_addr + tls_size;
-  ThreadState **thr_state_loc = cur_thread_location();
-  if (thr_state_loc == nullptr) {
+  uptr thread_identity = (uptr)pthread_self();
+  if (thread_identity == main_thread_identity) {
     MemoryRangeImitateWrite(thr, /*pc=*/2, tls_addr, tls_size);
   } else {
-    uptr thr_state_start = (uptr)thr_state_loc;
+    uptr thr_state_start = thread_identity;
     uptr thr_state_end = thr_state_start + sizeof(uptr);
     CHECK_GE(thr_state_start, tls_addr);
     CHECK_LE(thr_state_start, tls_addr + tls_size);
     CHECK_GE(thr_state_end, tls_addr);
     CHECK_LE(thr_state_end, tls_addr + tls_size);
     MemoryRangeImitateWrite(thr, /*pc=*/2, tls_addr,
                             thr_state_start - tls_addr);
     MemoryRangeImitateWrite(thr, /*pc=*/2, thr_state_end,

compiler-rt/trunk/lib/tsan/rtl/tsan_rtl.h

explicit ThreadState(Context *ctx, int tid, int unique_id, u64 epoch,
                        unsigned reuse_count,
                        uptr stk_addr, uptr stk_size,
                        uptr tls_addr, uptr tls_size);
 };
 
 #if !SANITIZER_GO
 #if SANITIZER_MAC || SANITIZER_ANDROID
 ThreadState *cur_thread();
+void set_cur_thread(ThreadState *thr);
 void cur_thread_finalize();
 INLINE void cur_thread_init() { }
 #else
 __attribute__((tls_model("initial-exec")))
 extern THREADLOCAL char cur_thread_placeholder[];
 INLINE ThreadState *cur_thread() {
   return reinterpret_cast<ThreadState *>(cur_thread_placeholder)->current;
 }

compiler-rt/trunk/lib/tsan/rtl/tsan_rtl_thread.cc

for (; size; addr++, size--) {
     Shadow cur(fast_state);
     cur.SetWrite(is_write);
     cur.SetAddr0AndSizeLog(addr & (kShadowCell - 1), kAccessSizeLog);
     MemoryAccessImpl(thr, addr, kAccessSizeLog, is_write, false,
         shadow_mem, cur);
   }
 }
 
-#if !SANITIZER_MAC && !SANITIZER_ANDROID && !SANITIZER_GO
+#if !SANITIZER_GO
 void FiberSwitchImpl(ThreadState *from, ThreadState *to) {
   Processor *proc = from->proc();
   ProcUnwire(proc, from);
   ProcWire(proc, to);
   set_cur_thread(to);
 }
 
 ThreadState *FiberCreate(ThreadState *thr, uptr pc, unsigned flags) {

compiler-rt/trunk/test/lsan/TestCases/swapcontext.cc

 // We can't unwind stack if we're running coroutines on heap-allocated
 // memory. Make sure we don't report these leaks.
 
 // RUN: %clangxx_lsan %s -o %t
 // RUN: %env_lsan_opts= %run %t 2>&1
 // RUN: %env_lsan_opts= not %run %t foo 2>&1 | FileCheck %s
 // UNSUPPORTED: arm,powerpc64
 
+#include "sanitizer_common/sanitizer_ucontext.h"
 #include <stdio.h>
-#if defined(__APPLE__)
-// Note: ucontext.h is deprecated on OSX, so this test may stop working
-// someday. We define _XOPEN_SOURCE to keep using ucontext.h for now.
-#define _XOPEN_SOURCE 1
-#endif
-#include <ucontext.h>
 #include <unistd.h>
 
 const int kStackSize = 1 << 20;
 
 void Child() {
   int child_stack;
   printf("Child: %p\n", &child_stack);
   int *leaked = new int[666];

compiler-rt/trunk/test/sanitizer_common/sanitizer_ucontext.h

+#ifdef __APPLE__
+// ucontext.h is deprecated on macOS, so tests that include it may stop working
+// someday. We define _XOPEN_SOURCE to keep using ucontext.h for now.
+#ifdef _STRUCT_UCONTEXT
+#error incomplete ucontext_t already defined, change #include order
+#endif
+#define _XOPEN_SOURCE 700
+#pragma clang diagnostic ignored "-Wdeprecated-declarations"
+#endif
+
+#include <ucontext.h>

compiler-rt/trunk/test/tsan/fiber_asm.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 // REQUIRES: x86_64-target-arch
-// UNSUPPORTED: darwin
+// UNSUPPORTED: tvos, watchos
 #include "test.h"
 
 struct ucontext {
   void *sp;
   void *fiber;
 };
 
 extern "C" {
   void ucontext_do_switch(void **save, void **load);
   void ucontext_trampoline();
 }
 
-__asm__(".global ucontext_do_switch\n"
-        "ucontext_do_switch:\n\t"
+__asm__(".global " ASM_SYMBOL(ucontext_do_switch) "\n"
+        ASM_SYMBOL(ucontext_do_switch) ":\n\t"
         "pushq %rbp\n\t"
         "pushq %r15\n\t"
         "pushq %r14\n\t"
         "pushq %r13\n\t"
         "pushq %r12\n\t"
         "pushq %rbx\n\t"
         "movq %rsp, (%rdi)\n\t"
         "movq (%rsi), %rsp\n\t"
         "popq %rbx\n\t"
         "popq %r12\n\t"
         "popq %r13\n\t"
         "popq %r14\n\t"
         "popq %r15\n\t"
         "popq %rbp\n\t"
         "retq");
 
-__asm__(".global ucontext_trampoline\n"
-        "ucontext_trampoline:\n\t"
+__asm__(".global " ASM_SYMBOL(ucontext_trampoline) "\n"
+        ASM_SYMBOL(ucontext_trampoline) ":\n\t"
         ".cfi_startproc\n\t"
         ".cfi_undefined rip\n\t"
         "movq %r12, %rdi\n\t"
         "jmpq *%rbx\n\t"
         ".cfi_endproc");
 
 void ucontext_init(ucontext *context, void *stack, unsigned stack_sz,
                    void (*func)(void*), void *arg) {

compiler-rt/trunk/test/tsan/fiber_from_thread.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
-// UNSUPPORTED: darwin
+// UNSUPPORTED: tvos, watchos
+#include "sanitizer_common/sanitizer_ucontext.h"
 #include "test.h"
-#include <ucontext.h>
 
 char stack[64 * 1024] __attribute__((aligned(16)));
 
 ucontext_t uc, orig_uc1, orig_uc2;
 void *fiber, *orig_fiber1, *orig_fiber2;
 
 int var;
 

compiler-rt/trunk/test/tsan/fiber_longjmp.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
-// UNSUPPORTED: darwin
+// UNSUPPORTED: tvos, watchos
+#include "sanitizer_common/sanitizer_ucontext.h"
 #include "test.h"
 #include <setjmp.h>
-#include <ucontext.h>
 
 char stack[64 * 1024] __attribute__((aligned(16)));
 
 sigjmp_buf jmpbuf, orig_jmpbuf[2];
 void *fiber, *orig_fiber[2];
 
 const unsigned N = 1000;
 

compiler-rt/trunk/test/tsan/fiber_race.cc

 // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
-// UNSUPPORTED: darwin
+// UNSUPPORTED: tvos, watchos
+#include "sanitizer_common/sanitizer_ucontext.h"
 #include "test.h"
-#include <ucontext.h>
 
 char stack[64 * 1024] __attribute__((aligned(16)));
 
 ucontext_t uc, orig_uc;
 void *fiber, *orig_fiber;
 
 int var;
 

compiler-rt/trunk/test/tsan/fiber_simple.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
-// UNSUPPORTED: darwin
+// UNSUPPORTED: tvos, watchos
+#include "sanitizer_common/sanitizer_ucontext.h"
 #include "test.h"
-#include <ucontext.h>
 
 char stack[64 * 1024] __attribute__((aligned(16)));
 
 ucontext_t uc, orig_uc;
 void *fiber, *orig_fiber;
 
 int var;
 

compiler-rt/trunk/test/tsan/fiber_two_threads.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
-// UNSUPPORTED: darwin
+// UNSUPPORTED: tvos, watchos
+#include "sanitizer_common/sanitizer_ucontext.h"
 #include "test.h"
-#include <ucontext.h>
 
 char stack[64 * 1024] __attribute__((aligned(16)));
 
 ucontext_t uc, orig_uc[2];
 void *fiber, *orig_fiber[2];
 
 const unsigned N = 1000;
 

compiler-rt/trunk/test/tsan/test.h

 #define ANNOTATE_RWLOCK_CREATE_STATIC(m) \
     AnnotateRWLockCreateStatic(__FILE__, __LINE__, m)
 #define ANNOTATE_RWLOCK_DESTROY(m) \
     AnnotateRWLockDestroy(__FILE__, __LINE__, m)
 #define ANNOTATE_RWLOCK_ACQUIRED(m, is_w) \
     AnnotateRWLockAcquired(__FILE__, __LINE__, m, is_w)
 #define ANNOTATE_RWLOCK_RELEASED(m, is_w) \
     AnnotateRWLockReleased(__FILE__, __LINE__, m, is_w)
+
+#ifdef __APPLE__
+#define ASM_SYMBOL(symbol) "_" #symbol
+#else
+#define ASM_SYMBOL(symbol) #symbol
+#endif