[tsan] Fix a crash when exiting the main thread (e.g. dispatch_main)

[kubamracek]

Summary

This patch fixes the custom ThreadState destruction on OS X to avoid crashing when dispatch_main calls pthread_exit which quits the main thread.

[dvyukov]

test/tsan/Darwin/dispatch_main.mm

27 ↗

(On Diff #51729)

Why do we need this sleep? Can we remove it? Can we use the invisible barrier instead?
This test will run for 3 seconds. I would like to reduce it's duration.

28 ↗

(On Diff #51729)

This is exit which should terminate whole process. How is calling pthread_exit? Where?

[kubamracek]

test/tsan/Darwin/dispatch_main.mm

27 ↗

(On Diff #51729)

I'll try to remove it.

28 ↗

(On Diff #51729)

pthread_exit is called immediately from dispatch_main below, not here.

[dvyukov]

test/tsan/Darwin/dispatch_main.mm

28 ↗

(On Diff #51729)

I am confused. You do "// CHECK: Done.", so NSLog is executed. But then exit is not executed? And this is executed on the main thread? How does dispatch_main calls pthread_exit while we are blocked in sleep in the very same thread? From a signal handler? Why does it do it?
I seem to be missing something.

[kubamracek]

test/tsan/Darwin/dispatch_main.mm

28 ↗

(On Diff #51729)

This is not executed on the main thread, the main thread is terminated from dispatch_main. This gets executed on a different thread, but technically still on the "main queue".

[kubamracek]

test/tsan/Darwin/dispatch_main.mm

28 ↗

(On Diff #51729)

Calling dispatch_main() from the main thread basically says: "I'm not using the main thread for anything else that libdispatch stuff." So libdispatch knows it's safe to treat the main queue as a "regular serial queue", exit the main thread and schedule stuff dispatched to the main queue on worker threads (but still in a serial, synchronized manner).

[dvyukov]

LGTM. But please document how pthread_exit gets called on the main thread and change sleep(3) to sleep(1) which should be enough to catch the crash in most executions.

compiler-rt/trunk/lib/tsan/rtl/tsan_platform_mac.cc

ThreadState *cur_thread() {
   return thr;
 }
 
 // TODO(kuba.brecka): This is not async-signal-safe. In particular, we call
 // munmap first and then clear `fake_tls`; if we receive a signal in between,
 // handler will try to access the unmapped ThreadState.
 void cur_thread_finalize() {
   uptr thread_identity = (uptr)pthread_self();
-  CHECK_NE(thread_identity, main_thread_identity);
+  if (thread_identity == main_thread_identity) {
+    // Calling dispatch_main() or xpc_main() actually invokes pthread_exit to
+    // exit the main thread. Let's keep the main thread's ThreadState.
+    return;
+  }
   ThreadState **fake_tls = (ThreadState **)MemToShadow(thread_identity);
   internal_munmap(*fake_tls, sizeof(ThreadState));
   *fake_tls = nullptr;
 }
 #endif
 
 uptr GetShadowMemoryConsumption() {
   return 0;

compiler-rt/trunk/test/tsan/Darwin/dispatch_main.mm

+// Check that we don't crash when dispatch_main calls pthread_exit which
+// quits the main thread.
+
+// RUN: %clang_tsan %s -o %t -framework Foundation
+// RUN: %env_tsan_opts=ignore_interceptors_accesses=1 %run %t 2>&1 | FileCheck %s
+
+#import <Foundation/Foundation.h>
+
+int main() {
+  NSLog(@"Hello world");
+
+  dispatch_queue_t q = dispatch_queue_create("my.queue", DISPATCH_QUEUE_SERIAL);
+
+  dispatch_async(q, ^{
+    NSLog(@"1");
+  });
+
+  dispatch_async(q, ^{
+    NSLog(@"2");
+  });
+
+  dispatch_async(q, ^{
+    NSLog(@"3");
+
+    dispatch_async(dispatch_get_main_queue(), ^{
+      NSLog(@"Done.");
+      sleep(1);
+      exit(0);
+    });
+  });
+
+  dispatch_main();
+}
+
+// CHECK: Hello world
+// CHECK: Done.
+// CHECK-NOT: WARNING: ThreadSanitizer
+// CHECK-NOT: CHECK failed