[tsan] Stop extending the block’s lifetime in dispatch_group_async

[kubamracek]

Summary

Our dispatch_group_async interceptor actually extends the lifetime of the executed block. This means the destructor of the block (and captured variables) is called *after* dispatch_group_leave, which changes the semantics of dispatch_group_async.

[dvyukov]

lib/tsan/rtl/tsan_libdispatch_mac.cc

289 ↗

(On Diff #62139)

What does keep the block alive currently? When is it destroyed without this change?

test/tsan/Darwin/gcd-groups-destructor.mm

43 ↗

(On Diff #62139)

AFAIK, filecheck checks line-by-line, so this ensures that there are no WARNINGS _after_ "Done." is printed. I think you need to move CHECK_NOT before Done CHECK.

[kubamracek]

lib/tsan/rtl/tsan_libdispatch_mac.cc

289 ↗

(On Diff #62139)

Without this change, the block named "block" is captured by this anonymous block that's submitted via dispatch_async, and it's also released only when this anonymous block is released (after dispatch completely executes it). That means the release only happens after the two lines below (call to dispatch_group_leave and dispatch_release).

With this change, we're avoiding the capture (with __block) by explicitly copying the block. It will be released/destroyed at the call to _Block_release (as long as there are no other references to the block).

test/tsan/Darwin/gcd-groups-destructor.mm

43 ↗

(On Diff #62139)

Ok

[dvyukov]

Is that pile of fixes happened because people started using tsan for their real programs?

lib/tsan/rtl/tsan_libdispatch_mac.cc

289 ↗

(On Diff #62139)

For my education: how does compiler/runtime know that it needs to capture the block with the current code, but does not need to capture with the new code? I mean in both cases a dispatch_block_t variable is referenced from ^(void){} block. So I would expect the capture happen in both cases.

[kubamracek]

lib/tsan/rtl/tsan_libdispatch_mac.cc

289 ↗

(On Diff #62139)

It’s the __block keyword. Objects with __block (blocks are objects) aren’t retained, they’re "captured directly" instead.

[kubamracek]

In D21816#469794, @dvyukov wrote:

Is that pile of fixes happened because people started using tsan for their real programs?

Yes. A lot of both internal and external developers are using TSan now and they’re finding real races.

compiler-rt/trunk/lib/tsan/rtl/tsan_libdispatch_mac.cc

TSAN_INTERCEPTOR(void, dispatch_group_leave, dispatch_group_t group) {
   REAL(dispatch_group_leave)(group);
 }
 
 TSAN_INTERCEPTOR(void, dispatch_group_async, dispatch_group_t group,
                  dispatch_queue_t queue, dispatch_block_t block) {
   SCOPED_TSAN_INTERCEPTOR(dispatch_group_async, group, queue, block);
   dispatch_retain(group);
   dispatch_group_enter(group);
+  __block dispatch_block_t block_copy = (dispatch_block_t)_Block_copy(block);
   WRAP(dispatch_async)(queue, ^(void) {
-    block();
+    block_copy();
+    _Block_release(block_copy);
     WRAP(dispatch_group_leave)(group);
     dispatch_release(group);
   });
 }
 
 TSAN_INTERCEPTOR(void, dispatch_group_async_f, dispatch_group_t group,
                  dispatch_queue_t queue, void *context,
                  dispatch_function_t work) {

compiler-rt/trunk/test/tsan/Darwin/gcd-groups-destructor.mm

+// RUN: %clang_tsan %s -o %t -framework Foundation
+// RUN: %env_tsan_opts=ignore_interceptors_accesses=1 %run %t 2>&1 | FileCheck %s
+
+#import <Foundation/Foundation.h>
+
+#import <memory>
+#import <stdatomic.h>
+
+_Atomic(long) destructor_counter = 0;
+
+struct MyStruct {
+  virtual ~MyStruct() {
+    usleep(10000);
+    atomic_fetch_add_explicit(&destructor_counter, 1, memory_order_relaxed);
+  }
+};
+
+int main(int argc, const char *argv[]) {
+  fprintf(stderr, "Hello world.\n");
+
+  dispatch_queue_t q = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
+  dispatch_group_t g = dispatch_group_create();
+
+  for (int i = 0; i < 100; i++) {
+    std::shared_ptr<MyStruct> shared(new MyStruct());
+
+    dispatch_group_async(g, q, ^{
+      shared.get(); // just to make sure the object is captured by the block
+    });
+  }
+
+  dispatch_group_wait(g, DISPATCH_TIME_FOREVER);
+
+  if (destructor_counter != 100) {
+    abort();
+  }
+
+  fprintf(stderr, "Done.\n");
+}
+
+// CHECK: Hello world.
+// CHECK-NOT: WARNING: ThreadSanitizer
+// CHECK: Done.