Handle NetBSD specific indirection of libpthread functions

[krytarowski]

Summary

Correct handling of libpthread(3) functions in TSan/NetBSD:

• pthread_cond_init(3),
• pthread_cond_signal(3),
• pthread_cond_broadcast(3),
• pthread_cond_wait(3),
• pthread_cond_destroy(3),
• pthread_mutex_init(3),
• pthread_mutex_destroy(3),
• pthread_mutex_trylock(3),
• pthread_rwlock_init(3),
• pthread_rwlock_destroy(3),
• pthread_rwlock_rdlock(3),
• pthread_rwlock_tryrdlock(3),
• pthread_rwlock_wrlock(3),
• pthread_rwlock_trywrlock(3),
• pthread_rwlock_unlock(3),
• pthread_once(3).

Code out of the libpthread(3) context uses the libc symbols
that are prefixed with libc_, for example: libc_cond_init.

This caused that these functions were invisible to sanitizers on NetBSD.
Intercept the libc-specific ones and add them as NetBSD-specific aliases
for the common pthread(3) ones.

NetBSD needs to intercept both functions, as the regularly named ones
are used internally in libpthread(3).

Sponsored by <The NetBSD Foundation>

[krytarowski]

There is an open question what to do with tests. They report results with the mangled name in stack trace, instead of the regular libpthread(3) ones.

$ ./mutex                                                                    
==================
WARNING: ThreadSanitizer: data race (pid=13274)
  Write of size 4 at 0x0000014f9ec0 by thread T1 (mutexes: write M76):
    #0 Thread1(void*) /public/llvm-build/mutex.cc:36:9 (mutex+0x46e5ec)

  Previous write of size 4 at 0x0000014f9ec0 by thread T2:
    #0 Thread2(void*) /public/llvm-build/mutex.cc:42:9 (mutex+0x46e6b7)

  Location is global 'Global' of size 4 at 0x0000014f9ec0 (mutex+0x0000014f9ec0)

  Mutex M76 (0x0000014f9ec8) created at:
    #0 __libc_mutex_init /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1137:3 (mutex+0x413ef3)
    #1 main /public/llvm-build/mutex.cc:66:3 (mutex+0x46e796)

  Thread T1 (tid=2, running) created by main thread at:
    #0 pthread_create /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:921:3 (mutex+0x412cf0)
    #1 main /public/llvm-build/mutex.cc:71:3 (mutex+0x46e7b5)

  Thread T2 (tid=3, finished) created by main thread at:
    #0 pthread_create /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:921:3 (mutex+0x412cf0)
    #1 main /public/llvm-build/mutex.cc:72:3 (mutex+0x46e7d4)

SUMMARY: ThreadSanitizer: data race /public/llvm-build/mutex.cc:36:9 in Thread1(void*)
==================
ThreadSanitizer: reported 1 warnings

__libc_mutex_init printed instead of pthread_mutex_init. This breaks tests that check for exact strings.

I will implement fix for tests (demangle names or change tests) in a dedicated review.

[dvyukov]

In D40243#930139, @krytarowski wrote:

There is an open question what to do with tests. They report results with the mangled name in stack trace, instead of the regular libpthread(3) ones.

$ ./mutex                                                                    
==================
WARNING: ThreadSanitizer: data race (pid=13274)
  Write of size 4 at 0x0000014f9ec0 by thread T1 (mutexes: write M76):
    #0 Thread1(void*) /public/llvm-build/mutex.cc:36:9 (mutex+0x46e5ec)

  Previous write of size 4 at 0x0000014f9ec0 by thread T2:
    #0 Thread2(void*) /public/llvm-build/mutex.cc:42:9 (mutex+0x46e6b7)

  Location is global 'Global' of size 4 at 0x0000014f9ec0 (mutex+0x0000014f9ec0)

  Mutex M76 (0x0000014f9ec8) created at:
    #0 __libc_mutex_init /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1137:3 (mutex+0x413ef3)
    #1 main /public/llvm-build/mutex.cc:66:3 (mutex+0x46e796)

  Thread T1 (tid=2, running) created by main thread at:
    #0 pthread_create /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:921:3 (mutex+0x412cf0)
    #1 main /public/llvm-build/mutex.cc:71:3 (mutex+0x46e7b5)

  Thread T2 (tid=3, finished) created by main thread at:
    #0 pthread_create /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:921:3 (mutex+0x412cf0)
    #1 main /public/llvm-build/mutex.cc:72:3 (mutex+0x46e7d4)

SUMMARY: ThreadSanitizer: data race /public/llvm-build/mutex.cc:36:9 in Thread1(void*)
==================
ThreadSanitizer: reported 1 warnings

__libc_mutex_init printed instead of pthread_mutex_init. This breaks tests that check for exact strings.

I will implement fix for tests (demangle names or change tests) in a dedicated review.

Check out PrintStack in tsan_report.cc. We already mess with interceptor names there (e.g. with interceptors they would like as __interceptor_pthread_mutex_lock even on linux). I think it's the right place to patch function names.

lib/tsan/rtl/tsan_interceptors.cc

2480

Perhaps we need some helper macro so this can be spelled along the lines of:

NETBSD_ALIAS(cond_init);

I think we could declare all alias functions as taking no arguments.

[krytarowski]

Add helper macros to make the code prettier and reduce copy-and-paste.

[krytarowski]

lib/tsan/rtl/tsan_interceptors.cc

2480

Done. I've decided to keep arguments.

lib/tsan/rtl/tsan_interceptors.cc

   DestroyThreadState();
   REAL(_lwp_exit)();
 }
 #define TSAN_MAYBE_INTERCEPT__LWP_EXIT TSAN_INTERCEPT(_lwp_exit)
 #else
 #define TSAN_MAYBE_INTERCEPT__LWP_EXIT
 #endif
 
+#if SANITIZER_NETBSD
+// NetBSD uses indirection for old threading functions for historical reasons
+TSAN_INTERCEPTOR(void, __libc_cond_init, void *c, void *a) \
+  ALIAS(WRAPPER_NAME(pthread_cond_init));
+TSAN_INTERCEPTOR(void, __libc_cond_signal, void *c) \
+  ALIAS(WRAPPER_NAME(pthread_cond_signal));
+TSAN_INTERCEPTOR(void, __libc_cond_broadcast, void *c) \
+  ALIAS(WRAPPER_NAME(pthread_cond_broadcast));
+TSAN_INTERCEPTOR(void, __libc_cond_wait, void *c, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_cond_wait));
+TSAN_INTERCEPTOR(void, __libc_cond_destroy, void *c) \
+  ALIAS(WRAPPER_NAME(pthread_cond_destroy));
+TSAN_INTERCEPTOR(void, __libc_mutex_init, void *m, void *a) \
+  ALIAS(WRAPPER_NAME(pthread_mutex_init));
+TSAN_INTERCEPTOR(void, __libc_mutex_destroy, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_mutex_destroy));
+TSAN_INTERCEPTOR(void, __libc_mutex_trylock, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_mutex_trylock));
+TSAN_INTERCEPTOR(void, __libc_rwlock_init, void *m, void *a) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_init));
+TSAN_INTERCEPTOR(void, __libc_rwlock_destroy, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_destroy));
+TSAN_INTERCEPTOR(void, __libc_rwlock_rdlock, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_rdlock));
+TSAN_INTERCEPTOR(void, __libc_rwlock_tryrdlock, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_tryrdlock));
+TSAN_INTERCEPTOR(void, __libc_rwlock_wrlock, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_wrlock));
+TSAN_INTERCEPTOR(void, __libc_rwlock_trywrlock, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_trywrlock));
+TSAN_INTERCEPTOR(void, __libc_rwlock_unlock, void *m) \
+  ALIAS(WRAPPER_NAME(pthread_rwlock_unlock));
+TSAN_INTERCEPTOR(void, __libc_thr_once, void *o, void (*f)()) \
+  ALIAS(WRAPPER_NAME(pthread_once));
+#endif
+
 namespace __tsan {
 
 static void finalize(void *arg) {
   ThreadState *thr = cur_thread();
   int status = Finalize(thr);
   // Make sure the output is not lost.
   FlushStreams();
   if (status)

 
 #if !SANITIZER_MAC && !SANITIZER_NETBSD
   if (pthread_key_create(&interceptor_ctx()->finalize_key, &thread_finalize)) {
     Printf("ThreadSanitizer: failed to create thread key\n");
     Die();
   }
 #endif
 
+#if SANITIZE_NETBSD
+  // NetBSD uses indirection for old threading functions for historical reasons
+  TSAN_INTERCEPT(__libc_cond_init);
+  TSAN_INTERCEPT(__libc_cond_signal);
+  TSAN_INTERCEPT(__libc_cond_broadcast);
+  TSAN_INTERCEPT(__libc_cond_wait);
+  TSAN_INTERCEPT(__libc_cond_destroy);
+  TSAN_INTERCEPT(__libc_mutex_init);
+  TSAN_INTERCEPT(__libc_mutex_destroy);
+  TSAN_INTERCEPT(__libc_mutex_trylock);
+  TSAN_INTERCEPT(__libc_rwlock_init);
+  TSAN_INTERCEPT(__libc_rwlock_destroy);
+  TSAN_INTERCEPT(__libc_rwlock_rdlock);
+  TSAN_INTERCEPT(__libc_rwlock_tryrdlock);
+  TSAN_INTERCEPT(__libc_rwlock_wrlock);
+  TSAN_INTERCEPT(__libc_rwlock_trywrlock);
+  TSAN_INTERCEPT(__libc_rwlock_unlock);
+  TSAN_INTERCEPT(__libc_thr_once);
+#endif
+
   FdInit();
 }
 
 }  // namespace __tsan
 
 // Invisible barrier for tests.
 // There were several unsuccessful iterations for this functionality:
 // 1. Initially it was implemented in user code using