[Asan] Attempt to fix for FreeBSD

[devnexen]

Summary

After several different approaches, asan hangs at AsanTSDInit, thus choosing a similar workflow as fuschia for example. Keeping the current workflow for other oses.

[krytarowski]

NetBSD is also affected. We shall go for a shared solution at least between BSDs and Linux.

A similar problem is in MSan, HWMSan etc.

[devnexen]

In D47448#1113955, @krytarowski wrote:

NetBSD is also affected. We shall go for a shared solution at least between BSDs and Linux.

A similar problem is in MSan, HWMSan etc.

I would say BSD at first, Linux works fine and this solution would probably not satisfy Linux anyway (e.g. lsan for example).
I can extend to NetBSD no worries as long you re ok with this solution.

[devnexen]

Apart of this is the MSan issue particular to NetBSD and what sort of issue is it ? e.g. it works relatively ok under FreeBSD.

[devnexen]

ping

[devnexen]

Using thread local solution for NetBSD and FreeBSD, keeping pthread_key_create and such for Linux and Solaris.

[devnexen]

@krytarowski What do you think of this approach ? I ve tried several approaches to keep pthread_key_create/pthread_getspecific/pthread_setspecific but did not work out.

[krytarowski]

I'm not sure whether this is equivalent.

I have a local patch to intercept _lwp_exit(2) and call destructor from this level. But I'm swamped by more serious issues like now, so I'm have scheduled it for later.

[devnexen]

In D47448#1123313, @krytarowski wrote:

I'm not sure whether this is equivalent.

I have a local patch to intercept _lwp_exit(2) and call destructor from this level. But I'm swamped by more serious issues like now, so I'm have scheduled it for later.

I understand. As long it s before August :)

[MaskRay]

BTW, how did you figure out it dead locked in __asan_init?

I used clang++ -fsanitize=address -Wl,-dynamic-linker,/tmp/c/libexec/ld-elf.so.1 -Wl,-R,/tmp/c/lib -xc =(printf 'int main(){}') -o aa
where ld-elf.so.1 and libthr.so.3 and their debug info were extracted from FreeBSD 12.0-CURRENT snapshot base.txz and base-dbg.txz.

The gdb aa terminal was unresponsive so I had to pkill -STOP aa in another shell. Then I could print the backtrace of the aa process.

[devnexen]

No debug info for me I worked at that time only with release versions but having ported softwares to FreeBSD I knew how libpthread is "delicate".

[krytarowski]

Fixed for FreeBSD? There is still a problem for NetBSD, but I'm still swamped with other things.

[MaskRay]

In D47448#1149017, @krytarowski wrote:

Fixed for FreeBSD? There is still a problem for NetBSD, but I'm still swamped with other things.

Fixed for FreeBSD 12.0-CURRENT (which lives in my qemu :( ). Other have told me OpenBSD is good. What's the issue for NetBSD?

[krytarowski]

In D47448#1149018, @MaskRay wrote:

In D47448#1149017, @krytarowski wrote:

Fixed for FreeBSD? There is still a problem for NetBSD, but I'm still swamped with other things.

Fixed for FreeBSD 12.0-CURRENT (which lives in my qemu :( ). Other have told me OpenBSD is good. What's the issue for NetBSD?

pthread functions are called too early, before initialization of the threading library

[devnexen]

In D47448#1149027, @krytarowski wrote:

In D47448#1149018, @MaskRay wrote:

In D47448#1149017, @krytarowski wrote:

Fixed for FreeBSD? There is still a problem for NetBSD, but I'm still swamped with other things.

Fixed for FreeBSD 12.0-CURRENT (which lives in my qemu :( ). Other have told me OpenBSD is good. What's the issue for NetBSD?

pthread functions are called too early, before initialization of the threading library

When you will have more time we can reexamine this fix or at least takng inspiration from if you wish.

lib/asan/asan_posix.cc

 void AsanOnDeadlySignal(int signo, void *siginfo, void *context) {
   StartReportDeadlySignal();
   SignalContext sig(siginfo, context);
   ReportDeadlySignal(sig);
 }
 
 // ---------------------- TSD ---------------- {{{1
 
+#if !SANITIZER_FREEBSD
 static pthread_key_t tsd_key;
+#else
+struct Tsd {
+  void *data;
+  void (*destructor)(void *tsd);
+  Tsd() : data(nullptr), destructor(nullptr) {}
+  ~Tsd() {
+    if (destructor)
+      destructor(data);
+  }
+};
+
+static thread_local Tsd _tsd;
+#endif
 static bool tsd_key_inited = false;
 void AsanTSDInit(void (*destructor)(void *tsd)) {
   CHECK(!tsd_key_inited);
   tsd_key_inited = true;
+#if !SANITIZER_FREEBSD
   CHECK_EQ(0, pthread_key_create(&tsd_key, destructor));
+#else
+  CHECK_EQ(destructor, &PlatformTSDDtor);
+  _tsd.destructor = destructor;
+#endif
 }
 
 void *AsanTSDGet() {
   CHECK(tsd_key_inited);
+#if !SANITIZER_FREEBSD
   return pthread_getspecific(tsd_key);
+#else
+  return _tsd.data;
+#endif
 }
 
 void AsanTSDSet(void *tsd) {
   CHECK(tsd_key_inited);
+#if !SANITIZER_FREEBSD
   pthread_setspecific(tsd_key, tsd);
+#else
+  _tsd.data = tsd;
+#endif
 }
 
 void PlatformTSDDtor(void *tsd) {
+#if !SANITIZER_FREEBSD
   AsanThreadContext *context = (AsanThreadContext*)tsd;
   if (context->destructor_iterations > 1) {
     context->destructor_iterations--;
     CHECK_EQ(0, pthread_setspecific(tsd_key, tsd));
     return;
   }
+#endif
   AsanThread::TSDDtor(tsd);
 }
 }  // namespace __asan
 
 #endif  // SANITIZER_POSIX