Allow initialization of Asan interceptors before the general Asan initialization takes place on FreeBSD

[kutuzov.viktor.84]

Here's how things currently look during initialization on FreeBSD:

#0  AsanInitInternal () at /usr/home/ik/llvm/llvm.current/projects/compiler-rt/lib/asan/asan_rtl.cc:695
#1  0x0000000000411d7a in __interceptor_strncmp () at sanitizer_common_interceptors.inc:178
#2  0x000000080109e35e in getenv () from /lib/libc.so.7
#3  0x0000000801302d03 in pthread_timedjoin_np () from /lib/libthr.so.3
#4  0x00000008013055d2 in __error () from /lib/libthr.so.3
#5  0x00000008012f9bd6 in .init () from /lib/libthr.so.3
#6  0x00007fffffffd0d0 in ?? ()
#7  0x000000080065d611 in r_debug_state () from /libexec/ld-elf.so.1
#8  0x000000080065cc97 in __tls_get_addr () from /libexec/ld-elf.so.1
#9  0x000000080065b089 in .text () from /libexec/ld-elf.so.1
#10 0x0000000000000000 in ?? ()

Note that an intercepted function is called before Asan has a chance to initialize itself.

The fix eliminates mass failures on Asan tests--both unit and lit ones.

[kcc]

This overcomplicates things that are already overcomplicated to my taste.
I especially dislike extra if(FREEBSD).

What is wrong with the stack you've shown?

[kutuzov.viktor.84]

What is wrong with the stack you've shown?

The problem is that strncmp() is called before REAL(strncmp) is initialized so we have to do InitializeAsanInterceptors() before we can refer to the real function. Calling AsanInitInternal() cannot proceed correctly as it actually called from within initialization of libthr while AsanInitInternal() itself tries to initialize the thread subsystem what lead us to recursive initialization of libthr.

[eugenis]

Can you use ENSURE_ASAN_INITED?

[kutuzov.viktor.84]

Can you use ENSURE_ASAN_INITED?

It would cause calling AsanInitInternal() which, in turn, will attempt to (recursively) initialize threads.

[eugenis]

OK.
So, InitializeAsanInterceptors() here is OK, and AsanInit() is not. Could this be solved by initializing interceptors earlier in AsanInit? This initialization sequence is kind of fragile, and I don't like that we arbitrarily move part of it upwards in a platform-dependent way.

[kutuzov.viktor.84]

Could this be solved by initializing interceptors earlier in AsanInit? This initialization sequence is kind of fragile, and I don't like that we arbitrarily move part of it upwards in a platform-dependent way.

By the time when the first of the intercepted functions is called, the libthr (the library Asan's thread rely on on FreeBSD) is being initialized already. AsanInitInternal() initializes both the interceptors and threads unconditionally and thus causes recursive initialization of libthr.

The overall process looks like this:

• the loader calls libthr's .init();
• .init() calls getenv() during initialization of the library;
• genenv() relies on strncmp();
• strncmp() is intercepted, but at this phase the REAL(strncmp) is not yet initialized;
• since Asan is not initialized, AsanInitInternal() takes place;
• AsanInitInternal() initializes the references to the real functions and then call AsanTSDInit();
• AsanTSDInit() calls pthread_key_create();
• pthread_key_create() sees that libthr is not yet initialized (what means its initialization is not yet completed) and launches initialization of libthr.

So the answer to your question is: unfortunately, it doesn't matter how early interceptors get initialized in AsanInitInternal(); the point is that on FreeBSD the initialization of interceptors and initialization of threads are separate phases that cannot be tied together.

[kcc]

nice loop.
Maybe we could break the loop by calling internal_strncmp?

INTERCEPTOR(int, strncmp, const char *s1, const char *s2, uptr size) {

if (COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED)
   return internal_strncmp(s1, s2, size);

we have a couple of similar hacks elsewhere

[kutuzov.viktor.84]

Yes, it worked. Thanks, Kostya. Updated.

[kcc]

LGTM with one small nit

lib/sanitizer_common/sanitizer_common_interceptors.inc

95

why new line?

[kutuzov.viktor.84]

Closed by commit rL213941 (authored by vkutuzov).

lib/asan/asan_interceptors.cc

 // can be slow.
 #define COMMON_INTERCEPTOR_SET_PTHREAD_NAME(ctx, thread, name) \
   do {                                                         \
   } while (false)
 #define COMMON_INTERCEPTOR_BLOCK_REAL(name) REAL(name)
 #define COMMON_INTERCEPTOR_ON_EXIT(ctx) OnExit()
 #define COMMON_INTERCEPTOR_LIBRARY_LOADED(filename, res) CovUpdateMapping()
 #define COMMON_INTERCEPTOR_LIBRARY_UNLOADED() CovUpdateMapping()
+#define COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED (!asan_inited)
 #include "sanitizer_common/sanitizer_common_interceptors.inc"
 
 #define COMMON_SYSCALL_PRE_READ_RANGE(p, s) ASAN_READ_RANGE(p, s)
 #define COMMON_SYSCALL_PRE_WRITE_RANGE(p, s) ASAN_WRITE_RANGE(p, s)
 #define COMMON_SYSCALL_POST_READ_RANGE(p, s) \
   do {                                       \
     (void)(p);                               \
     (void)(s);                               \

lib/sanitizer_common/sanitizer_common_interceptors.inc

 //   COMMON_INTERCEPTOR_FD_ACCESS
 //   COMMON_INTERCEPTOR_SET_THREAD_NAME
 //   COMMON_INTERCEPTOR_ON_EXIT
 //   COMMON_INTERCEPTOR_MUTEX_LOCK
 //   COMMON_INTERCEPTOR_MUTEX_UNLOCK
 //   COMMON_INTERCEPTOR_MUTEX_REPAIR
 //   COMMON_INTERCEPTOR_SET_PTHREAD_NAME
 //   COMMON_INTERCEPTOR_HANDLE_RECVMSG
+//   COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED
 //===----------------------------------------------------------------------===//
 #include "interception/interception.h"
 #include "sanitizer_addrhashmap.h"
 #include "sanitizer_placement_new.h"
 #include "sanitizer_platform_interceptors.h"
 #include "sanitizer_tls_get_addr.h"
 
 #include <stdarg.h>

 #define COMMON_INTERCEPTOR_LIBRARY_UNLOADED() {}
 #endif
 
 #ifndef COMMON_INTERCEPTOR_ENTER_NOIGNORE
 #define COMMON_INTERCEPTOR_ENTER_NOIGNORE(ctx, ...) \
   COMMON_INTERCEPTOR_ENTER(ctx, __VA_ARGS__)
 #endif
 
+#ifndef COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED
+#define COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED \
+  (0)
+#endif
+
 struct FileMetadata {
   // For open_memstream().
   char **addr;
   SIZE_T *size;
 };
 
 struct CommonInterceptorMetadata {
   enum {
for (i = 0;; i++) {
     if (c1 != c2 || c1 == '\0') break;
   }
   COMMON_INTERCEPTOR_READ_RANGE(ctx, s1, i + 1);
   COMMON_INTERCEPTOR_READ_RANGE(ctx, s2, i + 1);
   return CharCmpX(c1, c2);
 }
 
 INTERCEPTOR(int, strncmp, const char *s1, const char *s2, uptr size) {
+  if (COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED)
+    return internal_strncmp(s1, s2, size);
   void *ctx;
   COMMON_INTERCEPTOR_ENTER(ctx, strncmp, s1, s2, size);
   unsigned char c1 = 0, c2 = 0;
   uptr i;
   for (i = 0; i < size; i++) {
     c1 = (unsigned char)s1[i];
     c2 = (unsigned char)s2[i];
     if (c1 != c2 || c1 == '\0') break;