[msan] Don't unpoison phdrs on dlopen(NULL, 0)

[rnk]

Summary

dlopen(NULL, ...) is intended to give you back a handle to the
executable for use with dlsym. Casting it to link_map and using it with
ForEachMappedRegion results in a crash.

We also shouldn't unpoison the globals of a DSO that is already in
memory. This ensures that we don't do it for the executable, but in
general, MSan may have false positives if the DSO is already loaded.

[eugenis]

Did you mean "false negatives if the DSO is already loaded." ?
LGTM

[rnk]

Ah, yeah, false negatives. Thanks!

[rnk]

By the way, this fixes all the JIT crashes on the msan LLVM bot.

[eugenis]

In D14795#292323, @rnk wrote:

By the way, this fixes all the JIT crashes on the msan LLVM bot.

This is great. Thanks for working on this.

compiler-rt/trunk/lib/msan/msan_interceptors.cc

 #define COMMON_INTERCEPTOR_SET_THREAD_NAME(ctx, name) \
   do {                                                \
   } while (false)  // FIXME
 #define COMMON_INTERCEPTOR_SET_PTHREAD_NAME(ctx, thread, name) \
   do {                                                         \
   } while (false)  // FIXME
 #define COMMON_INTERCEPTOR_BLOCK_REAL(name) REAL(name)
 #define COMMON_INTERCEPTOR_ON_EXIT(ctx) OnExit()
-#define COMMON_INTERCEPTOR_LIBRARY_LOADED(filename, handle)  \
-  do {                                                       \
-    link_map *map = GET_LINK_MAP_BY_DLOPEN_HANDLE((handle)); \
-    if (map) ForEachMappedRegion(map, __msan_unpoison);      \
+#define COMMON_INTERCEPTOR_LIBRARY_LOADED(filename, handle)                    \
+  do {                                                                         \
+    link_map *map = GET_LINK_MAP_BY_DLOPEN_HANDLE((handle));                   \
+    if (filename && map)                                                       \
+      ForEachMappedRegion(map, __msan_unpoison);                               \
   } while (false)
 
 #define COMMON_INTERCEPTOR_GET_TLS_RANGE(begin, end)                           \
   if (MsanThread *t = GetCurrentThread()) {                                    \
     *begin = t->tls_begin();                                                   \
     *end = t->tls_end();                                                       \
   } else {                                                                     \
     *begin = *end = 0;                                                         \

compiler-rt/trunk/test/msan/dlopen_executable.cc

+// RUN: %clangxx_msan -O0 %s -o %t && not %run %t 2>&1 | FileCheck %s
+
+#include <assert.h>
+#include <dlfcn.h>
+#include <stdlib.h>
+
+static int my_global;
+
+int main(void) {
+  int *uninit = (int*)malloc(sizeof(int));
+  my_global = *uninit;
+  void *p = dlopen(0, RTLD_NOW);
+  assert(p && "failed to get handle to executable");
+  return my_global;
+  // CHECK: MemorySanitizer: use-of-uninitialized-value
+  // CHECK: #0 {{.*}} in main{{.*}}dlopen_executable.cc:[[@LINE-2]]
+}