[sanitizer_common] Ignore unloading of suppressed library

[protze.joachim]

Summary

When suppressing errors from uninstrumented MPI libraries, sanitizer runtime calls Die(), because some of the suppressed libraries are unloaded during execution.

This patch is to remove the hard failure and change the message to a verbose message.

[vitalybuka]

Could you please try to create a test which crashes here?

[protze.joachim]

I added a testcase. I added it just for Linux, because I have no machine ready to test dl-loading on other OS.

The test succeeds with the patch, but "Die"s when replacing the clang calls with release clang.
The test includes the library to be built as well as the application which loads and closes the library.

[vitalybuka]

LGTM
looks like this affects tsan only. @dvyukov do you know why original code was so strict? The patch https://reviews.llvm.org/rL191897 which added that Die() didn't explain that.

[dvyukov]

There are several reasons:

1. The address range is still ignored after unload. So if anything else loaded at that range it will be falsely ignore and this can lead to both false positives and false negatives.
2. The list of ignored libraries is append-only, so it's not possible to remove from it.
3. If a library is unloaded, it can be loaded again later. This will lead to unbounded growth of the list of ignored libraries. The list is fixed size, so soon we will crash with an obscure CHECK failure.

Nobody runs tsan with verbosity flag, so that message won't be helpful. We either need to resolve the problems, or at the very least do what we do with multithreaded fork -- fail loudly unless an explicit flag a-la enable_ignored_library_unloading_yes_I_know_what_I_am_doing_and_taking_full_responsibility is given.

[vitalybuka]

So we can have this as opt-in feature.

[protze.joachim]

@dvyukov thanks for the detailed reasoning.

I found the newly documented flag ignore_noninstrumented_module and guess that this would be the better option than listing all the MPI libraries in the suppression file.

[protze.joachim]

TSAN_OPTIONS="ignore_noninstrumented_module=1" works for us

lib/sanitizer_common/sanitizer_libignore.cc

for (const auto &mod : modules) {
         CHECK_LT(idx, ARRAY_SIZE(ignored_code_ranges_));
         ignored_code_ranges_[idx].begin = range.beg;
         ignored_code_ranges_[idx].end = range.end;
         atomic_store(&ignored_ranges_count_, idx + 1, memory_order_release);
         break;
       }
     }
     if (lib->loaded && !loaded) {
-      Report("%s: library '%s' that was matched against called_from_lib"
+      VReport(1, "%s: library '%s' that was matched against called_from_lib"
              " suppression '%s' is unloaded\n",
              SanitizerToolName, lib->name, lib->templ);
-      Die();
     }
   }
 
   // Track instrumented ranges.
   if (track_instrumented_libs_) {
     for (const auto &mod : modules) {
       if (!mod.instrumented())
         continue;

test/tsan/Linux/unload-suppressed-lib.c

+// RUN: %clang_tsan -O1 %s -o %t -DCODE && %clang_tsan -fPIC -shared -DLIB %s -o %t.so && echo "called_from_lib:%t.so" > %t.sup && env TSAN_OPTIONS="suppressions=%t.sup:$TSAN_OPTION" %run %t %t.so 2>&1
+// CHECK-NOT that was matched against called_from_lib suppression
+// CHECK foo(42)=
+// CHECK dlclose(lib)=
+
+#ifdef CODE
+
+#include <stdio.h>
+#include <dlfcn.h>
+int main(int argc, char** argv)
+{
+  void * lib = dlopen(argv[1], RTLD_LAZY);
+  int (*foo)(int) = dlsym(lib, "foo");
+  int res = foo(42);
+  printf("foo(42)=%i\n", res);
+  res = dlclose(lib);
+  printf("dlclose(lib)=%i\n", res);
+}
+
+#endif // library
+
+#ifdef LIB 
+int foo(int bar){
+    return bar+1;
+}
+#endif
+