sanitizer_common/symbolizer: fix crashes during exit

[dvyukov]

Summary

Override __cxa_atexit and ignore callbacks.
This prevents crashes in a configuration when the symbolizer
is built into sanitizer runtime and consequently into the test process.
LLVM libraries have some global objects destroyed during exit,
so if the test process triggers any bugs after that, the symbolizer crashes.
An example stack trace of such crash:

For the standalone llvm-symbolizer this does not hurt,
we just don't destroy few global objects on exit.

[dvyukov]

fix commit message to include stack

[kda]

LGTM, but I don't know if I know enough to see something bad.

[vitalybuka]

compiler-rt/test/tsan/atexit4.cpp

2

another solution is to build LLVM_ENABLE_THREADS=0
I assume this test triggers nothing in upstream, as upstream build is already with LLVM_ENABLE_THREADS=0?

[vitalybuka]

compiler-rt/lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp

134

I think more appropriate place is sanitizer_wrappers.cpp as here we keep interface and there libc hacks

[dvyukov]

compiler-rt/test/tsan/atexit4.cpp

2

another solution is to build LLVM_ENABLE_THREADS=0

But won't globals still registered for destruction at exit?

I assume this test triggers nothing in upstream, as upstream build is already with LLVM_ENABLE_THREADS=0?

This is not triggered upstream, because upstream we don't compile symbolizer into sanitizers. Symbolizer as standalone process is not affected by the problem entirely.

[vitalybuka]

compiler-rt/test/tsan/atexit4.cpp

2

another solution is to build LLVM_ENABLE_THREADS=0

But won't globals still registered for destruction at exit?

Yes, they would, LLVM_ENABLE_THREADS will avoid recursive_mutex and this crash.
I like you solution, it's more future proof then just LLVM_ENABLE_THREADS.

I assume this test triggers nothing in upstream, as upstream build is already with LLVM_ENABLE_THREADS=0?

This is not triggered upstream, because upstream we don't compile symbolizer into sanitizers. Symbolizer as standalone process is not affected by the problem entirely.

We do https://lab.llvm.org/buildbot/#/builders/37/builds/4500 "with symbolizer" tests, but they are LLVM_ENABLE_THREADS=0

compiler-rt/lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp

int __sanitizer_symbolize_demangle(const char *Name, char *Buffer,
   std::string Result =
       llvm::symbolize::LLVMSymbolizer::DemangleName(Name, nullptr);
   return __sanitizer::internal_snprintf(Buffer, MaxLength, "%s",
                                         Result.c_str()) < MaxLength
              ? static_cast<int>(Result.size() + 1)
              : 0;
 }
 
+// Override __cxa_atexit and ignore callbacks.
+// This prevents crashes in a configuration when the symbolizer
+// is built into sanitizer runtime and consequently into the test process.
+// LLVM libraries have some global objects destroyed during exit,
+// so if the test process triggers any bugs after that, the symbolizer crashes.
+// An example stack trace of such crash:
+//
+// #1  __cxa_throw
+// #2  std::__u::__throw_system_error
+// #3  std::__u::recursive_mutex::lock
+// #4  __sanitizer_llvm::ManagedStaticBase::RegisterManagedStatic
+// #5  __sanitizer_llvm::errorToErrorCode
+// #6  __sanitizer_llvm::getFileAux
+// #7  __sanitizer_llvm::MemoryBuffer::getFileOrSTDIN
+// #10 __sanitizer_llvm::symbolize::LLVMSymbolizer::getOrCreateModuleInfo
+// #13 __sanitizer::Symbolizer::SymbolizeData
+// #14 __tsan::SymbolizeData
+// #16 __tsan::ReportRace
+// #18 __tsan_write4
+// #19 race() () at test/tsan/atexit4.cpp
+// #20 cxa_at_exit_wrapper
+// #21 __cxa_finalize
+// #22 __do_fini
+//
+// For the standalone llvm-symbolizer this does not hurt,
+// we just don't destroy few global objects on exit.
+int __cxa_atexit(void (*f)(void *a), void *arg, void *dso) { return 0; }
+
 }  // extern "C"

compiler-rt/test/tsan/atexit4.cpp

+// RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t | FileCheck %s
+#include "test.h"
+
+void *thread(void *x) {
+  barrier_wait(&barrier);
+  *static_cast<int *>(x) = 2;
+  return nullptr;
+}
+
+static void race() {
+  int data = 0;
+  pthread_t t;
+  pthread_create(&t, nullptr, thread, &data);
+  data = 1;
+  barrier_wait(&barrier);
+  pthread_join(t, nullptr);
+}
+
+struct X {
+  X() { atexit(race); }
+} x;
+
+int main() {
+  barrier_init(&barrier, 2);
+  fprintf(stderr, "DONE\n");
+}
+
+// CHECK: DONE
+// CHECK: WARNING: ThreadSanitizer: data race