Plug dlerror() leak for swift_demangle

[krytarowski]

Summary

InitializeSwiftDemangler() attempts to resolve the
swift_demangle symbol. If this is not available, we
observe dlerror message leak.

Caught on NetBSD/amd64 in TSan.

Sponsored by <The NetBSD Foundation>

[marcan]

FYI, this introduces a subtle regression. dlerror() calls into gettext to translate the error. ASAN itself can be initialized from a random malloc intercept which can turn out to be in gettext, which is quite common since apps initialize gettext early, and some libraries even do so in loader init calls. This ends up re-entering into gettext and corrupting a rwmutex by trying to take the write lock while the read-side is locked. The unlock sequence leaves the rwlock in a bad state. Things then deadlock much later on the bad mutex.

[pcc]

In D40382#4188437, @marcan wrote:

FYI, this introduces a subtle regression. dlerror() calls into gettext to translate the error. ASAN itself can be initialized from a random malloc intercept which can turn out to be in gettext, which is quite common since apps initialize gettext early, and some libraries even do so in loader init calls. This ends up re-entering into gettext and corrupting a rwmutex by trying to take the write lock while the read-side is locked. The unlock sequence leaves the rwlock in a bad state. Things then deadlock much later on the bad mutex.

Looks like this was fixed by D128992 by removing the call to dlerror().

lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc

typedef char *(*swift_demangle_ft)(const char *mangledName,
                                    size_t *outputBufferSize, uint32_t flags);
 static swift_demangle_ft swift_demangle_f;
 
 // This must not happen lazily at symbolication time, because dlsym uses
 // malloc and thread-local storage, which is not a good thing to do during
 // symbolication.
 static void InitializeSwiftDemangler() {
   swift_demangle_f = (swift_demangle_ft)dlsym(RTLD_DEFAULT, "swift_demangle");
+  (void)dlerror(); // Cleanup error message in case of failure
 }
 
 // Attempts to demangle a Swift name. The demangler will return nullptr if a
 // non-Swift name is passed in.
 const char *DemangleSwift(const char *name) {
   if (!name) return nullptr;
 
   // Check if we are dealing with a Swift mangled name first.