sanitizers: call __cxa_demangle only on mangled external names

[emaste]

Summary

Explicitly check for mangled external names (starting with "_Z") as __cxa_demangle also handles type mangling, which conflicts with unmangled names such as "f" or "m".

For reference D2552 is the same issue in lldb. See also the discussion at https://lists.freebsd.org/pipermail/freebsd-toolchain/2014-June/001153.html.

[eugenis]

Will this work on Mac?
I think they add leading underscores, but it's only for C symbols, or is it not?

[emaste]

In D19818#418920, @eugenis wrote:

Will this work on Mac?
I think they add leading underscores, but it's only for C symbols, or is it not?

Hrm. Indeed, on OS X nm reports __Z2fni as the mangled name for fn(int). In the case of lldb the extra leading _ must be stripped off earlier.

[kubamracek]

Can we return nullptr in case 'name' is nullptr? __cxa_demangle does that and this removes this behavior.

[emaste]

Restore handling of name = nullptr.

[filcab]

Can you create a C test with a function called f, i, m, whatever that
triggered the demangler and doesn't trigger anymore?

Thank you,

Filipe

[samsonov]

I am not sure, but IIRC somewhere we depend on the ability of __cxa_demangle to demangle the type names.

[samsonov]

That is, even if we want to add this check, it should be closer to the call site, where we know for sure we're demangling the variable name.

[filcab]

AFAICT, the only place we might be expecting something that doesn't
start with _Z to be demangled is in lib/ubsan/ubsan_diag.cc:188
But I'm not sure we're calling it in that way.

Ed: I guess the best thing would be to either:

• Audit usages of Demangle (and its users :-) ), and see if we

actually use it for demangling type names.

• If we use it only once or twice for that, maybe another function

for those cases would work

• Or just do what Alexey asked an check before you call Demangle.

Thank you,

Filipe

lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc

 
 // Attempts to demangle the name via __cxa_demangle from __cxxabiv1.
 const char *DemangleCXXABI(const char *name) {
   // FIXME: __cxa_demangle aggressively insists on allocating memory.
   // There's not much we can do about that, short of providing our
   // own demangler (libc++abi's implementation could be adapted so that
   // it does not allocate). For now, we just call it anyway, and we leak
   // the returned value.
-  if (__cxxabiv1::__cxa_demangle)
+  // Explicitly check for mangled external names (starting with "_Z") as
+  // __cxa_demangle also handles type mangling, which conflicts with
+  // unmangled names such as "f" or "m".
+  if (__cxxabiv1::__cxa_demangle && name && name[0] == '_' && name[1] == 'Z')
     if (const char *demangled_name =
           __cxxabiv1::__cxa_demangle(name, 0, 0, 0))
       return demangled_name;
 
   return name;
 }
 
 // Attempts to demangle a Swift name. The demangler will return nullptr