[sanitizers] Teach the internal demangler about Swift names

[zaks.anna]

Summary

Add support for Swift names when symbolicating sanitizer traces. This is now relevant since TSan and ASan support have been added to Swift on OS X.

The swift demangler will return null if we are dealing with a non-Swift string, which allows to chain the CXX demangler. I am using swift_demangle function implemented here: https://github.com/apple/swift/pull/2169/commits/4ba7e418cb30e9fd10908bcbb6e9c91d28b8069c

(The test is only testing that no regressions are introduced to the non-Swift demangling since the Swift runtime is not accessible from compiler-rt. I plan on adding further testing of this in the Swift codebase.)

[kcc]

Anna,
Sadly, Alexey has left our team (he may still want to comment, but I wouldn't count on it).
I'll let Mike review this further.

lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc

77 ↗

(On Diff #53795)

does this mean that we call dlsym every time we are trying to demangle something?

lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

59 ↗

(On Diff #53795)

So, is a swift-specific test possible?

[zaks.anna]

lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc

77 ↗

(On Diff #53795)

Yes. I'll add a check for the Swift name prefix before calling dlsym.

lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

59 ↗

(On Diff #53795)

The dlsym will fail since compiler-rt does not depend on Swift runtime that provides the demangler. The test below is a Swift name, but we are only checking that we don't crash or turn it into something else here.

We should be able to add a test to the Swift repo, which is next on my list. (That is blocked on making sure the Swift buildbots checkout compiler-rt.)

[zaks.anna]

Only call dlsym if we have a Swift name.

[zaks.anna]

Addressed Kostya's comments.

[aizatsky]

lib/sanitizer_common/sanitizer_symbolizer_internal.h

31 ↗

(On Diff #53914)

should probably be DemangleSwiftAndCXX because of the demangling order.

lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

59 ↗

(On Diff #53914)

Let's have two separate test cases: DemangleCXX and DemangleSwift.

Let's also make the comment a statement.
// Swift names are not demangled in default llvm build because Swift runtime is not linked in.

[zaks.anna]

lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

59 ↗

(On Diff #53914)

Let's have two separate test cases: DemangleCXX and DemangleSwift.

It's important to test DemangleCXXAndSwift specifically since it's the only API that others will call and testing it ensures that the chaining works. (DemangleCXX and DemangleSwift are not exposed in the header and should not be.)

[aizatsky]

lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

59 ↗

(On Diff #53914)

I mean this:

TEST(Symbolizer, DemangleCXX) {
  EXPECT_STREQ("f1(char*, int)", DemangleCXXAndSwift("_Z2f1Pci"));
  EXPECT_STREQ("foo", DemangleCXXAndSwift("foo"));
  EXPECT_STREQ("", DemangleCXXAndSwift(""));
}

TEST(Symbolizer, DemangleSwift) {
  // ...
  EXPECT_STREQ("_TtSd", DemangleCXXAndSwift("_TtSd"));
}

[zaks.anna]

lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

59 ↗

(On Diff #53914)

But the following are not testing either Swift or CXX:

EXPECT_STREQ("foo", DemangleCXXAndSwift("foo"));
EXPECT_STREQ("", DemangleCXXAndSwift(""));

The point of this whole test is to check that the combined function works. We are not testing CXX or Swift detangling.. I can change it but I do not think that would be best.

compiler-rt/trunk/lib/sanitizer_common/sanitizer_symbolizer_internal.h

 // InternalAlloc) and null-terminataed buffer is returned. They return a pointer
 // to the next characted after the found delimiter.
 const char *ExtractToken(const char *str, const char *delims, char **result);
 const char *ExtractInt(const char *str, const char *delims, int *result);
 const char *ExtractUptr(const char *str, const char *delims, uptr *result);
 const char *ExtractTokenUpToDelimiter(const char *str, const char *delimiter,
                                       char **result);
 
-const char *DemangleCXXABI(const char *name);
+const char *DemangleSwiftAndCXX(const char *name);
 
 // SymbolizerTool is an interface that is implemented by individual "tools"
 // that can perform symbolication (external llvm-symbolizer, libbacktrace,
 // Windows DbgHelp symbolizer, etc.).
 class SymbolizerTool {
  public:
   // The main |Symbolizer| class implements a "fallback chain" of symbolizer
   // tools. In a request to symbolize an address, if one tool returns false,

compiler-rt/trunk/lib/sanitizer_common/sanitizer_symbolizer_mac.cc

 #include <sys/wait.h>
 #include <unistd.h>
 #include <util.h>
 
 bool DlAddrSymbolizer::SymbolizePC(uptr addr, SymbolizedStack *stack) {
   Dl_info info;
   int result = dladdr((const void *)addr, &info);
   if (!result) return false;
-  const char *demangled = DemangleCXXABI(info.dli_sname);
+  const char *demangled = DemangleSwiftAndCXX(info.dli_sname);
   stack->info.function = demangled ? internal_strdup(demangled) : nullptr;
   return true;
 }
 
 bool DlAddrSymbolizer::SymbolizeData(uptr addr, DataInfo *datainfo) {
   Dl_info info;
   int result = dladdr((const void *)addr, &info);
   if (!result) return false;
-  const char *demangled = DemangleCXXABI(info.dli_sname);
+  const char *demangled = DemangleSwiftAndCXX(info.dli_sname);
   datainfo->name = internal_strdup(demangled);
   datainfo->start = (uptr)info.dli_saddr;
   return true;
 }
 
 class AtosSymbolizerProcess : public SymbolizerProcess {
  public:
   explicit AtosSymbolizerProcess(const char *path, pid_t parent_pid)

compiler-rt/trunk/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc

 #include "sanitizer_linux.h"
 #include "sanitizer_placement_new.h"
 #include "sanitizer_posix.h"
 #include "sanitizer_procmaps.h"
 #include "sanitizer_symbolizer_internal.h"
 #include "sanitizer_symbolizer_libbacktrace.h"
 #include "sanitizer_symbolizer_mac.h"
 
+#include <dlfcn.h>   // for dlsym()
 #include <errno.h>
 #include <stdlib.h>
 #include <sys/wait.h>
 #include <unistd.h>
 
 #if SANITIZER_MAC
 #include <util.h>  // for forkpty()
 #endif  // SANITIZER_MAC
const char *DemangleCXXABI(const char *name) {
   if (__cxxabiv1::__cxa_demangle)
     if (const char *demangled_name =
           __cxxabiv1::__cxa_demangle(name, 0, 0, 0))
       return demangled_name;
 
   return name;
 }
 
+// Attempts to demangle a Swift name. The demangler will return nullptr
+/// if a non-Swift name is passed in.
+const char *DemangleSwift(const char *name) {
+  // Not to call dlsym every time we demangle, check if we are dealing with
+  // Swift mangled name first.
+  if (name[0] != '_' || name[1] != 'T') {
+    return nullptr;
+  }
+
+  // As of now, there are no headers for the Swift runtime. Once they are
+  // present, we will weakly link since we do not require Swift runtime to be
+  // linked.
+  typedef char *(*swift_demangle_ft)(const char *mangledName,
+                                     size_t mangledNameLength,
+                                     char *outputBuffer,
+                                     size_t *outputBufferSize,
+                                     uint32_t flags);
+  swift_demangle_ft swift_demangle_f =
+    (swift_demangle_ft) dlsym(RTLD_DEFAULT, "swift_demangle");
+  if (swift_demangle_f)
+    return swift_demangle_f(name, internal_strlen(name), 0, 0, 0);
+
+  return nullptr;
+}
+
+const char *DemangleSwiftAndCXX(const char *name) {
+  CHECK(name);
+  if (const char *swift_demangled_name = DemangleSwift(name))
+    return swift_demangled_name;
+  return DemangleCXXABI(name);
+}
+
 bool SymbolizerProcess::StartSymbolizerSubprocess() {
   if (!FileExists(path_)) {
     if (!reported_invalid_path_) {
       Report("WARNING: invalid path to external symbolizer!\n");
       reported_invalid_path_ = true;
     }
     return false;
   }

 class InternalSymbolizer : public SymbolizerTool {
  public:
   static InternalSymbolizer *get(LowLevelAllocator *alloc) { return 0; }
 };
 
 #endif  // SANITIZER_SUPPORTS_WEAK_HOOKS
 
 const char *Symbolizer::PlatformDemangle(const char *name) {
-  return DemangleCXXABI(name);
+  return DemangleSwiftAndCXX(name);
 }
 
 void Symbolizer::PlatformPrepareForSandboxing() {}
 
 static SymbolizerTool *ChooseExternalSymbolizer(LowLevelAllocator *allocator) {
   const char *path = common_flags()->external_symbolizer_path;
   const char *binary_name = path ? StripModuleName(path) : "";
   if (path && path[0] == '\0') {

compiler-rt/trunk/lib/sanitizer_common/tests/sanitizer_symbolizer_test.cc

TEST(Symbolizer, ExtractTokenUpToDelimiter) {
   char *token;
   const char *rest =
       ExtractTokenUpToDelimiter("aaa-+-bbb-+-ccc", "-+-", &token);
   EXPECT_STREQ("aaa", token);
   EXPECT_STREQ("bbb-+-ccc", rest);
   InternalFree(token);
 }
 
+TEST(Symbolizer, DemangleSwiftAndCXX) {
+  // Swift names are not demangled in default llvm build because Swift
+  // runtime is not linked in.
+  EXPECT_STREQ("_TtSd", DemangleSwiftAndCXX("_TtSd"));
+  // Check that the rest demangles properly.
+  EXPECT_STREQ("f1(char*, int)", DemangleSwiftAndCXX("_Z2f1Pci"));
+  EXPECT_STREQ("foo", DemangleSwiftAndCXX("foo"));
+  EXPECT_STREQ("", DemangleSwiftAndCXX(""));
+}
+
 }  // namespace __sanitizer