[llvm-symbolizer] Remove underscores and other C mangling on Windows

[rnk]

Summary

This makes it so that reports symbolized after the fact with
llvm-symbolizer are more similar to the ones we generate at runtime with
in-process dbghelp.

[samsonov]

tools/llvm-symbolizer/LLVMSymbolize.cpp

226 ↗

(On Diff #31410)

This is not the place we demangle the function names - see LLVMSymbolizer::DemangleName. At that point we already lose the knowledge of whether the module was ELF, or COFF, or MachO, but is it that important? Also, why doesn't ::UnDecorateSymbolName work for you?

[rnk]

tools/llvm-symbolizer/LLVMSymbolize.cpp

226 ↗

(On Diff #31410)

Yes, it's important that we only do this for 32-bit x86 symbols, which requires looking at the machine type of the module. See isWin32Module().

UnDecorateSymbolName only handles C++ symbols. This won't interfere because C++ symbols all start with '?'.

[samsonov]

tools/llvm-symbolizer/LLVMSymbolize.cpp

92 ↗

(On Diff #31410)

See below - if you make it a part of DemangleName , you can simplify the code with early-returns.

226 ↗

(On Diff #31410)

Still, let's keep all the demangling work in a single place - where we actually render DILineInfo entries. If you need to know smth. about the object file that produced these DILineInfo entries (which looks reasonable - demangling is obviously format-dependent) - just pass it down to printDILineInfo/DemangleName.

[rnk]

• Add IsWin32Symbol to DILineInfo

tools/llvm-symbolizer/LLVMSymbolize.cpp

226 ↗

(On Diff #31410)

I can do that, but it seems kind of silly to add a field to DILineInfo that is only used in LLVMSymbolize.cpp. There are other platforms with underscore prefixes, like MachO, that don't go this way. The underscore is removed earlier under an isMachO() in addSymbol().

[samsonov]

tools/llvm-symbolizer/LLVMSymbolize.cpp

188 ↗

(On Diff #31709)

Ah, sorry, I wasn't clear enough :(
See below.

217 ↗

(On Diff #31709)

You can pass ModuleInfo* (or the kind of that object file) to printDILineInfo here.

233 ↗

(On Diff #31709)

and here

542 ↗

(On Diff #31709)

and use the type of object file here to figure out which demangling you should actually do

[rnk]

• Pass ModuleInfo down into printDILineInfo

tools/llvm-symbolizer/LLVMSymbolize.cpp

188 ↗

(On Diff #31709)

Oh, that makes a lot more sense. :)

[samsonov]

LGTM with a note.

tools/llvm-symbolizer/LLVMSymbolize.cpp

568 ↗

(On Diff #31717)

This code will be unreachable if _MSC_VER is not defined.

llvm/trunk/test/tools/llvm-symbolizer/pdb/Inputs/test.cpp

 }
 
 int main() {
   foo();
   
   NS::Foo f;
   f.bar();
 }
+
+extern "C" {
+void __cdecl foo_cdecl() {}
+void __stdcall foo_stdcall() {}
+void __fastcall foo_fastcall() {}
+void __vectorcall foo_vectorcall() {}
+}

llvm/trunk/test/tools/llvm-symbolizer/pdb/Inputs/test.exe.input

+0x401000
+0x401010
+0x401070
 0x401030
 0x401040
+0x401050
 0x401060
 0x500000

llvm/trunk/test/tools/llvm-symbolizer/pdb/pdb.test

-RUN: llvm-symbolizer -obj="%p/Inputs/test.exe" < "%p/Inputs/test.exe.input" | \
-RUN:    FileCheck %s --check-prefix=CHECK
-RUN: llvm-symbolizer -obj="%p/Inputs/test.exe" -demangle=false < \
-RUN:    "%p/Inputs/test.exe.input" | FileCheck %s --check-prefix=CHECK-NO-DEMANGLE
-
-CHECK: foo(void)
-CHECK-NEXT: test.cpp:10
-CHECK: _main
-CHECK-NEXT: test.cpp:13:0
-CHECK: NS::Foo::bar(void)
-CHECK-NEXT: test.cpp:6:0
-
-CHECK-NO-DEMANGLE: foo
-CHECK-NO-DEMANGLE-NEXT: test.cpp:10
-CHECK-NO-DEMANGLE: _main
-CHECK-LINKAGE-NAME-NEXT: test.cpp:13:0
-CHECK-NO-DEMANGLE: bar
-CHECK-LINKAGE-NAME-NEXT: test.cpp:6:0
+RUN: llvm-symbolizer -obj="%p/Inputs/test.exe" < "%p/Inputs/test.exe.input" | \
+RUN:    FileCheck %s --check-prefix=CHECK
+RUN: llvm-symbolizer -obj="%p/Inputs/test.exe" -demangle=false < \
+RUN:    "%p/Inputs/test.exe.input" | FileCheck %s --check-prefix=CHECK-NO-DEMANGLE
+
+CHECK: foo(void)
+CHECK-NEXT: test.cpp:10
+CHECK: main
+CHECK-NEXT: test.cpp:13:0
+CHECK: NS::Foo::bar(void)
+CHECK-NEXT: test.cpp:6:0
+CHECK: {{^foo_cdecl$}}
+CHECK: {{^foo_stdcall$}}
+CHECK: {{^foo_fastcall$}}
+CHECK: {{^foo_vectorcall$}}
+
+CHECK-NO-DEMANGLE: ?foo@@YAXXZ
+CHECK-NO-DEMANGLE-NEXT: test.cpp:10
+CHECK-NO-DEMANGLE: _main
+CHECK-NO-DEMANGLE-NEXT: test.cpp:13
+CHECK-NO-DEMANGLE: ?bar@Foo@NS@@QAEXXZ
+CHECK-NO-DEMANGLE-NEXT: test.cpp:6
+CHECK-NO-DEMANGLE: _foo_cdecl
+CHECK-NO-DEMANGLE: _foo_stdcall@0
+CHECK-NO-DEMANGLE: @foo_fastcall@0
+CHECK-NO-DEMANGLE: foo_vectorcall@@0

llvm/trunk/tools/llvm-symbolizer/LLVMSymbolize.h

public:
 
   // Returns the result of symbolization for module name/offset as
   // a string (possibly containing newlines).
   std::string
   symbolizeCode(const std::string &ModuleName, uint64_t ModuleOffset);
   std::string
   symbolizeData(const std::string &ModuleName, uint64_t ModuleOffset);
   void flush();
-  static std::string DemangleName(const std::string &Name);
+  static std::string DemangleName(const std::string &Name, ModuleInfo *ModInfo);
+
 private:
   typedef std::pair<ObjectFile*, ObjectFile*> ObjectPair;
 
   ModuleInfo *getOrCreateModuleInfo(const std::string &ModuleName);
   ObjectFile *lookUpDsymFile(const std::string &Path, const MachOObjectFile *ExeObj,
                              const std::string &ArchName);
 
   /// \brief Returns pair of pointers to object and debug object.
   ObjectPair getOrCreateObjects(const std::string &Path,
                                 const std::string &ArchName);
   /// \brief Returns a parsed object file for a given architecture in a
   /// universal binary (or the binary itself if it is an object file).
   ObjectFile *getObjectFileFromBinary(Binary *Bin, const std::string &ArchName);
 
-  std::string printDILineInfo(DILineInfo LineInfo) const;
+  std::string printDILineInfo(DILineInfo LineInfo, ModuleInfo *ModInfo) const;
 
   // Owns all the parsed binaries and object files.
   SmallVector<std::unique_ptr<Binary>, 4> ParsedBinariesAndObjects;
   SmallVector<std::unique_ptr<MemoryBuffer>, 4> MemoryBuffers;
   void addOwningBinary(OwningBinary<Binary> OwningBin) {
     std::unique_ptr<Binary> Bin;
     std::unique_ptr<MemoryBuffer> MemBuf;
     std::tie(Bin, MemBuf) = OwningBin.takeBinary();
public:
 
   DILineInfo symbolizeCode(uint64_t ModuleOffset,
                            const LLVMSymbolizer::Options &Opts) const;
   DIInliningInfo symbolizeInlinedCode(
       uint64_t ModuleOffset, const LLVMSymbolizer::Options &Opts) const;
   bool symbolizeData(uint64_t ModuleOffset, std::string &Name, uint64_t &Start,
                      uint64_t &Size) const;
 
+  // Return true if this is a 32-bit x86 PE COFF module.
+  bool isWin32Module() const;
+
 private:
   bool getNameFromSymbolTable(SymbolRef::Type Type, uint64_t Address,
                               std::string &Name, uint64_t &Addr,
                               uint64_t &Size) const;
   // For big-endian PowerPC64 ELF, OpdAddress is the address of the .opd
   // (function descriptor) section and OpdExtractor refers to its contents.
   void addSymbol(const SymbolRef &Symbol, uint64_t SymbolSize,
                  DataExtractor *OpdExtractor = nullptr,

llvm/trunk/tools/llvm-symbolizer/LLVMSymbolize.cpp

 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Config/config.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
 #include "llvm/DebugInfo/PDB/PDB.h"
 #include "llvm/DebugInfo/PDB/PDBContext.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Object/MachO.h"
 #include "llvm/Object/SymbolSize.h"
+#include "llvm/Support/COFF.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Compression.h"
 #include "llvm/Support/DataExtractor.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include <sstream>
 #include <stdlib.h>
 
 #if defined(_MSC_VER)
 #include <Windows.h>
 #include <DbgHelp.h>
 #pragma comment(lib, "dbghelp.lib")
+
+// Windows.h conflicts with our COFF header definitions.
+#ifdef IMAGE_FILE_MACHINE_I386
+#undef IMAGE_FILE_MACHINE_I386
+#endif
 #endif
 
 namespace llvm {
 namespace symbolize {
 
 static bool error(std::error_code ec) {
   if (!ec)
     return false;
if (Module->isMachO() && SymbolName.size() > 0 && SymbolName[0] == '_')
     SymbolName = SymbolName.drop_front();
   // FIXME: If a function has alias, there are two entries in symbol table
   // with same address size. Make sure we choose the correct one.
   auto &M = SymbolType == SymbolRef::ST_Function ? Functions : Objects;
   SymbolDesc SD = { SymbolAddress, SymbolSize };
   M.insert(std::make_pair(SD, SymbolName));
 }
 
+// Return true if this is a 32-bit x86 PE COFF module.
+bool ModuleInfo::isWin32Module() const {
+  auto *CoffObject = dyn_cast<COFFObjectFile>(Module);
+  return CoffObject && CoffObject->getMachine() == COFF::IMAGE_FILE_MACHINE_I386;
+}
+
 bool ModuleInfo::getNameFromSymbolTable(SymbolRef::Type Type, uint64_t Address,
                                         std::string &Name, uint64_t &Addr,
                                         uint64_t &Size) const {
   const auto &SymbolMap = Type == SymbolRef::ST_Function ? Functions : Objects;
   if (SymbolMap.empty())
     return false;
   SymbolDesc SD = { Address, Address };
   auto SymbolIterator = SymbolMap.upper_bound(SD);

 }
 
 const char LLVMSymbolizer::kBadString[] = "??";
 
 std::string LLVMSymbolizer::symbolizeCode(const std::string &ModuleName,
                                           uint64_t ModuleOffset) {
   ModuleInfo *Info = getOrCreateModuleInfo(ModuleName);
   if (!Info)
-    return printDILineInfo(DILineInfo());
+    return printDILineInfo(DILineInfo(), Info);
   if (Opts.PrintInlining) {
     DIInliningInfo InlinedContext =
         Info->symbolizeInlinedCode(ModuleOffset, Opts);
     uint32_t FramesNum = InlinedContext.getNumberOfFrames();
     assert(FramesNum > 0);
     std::string Result;
     for (uint32_t i = 0; i < FramesNum; i++) {
       DILineInfo LineInfo = InlinedContext.getFrame(i);
-      Result += printDILineInfo(LineInfo);
+      Result += printDILineInfo(LineInfo, Info);
     }
     return Result;
   }
   DILineInfo LineInfo = Info->symbolizeCode(ModuleOffset, Opts);
-  return printDILineInfo(LineInfo);
+  return printDILineInfo(LineInfo, Info);
 }
 
 std::string LLVMSymbolizer::symbolizeData(const std::string &ModuleName,
                                           uint64_t ModuleOffset) {
   std::string Name = kBadString;
   uint64_t Start = 0;
   uint64_t Size = 0;
   if (Opts.UseSymbolTable) {
     if (ModuleInfo *Info = getOrCreateModuleInfo(ModuleName)) {
       if (Info->symbolizeData(ModuleOffset, Name, Start, Size) && Opts.Demangle)
-        Name = DemangleName(Name);
+        Name = DemangleName(Name, Info);
     }
   }
   std::stringstream ss;
   ss << Name << "\n" << Start << " " << Size << "\n";
   return ss.str();
 }
 
 void LLVMSymbolizer::flush() {
LLVMSymbolizer::getOrCreateModuleInfo(const std::string &ModuleName) {
   if (!Context)
     Context = new DWARFContextInMemory(*Objects.second);
   assert(Context);
   ModuleInfo *Info = new ModuleInfo(Objects.first, Context);
   Modules.insert(make_pair(ModuleName, Info));
   return Info;
 }
 
-std::string LLVMSymbolizer::printDILineInfo(DILineInfo LineInfo) const {
+std::string LLVMSymbolizer::printDILineInfo(DILineInfo LineInfo,
+                                            ModuleInfo *ModInfo) const {
   // By default, DILineInfo contains "<invalid>" for function/filename it
   // cannot fetch. We replace it to "??" to make our output closer to addr2line.
   static const std::string kDILineInfoBadString = "<invalid>";
   std::stringstream Result;
   if (Opts.PrintFunctions != FunctionNameKind::None) {
     std::string FunctionName = LineInfo.FunctionName;
     if (FunctionName == kDILineInfoBadString)
       FunctionName = kBadString;
     else if (Opts.Demangle)
-      FunctionName = DemangleName(FunctionName);
+      FunctionName = DemangleName(FunctionName, ModInfo);
     Result << FunctionName << "\n";
   }
   std::string Filename = LineInfo.FileName;
   if (Filename == kDILineInfoBadString)
     Filename = kBadString;
   Result << Filename << ":" << LineInfo.Line << ":" << LineInfo.Column << "\n";
   return Result.str();
 }
 
+// Undo these various manglings for Win32 extern "C" functions:
+// cdecl       - _foo
+// stdcall     - _foo@12
+// fastcall    - @foo@12
+// vectorcall  - foo@@12
+// These are all different linkage names for 'foo'.
+static StringRef demanglePE32ExternCFunc(StringRef SymbolName) {
+  // Remove any '_' or '@' prefix.
+  char Front = SymbolName.empty() ? '\0' : SymbolName[0];
+  if (Front == '_' || Front == '@')
+    SymbolName = SymbolName.drop_front();
+
+  // Remove any '@[0-9]+' suffix.
+  if (Front != '?') {
+    size_t AtPos = SymbolName.rfind('@');
+    if (AtPos != StringRef::npos &&
+        std::all_of(SymbolName.begin() + AtPos + 1, SymbolName.end(),
+                    [](char C) { return C >= '0' && C <= '9'; })) {
+      SymbolName = SymbolName.substr(0, AtPos);
+    }
+  }
+
+  // Remove any ending '@' for vectorcall.
+  if (SymbolName.endswith("@"))
+    SymbolName = SymbolName.drop_back();
+
+  return SymbolName;
+}
+
 #if !defined(_MSC_VER)
 // Assume that __cxa_demangle is provided by libcxxabi (except for Windows).
 extern "C" char *__cxa_demangle(const char *mangled_name, char *output_buffer,
                                 size_t *length, int *status);
 #endif
 
-std::string LLVMSymbolizer::DemangleName(const std::string &Name) {
+std::string LLVMSymbolizer::DemangleName(const std::string &Name,
+                                         ModuleInfo *ModInfo) {
 #if !defined(_MSC_VER)
   // We can spoil names of symbols with C linkage, so use an heuristic
   // approach to check if the name should be demangled.
-  if (Name.substr(0, 2) != "_Z")
-    return Name;
+  if (Name.substr(0, 2) == "_Z") {
     int status = 0;
     char *DemangledName = __cxa_demangle(Name.c_str(), nullptr, nullptr, &status);
     if (status != 0)
       return Name;
     std::string Result = DemangledName;
     free(DemangledName);
     return Result;
+  }
 #else
+  if (!Name.empty() && Name.front() == '?') {
+    // Only do MSVC C++ demangling on symbols starting with '?'.
     char DemangledName[1024] = {0};
     DWORD result = ::UnDecorateSymbolName(
         Name.c_str(), DemangledName, 1023,
         UNDNAME_NO_ACCESS_SPECIFIERS |       // Strip public, private, protected
             UNDNAME_NO_ALLOCATION_LANGUAGE | // Strip __thiscall, __stdcall, etc
             UNDNAME_NO_THROW_SIGNATURES |    // Strip throw() specifications
-          UNDNAME_NO_MEMBER_TYPE |      // Strip virtual, static, etc specifiers
-          UNDNAME_NO_MS_KEYWORDS |      // Strip all MS extension keywords
+            UNDNAME_NO_MEMBER_TYPE | // Strip virtual, static, etc specifiers
+            UNDNAME_NO_MS_KEYWORDS | // Strip all MS extension keywords
             UNDNAME_NO_FUNCTION_RETURNS); // Strip function return types
-
     return (result == 0) ? Name : std::string(DemangledName);
+  }
 #endif
+  if (ModInfo->isWin32Module())
+    return std::string(demanglePE32ExternCFunc(Name));
+  return Name;
 }
 
 } // namespace symbolize
 } // namespace llvm