[lldb] Allow SymbolTable regex search functions to match mangled name

[augusto2112]

Summary

It may be useful to search symbol table entries by mangled instead
of demangled names. Add this optional functionality in the SymbolTable
functions.

[aprantl]

It sounds like it should be possible to test this with lldb-test?

[augusto2112]

@aprantl do you mean dumping this with lldb-test? I'm not seeing a way to do this. If you're thinking about symbols that reads the symbol file not the symbol table.

[labath]

Who sets this flag? How do you intend to use it?

lldb/include/lldb/Core/Module.h

265

Could this be Mangled::NamePreference instead of bool ?

[augusto2112]

In D130803#3690754, @labath wrote:

Who sets this flag? How do you intend to use it?

I need this change downstream to search for certain swift mangled names by querying Module::FindSymbolsMatchingRegExAndType. It seemed general enough that we might want this upstream.

lldb/include/lldb/Core/Module.h

265

Ok! Wasn't aware of that enum. I'll change that.

[augusto2112]

Added test and updated functions to use Mangled::NamePreference instead of a bool

[JDevlieghere]

lldb/test/Shell/Symtab/Inputs/t.yaml

60 ↗

(On Diff #449440)

You could make this slightly smaller by dropping the content of the __unwind_info and __eh_frame below.

[JDevlieghere]

LGTM

[augusto2112]

Removed unwind_info and eh_frame sections from test yaml file.

[labath]

In D130803#3691363, @augusto2112 wrote:

In D130803#3690754, @labath wrote:

Who sets this flag? How do you intend to use it?

I need this change downstream to search for certain swift mangled names by querying Module::FindSymbolsMatchingRegExAndType. It seemed general enough that we might want this upstream.

That's ok. The reason I was asking was because it affects the ways in which one can test that.

lldb/test/Shell/Symtab/Inputs/t.yaml

1 ↗

(On Diff #449483)

Use a more descriptive name for the file.

I'd also consider using an elf file for this test, simply because their yaml form is much more concise (and you can strip out pretty much everything except the symtab, and use an unlinked .o file as the template, and still have it work).

[augusto2112]

Updated test to use elf object file.

lldb/include/lldb/Core/Module.h

public:
   ///     nullptr otherwise.
   const Symbol *FindFirstSymbolWithNameAndType(
       ConstString name, lldb::SymbolType symbol_type = lldb::eSymbolTypeAny);
 
   void FindSymbolsWithNameAndType(ConstString name,
                                   lldb::SymbolType symbol_type,
                                   SymbolContextList &sc_list);
 
-  void FindSymbolsMatchingRegExAndType(const RegularExpression &regex,
-                                       lldb::SymbolType symbol_type,
-                                       SymbolContextList &sc_list);
+  void FindSymbolsMatchingRegExAndType(
+      const RegularExpression &regex, lldb::SymbolType symbol_type,
+      SymbolContextList &sc_list,
+      Mangled::NamePreference mangling_preference = Mangled::ePreferDemangled);
 
   /// Find a function symbols in the object file's symbol table.
   ///
   /// \param[in] name
   ///     The name of the symbol that we are looking for.
   ///
   /// \param[in] name_type_mask
   ///     A mask that has one or more bitwise OR'ed values from the

lldb/include/lldb/Symbol/Symtab.h

public:
   uint32_t AppendSymbolIndexesWithNameAndType(ConstString symbol_name,
                                               lldb::SymbolType symbol_type,
                                               std::vector<uint32_t> &matches);
   uint32_t AppendSymbolIndexesWithNameAndType(ConstString symbol_name,
                                               lldb::SymbolType symbol_type,
                                               Debug symbol_debug_type,
                                               Visibility symbol_visibility,
                                               std::vector<uint32_t> &matches);
-  uint32_t
-  AppendSymbolIndexesMatchingRegExAndType(const RegularExpression &regex,
-                                          lldb::SymbolType symbol_type,
-                                          std::vector<uint32_t> &indexes);
+  uint32_t AppendSymbolIndexesMatchingRegExAndType(
+      const RegularExpression &regex, lldb::SymbolType symbol_type,
+      std::vector<uint32_t> &indexes,
+      Mangled::NamePreference name_preference = Mangled::ePreferDemangled);
   uint32_t AppendSymbolIndexesMatchingRegExAndType(
       const RegularExpression &regex, lldb::SymbolType symbol_type,
       Debug symbol_debug_type, Visibility symbol_visibility,
-      std::vector<uint32_t> &indexes);
+      std::vector<uint32_t> &indexes,
+      Mangled::NamePreference name_preference =
+          Mangled::NamePreference::ePreferDemangled);
   void FindAllSymbolsWithNameAndType(ConstString name,
                                      lldb::SymbolType symbol_type,
                                      std::vector<uint32_t> &symbol_indexes);
   void FindAllSymbolsWithNameAndType(ConstString name,
                                      lldb::SymbolType symbol_type,
                                      Debug symbol_debug_type,
                                      Visibility symbol_visibility,
                                      std::vector<uint32_t> &symbol_indexes);
   void FindAllSymbolsMatchingRexExAndType(
       const RegularExpression &regex, lldb::SymbolType symbol_type,
       Debug symbol_debug_type, Visibility symbol_visibility,
-      std::vector<uint32_t> &symbol_indexes);
+      std::vector<uint32_t> &symbol_indexes,
+      Mangled::NamePreference name_preference = Mangled::ePreferDemangled);
   Symbol *FindFirstSymbolWithNameAndType(ConstString name,
                                          lldb::SymbolType symbol_type,
                                          Debug symbol_debug_type,
                                          Visibility symbol_visibility);
   Symbol *FindSymbolAtFileAddress(lldb::addr_t file_addr);
   Symbol *FindSymbolContainingFileAddress(lldb::addr_t file_addr);
   void ForEachSymbolContainingFileAddress(
       lldb::addr_t file_addr, std::function<bool(Symbol *)> const &callback);

lldb/source/Core/Module.cpp

void Module::FindSymbolsWithNameAndType(ConstString name,
   // already thread safe.
   if (Symtab *symtab = GetSymtab()) {
     std::vector<uint32_t> symbol_indexes;
     symtab->FindAllSymbolsWithNameAndType(name, symbol_type, symbol_indexes);
     SymbolIndicesToSymbolContextList(symtab, symbol_indexes, sc_list);
   }
 }
 
-void Module::FindSymbolsMatchingRegExAndType(const RegularExpression &regex,
-                                             SymbolType symbol_type,
-                                             SymbolContextList &sc_list) {
+void Module::FindSymbolsMatchingRegExAndType(
+    const RegularExpression &regex, SymbolType symbol_type,
+    SymbolContextList &sc_list, Mangled::NamePreference mangling_preference) {
   // No need to protect this call using m_mutex all other method calls are
   // already thread safe.
   LLDB_SCOPED_TIMERF(
       "Module::FindSymbolsMatchingRegExAndType (regex = %s, type = %i)",
       regex.GetText().str().c_str(), symbol_type);
   if (Symtab *symtab = GetSymtab()) {
     std::vector<uint32_t> symbol_indexes;
     symtab->FindAllSymbolsMatchingRexExAndType(
         regex, symbol_type, Symtab::eDebugAny, Symtab::eVisibilityAny,
-        symbol_indexes);
+        symbol_indexes, mangling_preference);
     SymbolIndicesToSymbolContextList(symtab, symbol_indexes, sc_list);
   }
 }
 
 void Module::PreloadSymbols() {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
   SymbolFile *sym_file = GetSymbolFile();
   if (!sym_file)

lldb/source/Symbol/Symtab.cpp

while (pos != indexes.end()) {
         pos = indexes.erase(pos);
     }
   }
   return indexes.size();
 }
 
 uint32_t Symtab::AppendSymbolIndexesMatchingRegExAndType(
     const RegularExpression &regexp, SymbolType symbol_type,
-    std::vector<uint32_t> &indexes) {
+    std::vector<uint32_t> &indexes, Mangled::NamePreference name_preference) {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
 
   uint32_t prev_size = indexes.size();
   uint32_t sym_end = m_symbols.size();
 
   for (uint32_t i = 0; i < sym_end; i++) {
     if (symbol_type == eSymbolTypeAny ||
         m_symbols[i].GetType() == symbol_type) {
-      const char *name = m_symbols[i].GetName().AsCString();
+      const char *name =
+          m_symbols[i].GetMangled().GetName(name_preference).AsCString();
       if (name) {
         if (regexp.Execute(name))
           indexes.push_back(i);
       }
     }
   }
   return indexes.size() - prev_size;
 }
 
 uint32_t Symtab::AppendSymbolIndexesMatchingRegExAndType(
     const RegularExpression &regexp, SymbolType symbol_type,
     Debug symbol_debug_type, Visibility symbol_visibility,
-    std::vector<uint32_t> &indexes) {
+    std::vector<uint32_t> &indexes, Mangled::NamePreference name_preference) {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
 
   uint32_t prev_size = indexes.size();
   uint32_t sym_end = m_symbols.size();
 
   for (uint32_t i = 0; i < sym_end; i++) {
     if (symbol_type == eSymbolTypeAny ||
         m_symbols[i].GetType() == symbol_type) {
       if (!CheckSymbolAtIndex(i, symbol_debug_type, symbol_visibility))
         continue;
 
-      const char *name = m_symbols[i].GetName().AsCString();
+      const char *name =
+          m_symbols[i].GetMangled().GetName(name_preference).AsCString();
       if (name) {
         if (regexp.Execute(name))
           indexes.push_back(i);
       }
     }
   }
   return indexes.size() - prev_size;
 }
if (name) {
     AppendSymbolIndexesWithNameAndType(name, symbol_type, symbol_debug_type,
                                        symbol_visibility, symbol_indexes);
   }
 }
 
 void Symtab::FindAllSymbolsMatchingRexExAndType(
     const RegularExpression &regex, SymbolType symbol_type,
     Debug symbol_debug_type, Visibility symbol_visibility,
-    std::vector<uint32_t> &symbol_indexes) {
+    std::vector<uint32_t> &symbol_indexes,
+    Mangled::NamePreference name_preference) {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
 
   AppendSymbolIndexesMatchingRegExAndType(regex, symbol_type, symbol_debug_type,
-                                          symbol_visibility, symbol_indexes);
+                                          symbol_visibility, symbol_indexes,
+                                          name_preference);
 }
 
 Symbol *Symtab::FindFirstSymbolWithNameAndType(ConstString name,
                                                SymbolType symbol_type,
                                                Debug symbol_debug_type,
                                                Visibility symbol_visibility) {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
   LLDB_SCOPED_TIMER();

lldb/test/Shell/Symtab/Inputs/symbols.yaml

+--- !ELF
+FileHeader:
+  Class:           ELFCLASS64
+  Data:            ELFDATA2LSB
+  Type:            ET_REL
+  Machine:         EM_AARCH64
+  SectionHeaderStringTable: .strtab
+Sections:
+  - Name:            .text
+    Type:            SHT_PROGBITS
+    Flags:           [ SHF_ALLOC, SHF_EXECINSTR ]
+    AddressAlign:    0x4
+  - Type:            SectionHeaderTable
+    Sections:
+      - Name:            .text
+      - Name:            .strtab
+      - Name:            .symtab
+Symbols:
+  - Name:            _Z8someFunciii
+    Type:            STT_FUNC
+    Section:         .text
+    Binding:         STB_GLOBAL
+    Size:            0x1C
+  - Name:            _Z8someFuncci
+    Type:            STT_FUNC
+    Section:         .text
+    Binding:         STB_GLOBAL
+    Value:           0x1C
+    Size:            0x18
+  - Name:            _Z13someOtherFuncv
+    Type:            STT_FUNC
+    Section:         .text
+    Binding:         STB_GLOBAL
+    Value:           0x34
+    Size:            0x4
+  - Name:            _Z13someOtherFuncd
+    Type:            STT_FUNC
+    Section:         .text
+    Binding:         STB_GLOBAL
+    Value:           0x38
+    Size:            0x10
+  - Name:            _Z18ignoreThisFunctionv
+    Type:            STT_FUNC
+    Section:         .text
+    Binding:         STB_GLOBAL
+    Value:           0x48
+    Size:            0x8
+...

lldb/test/Shell/Symtab/symtab-regex-demangled.test

+# RUN: yaml2obj %S/Inputs/symbols.yaml -o %t
+#
+
+# RUN: lldb-test symtab --find-symbols-by-regex='.*some.*' --mangling-preference=demangled %t | Filecheck %s
+# CHECK: someFunc(int, int, int)
+# CHECK: someFunc(char, int)
+# CHECK: someOtherFunc()
+# CHECK: someOtherFunc(double)
+# CHECK-NOT: ignoreThisFunction()

lldb/test/Shell/Symtab/symtab-regex-mangled.test

+# RUN: yaml2obj %S/Inputs/symbols.yaml -o %t
+#
+
+# RUN: lldb-test symtab --find-symbols-by-regex='.*some.*' --mangling-preference=mangled %t | Filecheck %s
+# CHECK: _Z8someFunciii
+# CHECK: _Z8someFuncci
+# CHECK: _Z13someOtherFuncv
+# CHECK: _Z13someOtherFuncd
+# CHECK-NOT: _Z18ignoreThisFunctionv

lldb/tools/lldb-test/lldb-test.cpp

 #include "lldb/Core/Section.h"
 #include "lldb/Expression/IRMemoryMap.h"
 #include "lldb/Initialization/SystemLifetimeManager.h"
 #include "lldb/Interpreter/CommandInterpreter.h"
 #include "lldb/Interpreter/CommandReturnObject.h"
 #include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/LineTable.h"
 #include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Symbol/Symtab.h"
 #include "lldb/Symbol/TypeList.h"
 #include "lldb/Symbol/TypeMap.h"
 #include "lldb/Symbol/VariableList.h"
 #include "lldb/Target/Language.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/DataExtractor.h"
 #include "lldb/Utility/LLDBAssert.h"

 using namespace llvm;
 
 namespace opts {
 static cl::SubCommand BreakpointSubcommand("breakpoints",
                                            "Test breakpoint resolution");
 cl::SubCommand ObjectFileSubcommand("object-file",
                                     "Display LLDB object file information");
 cl::SubCommand SymbolsSubcommand("symbols", "Dump symbols for an object file");
+cl::SubCommand SymTabSubcommand("symtab",
+                                "Test symbol table functionality");
 cl::SubCommand IRMemoryMapSubcommand("ir-memory-map", "Test IRMemoryMap");
 cl::SubCommand AssertSubcommand("assert", "Test assert handling");
 
 cl::opt<std::string> Log("log", cl::desc("Path to a log file"), cl::init(""),
                          cl::sub(BreakpointSubcommand),
                          cl::sub(ObjectFileSubcommand),
                          cl::sub(SymbolsSubcommand),
+                         cl::sub(SymTabSubcommand),
                          cl::sub(IRMemoryMapSubcommand));
 
 /// Create a target using the file pointed to by \p Filename, or abort.
 TargetSP createTarget(Debugger &Dbg, const std::string &Filename);
 
 /// Read \p Filename into a null-terminated buffer, or abort.
 std::unique_ptr<MemoryBuffer> openFile(const std::string &Filename);
 

 cl::opt<bool> SectionDependentModules("dep-modules",
                                       cl::desc("Dump each dependent module"),
                                       cl::sub(ObjectFileSubcommand));
 cl::list<std::string> InputFilenames(cl::Positional, cl::desc("<input files>"),
                                      cl::OneOrMore,
                                      cl::sub(ObjectFileSubcommand));
 } // namespace object
 
+namespace symtab {
+
+/// The same enum as Mangled::NamePreference but with a default
+/// 'None' case. This is needed to disambiguate wheter "ManglingPreference" was
+/// explicitly set or not.
+enum class ManglingPreference {
+  None,
+  Mangled,
+  Demangled,
+  MangledWithoutArguments,
+};
+
+static cl::opt<std::string> FindSymbolsByRegex(
+    "find-symbols-by-regex",
+    cl::desc(
+        "Dump symbols found in the symbol table matching the specified regex."),
+    cl::sub(SymTabSubcommand));
+
+static cl::opt<ManglingPreference> ManglingPreference(
+    "mangling-preference",
+    cl::desc("Preference on mangling scheme the regex should match against and "
+             "dumped."),
+    cl::values(
+        clEnumValN(ManglingPreference::Mangled, "mangled", "Prefer mangled"),
+        clEnumValN(ManglingPreference::Demangled, "demangled",
+                   "Prefer demangled"),
+        clEnumValN(ManglingPreference::MangledWithoutArguments,
+                   "demangled-without-args", "Prefer mangled without args")),
+    cl::sub(SymTabSubcommand));
+
+static cl::opt<std::string> InputFile(cl::Positional, cl::desc("<input file>"),
+                                      cl::Required, cl::sub(SymTabSubcommand));
+
+/// Validate that the options passed make sense.
+static llvm::Optional<llvm::Error> validate();
+
+/// Transforms the selected mangling preference into a Mangled::NamePreference
+static Mangled::NamePreference getNamePreference();
+
+static int handleSymtabCommand(Debugger &Dbg);
+} // namespace symtab
+
 namespace symbols {
 static cl::opt<std::string> InputFile(cl::Positional, cl::desc("<input file>"),
                                       cl::Required, cl::sub(SymbolsSubcommand));
 
 static cl::opt<std::string>
     SymbolPath("symbol-file",
                cl::desc("The file from which to fetch symbol information."),
                cl::value_desc("file"), cl::sub(SymbolsSubcommand));
if (Line != 0)
       return make_string_error("Cannot search for variables "
                                "using line numbers.");
     return findVariables;
   }
 
   llvm_unreachable("Unsupported symbol action.");
 }
 
+llvm::Optional<llvm::Error> opts::symtab::validate() {
+  if (ManglingPreference != ManglingPreference::None &&
+      FindSymbolsByRegex.empty())
+    return make_string_error("Mangling preference set but no regex specified.");
+
+  return {};
+}
+
+static Mangled::NamePreference opts::symtab::getNamePreference() {
+  switch (ManglingPreference) {
+  case ManglingPreference::None:
+  case ManglingPreference::Mangled:
+    return Mangled::ePreferMangled;
+  case ManglingPreference::Demangled:
+    return Mangled::ePreferDemangled;
+  case ManglingPreference::MangledWithoutArguments:
+    return Mangled::ePreferDemangledWithoutArguments;
+  }
+}
+
+int opts::symtab::handleSymtabCommand(Debugger &Dbg) {
+  if (auto error = validate()) {
+    logAllUnhandledErrors(std::move(error.getValue()), WithColor::error(), "");
+    return 1;
+  }
+
+  if (!FindSymbolsByRegex.empty()) {
+    ModuleSpec Spec{FileSpec(InputFile)};
+
+    auto ModulePtr = std::make_shared<lldb_private::Module>(Spec);
+    auto *Symtab = ModulePtr->GetSymtab();
+    auto NamePreference = getNamePreference();
+    std::vector<uint32_t> Indexes;
+
+    Symtab->FindAllSymbolsMatchingRexExAndType(
+        RegularExpression(FindSymbolsByRegex), lldb::eSymbolTypeAny,
+        Symtab::eDebugAny, Symtab::eVisibilityAny, Indexes, NamePreference);
+    for (auto i : Indexes) {
+      auto *symbol = Symtab->SymbolAtIndex(i);
+      if (symbol) {
+        StreamString stream;
+        symbol->Dump(&stream, nullptr, i, NamePreference);
+        outs() << stream.GetString();
+      }
+    }
+  }
+
+  return 0;
+}
+
 int opts::symbols::dumpSymbols(Debugger &Dbg) {
   auto ActionOr = getAction();
   if (!ActionOr) {
     logAllUnhandledErrors(ActionOr.takeError(), WithColor::error(), "");
     return 1;
   }
   auto Action = *ActionOr;
 
if (!opts::Log.empty())
     Dbg->EnableLog("lldb", {"all"}, opts::Log, 0, 0, eLogHandlerStream, errs());
 
   if (opts::BreakpointSubcommand)
     return opts::breakpoint::evaluateBreakpoints(*Dbg);
   if (opts::ObjectFileSubcommand)
     return dumpObjectFiles(*Dbg);
   if (opts::SymbolsSubcommand)
     return opts::symbols::dumpSymbols(*Dbg);
+  if (opts::SymTabSubcommand)
+    return opts::symtab::handleSymtabCommand(*Dbg);
   if (opts::IRMemoryMapSubcommand)
     return opts::irmemorymap::evaluateMemoryMapCommands(*Dbg);
   if (opts::AssertSubcommand)
     return opts::assert::lldb_assert(*Dbg);
 
   WithColor::error() << "No command specified.\n";
   return 1;
 }