This is an archive of the discontinued LLVM Phabricator instance.

[NFC] Refactor symbol table parsing.
ClosedPublic

Authored by clayborg on Nov 15 2021, 10:50 PM.

Details

Summary

Symbol table parsing has evolved over the years and many plug-ins contained duplicate code in the ObjectFile::GetSymtab() that used to be pure virtual. With this change, the "Symbtab *ObjectFile::GetSymtab()" is no longer virtual and will end up calling a new "void ObjectFile::ParseSymtab(Symtab &symtab)" pure virtual function to actually do the parsing. This helps centralize the code for parsing the symbol table and allows the ObjectFile base class to do all of the common work, like taking the necessary locks and creating the symbol table object itself. Plug-ins now just need to parse when they are asked to parse as the ParseSymtab function will only get called once.

Diff Detail

Event Timeline

clayborg created this revision.Nov 15 2021, 10:50 PM
clayborg requested review of this revision.Nov 15 2021, 10:50 PM
Herald added a project: Restricted Project. · View Herald TranscriptNov 15 2021, 10:50 PM
labath accepted this revision.Nov 16 2021, 12:43 AM

This is a good cleanup independently of the caching feature.

lldb/include/lldb/Symbol/ObjectFile.h
344

I guess this should be protected, as people shouldn't be calling it directly.

This revision is now accepted and ready to land.Nov 16 2021, 12:43 AM
This revision was automatically updated to reflect the committed changes.

I had to revert this. There is a deadlock in:

void Module::PreloadSymbols() {
  std::lock_guard<std::recursive_mutex> guard(m_mutex);
  SymbolFile *sym_file = GetSymbolFile();
  if (!sym_file)
    return;

  // Prime the symbol file first, since it adds symbols to the symbol table.
  sym_file->PreloadSymbols();

  // Now we can prime the symbol table.
  if (Symtab *symtab = sym_file->GetSymtab())
    symtab->PreloadSymbols();
}

That happens only when we are using ELF files since they need to be manually indexed. What happens is:

  • Thread #1 calls "sym_file->PreloadSymbols();"
  • Thread #2 starts manually indexing the DWARF and tries to load section data for a DWARF section which has relocations and since Module::PreloadSymbols() from thread #1 has the lock.

The backtrace can be seen when debugging:

$ /Users/gclayton/Documents/src/llvm/clean/Debug/bin/lldb --no-lldbinit -S /Users/gclayton/Documents/src/llvm/clean/Debug/tools/lldb/test/Shell/lit-lldb-init debug_rnglists-dwo.o -o image lookup -v -s lookup_rnglists -o exit

This deadlocks with:

(lldb) bt all
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x00000001850f4548 libsystem_kernel.dylib`__psynch_cvwait + 8
    frame #1: 0x000000018512bdac libsystem_pthread.dylib`_pthread_cond_wait + 1248
    frame #2: 0x0000000185085efc libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
    frame #3: 0x0000000104ee97a8 LLDB`void std::__1::condition_variable::wait<llvm::ThreadPool::wait()::$_1>(this=0x000000016d3b03f0, __lk=0x000000016d3aff58, __pred=(anonymous class) @ 0x000000016d3aff28)::$_1) at __mutex_base:409:9
    frame #4: 0x0000000104ee9728 LLDB`llvm::ThreadPool::wait(this=0x000000016d3b0338) at ThreadPool.cpp:72:23
    frame #5: 0x0000000104b97d54 LLDB`lldb_private::ManualDWARFIndex::Index(this=0x00000001302129b0) at ManualDWARFIndex.cpp:109:8
    frame #6: 0x0000000104b0ef98 LLDB`lldb_private::ManualDWARFIndex::Preload(this=0x00000001302129b0) at ManualDWARFIndex.h:27:29
    frame #7: 0x0000000104bb5f1c LLDB`SymbolFileDWARF::PreloadSymbols(this=0x0000000130815400) at SymbolFileDWARF.cpp:2071:12
    frame #8: 0x0000000103c60dcc LLDB`lldb_private::Module::PreloadSymbols(this=0x000000013020d238) at Module.cpp:1383:13
    frame #9: 0x0000000104266f44 LLDB`lldb_private::TargetList::CreateTargetInternal(debugger=0x000000013302d200, user_exe_path=(Data = "debug_rnglists-dwo.o", Length = 20), specified_arch=0x000000016d3b1478, load_dependent_files=eLoadDependentsDefault, platform_sp=std::__1::shared_ptr<lldb_private::Platform>::element_type @ 0x0000000130705b70 strong=7 weak=1, target_sp=std::__1::shared_ptr<lldb_private::Target>::element_type @ 0x0000000130811600 strong=1 weak=2) at TargetList.cpp:371:24
    frame #10: 0x0000000104266648 LLDB`lldb_private::TargetList::CreateTargetInternal(debugger=0x000000013302d200, user_exe_path=(Data = "debug_rnglists-dwo.o", Length = 20), triple_str=(Data = "", Length = 0), load_dependent_files=eLoadDependentsDefault, platform_options=0x0000000133044dc0, target_sp=std::__1::shared_ptr<lldb_private::Target>::element_type @ 0x0000000130811600 strong=1 weak=2) at TargetList.cpp:280:10
    frame #11: 0x0000000104265ae0 LLDB`lldb_private::TargetList::CreateTarget(this=0x000000013302d2d0, debugger=0x000000013302d200, user_exe_path=(Data = "debug_rnglists-dwo.o", Length = 20), triple_str=(Data = "", Length = 0), load_dependent_files=eLoadDependentsDefault, platform_options=0x0000000133044dc0, target_sp=std::__1::shared_ptr<lldb_private::Target>::element_type @ 0x0000000130811600 strong=1 weak=2) at TargetList.cpp:52:17
    frame #12: 0x0000000107d2d2a8 LLDB`CommandObjectTargetCreate::DoExecute(this=0x0000000133044c00, command=0x000000016d3b22a0, result=0x000000016d3b28c0) at CommandObjectTarget.cpp:314:45
    frame #13: 0x0000000103efc228 LLDB`lldb_private::CommandObjectParsed::Execute(this=0x0000000133044c00, args_string="\"debug_rnglists-dwo.o\"", result=0x000000016d3b28c0) at CommandObject.cpp:995:19
    frame #14: 0x0000000103ebd234 LLDB`lldb_private::CommandInterpreter::HandleCommand(this=0x0000000132834c90, command_line="target create \"debug_rnglists-dwo.o\"", lazy_add_to_history=eLazyBoolCalculate, result=0x000000016d3b28c0) at CommandInterpreter.cpp:1971:14
    frame #15: 0x0000000103ec17fc LLDB`lldb_private::CommandInterpreter::IOHandlerInputComplete(this=0x0000000132834c90, io_handler=0x0000000130508c18, line="target create \"debug_rnglists-dwo.o\"") at CommandInterpreter.cpp:3021:3
    frame #16: 0x0000000103bffcd8 LLDB`lldb_private::IOHandlerEditline::Run(this=0x0000000130508c18) at IOHandler.cpp:576:22
    frame #17: 0x0000000103b8fbb8 LLDB`lldb_private::Debugger::RunIOHandlers(this=0x000000013302d200) at Debugger.cpp:877:16
    frame #18: 0x0000000103ec2eac LLDB`lldb_private::CommandInterpreter::RunCommandInterpreter(this=0x0000000132834c90, options=0x00000001305085a0) at CommandInterpreter.cpp:3267:16
    frame #19: 0x000000010349d85c LLDB`lldb::SBDebugger::RunCommandInterpreter(this=0x000000016d3b31e0, options=0x000000016d3b2ec0) at SBDebugger.cpp:1267:14
    frame #20: 0x0000000102a50da0 lldb`Driver::MainLoop(this=0x000000016d3b31c0) at Driver.cpp:635:20
    frame #21: 0x0000000102a51a60 lldb`main(argc=9, argv=0x000000016d3b3720) at Driver.cpp:965:26
    frame #22: 0x0000000185149430 libdyld.dylib`start + 4
  thread #2, name = 'lldb.debugger.event-handler'
    frame #0: 0x00000001850f4548 libsystem_kernel.dylib`__psynch_cvwait + 8
    frame #1: 0x000000018512bdac libsystem_pthread.dylib`_pthread_cond_wait + 1248
    frame #2: 0x0000000185085efc libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
    frame #3: 0x000000010435286c LLDB`lldb_private::Listener::GetEventInternal(this=0x0000000132834b50, timeout=0x000000016dbbae58, broadcaster=0x0000000000000000, broadcaster_names=0x0000000000000000, num_broadcaster_names=0, event_type_mask=0, event_sp=nullptr) at Listener.cpp:364:28
    frame #4: 0x0000000104352cd0 LLDB`lldb_private::Listener::GetEvent(this=0x0000000132834b50, event_sp=nullptr, timeout=0x000000016dbbae58) at Listener.cpp:399:10
    frame #5: 0x0000000103b930a0 LLDB`lldb_private::Debugger::DefaultEventHandler(this=0x000000013302d200) at Debugger.cpp:1510:22
    frame #6: 0x0000000103b933cc LLDB`lldb_private::Debugger::EventHandlerThread(arg=0x000000013302d200) at Debugger.cpp:1563:22
    frame #7: 0x0000000103e3060c LLDB`lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(arg=0x0000000130508780) at HostNativeThreadBase.cpp:65:10
    frame #8: 0x00000001081c90f0 LLDB`lldb_private::HostThreadMacOSX::ThreadCreateTrampoline(arg=0x0000000130508780) at HostThreadMacOSX.mm:67:10
    frame #9: 0x000000018512b878 libsystem_pthread.dylib`_pthread_start + 320
  thread #3
    frame #0: 0x00000001850f3a48 libsystem_kernel.dylib`__psynch_mutexwait + 8
    frame #1: 0x0000000185128918 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_wait + 88
    frame #2: 0x0000000185126244 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_slow + 232
    frame #3: 0x00000001850b8c88 libc++.1.dylib`std::__1::recursive_mutex::lock() + 16
    frame #4: 0x000000010401a354 LLDB`std::__1::lock_guard<std::__1::recursive_mutex>::lock_guard(this=0x000000016e3c6080, __m=0x000000013020d250) at __mutex_base:91:27
    frame #5: 0x0000000104017a40 LLDB`std::__1::lock_guard<std::__1::recursive_mutex>::lock_guard(this=0x000000016e3c6080, __m=0x000000013020d250) at __mutex_base:91:21
    frame #6: 0x0000000104016ef8 LLDB`lldb_private::ObjectFile::GetSymtab(this=0x0000000130706470) at ObjectFile.cpp:735:43
    frame #7: 0x0000000104771f1c LLDB`ObjectFileELF::RelocateSection(this=0x0000000130706470, section=0x0000000130706e10) at ObjectFileELF.cpp:2852:56
    frame #8: 0x000000010401771c LLDB`lldb_private::ObjectFile::ReadSectionData(this=0x0000000130706470, section=0x0000000130706e10, section_data=0x000000016e3c6538) at ObjectFile.cpp:527:5
    frame #9: 0x0000000104773eb8 LLDB`ObjectFileELF::ReadSectionData(this=0x0000000130706470, section=0x0000000130706e10, section_data=0x000000016e3c6538) at ObjectFileELF.cpp:3317:31
    frame #10: 0x0000000103cd0210 LLDB`lldb_private::Section::GetSectionData(this=0x0000000130706e10, section_data=0x000000016e3c6538) at Section.cpp:395:24
    frame #11: 0x0000000104b4a648 LLDB`LoadSection(section_list=0x0000000130706200, section_type=eSectionTypeDWARFDebugInfoDwo) at DWARFContext.cpp:26:15
    frame #12: 0x0000000104b4a4f0 LLDB`lldb_private::DWARFContext::LoadOrGetSection(this=0x000000016e3c67a0)::$_0::operator()() const at DWARFContext.cpp:36:19
    frame #13: 0x0000000104b4a444 LLDB`decltype(__f=0x000000016e3c67a0)::$_0>(fp)()) std::__1::__invoke<lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0>(lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0&&) at type_traits:3747:1
    frame #14: 0x0000000104b4a41c LLDB`void std::__1::__call_once_param<std::__1::tuple<lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0&&> >::__execute<>(this=0x000000016e3c6740, (null)=__tuple_indices<> @ 0x000000016e3c669f) at mutex:629:9
    frame #15: 0x0000000104b4a3ec LLDB`std::__1::__call_once_param<std::__1::tuple<lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0&&> >::operator(this=0x000000016e3c6740)() at mutex:621:9
    frame #16: 0x0000000104b4a2b8 LLDB`void std::__1::__call_once_proxy<std::__1::tuple<lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0&&> >(__vp=0x000000016e3c6740) at mutex:657:5
    frame #17: 0x00000001850b92b0 libc++.1.dylib`std::__1::__call_once(unsigned long volatile&, void*, void (*)(void*)) + 180
    frame #18: 0x0000000104b4a1f0 LLDB`void std::__1::call_once<lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0>(__flag=0x000000013200d1d0, __func=0x000000016e3c67a0)::$_0&&) at mutex:675:9
    frame #19: 0x0000000104b472b8 LLDB`void llvm::call_once<lldb_private::DWARFContext::LoadOrGetSection(llvm::Optional<lldb::SectionType>, llvm::Optional<lldb::SectionType>, lldb_private::DWARFContext::SectionData&)::$_0>(flag=0x000000013200d1d0, F=0x000000016e3c67a0)::$_0&&) at Threading.h:92:5
    frame #20: 0x0000000104b47274 LLDB`lldb_private::DWARFContext::LoadOrGetSection(this=0x000000013200d0b8, main_section_type=Optional<lldb::SectionType> @ 0x000000016e3c67d8, dwo_section_type=Optional<lldb::SectionType> @ 0x000000016e3c67d0, data=0x000000013200d1d0) at DWARFContext.cpp:34:3
    frame #21: 0x0000000104b47558 LLDB`lldb_private::DWARFContext::getOrLoadDebugInfoData(this=0x000000013200d0b8) at DWARFContext.cpp:69:10
    frame #22: 0x0000000104b5b3d8 LLDB`DWARFDebugInfo::ParseUnitsFor(this=0x00000001307068f0, section=DebugInfo) at DWARFDebugInfo.cpp:74:45
    frame #23: 0x0000000104b63980 LLDB`DWARFDebugInfo::ParseUnitHeadersIfNeeded(this=0x000000016e3c6a80)::$_0::operator()() const at DWARFDebugInfo.cpp:100:5
    frame #24: 0x0000000104b63918 LLDB`decltype(__f=0x000000016e3c6a80)::$_0>(fp)()) std::__1::__invoke<DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0>(DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0&&) at type_traits:3747:1
    frame #25: 0x0000000104b638f0 LLDB`void std::__1::__call_once_param<std::__1::tuple<DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0&&> >::__execute<>(this=0x000000016e3c6a20, (null)=__tuple_indices<> @ 0x000000016e3c697f) at mutex:629:9
    frame #26: 0x0000000104b638c0 LLDB`std::__1::__call_once_param<std::__1::tuple<DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0&&> >::operator(this=0x000000016e3c6a20)() at mutex:621:9
    frame #27: 0x0000000104b6378c LLDB`void std::__1::__call_once_proxy<std::__1::tuple<DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0&&> >(__vp=0x000000016e3c6a20) at mutex:657:5
    frame #28: 0x00000001850b92b0 libc++.1.dylib`std::__1::__call_once(unsigned long volatile&, void*, void (*)(void*)) + 180
    frame #29: 0x0000000104b636c4 LLDB`void std::__1::call_once<DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0>(__flag=0x0000000130706900, __func=0x000000016e3c6a80)::$_0&&) at mutex:675:9
    frame #30: 0x0000000104b5b9c0 LLDB`void llvm::call_once<DWARFDebugInfo::ParseUnitHeadersIfNeeded()::$_0>(flag=0x0000000130706900, F=0x000000016e3c6a80)::$_0&&) at Threading.h:92:5
    frame #31: 0x0000000104b5b984 LLDB`DWARFDebugInfo::ParseUnitHeadersIfNeeded(this=0x00000001307068f0) at DWARFDebugInfo.cpp:99:3
    frame #32: 0x0000000104b5c048 LLDB`DWARFDebugInfo::ContainsTypeUnits(this=0x00000001307068f0) at DWARFDebugInfo.cpp:173:3
    frame #33: 0x0000000104bff524 LLDB`SymbolFileDWARFDwo::FindSingleCompileUnit(this=0x000000013200d018) at SymbolFileDWARFDwo.cpp:62:19
    frame #34: 0x0000000104bff48c LLDB`SymbolFileDWARFDwo::GetDWOCompileUnitForHash(this=0x000000013200d018, hash=16045690984229367821) at SymbolFileDWARFDwo.cpp:49:26
    frame #35: 0x0000000104b7ce44 LLDB`DWARFUnit::ExtractUnitDIEIfNeeded(this=0x000000013081e400) at DWARFUnit.cpp:84:40
    frame #36: 0x0000000104b804a4 LLDB`DWARFUnit::GetDwoSymbolFile(this=0x000000013081e400) at DWARFUnit.cpp:817:3
    frame #37: 0x0000000104b98538 LLDB`lldb_private::ManualDWARFIndex::IndexUnit(this=0x00000001302129b0, unit=0x000000013081e400, dwp=0x0000000000000000, set=0x0000000130213240) at ManualDWARFIndex.cpp:144:50
    frame #38: 0x0000000104ba2354 LLDB`lldb_private::ManualDWARFIndex::Index(this=0x00000001303047c8, cu_idx=0)::$_1::operator()(unsigned long) const at ManualDWARFIndex.cpp:80:5
    frame #39: 0x0000000104ba22e0 LLDB`decltype(__f=0x00000001303047c8, __args=0x00000001303047f0)::$_1&>(fp)(std::__1::forward<unsigned long&>(fp0))) std::__1::__invoke<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&>(lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&) at type_traits:3747:1
    frame #40: 0x0000000104ba229c LLDB`std::__1::__bind_return<lldb_private::ManualDWARFIndex::Index()::$_1, std::__1::tuple<unsigned long>, std::__1::tuple<>, __is_valid_bind_return<lldb_private::ManualDWARFIndex::Index()::$_1, std::__1::tuple<unsigned long>, std::__1::tuple<> >::value>::type std::__1::__apply_functor<lldb_private::ManualDWARFIndex::Index(__f=0x00000001303047c8, __bound_args=size=1, (null)=__tuple_indices<0> @ 0x000000016e3c6d5f, __args=size=0)::$_1, std::__1::tuple<unsigned long>, 0ul, std::__1::tuple<> >(lldb_private::ManualDWARFIndex::Index()::$_1&, std::__1::tuple<unsigned long>&, std::__1::__tuple_indices<0ul>, std::__1::tuple<>&&) at functional:2852:12
    frame #41: 0x0000000104ba2254 LLDB`std::__1::__bind_return<lldb_private::ManualDWARFIndex::Index()::$_1, std::__1::tuple<unsigned long>, std::__1::tuple<>, __is_valid_bind_return<lldb_private::ManualDWARFIndex::Index()::$_1, std::__1::tuple<unsigned long>, std::__1::tuple<> >::value>::type std::__1::__bind<lldb_private::ManualDWARFIndex::Index(this=0x00000001303047c8)::$_1&, unsigned long&>::operator()<>() at functional:2885:20
    frame #42: 0x0000000104ba2210 LLDB`decltype(__f=0x00000001303047c8)::$_1&, unsigned long&>&>(fp)()) std::__1::__invoke<std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&>&>(std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&>&) at type_traits:3747:1
    frame #43: 0x0000000104ba21c4 LLDB`void std::__1::__invoke_void_return_wrapper<void>::__call<std::__1::__bind<lldb_private::ManualDWARFIndex::Index(__args=0x00000001303047c8)::$_1&, unsigned long&>&>(std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&>&) at __functional_base:348:9
    frame #44: 0x0000000104ba219c LLDB`std::__1::__function::__alloc_func<std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&>, std::__1::allocator<std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&> >, void ()>::operator(this=0x00000001303047c8)() at functional:1553:16
    frame #45: 0x0000000104ba1054 LLDB`std::__1::__function::__func<std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&>, std::__1::allocator<std::__1::__bind<lldb_private::ManualDWARFIndex::Index()::$_1&, unsigned long&> >, void ()>::operator(this=0x00000001303047c0)() at functional:1727:12
    frame #46: 0x0000000103f4f47c LLDB`std::__1::__function::__value_func<void ()>::operator(this=0x0000000130304898)() const at functional:1880:16
    frame #47: 0x0000000103f4f414 LLDB`std::__1::function<void ()>::operator(this=0x0000000130304898)() const at functional:2555:12
    frame #48: 0x0000000104eee758 LLDB`decltype(__f=0x0000000130304898)>&>(fp)()) std::__1::__invoke<std::__1::function<void ()>&>(std::__1::function<void ()>&) at type_traits:3747:1
    frame #49: 0x0000000104eee288 LLDB`std::__1::__packaged_task_func<std::__1::function<void ()>, std::__1::allocator<std::__1::function<void ()> >, void ()>::operator(this=0x0000000130304890)() at future:1683:12
    frame #50: 0x0000000104eed45c LLDB`std::__1::__packaged_task_function<void ()>::operator(this=0x000000016e3c6f20)() const at future:1865:12
    frame #51: 0x0000000104eecc58 LLDB`std::__1::packaged_task<void ()>::operator(this=0x000000016e3c6f20)() at future:2085:9
    frame #52: 0x0000000104eeca24 LLDB`llvm::ThreadPool::ThreadPool(this=0x00000001302128c0)::$_0::operator()() const at ThreadPool.cpp:51:9
    frame #53: 0x0000000104eec900 LLDB`void llvm::thread::Apply<llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0>(Callee=size=1, (null)=std::__1::index_sequence<> @ 0x000000016e3c6f6f)::$_0>&, std::__1::integer_sequence<unsigned long>) at thread.h:42:5
    frame #54: 0x0000000104eec8cc LLDB`void llvm::thread::GenericThreadProxy<std::__1::tuple<llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(Ptr=0x00000001302128c0) at thread.h:50:5
    frame #55: 0x0000000104eec5bc LLDB`void* llvm::thread::ThreadProxy<std::__1::tuple<llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(Ptr=0x00000001302128c0) at thread.h:60:5
    frame #56: 0x000000018512b878 libsystem_pthread.dylib`_pthread_start + 320

The issue is the ObjectFileELF::GetSymtab() used to _not_ take the module lock, which is really bad. With this new patch is did take the module lock. So we had very dangerous multi-threading issues before with ObjectFileELF::GetSymtab() not being threadsafe which could explain flakiness.

The main issue is how to fix this. @labath: let me know if you have any ideas on how to fix this. We are asking the symbol file to pre-load the symbols first, and _then_ asking the symbol table to preload itself. Do you remember why? The comment "Prime the symbol file first, since it adds symbols to the symbol table" but the SymbolFile::PreloadSymbols() doesn't seem to do that anymore. Maybe this was changed? I see the symbol file can add symbols via SymbolFile::AddSymbols() as seen in:

Symtab *SymbolFile::GetSymtab() {
  std::lock_guard<std::recursive_mutex> guard(GetModuleMutex());
  if (m_symtab)
    return m_symtab;

  // Fetch the symtab from the main object file.
  m_symtab = GetMainObjectFile()->GetSymtab();

  // Then add our symbols to it.
  if (m_symtab)
    AddSymbols(*m_symtab);

  return m_symtab;
}

So one solution is to load the normal symbol table first in Module::PreloadSymbols():

void Module::PreloadSymbols() {
  std::lock_guard<std::recursive_mutex> guard(m_mutex);
  SymbolFile *sym_file = GetSymbolFile();
  if (!sym_file)
    return;
  // Load the symbol table first so it can be available to any thread without a lock
  if (Symtab *symtab = sym_file->GetSymtab())
    symtab->PreloadSymbols();

  // Now let the symbol file preload its symbols
  sym_file->PreloadSymbols();
}

Then we would need to modify the ObjectFile::GetSymtab() to be able to check m_symtab_up without taking the module lock:

Symtab *ObjectFile::GetSymtab() {
  // We don't need to take the module lock if the symbol table pointer is
  // already set in the std::unique_ptr since it is thread safe for access. This
  // prevents deadlocks where the symbol table has already been parsed and
  // someone has the module lock acquired and another thread spins up to do
  // something that requires the symbol table. This currently happens in the
  // manual DWARF indexing where it will spin up a thread to index the DWARF and
  // that thread might ask for the symbol table in
  // ObjectFileELF::RelocateSection() which requires the symbol table and causes
  // a deadlock.
  if (auto symtab = m_symtab_up.get())
    return symtab;

  ModuleSP module_sp(GetModule());
  if (module_sp) {
    std::lock_guard<std::recursive_mutex> guard(module_sp->GetMutex());
    // Check the unique pointer again in case two threads made it to the above
    // lock guard and one thread actually acquired the lock first and was
    // populating the symbol table. By the time we get to this line of code
    // the symbol table will either already be parsed or we need to parse it.

    // Here we detect if the symbol table was parsed while the lock was held
    // on this thread.
    if (auto symtab = m_symtab_up.get())
      return symtab;

    // We haven't parse the symbol table yet.
    ElapsedTime elapsed(module_sp->GetSymtabParseTime());
    // Store the symbol table in a local unique pointer until the symbol table
    // has been parsed, and then store the value into m_symtab_up.
    auto symtab_up = std::make_unique<Symtab>(this);
    std::lock_guard<std::recursive_mutex> symtab_guard(
        symtab_up->GetMutex());
    ParseSymtab(*symtab_up);
    symtab_up->Finalize();
    m_symtab_up = std::move(symtab_up);
  }
  return m_symtab_up.get();
}

Any input would be appreciated!