Advanced guessing of rendezvous breakpoint

[eugene]

Summary

When rendezvous structure is not initialized we need to set up rendezvous breakpoint anyway. In this case the code will locate dynamic loader (interpreter) and look for known function names.
Bug: 25806

[labath]

packages/Python/lldbsuite/test/functionalities/load_unload/TestLoadUnload.py

371 ↗

(On Diff #127981)

This test has been disabled for a couple of years. I wouldn't be surprised if the problems you are running into with it are due to bitrot.

I didn't know this test exists, so I've added TestBreakpointInGlobalConstructor.py to test this behavior a week or two ago. That one is known to pass on mac, so it shouldn't suffer from bitrot. If that's the case then maybe we can just remove this test (or the other one -- In any case, I think we don't need both of them).

source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

265

So, the way I understand this, the main purpose of the entry breakpoint is to set the rendezvous breakpoint (the LoadAllCurrentModules is just a side-effect of us not being able to catch the library load early enough -- if we resolve the rendezvous breakpoint this way, it should not be needed.

I expect we can just omit the entry breakpoint altogether if we've managed to set the rendezvous breakpoint this way (we can probably keep the code as a fallback, but use it only for cases where we don't locate any of the rendezvous symbols).

345

s/tring/trying

384

the LLDB_LOG will automatically insert file+function (if you do log enable --file-function) so you can just drop this __FUNCTION__ part

599

s/moduel/module

[eugene]

• Addressing review comments
• Making sure all tests pass on Linux
• Reformating

[eugene]

packages/Python/lldbsuite/test/functionalities/load_unload/TestLoadUnload.py

371 ↗

(On Diff #127981)

I fixed both of them, I think it's fine to keep both.

[eugene]

Don't set entry breakpoint if rendezvous breakpoint was set right after launch.

[eugene]

Fix tests.

[labath]

I'm adding some people with non-linux non-x86/arm targets who use this plugin for more visibility. I have a couple of small comments, but in general, I am happy with this.

packages/Python/lldbsuite/test/functionalities/breakpoint/global_constructor/TestBreakpointInGlobalConstructor.py

24 ↗

(On Diff #129002)

The right way to handle this is to use the environment which is returned from self.registerSharedLibrariesWithTarget as the launch environment (you'll need to launch using the SB API). The reason that TestLoadUnload does not handle this is probably because it predates this function.

source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

223

s/enty/entry

417

It should be possible to create the breakpoint via Target::CreateBreakpoint (the overload that lets you specify a list of function names) instead of manually resolving the symbols... Have you tried doing that?

[clayborg]

Looks fine. Set the breakpoint using the list of names and delete the breakpoint if you get no locations and this will be good to go.

source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

403–406

Will only one of these ever be present?

406

You should delete the breakpoint if the number of locations is zero (for this one by address and also in case you end up setting the breakpoint using a list of names). Otherwise you will have a dangling breakpoint that no one wants.

417

Agreed, I would use the breakpoint setting function in target that allows you to specify one or more names as labath suggested. One question about this code though: how would we ever get a load address from a module if the dynamic loader doesn't load it itself? Are these symbols in the "[vdso]" module and is the [vdso]" module loaded differently?

[eugene]

source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

403–406

Yes. These are the names of rendezvous functions in different known linkers (Android, Linux, BSD) None of them should have more than one function from this list.

417

OS loads the dynamic loader, if it is requested by the executable, and passes its base address via AT_BASE value in the auxiliary vector (https://refspecs.linuxfoundation.org/LSB_1.3.0/IA64/spec/auxiliaryvector.html). That's how we know where to look for it. (see EvalSpecialModulesStatus)

vdso is yet another special module, but it is not related to the dynamic loader.

[eugene]

Addressing code review comments. Switching from manual symbol resolution to the appropriate overload of CreateBreakpoint.

[labath]

looks good, thanks.

[ted]

Thanks for adding me, Pavel.

Hexagon running Linux uses this plugin. These changes lgtm.

Standalone Hexagon uses its own dyld plugin; I need to look at it and see if I want to pull any of these ideas into it.

[clayborg]

Looks good.

source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

if (!SOEntryIsMainExecutable(entry))
       m_soentries.push_back(entry);
   }
 
   m_loaded_modules = module_list;
   return true;
 }
 
 bool DYLDRendezvous::AddSOEntriesFromRemote(LoadedModuleInfoList &module_list) {
+  Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
   for (auto const &modInfo : module_list.m_list) {
     bool found = false;
     for (auto const &existing : m_loaded_modules.m_list) {
       if (modInfo == existing) {
         found = true;
         break;
       }
     }
 
     if (found)
       continue;
 
     SOEntry entry;
     if (!FillSOEntryFromModuleInfo(modInfo, entry))
       return false;
 
     // Only add shared libraries and not the executable.
-    if (!SOEntryIsMainExecutable(entry))
+    if (!SOEntryIsMainExecutable(entry)) {
       m_soentries.push_back(entry);
     }
+  }
 
   m_loaded_modules = module_list;
   return true;
 }
 
 bool DYLDRendezvous::RemoveSOEntriesFromRemote(
     LoadedModuleInfoList &module_list) {
   for (auto const &existing : m_loaded_modules.m_list) {
bool DYLDRendezvous::RemoveSOEntriesFromRemote(
 
   m_loaded_modules = module_list;
   return true;
 }
 
 bool DYLDRendezvous::AddSOEntries() {
   SOEntry entry;
   iterator pos;
+  Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
 
   assert(m_previous.state == eAdd);
 
   if (m_current.map_addr == 0)
     return false;
 
   for (addr_t cursor = m_current.map_addr; cursor != 0; cursor = entry.next) {
     if (!ReadSOEntryFromMemory(cursor, entry))
case llvm::Triple::Linux:
     return !entry.file_spec;
   default:
     return false;
   }
 }
 
 bool DYLDRendezvous::TakeSnapshot(SOEntryList &entry_list) {
   SOEntry entry;
+  Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
 
   if (m_current.map_addr == 0)
     return false;
 
   // Clear previous entries since we are about to obtain an up to date list.
   entry_list.clear();
 
   for (addr_t cursor = m_current.map_addr; cursor != 0; cursor = entry.next) {
     if (!ReadSOEntryFromMemory(cursor, entry))
       return false;
 
     // Only add shared libraries and not the executable.
     if (SOEntryIsMainExecutable(entry))
       continue;
 
+    LLDB_LOG(log, "TakeSnapshot. New SO entry {0}",
+             entry.file_spec.GetFilename());
     entry_list.push_back(entry);
   }
 
   return true;
 }
 
 addr_t DYLDRendezvous::ReadWord(addr_t addr, uint64_t *dst, size_t size) {
   Status error;

source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.h

protected:
 
   /// Rendezvous breakpoint.
   lldb::break_id_t m_dyld_bid;
 
   /// Contains AT_SYSINFO_EHDR, which means a vDSO has been
   /// mapped to the address space
   lldb::addr_t m_vdso_base;
 
+  /// Contains AT_BASE, which means a dynamic loader has been
+  /// mapped to the address space
+  lldb::addr_t m_interpreter_base;
+
   /// Loaded module list. (link map for each module)
   std::map<lldb::ModuleWP, lldb::addr_t, std::owner_less<lldb::ModuleWP>>
       m_loaded_modules;
 
   /// Enables a breakpoint on a function called by the runtime
   /// linker each time a module is loaded or unloaded.
   virtual void SetRendezvousBreakpoint();
 
protected:
   EntryBreakpointHit(void *baton,
                      lldb_private::StoppointCallbackContext *context,
                      lldb::user_id_t break_id, lldb::user_id_t break_loc_id);
 
   /// Helper for the entry breakpoint callback.  Resolves the load addresses
   /// of all dependent modules.
   virtual void LoadAllCurrentModules();
 
-  void LoadVDSO(lldb_private::ModuleList &modules);
+  void LoadVDSO();
+
+  // Loading an interpreter module (if present) assumming m_interpreter_base
+  // already points to its base address.
+  lldb::ModuleSP LoadInterpreterModule();
 
   /// Computes a value for m_load_offset returning the computed address on
   /// success and LLDB_INVALID_ADDRESS on failure.
   lldb::addr_t ComputeLoadOffset();
 
   /// Computes a value for m_entry_point returning the computed address on
   /// success and LLDB_INVALID_ADDRESS on failure.
   lldb::addr_t GetEntryPoint();
 
-  /// Evaluate if Aux vectors contain vDSO information
+  /// Evaluate if Aux vectors contain vDSO and LD information
   /// in case they do, read and assign the address to m_vdso_base
-  void EvalVdsoStatus();
+  /// and m_interpreter_base.
+  void EvalSpecialModulesStatus();
 
   /// Loads Module from inferior process.
   void ResolveExecutableModule(lldb::ModuleSP &module_sp);
 
   bool AlwaysRelyOnEHUnwindInfo(lldb_private::SymbolContext &sym_ctx) override;
 
 private:
   DISALLOW_COPY_AND_ASSIGN(DynamicLoaderPOSIXDYLD);
 };
 
 #endif // liblldb_DynamicLoaderPOSIXDYLD_h_

source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

if (log)
     log->Printf("DynamicLoaderPOSIXDYLD::%s pid %" PRIu64
                 " executable '%s', load_offset 0x%" PRIx64,
                 __FUNCTION__,
                 m_process ? m_process->GetID() : LLDB_INVALID_PROCESS_ID,
                 executable_sp ? executable_sp->GetFileSpec().GetPath().c_str()
                               : "<null executable>",
                 load_offset);
 
-  EvalVdsoStatus();
+  EvalSpecialModulesStatus();
 
   // if we dont have a load address we cant re-base
   bool rebase_exec = (load_offset == LLDB_INVALID_ADDRESS) ? false : true;
 
   // if we have a valid executable
   if (executable_sp.get()) {
     lldb_private::ObjectFile *obj = executable_sp->GetObjectFile();
     if (obj) {
void DynamicLoaderPOSIXDYLD::DidLaunch() {
 
   ModuleSP executable;
   addr_t load_offset;
 
   m_auxv.reset(new AuxVector(m_process));
 
   executable = GetTargetExecutable();
   load_offset = ComputeLoadOffset();
-  EvalVdsoStatus();
+  EvalSpecialModulesStatus();
 
   if (executable.get() && load_offset != LLDB_INVALID_ADDRESS) {
     ModuleList module_list;
     module_list.Append(executable);
     UpdateLoadedSections(executable, LLDB_INVALID_ADDRESS, load_offset, true);
 
     if (log)
       log->Printf("DynamicLoaderPOSIXDYLD::%s about to call ProbeEntry()",
                   __FUNCTION__);
     ProbeEntry();
 
     m_process->GetTarget().ModulesDidLoad(module_list);
   }
 }
 
 Status DynamicLoaderPOSIXDYLD::CanLoadImage() { return Status(); }
 
 void DynamicLoaderPOSIXDYLD::UpdateLoadedSections(ModuleSP module,
                                                   addr_t link_map_addr,
                                                   addr_t base_addr,
                                                   bool base_addr_is_offset) {
if (log)
     log->Printf("DynamicLoaderPOSIXDYLD::%s pid %" PRIu64
                 " GetEntryPoint() returned address 0x%" PRIx64
                 ", setting entry breakpoint",
                 __FUNCTION__,
                 m_process ? m_process->GetID() : LLDB_INVALID_PROCESS_ID,
                 entry);
 
   if (m_process) {
     Breakpoint *const entry_break =
         m_process->GetTarget().CreateBreakpoint(entry, true, false).get();
     entry_break->SetCallback(EntryBreakpointHit, this, true);
     entry_break->SetBreakpointKind("shared-library-event");
 
     // Shoudn't hit this more than once.
     entry_break->SetOneShot(true);
+
+    SetRendezvousBreakpoint();
   }
 }
 
 // The runtime linker has run and initialized the rendezvous structure once the
 // process has hit its entry point.  When we hit the corresponding breakpoint we
 // interrogate the rendezvous structure to get the load addresses of all
 // dependent modules for the process.  Similarly, we can discover the runtime
 // linker function and setup a breakpoint to notify us of any dynamically loaded
bool DynamicLoaderPOSIXDYLD::EntryBreakpointHit(
   dyld_instance->LoadAllCurrentModules();
   dyld_instance->SetRendezvousBreakpoint();
   return false; // Continue running.
 }
 
 void DynamicLoaderPOSIXDYLD::SetRendezvousBreakpoint() {
   Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
 
-  addr_t break_addr = m_rendezvous.GetBreakAddress();
+  addr_t break_addr;
   Target &target = m_process->GetTarget();
 
+  if (m_rendezvous.IsValid()) {
+    break_addr = m_rendezvous.GetBreakAddress();
+  } else {
+    LLDB_LOG(log,
+             "Rendezvous structure is not set up yet. "
+             "Tring to locate rendezvous breakpoint in the interpreter "
+             "by symbol name.");
+    ModuleSP interpreter = LoadInterpreterModule();
+    if (!interpreter) {
+      LLDB_LOG(log, "Can't find interpreter, rendezvous breakpoint isn't set.");
+      return;
+    }
+
+    const Symbol *symbol = nullptr;
+    static const char *DebugStateCandidates[] = {
+        "_dl_debug_state", "rtld_db_dlactivity", "__dl_rtld_db_dlactivity",
+        "r_debug_state",   "_r_debug_state",     "_rtld_debug_state",
+    };
+
+    for (auto name : DebugStateCandidates) {
+      symbol = interpreter->FindFirstSymbolWithNameAndType(ConstString(name));
+      if (symbol) break;
+    }
+    if (symbol == nullptr) {
+      LLDB_LOG(log, "Can't find debug state function in the interpreter.");
+      return;
+    }
+    break_addr = symbol->GetLoadAddress(&m_process->GetTarget());
+    if (break_addr == LLDB_INVALID_ADDRESS) {
+      LLDB_LOG(log, "Can't resolve address of debug state function {0}.",
+               symbol->GetName());
+      return;
+    } else {
+      LLDB_LOG(
+          log,
+          "Function {0} from module {1} will be used as rendezvous breakpoint.",
+          symbol->GetName(), interpreter->GetFileSpec().GetFilename());
+    }
+  }
+
   if (m_dyld_bid == LLDB_INVALID_BREAK_ID) {
-    if (log)
-      log->Printf("DynamicLoaderPOSIXDYLD::%s pid %" PRIu64
-                  " setting rendezvous break address at 0x%" PRIx64,
-                  __FUNCTION__,
-                  m_process ? m_process->GetID() : LLDB_INVALID_PROCESS_ID,
-                  break_addr);
+    LLDB_LOG(log,
+             "DynamicLoaderPOSIXDYLD::{0} pid {1}"
+             " setting rendezvous break address at {2:x}",
     Breakpoint *dyld_break =
         target.CreateBreakpoint(break_addr, true, false).get();
     dyld_break->SetCallback(RendezvousBreakpointHit, this, true);
     dyld_break->SetBreakpointKind("shared-library-event");
     m_dyld_bid = dyld_break->GetID();
   } else {
-    if (log)
-      log->Printf("DynamicLoaderPOSIXDYLD::%s pid %" PRIu64
-                  " reusing break id %" PRIu32 ", address at 0x%" PRIx64,
-                  __FUNCTION__,
-                  m_process ? m_process->GetID() : LLDB_INVALID_PROCESS_ID,
-                  m_dyld_bid, break_addr);
+    LLDB_LOG(log,
+             "DynamicLoaderPOSIXDYLD::{0} pid {1}"
+             " reusing break id {2}, address at {3:x}",
   }
-
-  // Make sure our breakpoint is at the right address.
-  assert(target.GetBreakpointByID(m_dyld_bid)
-             ->FindLocationByAddress(break_addr)
-             ->GetBreakpoint()
-             .GetID() == m_dyld_bid);
 }
 
 bool DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit(
     void *baton, StoppointCallbackContext *context, user_id_t break_id,
     user_id_t break_loc_id) {
   assert(baton && "null baton");
   if (!baton)
     return false;
 
   Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
   DynamicLoaderPOSIXDYLD *const dyld_instance =
       static_cast<DynamicLoaderPOSIXDYLD *>(baton);
   if (log)
     log->Printf("DynamicLoaderPOSIXDYLD::%s called for pid %" PRIu64,
                 __FUNCTION__,
                 dyld_instance->m_process ? dyld_instance->m_process->GetID()
                                          : LLDB_INVALID_PROCESS_ID);
 
   dyld_instance->RefreshModules();
 
   // Return true to stop the target, false to just let the target run.
   const bool stop_when_images_change = dyld_instance->GetStopWhenImagesChange();
   if (log)
     log->Printf("DynamicLoaderPOSIXDYLD::%s pid %" PRIu64
                 " stop_when_images_change=%s",
                 __FUNCTION__,
if (addrs.size() > 0) {
     std::sort(start, end);
     addrs.erase(std::unique(start, end), end);
     thread_plan_sp.reset(new ThreadPlanRunToAddress(thread, addrs, stop));
   }
 
   return thread_plan_sp;
 }
 
-void DynamicLoaderPOSIXDYLD::LoadVDSO(ModuleList &modules) {
+void DynamicLoaderPOSIXDYLD::LoadVDSO() {
   if (m_vdso_base == LLDB_INVALID_ADDRESS)
     return;
 
   FileSpec file("[vdso]", false);
 
   MemoryRegionInfo info;
   Status status = m_process->GetMemoryRegionInfo(m_vdso_base, info);
   if (status.Fail()) {
     Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
     LLDB_LOG(log, "Failed to get vdso region info: {0}", status);
     return;
   }
 
   if (ModuleSP module_sp = m_process->ReadModuleFromMemory(
           file, m_vdso_base, info.GetRange().GetByteSize())) {
     UpdateLoadedSections(module_sp, LLDB_INVALID_ADDRESS, m_vdso_base, false);
     m_process->GetTarget().GetImages().AppendIfNeeded(module_sp);
   }
 }
 
+ModuleSP DynamicLoaderPOSIXDYLD::LoadInterpreterModule() {
+  if (m_interpreter_base == LLDB_INVALID_ADDRESS) return nullptr;
+
+  MemoryRegionInfo info;
+  Target &target = m_process->GetTarget();
+  Status status = m_process->GetMemoryRegionInfo(m_interpreter_base, info);
+  if (status.Fail()) {
+    Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
+    LLDB_LOG(log, "Failed to get interpreter region info: {0}", status);
+    return nullptr;
+  }
+
+  FileSpec file(info.GetName().GetCString(), false);
+  ModuleSpec module_spec(file, target.GetArchitecture());
+
+  if (ModuleSP module_sp = target.GetSharedModule(module_spec)) {
+    UpdateLoadedSections(module_sp, LLDB_INVALID_ADDRESS, m_interpreter_base,
+                         false);
+    target.GetImages().AppendIfNeeded(module_sp);
+    return module_sp;
+  }
+  return nullptr;
+}
+
 void DynamicLoaderPOSIXDYLD::LoadAllCurrentModules() {
   DYLDRendezvous::iterator I;
   DYLDRendezvous::iterator E;
   ModuleList module_list;
+  Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
 
   if (!m_rendezvous.Resolve()) {
-    Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
     if (log)
       log->Printf("DynamicLoaderPOSIXDYLD::%s unable to resolve POSIX DYLD "
                   "rendezvous address",
                   __FUNCTION__);
     return;
   }
 
   // The rendezvous class doesn't enumerate the main module, so track
   // that ourselves here.
   ModuleSP executable = GetTargetExecutable();
   m_loaded_modules[executable] = m_rendezvous.GetLinkMapAddress();
-  LoadVDSO(module_list);
+  LoadVDSO();
 
   std::vector<FileSpec> module_names;
   for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I)
     module_names.push_back(I->file_spec);
   m_process->PrefetchModuleSpecs(
       module_names, m_process->GetTarget().GetArchitecture().GetTriple());
 
   for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) {
     ModuleSP module_sp =
         LoadModuleAtAddress(I->file_spec, I->link_addr, I->base_addr, true);
     if (module_sp.get()) {
+      LLDB_LOG(log, "LoadAllCurrentModules loading moduel: {0}",
+               I->file_spec.GetFilename());
       module_list.Append(module_sp);
     } else {
       Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
       if (log)
         log->Printf(
             "DynamicLoaderPOSIXDYLD::%s failed loading module %s at 0x%" PRIx64,
             __FUNCTION__, I->file_spec.GetCString(), I->base_addr);
     }
addr_t DynamicLoaderPOSIXDYLD::ComputeLoadOffset() {
 
   if (!file_entry.IsValid())
     return LLDB_INVALID_ADDRESS;
 
   m_load_offset = virt_entry - file_entry.GetFileAddress();
   return m_load_offset;
 }
 
-void DynamicLoaderPOSIXDYLD::EvalVdsoStatus() {
-  AuxVector::iterator I = m_auxv->FindEntry(AuxVector::AUXV_AT_SYSINFO_EHDR);
-
+void DynamicLoaderPOSIXDYLD::EvalSpecialModulesStatus() {
+  auto I = m_auxv->FindEntry(AuxVector::AUXV_AT_SYSINFO_EHDR);
   if (I != m_auxv->end())
     m_vdso_base = I->value;
+
+  I = m_auxv->FindEntry(AuxVector::AUXV_AT_BASE);
+  if (I != m_auxv->end()) m_interpreter_base = I->value;
 }
 
 addr_t DynamicLoaderPOSIXDYLD::GetEntryPoint() {
   if (m_entry_point != LLDB_INVALID_ADDRESS)
     return m_entry_point;
 
   if (m_auxv.get() == NULL)
     return LLDB_INVALID_ADDRESS;