[lldb] [llgs client] Support minimal fork/vfork handling

[mgorny]

Summary

Add a support for handling fork/vfork stops in LLGS client. At this
point, it only sends a detach packet for the newly forked child
(and implicitly resumes the parent).

[mgorny]

Base class Did*Fork() definitions have been moved to the earlier patch.

[mgorny]

Merged D100541 and client fork/vfork stop infos from D100208 into this one. Included fork-events+ and vfork-events+ in client qSupported.

In other words, this is a complete commit that moves fork/vfork handling from server to client. qSupported enables Process plugins to start emitting fork/vfork stop reasons. The stop handler triggers DidFork() and DidVFork() handlers in gdb-remote process plugin, that send detach packet with the specific PID.

The functionality is covered by existing tests in test/Shell/Subprocess.

[mgorny]

@labath, now's a curious problem I've seen very rarely (I mean, 1 out of 150 attempts or so):

 1: (lldb) command source -s 0 '/home/mgorny/git/llvm-project/build.gentoo/tools/lldb/test/Shell/lit-lldb-init' 
 2: Executing commands in '/home/mgorny/git/llvm-project/build.gentoo/tools/lldb/test/Shell/lit-lldb-init'. 
 3: (lldb) # LLDB init file for the LIT tests. 
 4: (lldb) settings set symbols.enable-external-lookup false 
 5: (lldb) settings set plugin.process.gdb-remote.packet-timeout 60 
 6: (lldb) settings set interpreter.echo-comment-commands false 
 7: (lldb) settings set symbols.clang-modules-cache-path "/home/mgorny/git/llvm-project/build.gentoo/lldb-test-build.noindex/module-cache-lldb" 
 8: (lldb) settings set target.auto-apply-fixits false 
 9: (lldb) settings set target.inherit-tcc true 
10: (lldb) settings set target.detach-on-error false 
11: (lldb) target create "/home/mgorny/git/llvm-project/build.gentoo/tools/lldb/test/Shell/Subprocess/Output/vfork-follow-parent-wp.test.tmp" 
12: Current executable set to '/home/mgorny/git/llvm-project/build.gentoo/tools/lldb/test/Shell/Subprocess/Output/vfork-follow-parent-wp.test.tmp' (x86_64). 
13: (lldb) command source -s 0 '/home/mgorny/git/llvm-project/lldb/test/Shell/Subprocess/vfork-follow-parent-wp.test' 
14: Executing commands in '/home/mgorny/git/llvm-project/lldb/test/Shell/Subprocess/vfork-follow-parent-wp.test'. 
15: (lldb) process launch -s 
16: Process 240829 launched: '/home/mgorny/git/llvm-project/build.gentoo/tools/lldb/test/Shell/Subprocess/Output/vfork-follow-parent-wp.test.tmp' (x86_64) 
17: (lldb) watchpoint set variable -w write g_val 
18: Watchpoint created: Watchpoint 1: addr = 0x00404054 size = 4 state = enabled type = w 
19:  watchpoint spec = 'g_val' 
20:  new value: 0 
21: (lldb) continue 
22: Process 240829 resuming 
23: Process 240829 stopped 
24: * thread #1, name = 'vfork-follow-pa', stop reason = signal SIGSTKFLT 
25:  frame #0: 0x00007ffff7b3b708 libc.so.6`__vfork + 8 
26: libc.so.6`__vfork: 
27: -> 0x7ffff7b3b708 <+8>: pushq %rdi 
28:  0x7ffff7b3b709 <+9>: cmpl $0xfffff001, %eax ; imm = 0xFFFFF001  
29:  0x7ffff7b3b70e <+14>: jae 0x7ffff7b3b711 ; <+17> 
30:  0x7ffff7b3b710 <+16>: retq

i.e. vfork() emits SIGSTKFLT for some reason and this confuses the test. I have no clue why this happens — signal(7) says that this signal is unused. It might be a bug in glibc, or in glibc-2.33 more specifically.

[mgorny]

Add gdb_remote_client tests for fork/vfork stop reason handling.

[mgorny]

Rebased and added VForKDone, for completeness.

[mgorny]

Rebase.

[mgorny]

Rebased.

[mgorny]

@JDevlieghere, any chance you could look at this, please?

[mgorny]

Ping.

[mgorny]

@JDevlieghere, ping.

[krytarowski]

Looks good.

lldb/include/lldb/Target/Process.h

/// \see Process::Notifications
   /// lldb_private::Process class needs to remove its dynamic loader, runtime,
   /// ABI and other plug-ins, as well as unload all shared libraries.
   virtual void DidExec();
 
   /// Subclasses of Process should implement this function if they need to do
   /// anything after a process exec's itself.
   virtual void DoDidExec() {}
 
+  /// Called after a reported fork.
+  virtual void DidFork(lldb::pid_t child_pid, lldb::tid_t child_tid) {}
+
+  /// Called after a reported vfork.
+  virtual void DidVFork(lldb::pid_t child_pid, lldb::tid_t child_tid) {}
+
+  /// Called after reported vfork completion.
+  virtual void DidVForkDone() {}
+
   /// Called before launching to a process.
   ///
   /// Allow Process plug-ins to execute some code before launching a process.
   ///
   /// \return
   ///     Returns an error object.
   virtual Status WillLaunch(Module *module) { return Status(); }
 

lldb/include/lldb/Target/StopInfo.h

public:
   static lldb::StopInfoSP
   CreateStopReasonWithException(Thread &thread, const char *description);
 
   static lldb::StopInfoSP CreateStopReasonWithExec(Thread &thread);
 
   static lldb::StopInfoSP
   CreateStopReasonProcessorTrace(Thread &thread, const char *description);
 
+  static lldb::StopInfoSP CreateStopReasonFork(Thread &thread,
+                                               lldb::pid_t child_pid,
+                                               lldb::tid_t child_tid);
+
+  static lldb::StopInfoSP CreateStopReasonVFork(Thread &thread,
+                                                lldb::pid_t child_pid,
+                                                lldb::tid_t child_tid);
+
+  static lldb::StopInfoSP CreateStopReasonVForkDone(Thread &thread);
+
   static lldb::ValueObjectSP
   GetReturnValueObject(lldb::StopInfoSP &stop_info_sp);
 
   static lldb::ExpressionVariableSP
   GetExpressionVariable(lldb::StopInfoSP &stop_info_sp);
 
   static lldb::ValueObjectSP
   GetCrashingDereference(lldb::StopInfoSP &stop_info_sp,

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.h

public:
   /// \return
   ///     Boolean for success
   bool GetWorkingDir(FileSpec &working_dir);
 
   lldb::addr_t AllocateMemory(size_t size, uint32_t permissions);
 
   bool DeallocateMemory(lldb::addr_t addr);
 
-  Status Detach(bool keep_stopped);
+  Status Detach(bool keep_stopped, lldb::pid_t pid = LLDB_INVALID_PROCESS_ID);
 
   Status GetMemoryRegionInfo(lldb::addr_t addr, MemoryRegionInfo &range_info);
 
   Status GetWatchpointSupportInfo(uint32_t &num);
 
   Status GetWatchpointSupportInfo(uint32_t &num, bool &after,
                                   const ArchSpec &arch);
 

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp

void GDBRemoteCommunicationClient::GetRemoteQSupported() {
   m_supports_QPassSignals = eLazyBoolNo;
   m_supports_memory_tagging = eLazyBoolNo;
 
   m_max_packet_size = UINT64_MAX; // It's supposed to always be there, but if
                                   // not, we assume no limit
 
   // build the qSupported packet
   std::vector<std::string> features = {"xmlRegisters=i386,arm,mips,arc",
-                                       "multiprocess+"};
+                                       "multiprocess+", "fork-events+",
+                                       "vfork-events+"};
   StreamString packet;
   packet.PutCString("qSupported");
   for (uint32_t i = 0; i < features.size(); ++i) {
     packet.PutCString(i == 0 ? ":" : ";");
     packet.PutCString(features[i]);
   }
 
   StringExtractorGDBRemote response;
if (SendPacketAndWaitForResponse(packet, response) ==
         return true;
     } else {
       m_supports_alloc_dealloc_memory = eLazyBoolNo;
     }
   }
   return false;
 }
 
-Status GDBRemoteCommunicationClient::Detach(bool keep_stopped) {
+Status GDBRemoteCommunicationClient::Detach(bool keep_stopped,
+                                            lldb::pid_t pid) {
   Status error;
+  lldb_private::StreamString packet;
 
+  packet.PutChar('D');
   if (keep_stopped) {
     if (m_supports_detach_stay_stopped == eLazyBoolCalculate) {
       char packet[64];
       const int packet_len =
           ::snprintf(packet, sizeof(packet), "qSupportsDetachAndStayStopped:");
       assert(packet_len < (int)sizeof(packet));
       UNUSED_IF_ASSERT_DISABLED(packet_len);
       StringExtractorGDBRemote response;
       if (SendPacketAndWaitForResponse(packet, response) ==
               PacketResult::Success &&
           response.IsOKResponse()) {
         m_supports_detach_stay_stopped = eLazyBoolYes;
       } else {
         m_supports_detach_stay_stopped = eLazyBoolNo;
       }
     }
 
     if (m_supports_detach_stay_stopped == eLazyBoolNo) {
       error.SetErrorString("Stays stopped not supported by this target.");
       return error;
     } else {
-      StringExtractorGDBRemote response;
-      PacketResult packet_result = SendPacketAndWaitForResponse("D1", response);
-      if (packet_result != PacketResult::Success)
-        error.SetErrorString("Sending extended disconnect packet failed.");
+      packet.PutChar('1');
     }
-  } else {
-    StringExtractorGDBRemote response;
-    PacketResult packet_result = SendPacketAndWaitForResponse("D", response);
-    if (packet_result != PacketResult::Success)
-      error.SetErrorString("Sending disconnect packet failed.");
-  }
+  }
+
+  if (pid != LLDB_INVALID_PROCESS_ID) {
+    if (!m_supports_multiprocess) {
+      error.SetErrorString(
+          "Multiprocess extension not supported by the server.");
+      return error;
+    }
+    packet.PutChar(';');
+    packet.PutHex64(pid);
+  }
+
+  PacketResult packet_result =
+      SendPacketAndWaitForResponse(packet.GetString(), response);
+    error.SetErrorString("Sending isconnect packet failed.");
   return error;
 }
 
 Status GDBRemoteCommunicationClient::GetMemoryRegionInfo(
     lldb::addr_t addr, lldb_private::MemoryRegionInfo &region_info) {
   Status error;
   region_info.Clear();
 

lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.h

public:
   StructuredData::ObjectSP
   GetLoadedDynamicLibrariesInfos_sender(StructuredData::ObjectSP args);
 
   StructuredData::ObjectSP GetSharedCacheInfo() override;
 
   std::string HarmonizeThreadIdsForProfileData(
       StringExtractorGDBRemote &inputStringExtractor);
 
+  void DidFork(lldb::pid_t child_pid, lldb::tid_t child_tid) override;
+  void DidVFork(lldb::pid_t child_pid, lldb::tid_t child_tid) override;
+
 protected:
   friend class ThreadGDBRemote;
   friend class GDBRemoteCommunicationClient;
   friend class GDBRemoteRegisterContext;
 
   bool SupportsMemoryTagging() override;
 
   /// Broadcaster event bits definitions.

lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp

if (thread_sp) {
             } else if (reason == "exec") {
               did_exec = true;
               thread_sp->SetStopInfo(
                   StopInfo::CreateStopReasonWithExec(*thread_sp));
               handled = true;
             } else if (reason == "processor trace") {
               thread_sp->SetStopInfo(StopInfo::CreateStopReasonProcessorTrace(
                   *thread_sp, description.c_str()));
+            } else if (reason == "fork") {
+              StringExtractor desc_extractor(description.c_str());
+              lldb::pid_t child_pid = desc_extractor.GetU64(
+                  LLDB_INVALID_PROCESS_ID);
+              lldb::tid_t child_tid = desc_extractor.GetU64(
+                  LLDB_INVALID_THREAD_ID);
+              thread_sp->SetStopInfo(StopInfo::CreateStopReasonFork(
+                  *thread_sp, child_pid, child_tid));
+              handled = true;
+            } else if (reason == "vfork") {
+              StringExtractor desc_extractor(description.c_str());
+              lldb::pid_t child_pid = desc_extractor.GetU64(
+                  LLDB_INVALID_PROCESS_ID);
+              lldb::tid_t child_tid = desc_extractor.GetU64(
+                  LLDB_INVALID_THREAD_ID);
+              thread_sp->SetStopInfo(StopInfo::CreateStopReasonVFork(
+                  *thread_sp, child_pid, child_tid));
+              handled = true;
+            } else if (reason == "vforkdone") {
+              thread_sp->SetStopInfo(
+                  StopInfo::CreateStopReasonVForkDone(*thread_sp));
+              handled = true;
             }
           } else if (!signo) {
             addr_t pc = thread_sp->GetRegisterContext()->GetPC();
             lldb::BreakpointSiteSP bp_site_sp =
                 thread_sp->GetProcess()->GetBreakpointSiteList().FindByAddress(
                     pc);
 
             // If the current pc is a breakpoint site then the StopInfo should
while (stop_packet.GetNameColonValue(key, value)) {
         description = std::string(ostr.GetString());
       } else if (key.compare("library") == 0) {
         auto error = LoadModules();
         if (error) {
           Log *log(
               ProcessGDBRemoteLog::GetLogIfAllCategoriesSet(GDBR_LOG_PROCESS));
           LLDB_LOG_ERROR(log, std::move(error), "Failed to load modules: {0}");
         }
+      } else if (key.compare("fork") == 0 || key.compare("vfork") == 0) {
+        // fork includes child pid/tid in thread-id format
+        StringExtractorGDBRemote thread_id{value};
+        auto pid_tid = thread_id.GetPidTid(LLDB_INVALID_PROCESS_ID);
+        if (!pid_tid) {
+          Log *log(
+              ProcessGDBRemoteLog::GetLogIfAllCategoriesSet(GDBR_LOG_PROCESS));
+          LLDB_LOG(log, "Invalid PID/TID to fork: {0}", value);
+          pid_tid = {{LLDB_INVALID_PROCESS_ID, LLDB_INVALID_THREAD_ID}};
+        }
+
+        reason = key.str();
+        StreamString ostr;
+        ostr.Printf("%" PRIu64 " %" PRIu64, pid_tid->first, pid_tid->second);
+        description = std::string(ostr.GetString());
       } else if (key.size() == 2 && ::isxdigit(key[0]) && ::isxdigit(key[1])) {
         uint32_t reg = UINT32_MAX;
         if (!key.getAsInteger(16, reg))
           expedited_register_map[reg] = std::string(std::move(value));
       }
     }
 
     if (stop_pid != LLDB_INVALID_PROCESS_ID && stop_pid != pid) {

 };
 
 CommandObject *ProcessGDBRemote::GetPluginCommandObject() {
   if (!m_command_sp)
     m_command_sp = std::make_shared<CommandObjectMultiwordProcessGDBRemote>(
         GetTarget().GetDebugger().GetCommandInterpreter());
   return m_command_sp.get();
 }
+
+void ProcessGDBRemote::DidFork(lldb::pid_t child_pid, lldb::tid_t child_tid) {
+  Log *log(ProcessGDBRemoteLog::GetLogIfAllCategoriesSet(GDBR_LOG_PROCESS));
+
+  LLDB_LOG(log, "Detaching forked child {0}", child_pid);
+  Status error = m_gdb_comm.Detach(false, child_pid);
+  if (error.Fail()) {
+      LLDB_LOG(log,
+               "ProcessGDBRemote::DidFork() detach packet send failed: {0}",
+                error.AsCString() ? error.AsCString() : "<unknown error>");
+      return;
+  }
+}
+
+void ProcessGDBRemote::DidVFork(lldb::pid_t child_pid, lldb::tid_t child_tid) {
+  Log *log(ProcessGDBRemoteLog::GetLogIfAllCategoriesSet(GDBR_LOG_PROCESS));
+
+  LLDB_LOG(log, "Detaching forked child {0}", child_pid);
+  Status error = m_gdb_comm.Detach(false, child_pid);
+  if (error.Fail()) {
+      LLDB_LOG(log,
+               "ProcessGDBRemote::DidFork() detach packet send failed: {0}",
+                error.AsCString() ? error.AsCString() : "<unknown error>");
+      return;
+  }
+}

lldb/source/Target/StopInfo.cpp

void PerformAction(Event *event_ptr) override {
     ThreadSP thread_sp(m_thread_wp.lock());
     if (thread_sp)
       thread_sp->GetProcess()->DidExec();
   }
 
   bool m_performed_action;
 };
 
+// StopInfoFork
+
+class StopInfoFork : public StopInfo {
+public:
+  StopInfoFork(Thread &thread, lldb::pid_t child_pid, lldb::tid_t child_tid)
+      : StopInfo(thread, child_pid), m_performed_action(false),
+        m_child_pid(child_pid), m_child_tid(child_tid) {}
+
+  ~StopInfoFork() override = default;
+
+  bool ShouldStop(Event *event_ptr) override { return false; }
+
+  StopReason GetStopReason() const override { return eStopReasonFork; }
+
+  const char *GetDescription() override { return "fork"; }
+
+protected:
+  void PerformAction(Event *event_ptr) override {
+    // Only perform the action once
+    if (m_performed_action)
+      return;
+    m_performed_action = true;
+    ThreadSP thread_sp(m_thread_wp.lock());
+    if (thread_sp)
+      thread_sp->GetProcess()->DidFork(m_child_pid, m_child_tid);
+  }
+
+  bool m_performed_action;
+
+private:
+  lldb::pid_t m_child_pid;
+  lldb::tid_t m_child_tid;
+};
+
+// StopInfoVFork
+
+class StopInfoVFork : public StopInfo {
+public:
+  StopInfoVFork(Thread &thread, lldb::pid_t child_pid, lldb::tid_t child_tid)
+      : StopInfo(thread, child_pid), m_performed_action(false),
+        m_child_pid(child_pid), m_child_tid(child_tid) {}
+
+  ~StopInfoVFork() override = default;
+
+  bool ShouldStop(Event *event_ptr) override { return false; }
+
+  StopReason GetStopReason() const override { return eStopReasonVFork; }
+
+  const char *GetDescription() override { return "vfork"; }
+
+protected:
+  void PerformAction(Event *event_ptr) override {
+    // Only perform the action once
+    if (m_performed_action)
+      return;
+    m_performed_action = true;
+    ThreadSP thread_sp(m_thread_wp.lock());
+    if (thread_sp)
+      thread_sp->GetProcess()->DidVFork(m_child_pid, m_child_tid);
+  }
+
+  bool m_performed_action;
+
+private:
+  lldb::pid_t m_child_pid;
+  lldb::tid_t m_child_tid;
+};
+
+// StopInfoVForkDone
+
+class StopInfoVForkDone : public StopInfo {
+public:
+  StopInfoVForkDone(Thread &thread)
+      : StopInfo(thread, 0), m_performed_action(false) {}
+
+  ~StopInfoVForkDone() override = default;
+
+  bool ShouldStop(Event *event_ptr) override { return false; }
+
+  StopReason GetStopReason() const override { return eStopReasonVForkDone; }
+
+  const char *GetDescription() override { return "vforkdone"; }
+
+protected:
+  void PerformAction(Event *event_ptr) override {
+    // Only perform the action once
+    if (m_performed_action)
+      return;
+    m_performed_action = true;
+    ThreadSP thread_sp(m_thread_wp.lock());
+    if (thread_sp)
+      thread_sp->GetProcess()->DidVForkDone();
+  }
+
+  bool m_performed_action;
+};
+
 } // namespace lldb_private
 
 StopInfoSP StopInfo::CreateStopReasonWithBreakpointSiteID(Thread &thread,
                                                           break_id_t break_id) {
   return StopInfoSP(new StopInfoBreakpoint(thread, break_id));
 }
 
 StopInfoSP StopInfo::CreateStopReasonWithBreakpointSiteID(Thread &thread,
StopInfoSP StopInfo::CreateStopReasonProcessorTrace(Thread &thread,
                                                     const char *description) {
   return StopInfoSP(new StopInfoProcessorTrace(thread, description));
 }
 
 StopInfoSP StopInfo::CreateStopReasonWithExec(Thread &thread) {
   return StopInfoSP(new StopInfoExec(thread));
 }
 
+StopInfoSP StopInfo::CreateStopReasonFork(Thread &thread,
+                                          lldb::pid_t child_pid,
+                                          lldb::tid_t child_tid) {
+  return StopInfoSP(new StopInfoFork(thread, child_pid, child_tid));
+}
+
+
+StopInfoSP StopInfo::CreateStopReasonVFork(Thread &thread,
+                                           lldb::pid_t child_pid,
+                                           lldb::tid_t child_tid) {
+  return StopInfoSP(new StopInfoVFork(thread, child_pid, child_tid));
+}
+
+StopInfoSP StopInfo::CreateStopReasonVForkDone(Thread &thread) {
+  return StopInfoSP(new StopInfoVForkDone(thread));
+}
+
 ValueObjectSP StopInfo::GetReturnValueObject(StopInfoSP &stop_info_sp) {
   if (stop_info_sp &&
       stop_info_sp->GetStopReason() == eStopReasonPlanComplete) {
     StopInfoThreadPlan *plan_stop_info =
         static_cast<StopInfoThreadPlan *>(stop_info_sp.get());
     return plan_stop_info->GetReturnValueObject();
   } else
     return ValueObjectSP();

lldb/test/API/functionalities/gdb_remote_client/TestFork.py

+from __future__ import print_function
+import lldb
+import unittest
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test.decorators import *
+from gdbclientutils import *
+
+
+class TestMultiprocess(GDBRemoteTestBase):
+    def base_test(self, variant):
+        class MyResponder(MockGDBServerResponder):
+            def __init__(self):
+                super().__init__()
+                self.first = True
+                self.detached = None
+                self.property = "{}-events+".format(variant)
+
+            def qSupported(self, client_supported):
+                assert "multiprocess+" in client_supported
+                assert self.property in client_supported
+                return "{};multiprocess+;{}".format(
+                    super().qSupported(client_supported), self.property)
+
+            def qfThreadInfo(self):
+                return "mp400.10200"
+
+            def cont(self):
+                if self.first:
+                    self.first = False
+                    return ("T0fthread:p400.10200;reason:{0};{0}:p401.10400;"
+                            .format(variant))
+                return "W00"
+
+            def D(self, packet):
+                self.detached = packet
+                return "OK"
+
+        self.server.responder = MyResponder()
+        target = self.dbg.CreateTarget('')
+        if self.TraceOn():
+          self.runCmd("log enable gdb-remote packets")
+          self.addTearDownHook(
+                lambda: self.runCmd("log disable gdb-remote packets"))
+        process = self.connect(target)
+        process.Continue()
+        self.assertRegex(self.server.responder.detached, r"D;0*401")
+
+    def test_fork(self):
+        self.base_test("fork")
+
+    def test_vfork(self):
+        self.base_test("vfork")

lldb/test/API/functionalities/gdb_remote_client/gdbclientutils.py

def respond(self, packet):
         if packet is MockGDBServer.PACKET_INTERRUPT:
             return self.interrupt()
         if packet == "c":
             return self.cont()
         if packet.startswith("vCont;c"):
             return self.vCont(packet)
         if packet[0] == "A":
             return self.A(packet)
+        if packet[0] == "D":
+            return self.D(packet)
         if packet[0] == "g":
             return self.readRegisters()
         if packet[0] == "G":
             # Gxxxxxxxxxxx
             # Gxxxxxxxxxxx;thread:1234;
             return self.writeRegisters(packet[1:].split(';')[0])
         if packet[0] == "p":
             regnum = packet[1:].split(';')[0]
def cont(self):
         raise self.UnexpectedPacketException()
 
     def vCont(self, packet):
         raise self.UnexpectedPacketException()
 
     def A(self, packet):
         return ""
 
+    def D(self, packet):
+        return "OK"
+
     def readRegisters(self):
         return "00000000" * self.registerCount
 
     def readRegister(self, register):
         return "00000000"
 
     def writeRegisters(self, registers_hex):
         return "OK"