[lldb/Plugins] Add support of multiple ScriptedThreads in a ScriptedProcess

[mib]

Summary

This patch adds support of multiple Scripted Threads in a ScriptedProcess.

This is done by fetching the Scripted Threads info dictionary at every
ScriptedProcess::DoUpdateThreadList and iterate over each element to
create a new ScriptedThread using the object instance, if it was not
already available.

This patch also adds the ability to pass a pointer of a script interpreter
object instance to initialize a ScriptedInterface instead of having to call
the script object initializer in the ScriptedInterface constructor.

This is used to instantiate the ScriptedThreadInterface from the
ScriptedThread constructor, to be able to perform call on that script
interpreter object instance.

Finally, the patch also updates the scripted process test to check for
multiple threads.

rdar://84507704

Signed-off-by: Med Ismail Bennani <[e-mail]>

[mib]

Update invalid_scripted_process.py

[mib]

lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp

332

@labath This is why I switched to a raw pointer in D117065. Basically, I fetch a python dictionary for the thread_info_sp. It gets converted to a StructuredData::Dictionary and every element gets re-allocated in pythonObject::CreateStructuredDictionary. Then I iterate over each of element of the StructuredData::Dictionary and passes the StructuredData::Object to the ScriptedThread constructor.

The ScriptedThread needs a valid reference to the python object instance to perform calls on it. However, when the structured dictionary goes out of scope (at the end of the function), all the objects it holds get destroyed so when the ScriptedThread tries to make a call on the python object is causes a heap-use-after-free.

[mib]

Update after abandoning D117065 for D117139.

To create the ScriptedThread, we pass a StructuredData::Object* that contains a borrowed reference of the python ScriptedThread instance, to the constructor.

Previously, we just re-assigned the ScriptedThread StructuredData script object shared_ptr with the StructuredData::Object*.
Now, we "extract" the python borrowed reference from the StructuredData::Object* and create a new StructuredData object with it, to make sure we can access it even after the StructuredData::Object* gets freed when exiting ScriptedProcess::DoUpdateThreadList

[mib]

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

51

Instead of re-assigning m_script_object_sp with script_object directly, we get the python object ptr and pass it to CreatePluginObject that will create a new StructuredData::GenericSP holding that ref.

[JDevlieghere]

Never thought I'd ask someone to merge two patches, but I think it might make reviewing easier if you merge D117139 into this patch. :-)

If I understand the patch correctly, you're getting the underlying PyObject out of the PythonDateObject when passing it to the scripted thread interface. I assume that works because the objects are retained by being stored in a dict on the Python side (referring to threads in ScriptedProcess, from D117068). Why can't we pass the PythonObject around instead? That seems simpler but more importantly would guarantee the underlying object remains alive (even if it weren't stored on the Python side) by keeping the ref-count incremented.

[mib]

Address @JDevlieghere comment by merging D117139 into D117071

[mib]

In D117071#3238816, @JDevlieghere wrote:

Never thought I'd ask someone to merge two patches, but I think it might make reviewing easier if you merge D117139 into this patch. :-)

Done ^^

If I understand the patch correctly, you're getting the underlying PyObject out of the PythonDateObject when passing it to the scripted thread interface. I assume that works because the objects are retained by being stored in a dict on the Python side (referring to threads in ScriptedProcess, from D117068). Why can't we pass the PythonObject around instead? That seems simpler but more importantly would guarantee the underlying object remains alive (even if it weren't stored on the Python side) by keeping the ref-count incremented.

Right. Sure, it would be easier to pass the PythonObject around but the Scripted(Thread)Interface tries to be as language-agnostic as possible, that's why we use StructuredData::Generic instead. I think it should remain this way.

[labath]

Continuing the discussion from the other patch, I've made some comments about who I think owns various objects going around in this patch. They should be read in the control-flow order (which happens to coincide with how they appear in the source code), not in the order the phabricator puts them at the bottom of the page. I'd appreciate it if you could go through them and point out any errors in my reasoning/assumptions.

lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp

304

This looks like the "start" of this patch. IIUC, this shared_ptr is the only shared_ptr pointing to this object (use_count() == 1).

313

this object appears to come out of thread_info_sp. I am going to assume it is owned by that object.

332

This just does a cast so it is still owned by that object.

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

32

Control continues here. script_object is not owned by this function/class.

49

This is when things start to get fuzzy, as this function seems to support both a nullptr and a non-nullptr argument. Not necessarily bad, but it makes reasoning about anything harder. It'd be better if this function could always be called with a valid object.

51

I'm assuming this is a PyObject disguised as a void* and that it is still being owned(increffed) by the initial shared_ptr.

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp

33–34

continuing here

46

Now it gets really confusing. *If* instance_obj was non-null, then it will point to an object which is not owned by this function. *If* it was null, then it set to point to an object which *is* owned (or at least *should be* owned -- as noone else owns it) by this function.

This doesn't seem correct to me, but even if it is, it sure is confusing.

59

now this maybe-owned reference gets passed into a StructuredPythonObject which assumes it is getting a borrowed reference (it does an incref).

That seems correct in the instance_obj@entry != nullptr case, but not in the other one (=> leak).

[mib]

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

49

I understand but even if it might be possible to do it for ScriptedThreads, we want to give the user multiple way of creating them (in lldb or directly from python).

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp

46

Correct, when instance_obj is null at entry, it gets allocated by LLDBSwigPythonCreateScriptedThread which creates an owned reference as you mentioned in https://reviews.llvm.org/D117065#inline-1120555, and that reference gets passed to m_object_instance_sp.

So in my understanding, at this stage, instance_obj use_count() == 1.

59

I might be missing something but I don't understand why does the StructuredPythonObject expects a borrowed reference ?
Why can't it wrap an owned reference ?

I think checking the PyObject type (Borrowed/Owned) on the StructuredPythonObject constructor would allow us to incref it only in the case of a borrowed ref, and fix the leak in the case of an owned ref, right ?

[mib]

In D117071#3241300, @labath wrote:

Continuing the discussion from the other patch, I've made some comments about who I think owns various objects going around in this patch. They should be read in the control-flow order (which happens to coincide with how they appear in the source code), not in the order the phabricator puts them at the bottom of the page. I'd appreciate it if you could go through them and point out any errors in my reasoning/assumptions.

I've marked as done all of the comments that sounded good to me, and left some comments and questions myself.

I think this patch can stay as is, as long as add support for owned references in StructuredPythonObject. I don't see any reason why this should not be allowed, but I might be missing something. If this solution sounds good to you, I can write a separate patch for that.

[labath]

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

49

While not ideal, I don't think this is a big deal. I might consider passing the script_object to the CreatePluginObject though -- it would make a typed interface, and avoid a branch here.

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp

59

Why can't it wrap an owned reference ?

There's no reason it *can't* do that. It's just that it doesn't do that now.

checking the PyObject type (Borrowed/Owned)

borrowedness/ownedness is not a real property of the PyObject. It's just an abstract notion describing the relationship between a reference (PyObject instance) and code handling that reference. If some code "owns" a reference it is responsible for (eventually) freeing (decreffing) it (or passing the ownership onto someone else, etc.) If some code "borrows" a reference, then it must not free it, and it must be careful to not use it after the actual owner frees it.

The reference received through the instance_obj is borrowed since the thread_info_sp destructor will eventually free it. The reference created through LLDBSwigPythonCreateScriptedThread is owned. The fact that you have owning and non-owning code paths converging means one of them is incorrect.

The fix is actually pretty straightforward -- you can change a borrowed reference into an owned one by increffing it yourself. Ideally the PythonObject helper class, as that way the ownership will be explicitly managed.

[mib]

Address @labath feedbacks:

• Pass StructuredData::Generic *script_object to ScriptedInterface::CreatePluginObject.
• IncRef the borrowed reference to make it an owned reference.

[labath]

Unifying the two paths by making turning the other reference into an owned one is a step in the right direction, but it is not enough, as StructuredPythonObject expects a *borrowed* reference. So, instead of fixing things, you've made both code paths leak. :)

I've created D117462, which should make reasoning about ownership much easier. StructuredPythonObject and all of the LLDBSwigPython functions now tracks ownership automatically, and one only needs to be careful when converting to/from a void *. Ideally those would be removed too, but that would require a bigger refactor. If you rebase this patch on top of that, you should be able to transfer ownership correctly.

While working on the patch I also realized that pretty much all of our StructuredPythonObject interfaces were leaking. So, in a way, you were very unfortunate to hit a use after free here (like, it it must have taken multiple decrefs on borrowed references (one for each thread, I guess) to undo all of those leaks). OTOH, we were also fortunate that you've found this, as we can now improve the interfaces and fix those leaks.

[mib]

Rebase

[labath]

I think this is as good as we can make it. Thanks for your patience.

[mib]

In D117071#3251124, @labath wrote:

I think this is as good as we can make it. Thanks for your patience.

Thanks for your help :) !

lldb/examples/python/scripted_process/scripted_process.py

class ScriptedProcess:
     @abstractmethod
     def get_thread_with_id(self, tid):
         """ Get the scripted process thread with a specific ID.
 
         Args:
             tid (int): Thread ID to look for in the scripted process.
 
         Returns:
-            Dict: The thread represented as a dictionary, withr the
+            Dict: The thread represented as a dictionary, with the
                 tid thread ID. None if tid doesn't match any of the scripted
                 process threads.
         """
         pass
 
     @abstractmethod
     def get_registers_for_thread(self, tid):
         """ Get the register context dictionary for a certain thread of
def __init__(self, process, args):
         Args:
             process (lldb.SBProcess): The scripted process owning this thread.
             args (lldb.SBStructuredData): A Dictionary holding arbitrary
                 key/value pairs used by the scripted thread.
         """
         self.target = None
         self.process = None
         self.args = None
-        if isinstance(process, lldb.SBProcess) and process.IsValid():
-            self.process = process
-            self.target = process.GetTarget()
+        if isinstance(process, ScriptedProcess):
+            self.target = process.target
+            self.process = self.target.GetProcess()
 
         self.id = None
+        self.idx = None
         self.name = None
         self.queue = None
         self.state = None
         self.stop_reason = None
         self.register_info = None
         self.register_ctx = {}
         self.frames = []
 

lldb/include/lldb/Interpreter/ScriptedInterface.h

 namespace lldb_private {
 class ScriptedInterface {
 public:
   ScriptedInterface() = default;
   virtual ~ScriptedInterface() = default;
 
   virtual StructuredData::GenericSP
   CreatePluginObject(llvm::StringRef class_name, ExecutionContext &exe_ctx,
-                     StructuredData::DictionarySP args_sp) = 0;
+                     StructuredData::DictionarySP args_sp,
+                     StructuredData::Generic *script_obj = nullptr) = 0;
 
   template <typename Ret>
   Ret ErrorWithMessage(llvm::StringRef caller_name, llvm::StringRef error_msg,
                        Status &error,
                        uint32_t log_caterogy = LIBLLDB_LOG_PROCESS) {
     LLDB_LOGF(GetLogIfAllCategoriesSet(log_caterogy), "%s ERROR = %s",
               caller_name.data(), error_msg.data());
     error.SetErrorString(llvm::Twine(caller_name + llvm::Twine(" ERROR = ") +

lldb/include/lldb/Interpreter/ScriptedProcessInterface.h

 
 #include <string>
 
 namespace lldb_private {
 class ScriptedProcessInterface : virtual public ScriptedInterface {
 public:
   StructuredData::GenericSP
   CreatePluginObject(llvm::StringRef class_name, ExecutionContext &exe_ctx,
-                     StructuredData::DictionarySP args_sp) override {
+                     StructuredData::DictionarySP args_sp,
+                     StructuredData::Generic *script_obj = nullptr) override {
     return nullptr;
   }
 
   virtual Status Launch() { return Status("ScriptedProcess did not launch"); }
 
   virtual Status Resume() { return Status("ScriptedProcess did not resume"); }
 
   virtual bool ShouldStop() { return true; }
virtual lldb::ScriptedThreadInterfaceSP CreateScriptedThreadInterface() {
     return nullptr;
   }
 };
 
 class ScriptedThreadInterface : virtual public ScriptedInterface {
 public:
   StructuredData::GenericSP
   CreatePluginObject(llvm::StringRef class_name, ExecutionContext &exe_ctx,
-                     StructuredData::DictionarySP args_sp) override {
+                     StructuredData::DictionarySP args_sp,
+                     StructuredData::Generic *script_obj = nullptr) override {
     return nullptr;
   }
 
   virtual lldb::tid_t GetThreadID() { return LLDB_INVALID_THREAD_ID; }
 
   virtual llvm::Optional<std::string> GetName() { return llvm::None; }
 
   virtual lldb::StateType GetState() { return lldb::eStateInvalid; }

lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp

Status ScriptedProcess::DoLaunch(Module *exe_module,
 
   // TODO: Fetch next state from stopped event queue then send stop event
   //  const StateType state = SetThreadStopInfo(response);
   //  if (state != eStateInvalid) {
   //    SetPrivateState(state);
 
   SetPrivateState(eStateStopped);
 
-  UpdateThreadListIfNeeded();
-  GetThreadList();
-
   return {};
 }
 
 void ScriptedProcess::DidLaunch() {
   CheckInterpreterAndScriptObject();
   m_pid = GetInterface().GetProcessID();
 }
 
if (language != eScriptLanguagePython)
     return GetInterface().ErrorWithMessage<bool>(
         LLVM_PRETTY_FUNCTION,
         llvm::Twine("ScriptInterpreter language (" +
                     llvm::Twine(m_interpreter->LanguageToString(language)) +
                     llvm::Twine(") not supported."))
             .str(),
         error);
 
-  lldb::ThreadSP thread_sp;
-  thread_sp = std::make_shared<ScriptedThread>(*this, error);
+  StructuredData::DictionarySP thread_info_sp = GetInterface().GetThreadsInfo();
+
+  if (!thread_info_sp)
+    return GetInterface().ErrorWithMessage<bool>(
+        LLVM_PRETTY_FUNCTION,
+        "Couldn't fetch thread list from Scripted Process.", error);
+
+  auto create_scripted_thread =
+      [this, &old_thread_list, &error,
+       &new_thread_list](ConstString key, StructuredData::Object *val) -> bool {
+    if (!val)
+      return GetInterface().ErrorWithMessage<bool>(
+          LLVM_PRETTY_FUNCTION, "Invalid thread info object", error);
+
+    lldb::tid_t tid = LLDB_INVALID_THREAD_ID;
+    if (!llvm::to_integer(key.AsCString(), tid))
+      return GetInterface().ErrorWithMessage<bool>(LLVM_PRETTY_FUNCTION,
+                                                   "Invalid thread id", error);
+
+    if (ThreadSP thread_sp =
+            old_thread_list.FindThreadByID(tid, false /*=can_update*/)) {
+      // If the thread was already in the old_thread_list,
+      // just add it back to the new_thread_list.
+      new_thread_list.AddThread(thread_sp);
+      return true;
+    }
+
+    lldb::ThreadSP thread_sp =
+        std::make_shared<ScriptedThread>(*this, error, val->GetAsGeneric());
 
     if (!thread_sp || error.Fail())
       return GetInterface().ErrorWithMessage<bool>(LLVM_PRETTY_FUNCTION,
                                                    error.AsCString(), error);
 
     RegisterContextSP reg_ctx_sp = thread_sp->GetRegisterContext();
     if (!reg_ctx_sp)
       return GetInterface().ErrorWithMessage<bool>(
-        LLVM_PRETTY_FUNCTION, "Invalid Register Context", error);
+          LLVM_PRETTY_FUNCTION,
+          llvm::Twine("Invalid Register Context for thread " +
+                      llvm::Twine(key.AsCString()))
+              .str(),
+          error);
 
     new_thread_list.AddThread(thread_sp);
 
+    return true;
+  };
+
+  thread_info_sp->ForEach(create_scripted_thread);
+
   return new_thread_list.GetSize(false) > 0;
 }
 
 void ScriptedProcess::RefreshStateAfterStop() {
   // Let all threads recover from stopping and do any clean up based on the
   // previous thread state (if any).
   m_thread_list.RefreshStateAfterStop();
 }

lldb/source/Plugins/Process/scripted/ScriptedThread.h

 namespace lldb_private {
 class ScriptedProcess;
 }
 
 namespace lldb_private {
 
 class ScriptedThread : public lldb_private::Thread {
 public:
-  ScriptedThread(ScriptedProcess &process, Status &error);
+  ScriptedThread(ScriptedProcess &process, Status &error,
+                 StructuredData::Generic *script_object = nullptr);
 
   ~ScriptedThread() override;
 
   lldb::RegisterContextSP GetRegisterContext() override;
 
   lldb::RegisterContextSP
   CreateRegisterContextForFrame(lldb_private::StackFrame *frame) override;
 

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

 using namespace lldb;
 using namespace lldb_private;
 
 void ScriptedThread::CheckInterpreterAndScriptObject() const {
   lldbassert(m_script_object_sp && "Invalid Script Object.");
   lldbassert(GetInterface() && "Invalid Scripted Thread Interface.");
 }
 
-ScriptedThread::ScriptedThread(ScriptedProcess &process, Status &error)
+ScriptedThread::ScriptedThread(ScriptedProcess &process, Status &error,
+                               StructuredData::Generic *script_object)
     : Thread(process, LLDB_INVALID_THREAD_ID), m_scripted_process(process),
       m_scripted_thread_interface_sp(
           m_scripted_process.GetInterface().CreateScriptedThreadInterface()) {
   if (!process.IsValid()) {
     error.SetErrorString("Invalid scripted process");
     return;
   }
 
   process.CheckInterpreterAndScriptObject();
 
   auto scripted_thread_interface = GetInterface();
   if (!scripted_thread_interface) {
     error.SetErrorString("Failed to get scripted thread interface.");
     return;
   }
 
   llvm::Optional<std::string> class_name =
       process.GetInterface().GetScriptedThreadPluginName();
   if (!class_name || class_name->empty()) {
     error.SetErrorString("Failed to get scripted thread class name.");
     return;
   }
 
   ExecutionContext exe_ctx(process);
 
-  StructuredData::GenericSP object_sp =
-      scripted_thread_interface->CreatePluginObject(
-          class_name->c_str(), exe_ctx,
-          process.m_scripted_process_info.GetArgsSP());
-  if (!object_sp || !object_sp->IsValid()) {
-    error.SetErrorString("Failed to create valid script object");
+  m_script_object_sp = scripted_thread_interface->CreatePluginObject(
+      class_name->c_str(), exe_ctx, process.m_scripted_process_info.GetArgsSP(),
+      script_object);
+
+  if (!m_script_object_sp) {
+    error.SetErrorString("Failed to create script object");
     return;
   }
 
-  m_script_object_sp = object_sp;
+  if (!m_script_object_sp->IsValid()) {
+    m_script_object_sp = nullptr;
+    error.SetErrorString("Created script object is invalid");
+    return;
+  }
 
-  SetID(scripted_thread_interface->GetThreadID());
+  lldb::tid_t tid = scripted_thread_interface->GetThreadID();
+  SetID(tid);
 }
 
 ScriptedThread::~ScriptedThread() { DestroyThread(); }
 
 const char *ScriptedThread::GetName() {
   CheckInterpreterAndScriptObject();
   llvm::Optional<std::string> thread_name = GetInterface()->GetName();
   if (!thread_name)

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.h

 class ScriptedProcessPythonInterface : public ScriptedProcessInterface,
                                        public ScriptedPythonInterface {
 public:
   ScriptedProcessPythonInterface(ScriptInterpreterPythonImpl &interpreter);
 
   StructuredData::GenericSP
   CreatePluginObject(const llvm::StringRef class_name,
                      ExecutionContext &exe_ctx,
-                     StructuredData::DictionarySP args_sp) override;
+                     StructuredData::DictionarySP args_sp,
+                     StructuredData::Generic *script_obj = nullptr) override;
 
   Status Launch() override;
 
   Status Resume() override;
 
   bool ShouldStop() override;
 
   Status Stop() override;

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp

 using Locker = ScriptInterpreterPythonImpl::Locker;
 
 ScriptedProcessPythonInterface::ScriptedProcessPythonInterface(
     ScriptInterpreterPythonImpl &interpreter)
     : ScriptedProcessInterface(), ScriptedPythonInterface(interpreter) {}
 
 StructuredData::GenericSP ScriptedProcessPythonInterface::CreatePluginObject(
     llvm::StringRef class_name, ExecutionContext &exe_ctx,
-    StructuredData::DictionarySP args_sp) {
+    StructuredData::DictionarySP args_sp, StructuredData::Generic *script_obj) {
   if (class_name.empty())
     return {};
 
   TargetSP target_sp = exe_ctx.GetTargetSP();
   StructuredDataImpl args_impl(args_sp);
   std::string error_string;
 
   Locker py_lock(&m_interpreter, Locker::AcquireLock | Locker::NoSTDIN,
                  Locker::FreeLock);
 
   PythonObject ret_val = LLDBSwigPythonCreateScriptedProcess(
       class_name.str().c_str(), m_interpreter.GetDictionaryName(), target_sp,
       args_impl, error_string);
 
-  if (!ret_val)
-    return {};
-
   m_object_instance_sp =
       StructuredData::GenericSP(new StructuredPythonObject(std::move(ret_val)));
 
   return m_object_instance_sp;
 }
 
 Status ScriptedProcessPythonInterface::Launch() {
   return GetStatusFromMethod("launch");

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.h

 namespace lldb_private {
 class ScriptedThreadPythonInterface : public ScriptedThreadInterface,
                                       public ScriptedPythonInterface {
 public:
   ScriptedThreadPythonInterface(ScriptInterpreterPythonImpl &interpreter);
 
   StructuredData::GenericSP
   CreatePluginObject(llvm::StringRef class_name, ExecutionContext &exe_ctx,
-                     StructuredData::DictionarySP args_sp) override;
+                     StructuredData::DictionarySP args_sp,
+                     StructuredData::Generic *script_obj = nullptr) override;
 
   lldb::tid_t GetThreadID() override;
 
   llvm::Optional<std::string> GetName() override;
 
   lldb::StateType GetState() override;
 
   llvm::Optional<std::string> GetQueue() override;

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp

 using namespace lldb_private::python;
 using Locker = ScriptInterpreterPythonImpl::Locker;
 
 ScriptedThreadPythonInterface::ScriptedThreadPythonInterface(
     ScriptInterpreterPythonImpl &interpreter)
     : ScriptedThreadInterface(), ScriptedPythonInterface(interpreter) {}
 
 StructuredData::GenericSP ScriptedThreadPythonInterface::CreatePluginObject(
     const llvm::StringRef class_name, ExecutionContext &exe_ctx,
-    StructuredData::DictionarySP args_sp) {
-
+    StructuredData::DictionarySP args_sp, StructuredData::Generic *script_obj) {
   if (class_name.empty())
     return {};
 
   ProcessSP process_sp = exe_ctx.GetProcessSP();
   StructuredDataImpl args_impl(args_sp);
   std::string error_string;
 
   Locker py_lock(&m_interpreter, Locker::AcquireLock | Locker::NoSTDIN,
                  Locker::FreeLock);
 
-  PythonObject ret_val = LLDBSwigPythonCreateScriptedThread(
+  PythonObject ret_val;
+
+  if (!script_obj)
+    ret_val = LLDBSwigPythonCreateScriptedThread(
         class_name.str().c_str(), m_interpreter.GetDictionaryName(), process_sp,
         args_impl, error_string);
+  else
+    ret_val = PythonObject(PyRefType::Borrowed,
+                           static_cast<PyObject *>(script_obj->GetValue()));
 
   if (!ret_val)
     return {};
 
   m_object_instance_sp =
       StructuredData::GenericSP(new StructuredPythonObject(std::move(ret_val)));
 
   return m_object_instance_sp;
 }
 
 lldb::tid_t ScriptedThreadPythonInterface::GetThreadID() {
   Status error;
   StructuredData::ObjectSP obj = Dispatch("get_thread_id", error);
 

lldb/test/API/functionalities/scripted_process/Makefile

-C_SOURCES := main.c
-
+CXX_SOURCES := main.cpp
+ENABLE_THREADS := YES
 include Makefile.rules
 

lldb/test/API/functionalities/scripted_process/TestScriptedProcess.py

def test_scripted_process_and_scripted_thread(self):
 
         self.assertTrue(GPRs, "Invalid General Purpose Registers Set")
         self.assertEqual(GPRs.GetNumChildren(), 21)
         for idx, reg in enumerate(GPRs, start=1):
             self.assertEqual(idx, int(reg.value, 16))
 
     def create_stack_skinny_corefile(self, file):
         self.build()
-        target, process, thread, _ = lldbutil.run_to_source_breakpoint(self, "// break here", lldb.SBFileSpec("main.c"))
+        target, process, thread, _ = lldbutil.run_to_source_breakpoint(self, "// break here",
+                                                                       lldb.SBFileSpec("main.cpp"))
         self.assertTrue(process.IsValid(), "Process is invalid.")
         # FIXME: Use SBAPI to save the process corefile.
         self.runCmd("process save-core -s stack  " + file)
         self.assertTrue(os.path.exists(file), "No stack-only corefile found.")
         self.assertTrue(self.dbg.DeleteTarget(target), "Couldn't delete target")
 
     @skipUnlessDarwin
     @skipIfOutOfTreeDebugserver
def test_launch_scripted_process_stack_frames(self):
         launch_info.SetScriptedProcessDictionary(structured_data)
 
         error = lldb.SBError()
         process = target.Launch(launch_info, error)
         self.assertTrue(error.Success(), error.GetCString())
         self.assertTrue(process, PROCESS_IS_VALID)
         self.assertEqual(process.GetProcessID(), 42)
 
-        self.assertEqual(process.GetNumThreads(), 1)
+        self.assertEqual(process.GetNumThreads(), 3)
         thread = process.GetSelectedThread()
         self.assertTrue(thread, "Invalid thread.")
-        self.assertEqual(thread.GetName(), "StackCoreScriptedThread.thread-1")
+        self.assertEqual(thread.GetName(), "StackCoreScriptedThread.thread-0")
 
-        self.assertEqual(thread.GetNumFrames(), 3)
+        self.assertEqual(thread.GetNumFrames(), 2)
         frame = thread.GetSelectedFrame()
         self.assertTrue(frame, "Invalid frame.")
-        self.assertEqual(frame.GetFunctionName(), "bar")
-        self.assertEqual(int(frame.FindValue("i", lldb.eValueTypeVariableArgument).GetValue()), 42)
-        self.assertEqual(int(frame.FindValue("j", lldb.eValueTypeVariableLocal).GetValue()), 42 * 42)
+        # self.assertEqual(frame.GetFunctionName(), "bar")
+        # self.assertEqual(int(frame.FindValue("i", lldb.eValueTypeVariableArgument).GetValue()), 42)
+        # self.assertEqual(int(frame.FindValue("j", lldb.eValueTypeVariableLocal).GetValue()), 42 * 42)

lldb/test/API/functionalities/scripted_process/invalid_scripted_process.py

 import os,struct, signal
 
 from typing import Any, Dict
 
 import lldb
 from lldb.plugins.scripted_process import ScriptedProcess
 from lldb.plugins.scripted_process import ScriptedThread
 
 class InvalidScriptedProcess(ScriptedProcess):
     def __init__(self, target: lldb.SBTarget, args : lldb.SBStructuredData):
         super().__init__(target, args)
+        self.threads[0] = InvalidScriptedThread(self, None)
 
     def get_memory_region_containing_address(self, addr: int) -> lldb.SBMemoryRegionInfo:
         return None
 
     def get_thread_with_id(self, tid: int):
         return {}
 
     def get_registers_for_thread(self, tid: int):

 
 def __lldb_init_module(debugger, dict):
     if not 'SKIP_SCRIPTED_PROCESS_LAUNCH' in os.environ:
         debugger.HandleCommand(
             "process launch -C %s.%s" % (__name__,
                                      InvalidScriptedProcess.__name__))
     else:
         print("Name of the class that will manage the scripted process: '%s.%s'"
-                % (__name__, InvalidScriptedProcess.__name__))
+                % (__name__, InvalidScriptedProcess.__name__))
 

lldb/test/API/functionalities/scripted_process/main.c

-int bar(int i) {
-  int j = i * i;
-  return j; // break here
-}
-
-int foo(int i) { return bar(i); }
-
-int main() { return foo(42); }

lldb/test/API/functionalities/scripted_process/main.cpp

+#include <iostream>
+#include <mutex>
+#include <thread>
+
+int bar(int i) {
+  int j = i * i;
+  return j; // break here
+}
+
+int foo(int i) { return bar(i); }
+
+void call_and_wait(int &n) {
+  std::cout << "waiting for computation!" << std::endl;
+  while (n != 42 * 42)
+    ;
+  std::cout << "finished computation!" << std::endl;
+}
+
+void compute_pow(int &n) { n = foo(n); }
+
+int main() {
+  int n = 42;
+  std::mutex mutex;
+  std::unique_lock<std::mutex> lock(mutex);
+
+  std::thread thread_1(call_and_wait, std::ref(n));
+  std::thread thread_2(compute_pow, std::ref(n));
+  lock.unlock();
+
+  thread_1.join();
+  thread_2.join();
+
+  return 0;
+}

lldb/test/API/functionalities/scripted_process/stack_core_scripted_process.py

-import os,struct,signal
+import os,json,struct,signal
 
 from typing import Any, Dict
 
 import lldb
 from lldb.plugins.scripted_process import ScriptedProcess
 from lldb.plugins.scripted_process import ScriptedThread
 
 class StackCoreScriptedProcess(ScriptedProcess):
     def __init__(self, target: lldb.SBTarget, args : lldb.SBStructuredData):
         super().__init__(target, args)
 
         self.backing_target_idx = args.GetValueForKey("backing_target_idx")
 
         self.corefile_target = None
         self.corefile_process = None
         if (self.backing_target_idx and self.backing_target_idx.IsValid()):
             if self.backing_target_idx.GetType() == lldb.eStructuredDataTypeInteger:
                 idx = self.backing_target_idx.GetIntegerValue(42)
             if self.backing_target_idx.GetType() == lldb.eStructuredDataTypeString:
                 idx = int(self.backing_target_idx.GetStringValue(100))
             self.corefile_target = target.GetDebugger().GetTargetAtIndex(idx)
             self.corefile_process = self.corefile_target.GetProcess()
+            for corefile_thread in self.corefile_process:
+                structured_data = lldb.SBStructuredData()
+                structured_data.SetFromJSON(json.dumps({
+                    "backing_target_idx" : idx,
+                    "thread_idx" : corefile_thread.GetIndexID()
+                }))
+
+                self.threads[corefile_thread.GetThreadID()] = StackCoreScriptedThread(self, structured_data)
 
     def get_memory_region_containing_address(self, addr: int) -> lldb.SBMemoryRegionInfo:
         mem_region = lldb.SBMemoryRegionInfo()
         error = self.corefile_process.GetMemoryRegionInfo(addr, mem_region)
         if error.Fail():
             return None
         return mem_region
 
class StackCoreScriptedProcess(ScriptedProcess):
 
     def get_scripted_thread_plugin(self):
         return StackCoreScriptedThread.__module__ + "." + StackCoreScriptedThread.__name__
 
 
 class StackCoreScriptedThread(ScriptedThread):
     def __init__(self, process, args):
         super().__init__(process, args)
-        self.backing_target_idx = args.GetValueForKey("backing_target_idx")
+        backing_target_idx = args.GetValueForKey("backing_target_idx")
+        thread_idx = args.GetValueForKey("thread_idx")
+
+        def extract_value_from_structured_data(data, default_val):
+            if data and data.IsValid():
+                if data.GetType() == lldb.eStructuredDataTypeInteger:
+                    return data.GetIntegerValue(default_val)
+                if data.GetType() == lldb.eStructuredDataTypeString:
+                    return int(data.GetStringValue(100))
+            return None
+
+        #TODO: Change to Walrus operator (:=) with oneline if assignment
+        # Requires python 3.8
+        val = extract_value_from_structured_data(thread_idx, 0)
+        if val is not None:
+            self.idx = val
 
         self.corefile_target = None
         self.corefile_process = None
-        if (self.backing_target_idx and self.backing_target_idx.IsValid()):
-            if self.backing_target_idx.GetType() == lldb.eStructuredDataTypeInteger:
-                idx = self.backing_target_idx.GetIntegerValue(42)
-            if self.backing_target_idx.GetType() == lldb.eStructuredDataTypeString:
-                idx = int(self.backing_target_idx.GetStringValue(100))
-            self.corefile_target = self.target.GetDebugger().GetTargetAtIndex(idx)
+        self.corefile_thread = None
+
+        #TODO: Change to Walrus operator (:=) with oneline if assignment
+        # Requires python 3.8
+        val = extract_value_from_structured_data(backing_target_idx, 42)
+        if val is not None:
+            self.corefile_target = self.target.GetDebugger().GetTargetAtIndex(val)
             self.corefile_process = self.corefile_target.GetProcess()
+            self.corefile_thread = self.corefile_process.GetThreadByIndexID(self.idx)
+
+        if self.corefile_thread:
+            self.id = self.corefile_thread.GetThreadID()
 
     def get_thread_id(self) -> int:
-        return 0x19
+        return self.id
 
     def get_name(self) -> str:
-        return StackCoreScriptedThread.__name__ + ".thread-1"
+        return StackCoreScriptedThread.__name__ + ".thread-" + str(self.id)
 
     def get_stop_reason(self) -> Dict[str, Any]:
         return { "type": lldb.eStopReasonSignal, "data": {
             "signal": signal.SIGINT
         } }
 
     def get_stackframes(self):
         class ScriptedStackFrame:
             def __init__(idx, cfa, pc, symbol_ctx):
                 self.idx = idx
                 self.cfa = cfa
                 self.pc = pc
                 self.symbol_ctx = symbol_ctx
 
 
         symbol_ctx = lldb.SBSymbolContext()
         frame_zero = ScriptedStackFrame(0, 0x42424242, 0x5000000, symbol_ctx)
         self.frames.append(frame_zero)
 
         return self.frame_zero[0:0]
 
     def get_register_context(self) -> str:
-        thread = self.corefile_process.GetSelectedThread()
-        if not thread or thread.GetNumFrames() == 0:
+        if not self.corefile_thread or self.corefile_thread.GetNumFrames() == 0:
             return None
-        frame = thread.GetFrameAtIndex(0)
+        frame = self.corefile_thread.GetFrameAtIndex(0)
 
         GPRs = None
         registerSet = frame.registers # Returns an SBValueList.
         for regs in registerSet:
             if 'general purpose' in regs.name.lower():
                 GPRs = regs
                 break