[lldb] Rework the Python threading initialization

[teemperor]

Summary

Python 3.11 will remove the functions that initialize the threading support: https://docs.python.org/3.9/c-api/init.html#c.PyEval_InitThreads

The threading support init work is done automatically as part of Py_Initialize since Python 3.7. With Python 3.9
the functions are now also no-ops and marked as deprecated to warn users[1].

This patch reworks the current initialization code so that it hopefully compiles and does the
right thing for all Python versions:

• Python<3.7 we keep the old behaviour.
• Python>3.7 we don't do any calls to the potentially deleted functions and just try to acquire the GIL to preserve the side-effect of the old implementation.

[1] We disabled deprecation warnings globally in LLDB so that's why we didn't see this yet.

[tatyana-krasnukha]

Looks reasonable

lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp

355

I would add return here to make it clearly evident that the function doesn't do anything more in this case.

lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp

 //===-- ScriptInterpreterPython.cpp ---------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Host/Config.h"
 #include "lldb/lldb-enumerations.h"
 
 #if LLDB_ENABLE_PYTHON
 
 // LLDB Python header must be included first
 #include "lldb-python.h"
 
 #include "PythonDataObjects.h"
#if defined(__APPLE__) && PY_MAJOR_VERSION == 2 && PY_MINOR_VERSION == 7
     // the PythonHome to be right, as it's not going to change.
     static char path[] =
         "/System/Library/Frameworks/Python.framework/Versions/2.7";
     Py_SetPythonHome(path);
 #endif
 #endif
   }
 
-  void InitializeThreadsPrivate() {
-// Since Python 3.7 `Py_Initialize` calls `PyEval_InitThreads` inside itself,
-// so there is no way to determine whether the embedded interpreter
-// was already initialized by some external code. `PyEval_ThreadsInitialized`
-// would always return `true` and `PyGILState_Ensure/Release` flow would be
-// executed instead of unlocking GIL with `PyEval_SaveThread`. When
-// an another thread calls `PyGILState_Ensure` it would get stuck in deadlock.
-#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 7) || (PY_MAJOR_VERSION > 3)
-    // The only case we should go further and acquire the GIL: it is unlocked.
+  /// Acquires the GIL if it hasn't already been acquired by someone else.
+  void EnsureGIL() {
+    // If this thread already holds the GIL then there is nothing to initialize
+    // or set up.
     if (PyGILState_Check())
       return;
-#endif
-
-    if (PyEval_ThreadsInitialized()) {
-      Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_SCRIPT));
-
-      m_was_already_initialized = true;
-      m_gil_state = PyGILState_Ensure();
-      LLDB_LOGV(log, "Ensured PyGILState. Previous state = {0}locked\n",
-                m_gil_state == PyGILState_UNLOCKED ? "un" : "");
+    // Acquire the GIL.
+  }
+
+  /// Initializes Python's threading support if needed and acquires the GIL.
+  void InitializeThreadsPrivate() {
+    // With Python 3.9 PyEval_InitThreads does nothing and is marked as
+    // deprecated. With Python 3.11 the function also got removed.
+    // PyEval_InitThreads is always called from Py_Initialize/Py_InitializeEx
+    // since Python 3.7 so we don't need to call it manually.
+
+    // The only thing we need to do for Python>=3.7 is to acquire the GIL as
+    // that's we promised the caller of this function.
+#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 7) || (PY_MAJOR_VERSION > 3)
+    EnsureGIL();
+#else
+    // For Python<3.7 call PyEval_InitThreads if this hasn't already done
+    // before by someone else (as indicated by PyEval_ThreadsInitialized).
+    // Manually try to take the GIL so that we fulfill our contract with the
+    // caller in case threading support has already been initialized.
+    if (PyEval_ThreadsInitialized()) {
+      EnsureGIL();
       return;
     }
 
-    // InitThreads acquires the GIL if it hasn't been called before.
+    // InitThreads acquires the GIL if it hasn't been called before so no
+    // need to manually acquire it.
     PyEval_InitThreads();
+#endif
   }
 
   TerminalState m_stdin_tty_state;
   PyGILState_STATE m_gil_state = PyGILState_UNLOCKED;
   bool m_was_already_initialized = false;
 };
 } // namespace