[lldb/Plugins] Improve error reporting when reading memory in Scripted Process

[mib]

Summary

This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python implementation.

This will allow, for instance, the Scripted Process python implementation
to report errors, when reading memory, back to lldb.

To do so, the Dispatch method will transform the private types in the
parameter pack into PythonObjects to be able to pass them down to the
python methods.

Then, if the call succeeded, the transformed arguments will be converted
back to their original type and re-assigned in the parameter pack, to
ensure pointers and references behaviours are preserved.

This patch also updates various scripted process python class and tests
to reflect this change.

rdar://100030995

Signed-off-by: Med Ismail Bennani <[e-mail]>

[mib]

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp

152

@labath In order to pass down the Status& to python, I create a StatusSP (to avoid leaking the lldb_private::Status type to python), and turn that into a python::PyObject by calling ToSWIGWrapper. This is why I moved its declaration to SWIGPythonBridge.h.

Finally, the python::PyObject is wrapped into another PythonObject* so it can be passed to GetPythonValueFormatString and PyObject_CallMethod in ScriptedPythonInterface::Dispatch

I tried to follow the logic explained in 7f09ab0, however, when ToSWIGWrapper is called at runtime, it crashes in Python:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
    frame #0: 0x00000001056f3c3c Python`PyTuple_New + 212
    frame #1: 0x0000000135524720 liblldb.16.0.0git.dylib`SWIG_Python_NewShadowInstance(data=0x00006000011d6340, swig_this=0x0000000105ec5d70) at LLDBWrapPython.cpp:2284:28
  * frame #2: 0x0000000135515a00 liblldb.16.0.0git.dylib`SWIG_Python_NewPointerObj(self=0x0000000000000000, ptr=0x0000600001dc4040, type=0x000000014448c140, flags=1) at LLDBWrapPython.cpp:2395:22
    frame #3: 0x00000001355157f4 liblldb.16.0.0git.dylib`lldb_private::python::ToSWIGHelper(obj=0x0000600001dc4040, info=0x000000014448c140) at python-swigsafecast.swig:5:29
    frame #4: 0x0000000135515e14 liblldb.16.0.0git.dylib`lldb_private::python::ToSWIGWrapper(status_sp=std::__1::shared_ptr<lldb_private::Status>::element_type @ 0x0000600000ac9a18 strong=3 weak=1) at python-swigsafecast.swig:37:10
    frame #5: 0x00000001367383c4 liblldb.16.0.0git.dylib`lldb_private::ScriptedProcessPythonInterface::ReadMemoryAtAddress(this=0x00006000011cd2c0, address=8034160640, size=512, error=0x000000016b35c650) at ScriptedProcessPythonInterface.cpp:161:45

Am I doing something wrong or maybe I'm missing something ?

[labath]

lldb/include/lldb/API/SBError.h

99

This is technically an ABI break (changes sizeof(SBError)). I don't care, but someone might.

lldb/source/Plugins/ScriptInterpreter/Python/SWIGPythonBridge.cpp

27 ↗

(On Diff #460711)

Yes, it should.

lldb/source/Plugins/ScriptInterpreter/Python/SWIGPythonBridge.h

35

we don't indent namespaces

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp

152

Well.. I'm not sure if this is the cause of the crash, but one problem I see is the passing of PythonObject* to the Dispatch function, which forwards it unmodified to the PyObject_CallMethod function. Python clearly does not know anything about our lldb_private::python::PythonObject wrappers. You'll either need to pass a PyObject here, or teach Dispatch how to unwrap a PythonObject.

I also don't think that it was necessary to move all of this code around. I don't see why would Status be any different from say StructuredDataImpl, which is already passed as a plain reference (no smart pointers). Using an lldb-private object inside python-swigsafecast.swig is still ok, as that code is still part of liblldb. We already include some internal headers from the swig files, but I think that in this case a forward declaration should be enough as well (you may need to add a SBError(const Status &) constructor). The opposite (moving this code to "regular" files) is not ideal as well, as you've needed to add a bunch of SB forward declarations into SWIGPythonBridge, and split the implementation of ToSWIGWrapper into two files.

I also don't think the shared_ptr transition is helping with anything (and it is an ABI break). It might be necessary if the problem was that the SBError object was being copied, and you couldn't get a hold of the copy which the user code has modified, but in that case you'd also have to modify the objects copy constructors to do a shallow (instead of a deep) copy (which in turn could break other code which was relying on this).

[mib]

Re-did the whole patch (taking into account @labath's previous comments)

It makes use of:

• Generic Variadic Lambdas
• Fold Expressions
• Index Sequences
• Template Partial Specializations
• Variadic template parameters and tuples

...

[labath]

I kinda like it. One thing that I think would help with the readability is if the "transformation" methods were grouped according to the type of the object being transformed, rather than according to the direction. So something like:

// general transform
// general reverse
// Status transform
// Status reverse

instead of

// general transform
// Status transform
// general reverse
// Status reverse

Also I don't think that these structs (transformation, reverse_transformation) wrapping the transformation functions are really necessary. It's true that one cannot partially specialize functions, but one of the reasons for that is this is normally not necessary -- regular function overloading can do most of that as well.

lldb/bindings/python/python-swigsafecast.swig

36

add const ?

lldb/source/API/SBError.cpp

29

That's cute, but you can just call the copy constructor (new Status(status))

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp

125

or assign it to the error argument instead?

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h

11–22

move this inside #if LLDB_ENABLE_PYTHON

103–132

Can this be a call to PythonObject::CallMethod (via std::apply and everything). If not, why not?

184

PythonObject::CallMethod knows how to unwrap a PythonObject argument, so ideally, we'd return py_obj directly here.

[mib]

Address @labath comments.

[mib]

In D134033#3933936, @labath wrote:

I kinda like it. One thing that I think would help with the readability is if the "transformation" methods were grouped according to the type of the object being transformed, rather than according to the direction. So something like:

// general transform
// general reverse
// Status transform
// Status reverse

instead of

// general transform
// Status transform
// general reverse
// Status reverse

Also I don't think that these structs (transformation, reverse_transformation) wrapping the transformation functions are really necessary. It's true that one cannot partially specialize functions, but one of the reasons for that is this is normally not necessary -- regular function overloading can do most of that as well.

Thanks for the feedback! I really appreciate it :) For now I reordered the structs as you suggested but I'm thinking of moving all the transformation related code to an anonymous namespace above the class. I'll do that in a follow-up patch :)

lldb/include/lldb/API/SBError.h

99

Luckily, I didn't have to change this.

[mib]

Fix failure when building PythonScriptInterpreter unit test libraries

[labath]

Cool. Thanks.

In D134033#3935424, @mib wrote:

In D134033#3933936, @labath wrote:

I kinda like it. One thing that I think would help with the readability is if the "transformation" methods were grouped according to the type of the object being transformed, rather than according to the direction. So something like:

// general transform
// general reverse
// Status transform
// Status reverse

instead of

// general transform
// Status transform
// general reverse
// Status reverse

Also I don't think that these structs (transformation, reverse_transformation) wrapping the transformation functions are really necessary. It's true that one cannot partially specialize functions, but one of the reasons for that is this is normally not necessary -- regular function overloading can do most of that as well.

Thanks for the feedback! I really appreciate it :) For now I reordered the structs as you suggested but I'm thinking of moving all the transformation related code to an anonymous namespace above the class. I'll do that in a follow-up patch :)

You can't have an anonymous namespace in a header (well.. you can, but it will break in very interesting ways). You can put it inside some detail or internal namespace, if you think it helps.

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h

167–168

this could be a static_assert

[mgorny]

LLDB no longer builds for me after this change (could be related to swig 4.1.0):

[1/5] Building CXX object tools/lldb/source/Plugins/ScriptInterpre...ldbPluginScriptInterpreterPython.dir/ScriptedPythonInterface.cpp.o
FAILED: tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedPythonInterface.cpp.o 
CCACHE_CPP2=yes CCACHE_HASHDIR=yes /usr/bin/ccache /usr/bin/c++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLLDB_PYTHON_EXE_RELATIVE_PATH=\"bin/python3.10\" -DLLDB_PYTHON_RELATIVE_LIBDIR=\"lib/python3.10/site-packages\" -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/mgorny/git/llvm-project/build/tools/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/include -I/home/mgorny/git/llvm-project/build/tools/lldb/include -I/home/mgorny/git/llvm-project/build/include -I/home/mgorny/git/llvm-project/llvm/include -I/usr/include/python3.10 -I/home/mgorny/git/llvm-project/llvm/../clang/include -I/home/mgorny/git/llvm-project/build/tools/lldb/../clang/include -I/home/mgorny/git/llvm-project/lldb/source -I/home/mgorny/git/llvm-project/build/tools/lldb/source -isystem /usr/include/libxml2 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-stringop-truncation -Os -DNDEBUG  -fno-exceptions -fno-rtti -UNDEBUG -std=c++17 -MD -MT tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedPythonInterface.cpp.o -MF tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedPythonInterface.cpp.o.d -o tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedPythonInterface.cpp.o -c /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.cpp
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.cpp:20:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  113 |   template <> struct transformation<Status> {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:22: error: too few template-parameter-lists
  113 |   template <> struct transformation<Status> {
      |                      ^~~~~~~~~~~~~~~~~~~~~~
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:129:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  129 |   template <> typename transformation<Status>::type Transform(Status arg) {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h: In member function ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type lldb_private::ScriptedPythonInterface::Transform(lldb_private::Status)’:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:131:33: error: could not convert ‘lldb_private::python::ToSWIGWrapper(const lldb_private::Status&)()’ from ‘lldb_private::python::PythonObject’ to ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type’ {aka ‘lldb_private::Status’}
  131 |     return python::ToSWIGWrapper(arg);
      |            ~~~~~~~~~~~~~~~~~~~~~^~~~~
      |                                 |
      |                                 lldb_private::python::PythonObject
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.cpp:16:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h: At global scope:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h:16:30: warning: ‘g_fcxx_modules_workaround’ defined but not used [-Wunused-variable]
   16 | static llvm::Expected<bool> *g_fcxx_modules_workaround;
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~
[2/5] Building CXX object tools/lldb/source/Plugins/ScriptInterpre...ginScriptInterpreterPython.dir/ScriptedThreadPythonInterface.cpp.o
FAILED: tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedThreadPythonInterface.cpp.o 
CCACHE_CPP2=yes CCACHE_HASHDIR=yes /usr/bin/ccache /usr/bin/c++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLLDB_PYTHON_EXE_RELATIVE_PATH=\"bin/python3.10\" -DLLDB_PYTHON_RELATIVE_LIBDIR=\"lib/python3.10/site-packages\" -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/mgorny/git/llvm-project/build/tools/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/include -I/home/mgorny/git/llvm-project/build/tools/lldb/include -I/home/mgorny/git/llvm-project/build/include -I/home/mgorny/git/llvm-project/llvm/include -I/usr/include/python3.10 -I/home/mgorny/git/llvm-project/llvm/../clang/include -I/home/mgorny/git/llvm-project/build/tools/lldb/../clang/include -I/home/mgorny/git/llvm-project/lldb/source -I/home/mgorny/git/llvm-project/build/tools/lldb/source -isystem /usr/include/libxml2 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-stringop-truncation -Os -DNDEBUG  -fno-exceptions -fno-rtti -UNDEBUG -std=c++17 -MD -MT tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedThreadPythonInterface.cpp.o -MF tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedThreadPythonInterface.cpp.o.d -o tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedThreadPythonInterface.cpp.o -c /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.h:16,
                 from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp:20:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  113 |   template <> struct transformation<Status> {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:22: error: too few template-parameter-lists
  113 |   template <> struct transformation<Status> {
      |                      ^~~~~~~~~~~~~~~~~~~~~~
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:129:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  129 |   template <> typename transformation<Status>::type Transform(Status arg) {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h: In member function ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type lldb_private::ScriptedPythonInterface::Transform(lldb_private::Status)’:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:131:33: error: could not convert ‘lldb_private::python::ToSWIGWrapper(const lldb_private::Status&)()’ from ‘lldb_private::python::PythonObject’ to ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type’ {aka ‘lldb_private::Status’}
  131 |     return python::ToSWIGWrapper(arg);
      |            ~~~~~~~~~~~~~~~~~~~~~^~~~~
      |                                 |
      |                                 lldb_private::python::PythonObject
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedThreadPythonInterface.cpp:16:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h: At global scope:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h:16:30: warning: ‘g_fcxx_modules_workaround’ defined but not used [-Wunused-variable]
   16 | static llvm::Expected<bool> *g_fcxx_modules_workaround;
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~
[3/5] Building CXX object tools/lldb/source/Plugins/ScriptInterpre...inScriptInterpreterPython.dir/ScriptedProcessPythonInterface.cpp.o
FAILED: tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedProcessPythonInterface.cpp.o 
CCACHE_CPP2=yes CCACHE_HASHDIR=yes /usr/bin/ccache /usr/bin/c++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLLDB_PYTHON_EXE_RELATIVE_PATH=\"bin/python3.10\" -DLLDB_PYTHON_RELATIVE_LIBDIR=\"lib/python3.10/site-packages\" -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/mgorny/git/llvm-project/build/tools/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/include -I/home/mgorny/git/llvm-project/build/tools/lldb/include -I/home/mgorny/git/llvm-project/build/include -I/home/mgorny/git/llvm-project/llvm/include -I/usr/include/python3.10 -I/home/mgorny/git/llvm-project/llvm/../clang/include -I/home/mgorny/git/llvm-project/build/tools/lldb/../clang/include -I/home/mgorny/git/llvm-project/lldb/source -I/home/mgorny/git/llvm-project/build/tools/lldb/source -isystem /usr/include/libxml2 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-stringop-truncation -Os -DNDEBUG  -fno-exceptions -fno-rtti -UNDEBUG -std=c++17 -MD -MT tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedProcessPythonInterface.cpp.o -MF tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedProcessPythonInterface.cpp.o.d -o tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptedProcessPythonInterface.cpp.o -c /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.h:16,
                 from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp:21:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  113 |   template <> struct transformation<Status> {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:22: error: too few template-parameter-lists
  113 |   template <> struct transformation<Status> {
      |                      ^~~~~~~~~~~~~~~~~~~~~~
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:129:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  129 |   template <> typename transformation<Status>::type Transform(Status arg) {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h: In member function ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type lldb_private::ScriptedPythonInterface::Transform(lldb_private::Status)’:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:131:33: error: could not convert ‘lldb_private::python::ToSWIGWrapper(const lldb_private::Status&)()’ from ‘lldb_private::python::PythonObject’ to ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type’ {aka ‘lldb_private::Status’}
  131 |     return python::ToSWIGWrapper(arg);
      |            ~~~~~~~~~~~~~~~~~~~~~^~~~~
      |                                 |
      |                                 lldb_private::python::PythonObject
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/SWIGPythonBridge.h:21,
                 from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp:19:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h: In instantiation of ‘llvm::Expected<lldb_private::python::PythonObject> lldb_private::python::PythonObject::CallMethod(const char*, const T& ...) const [with T = {long unsigned int, long unsigned int, lldb_private::Status}]’:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:72:34:   required from ‘lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)> [with auto:32 = {long unsigned int&, long unsigned int&, lldb_private::Status&}]’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/type_traits:2565:26:   required by substitution of ‘template<class _Fn, class ... _Args> static std::__result_of_success<decltype (declval<_Fn>()((declval<_Args>)()...)), std::__invoke_other> std::__result_of_other_impl::_S_test(int) [with _Fn = lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>; _Args = {long unsigned int&, long unsigned int&, lldb_private::Status&}]’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/type_traits:2576:55:   required from ‘struct std::__result_of_impl<false, false, lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, long unsigned int&, long unsigned int&, lldb_private::Status&>’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/type_traits:167:12:   recursively required by substitution of ‘template<class _Result, class _Ret> struct std::__is_invocable_impl<_Result, _Ret, true, std::__void_t<typename _CTp::type> > [with _Result = std::__invoke_result<lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, long unsigned int&, long unsigned int&, lldb_private::Status&>; _Ret = void]’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/type_traits:167:12:   required from ‘struct std::__and_<std::__is_invocable_impl<std::__invoke_result<lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, long unsigned int&, long unsigned int&, lldb_private::Status&>, void, true, void>, std::__call_is_nothrow<std::__invoke_result<lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, long unsigned int&, long unsigned int&, lldb_private::Status&>, lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, long unsigned int&, long unsigned int&, lldb_private::Status&> >’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/type_traits:3064:12:   required from ‘struct std::is_nothrow_invocable<lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, long unsigned int&, long unsigned int&, lldb_private::Status&>’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/tuple:1836:31:   required from ‘constexpr const bool std::__unpack_std_tuple<template<class _Fn, class ... _ArgTypes> struct std::is_nothrow_invocable, lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>, std::tuple<long unsigned int, long unsigned int, lldb_private::Status>&>’
/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12/tuple:1859:14:   required from ‘constexpr decltype(auto) std::apply(_Fn&&, _Tuple&&) [with _Fn = lldb_private::ScriptedPythonInterface::Dispatch<std::shared_ptr<lldb_private::DataExtractor>, long unsigned int&, long unsigned int&, lldb_private::Status&>(llvm::StringRef, lldb_private::Status&, long unsigned int&, long unsigned int&, lldb_private::Status&)::<lambda(auto:32&& ...)>; _Tuple = tuple<long unsigned int, long unsigned int, lldb_private::Status>&]’
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:69:15:   required from ‘T lldb_private::ScriptedPythonInterface::Dispatch(llvm::StringRef, lldb_private::Status&, Args&& ...) [with T = std::shared_ptr<lldb_private::DataExtractor>; Args = {long unsigned int&, long unsigned int&, lldb_private::Status&}]’
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp:125:66:   required from here
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h:317:50: error: incomplete type ‘lldb_private::python::PythonFormat<lldb_private::Status, void>’ used in nested name specifier
  317 |     const char format[] = {'(', PythonFormat<T>::format..., ')', 0};
      |                                                  ^~~~~~
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/PythonDataObjects.h:320:73: error: incomplete type ‘lldb_private::python::PythonFormat<lldb_private::Status, void>’ used in nested name specifier
  320 |                             py2_const_cast(format), PythonFormat<T>::get(t)...);
      |                                                     ~~~~~~~~~~~~~~~~~~~~^~~
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp:17:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h: At global scope:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h:16:30: warning: ‘g_fcxx_modules_workaround’ defined but not used [-Wunused-variable]
   16 | static llvm::Expected<bool> *g_fcxx_modules_workaround;
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~
[4/5] Building CXX object tools/lldb/source/Plugins/ScriptInterpre...ldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o
FAILED: tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o 
CCACHE_CPP2=yes CCACHE_HASHDIR=yes /usr/bin/ccache /usr/bin/c++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLLDB_PYTHON_EXE_RELATIVE_PATH=\"bin/python3.10\" -DLLDB_PYTHON_RELATIVE_LIBDIR=\"lib/python3.10/site-packages\" -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/mgorny/git/llvm-project/build/tools/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python -I/home/mgorny/git/llvm-project/lldb/include -I/home/mgorny/git/llvm-project/build/tools/lldb/include -I/home/mgorny/git/llvm-project/build/include -I/home/mgorny/git/llvm-project/llvm/include -I/usr/include/python3.10 -I/home/mgorny/git/llvm-project/llvm/../clang/include -I/home/mgorny/git/llvm-project/build/tools/lldb/../clang/include -I/home/mgorny/git/llvm-project/lldb/source -I/home/mgorny/git/llvm-project/build/tools/lldb/source -isystem /usr/include/libxml2 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-stringop-truncation -Os -DNDEBUG  -fno-exceptions -fno-rtti -UNDEBUG -std=c++17 -MD -MT tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o -MF tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o.d -o tools/lldb/source/Plugins/ScriptInterpreter/Python/CMakeFiles/lldbPluginScriptInterpreterPython.dir/ScriptInterpreterPython.cpp.o -c /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.h:16,
                 from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:21:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  113 |   template <> struct transformation<Status> {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:113:22: error: too few template-parameter-lists
  113 |   template <> struct transformation<Status> {
      |                      ^~~~~~~~~~~~~~~~~~~~~~
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:129:13: error: explicit specialization in non-namespace scope ‘class lldb_private::ScriptedPythonInterface’
  129 |   template <> typename transformation<Status>::type Transform(Status arg) {
      |             ^
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h: In member function ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type lldb_private::ScriptedPythonInterface::Transform(lldb_private::Status)’:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h:131:33: error: could not convert ‘lldb_private::python::ToSWIGWrapper(const lldb_private::Status&)()’ from ‘lldb_private::python::PythonObject’ to ‘lldb_private::ScriptedPythonInterface::transformation<lldb_private::Status>::type’ {aka ‘lldb_private::Status’}
  131 |     return python::ToSWIGWrapper(arg);
      |            ~~~~~~~~~~~~~~~~~~~~~^~~~~
      |                                 |
      |                                 lldb_private::python::PythonObject
In file included from /home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp:15:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h: At global scope:
/home/mgorny/git/llvm-project/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h:16:30: warning: ‘g_fcxx_modules_workaround’ defined but not used [-Wunused-variable]
   16 | static llvm::Expected<bool> *g_fcxx_modules_workaround;
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: cannot make progress due to previous errors.

[dyung]

This seems to have broken the cross-project-test buildbot as well: https://lab.llvm.org/buildbot/#/builders/217/builds/15302

[mib]

@mgorny @dyung error: too few template-parameter-lists this seems to be a g++ error. Could you try building with clang to confirm my theory ? I'll try to fix the issue but I have no way to reproduce it

[mgorny]

In D134033#3939158, @mib wrote:

@mgorny @dyung error: too few template-parameter-lists this seems to be a g++ error. Could you try building with clang to confirm my theory ? I'll try to fix the issue but I have no way to reproduce it

I'm going to try. Can we revert the change in the meantime?

[mib]

In D134033#3939186, @mgorny wrote:

In D134033#3939158, @mib wrote:

@mgorny @dyung error: too few template-parameter-lists this seems to be a g++ error. Could you try building with clang to confirm my theory ? I'll try to fix the issue but I have no way to reproduce it

I'm going to try. Can we revert the change in the meantime?

I was able to get my hands on a linux box, if I can reproduce the build failure I'll try to fix it otherwise I'll revert my patches.

[mgorny]

I can confirm that it doesn't fail with clang.

[mib]

In D134033#3939219, @mgorny wrote:

I can confirm that it doesn't fail with clang.

I think I have a fix!

[mib]

@mgorny @dyung 41f3e6b74df1 should fix this

[mgorny]

Thanks, it builds fine for me now!

lldb/bindings/python/python-swigsafecast.swig

 namespace lldb_private {
 namespace python {
 
 PythonObject ToSWIGHelper(void *obj, swig_type_info *info) {
   return {PyRefType::Owned, SWIG_NewPointerObj(obj, info, SWIG_POINTER_OWN)};
 }
 
-/// A class that automatically clears an SB object when it goes out of scope.
-/// Use for cases where the SB object points to a temporary/unowned entity.
-template <typename T> class ScopedPythonObject : PythonObject {
-public:
-  ScopedPythonObject(T *sb, swig_type_info *info)
-      : PythonObject(ToSWIGHelper(sb, info)), m_sb(sb) {}
-  ~ScopedPythonObject() {
-    if (m_sb)
-      *m_sb = T();
-  }
-  ScopedPythonObject(ScopedPythonObject &&rhs)
-      : PythonObject(std::move(rhs)), m_sb(std::exchange(rhs.m_sb, nullptr)) {}
-  ScopedPythonObject(const ScopedPythonObject &) = delete;
-  ScopedPythonObject &operator=(const ScopedPythonObject &) = delete;
-  ScopedPythonObject &operator=(ScopedPythonObject &&) = delete;
-
-  const PythonObject &obj() const { return *this; }
-
-private:
-  T *m_sb;
-};
-
 PythonObject ToSWIGWrapper(std::unique_ptr<lldb::SBValue> value_sb) {
   return ToSWIGHelper(value_sb.release(), SWIGTYPE_p_lldb__SBValue);
 }
 
 PythonObject ToSWIGWrapper(lldb::ValueObjectSP value_sp) {
   return ToSWIGWrapper(std::make_unique<lldb::SBValue>(std::move(value_sp)));
 }
 
return ToSWIGHelper(new lldb::SBThreadPlan(std::move(thread_plan_sp)),
                       SWIGTYPE_p_lldb__SBThreadPlan);
 }
 
 PythonObject ToSWIGWrapper(lldb::BreakpointSP breakpoint_sp) {
   return ToSWIGHelper(new lldb::SBBreakpoint(std::move(breakpoint_sp)),
                       SWIGTYPE_p_lldb__SBBreakpoint);
 }
 
+PythonObject ToSWIGWrapper(const Status& status) {
+  return ToSWIGHelper(new lldb::SBError(status), SWIGTYPE_p_lldb__SBError);
+}
+
 PythonObject ToSWIGWrapper(std::unique_ptr<lldb::SBStream> stream_sb) {
   return ToSWIGHelper(stream_sb.release(), SWIGTYPE_p_lldb__SBStream);
 }
 
 PythonObject ToSWIGWrapper(std::unique_ptr<lldb::SBStructuredData> data_sb) {
   return ToSWIGHelper(data_sb.release(), SWIGTYPE_p_lldb__SBStructuredData);
 }
 

lldb/examples/python/scripted_process/crashlog_scripted_process.py

def get_memory_region_containing_address(self, addr: int) -> lldb.SBMemoryRegionInfo:
         return None
 
     def get_thread_with_id(self, tid: int):
         return {}
 
     def get_registers_for_thread(self, tid: int):
         return {}
 
-    def read_memory_at_address(self, addr: int, size: int) -> lldb.SBData:
+    def read_memory_at_address(self, addr: int, size: int, error: lldb.SBError) -> lldb.SBData:
         # NOTE: CrashLogs don't contain any memory.
         return lldb.SBData()
 
     def get_loaded_images(self):
         # TODO: Iterate over corefile_target modules and build a data structure
         # from it.
         return self.loaded_images
 

lldb/examples/python/scripted_process/scripted_process.py

def get_registers_for_thread(self, tid):
         Returns:
             Dict: The register context represented as a dictionary, for the
                 tid thread. None if tid doesn't match any of the scripted
                 process threads.
         """
         pass
 
     @abstractmethod
-    def read_memory_at_address(self, addr, size):
+    def read_memory_at_address(self, addr, size, error):
         """ Get a memory buffer from the scripted process at a certain address,
             of a certain size.
 
         Args:
             addr (int): Address from which we should start reading.
             size (int): Size of the memory to read.
+            error (lldb.SBError): Error object.
 
         Returns:
             lldb.SBData: An `lldb.SBData` buffer with the target byte size and
                 byte order storing the memory read.
         """
         pass
 
     def get_loaded_images(self):

lldb/include/lldb/API/SBError.h

 namespace lldb {
 
 class LLDB_API SBError {
 public:
   SBError();
 
   SBError(const lldb::SBError &rhs);
 
+  SBError(const lldb_private::Status &error);
+
   ~SBError();
 
   const SBError &operator=(const lldb::SBError &rhs);
 
   /// Get the error string as a NULL terminated UTF8 c-string.
   ///
   /// This SBError object owns the returned string and this object must be kept
   /// around long enough to use the returned string.
protected:
 
   const lldb_private::Status &operator*() const;
 
   lldb_private::Status &ref();
 
   void SetError(const lldb_private::Status &lldb_error);
 
 private:
   std::unique_ptr<lldb_private::Status> m_opaque_up;
 
   void CreateIfNeeded();
 };
 
 } // namespace lldb
 
 #endif // LLDB_API_SBERROR_H

lldb/source/API/SBError.cpp

 SBError::SBError() { LLDB_INSTRUMENT_VA(this); }
 
 SBError::SBError(const SBError &rhs) {
   LLDB_INSTRUMENT_VA(this, rhs);
 
   m_opaque_up = clone(rhs.m_opaque_up);
 }
 
+SBError::SBError(const lldb_private::Status &status)
+    : m_opaque_up(new Status(status)) {
+  LLDB_INSTRUMENT_VA(this, status);
+}
+
 SBError::~SBError() = default;
 
 const SBError &SBError::operator=(const SBError &rhs) {
   LLDB_INSTRUMENT_VA(this, rhs);
 
   if (this != &rhs)
     m_opaque_up = clone(rhs.m_opaque_up);
   return *this;

lldb/source/Plugins/ScriptInterpreter/Python/SWIGPythonBridge.h

 // LLDB Python header must be included first
 #include "lldb-python.h"
 
 #include "Plugins/ScriptInterpreter/Python/PythonDataObjects.h"
 #include "lldb/lldb-forward.h"
 #include "lldb/lldb-types.h"
 #include "llvm/Support/Error.h"
 
+namespace lldb {
+class SBEvent;
+class SBCommandReturnObject;
+class SBValue;
+class SBStream;
+class SBStructuredData;
+} // namespace lldb
+
 namespace lldb_private {
+namespace python {
+
+typedef struct swig_type_info swig_type_info;
+
+python::PythonObject ToSWIGHelper(void *obj, swig_type_info *info);
+
+/// A class that automatically clears an SB object when it goes out of scope.
+/// Use for cases where the SB object points to a temporary/unowned entity.
+template <typename T> class ScopedPythonObject : PythonObject {
+public:
+  ScopedPythonObject(T *sb, swig_type_info *info)
+      : PythonObject(ToSWIGHelper(sb, info)), m_sb(sb) {}
+  ~ScopedPythonObject() {
+    if (m_sb)
+      *m_sb = T();
+  }
+  ScopedPythonObject(ScopedPythonObject &&rhs)
+      : PythonObject(std::move(rhs)), m_sb(std::exchange(rhs.m_sb, nullptr)) {}
+  ScopedPythonObject(const ScopedPythonObject &) = delete;
+  ScopedPythonObject &operator=(const ScopedPythonObject &) = delete;
+  ScopedPythonObject &operator=(ScopedPythonObject &&) = delete;
+
+  const PythonObject &obj() const { return *this; }
+
+private:
+  T *m_sb;
+};
+
+PythonObject ToSWIGWrapper(lldb::ValueObjectSP value_sp);
+PythonObject ToSWIGWrapper(lldb::TargetSP target_sp);
+PythonObject ToSWIGWrapper(lldb::ProcessSP process_sp);
+PythonObject ToSWIGWrapper(lldb::ThreadPlanSP thread_plan_sp);
+PythonObject ToSWIGWrapper(lldb::BreakpointSP breakpoint_sp);
+PythonObject ToSWIGWrapper(const Status &status);
+PythonObject ToSWIGWrapper(const StructuredDataImpl &data_impl);
+PythonObject ToSWIGWrapper(lldb::ThreadSP thread_sp);
+PythonObject ToSWIGWrapper(lldb::StackFrameSP frame_sp);
+PythonObject ToSWIGWrapper(lldb::DebuggerSP debugger_sp);
+PythonObject ToSWIGWrapper(lldb::WatchpointSP watchpoint_sp);
+PythonObject ToSWIGWrapper(lldb::BreakpointLocationSP bp_loc_sp);
+PythonObject ToSWIGWrapper(lldb::ExecutionContextRefSP ctx_sp);
+PythonObject ToSWIGWrapper(const TypeSummaryOptions &summary_options);
+PythonObject ToSWIGWrapper(const SymbolContext &sym_ctx);
+
+PythonObject ToSWIGWrapper(std::unique_ptr<lldb::SBValue> value_sb);
+PythonObject ToSWIGWrapper(std::unique_ptr<lldb::SBStream> stream_sb);
+PythonObject ToSWIGWrapper(std::unique_ptr<lldb::SBStructuredData> data_sb);
+
+python::ScopedPythonObject<lldb::SBCommandReturnObject>
+ToSWIGWrapper(CommandReturnObject &cmd_retobj);
+python::ScopedPythonObject<lldb::SBEvent> ToSWIGWrapper(Event *event);
+
+} // namespace python
 
 // GetPythonValueFormatString provides a system independent type safe way to
 // convert a variable's type into a python value format. Python value formats
 // are defined in terms of builtin C types and could change from system to as
 // the underlying typedef for uint* types, size_t, off_t and other values
 // change.
 
 template <typename T> const char *GetPythonValueFormatString(T t);

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedProcessPythonInterface.cpp

 //===-- ScriptedProcessPythonInterface.cpp --------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Host/Config.h"
 #include "lldb/Utility/Log.h"
+#include "lldb/Utility/Status.h"
 #include "lldb/lldb-enumerations.h"
 
 #if LLDB_ENABLE_PYTHON
 
 // LLDB Python header must be included first
 #include "lldb-python.h"
 
 #include "SWIGPythonBridge.h"

 StructuredData::DictionarySP
 ScriptedProcessPythonInterface::GetRegistersForThread(lldb::tid_t tid) {
   // TODO: Implement
   return {};
 }
 
 lldb::DataExtractorSP ScriptedProcessPythonInterface::ReadMemoryAtAddress(
     lldb::addr_t address, size_t size, Status &error) {
-  return Dispatch<lldb::DataExtractorSP>("read_memory_at_address", error,
-                                         address, size);
+  Status py_error;
+  lldb::DataExtractorSP data_sp = Dispatch<lldb::DataExtractorSP>(
+      "read_memory_at_address", py_error, address, size, error);
+
+  // If there was an error on the python call, surface it to the user.
+  if (py_error.Fail())
+    error = py_error;
+
+  return data_sp;
 }
 
 StructuredData::ArraySP ScriptedProcessPythonInterface::GetLoadedImages() {
   Status error;
   StructuredData::ArraySP array =
       Dispatch<StructuredData::ArraySP>("get_loaded_images", error);
 
   if (!array || !array->IsValid() || error.Fail()) {
     return ScriptedInterface::ErrorWithMessage<StructuredData::ArraySP>(
         LLVM_PRETTY_FUNCTION,
         llvm::Twine("Null or invalid object (" +
                     llvm::Twine(error.AsCString()) + llvm::Twine(")."))
             .str(),
         error);
   }
 
   return array;
 }
 
 lldb::pid_t ScriptedProcessPythonInterface::GetProcessID() {
   Status error;
   StructuredData::ObjectSP obj = Dispatch("get_process_id", error);
 
   if (!CheckStructuredDataObject(LLVM_PRETTY_FUNCTION, obj, error))
     return LLDB_INVALID_PROCESS_ID;
 
   return obj->GetIntegerValue(LLDB_INVALID_PROCESS_ID);
 }

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.h

 //===-- ScriptedPythonInterface.h -------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_PLUGINS_SCRIPTINTERPRETER_PYTHON_SCRIPTEDPYTHONINTERFACE_H
 #define LLDB_PLUGINS_SCRIPTINTERPRETER_PYTHON_SCRIPTEDPYTHONINTERFACE_H
 
-#include "lldb/Host/Config.h"
-
 #if LLDB_ENABLE_PYTHON
 
+#include <sstream>
+#include <tuple>
+#include <type_traits>
+#include <utility>
+
+#include "lldb/Host/Config.h"
 #include "lldb/Interpreter/ScriptedInterface.h"
 #include "lldb/Utility/DataBufferHeap.h"
 
 #include "PythonDataObjects.h"
 #include "SWIGPythonBridge.h"
 #include "ScriptInterpreterPythonImpl.h"
 
 namespace lldb_private {
 class ScriptInterpreterPythonImpl;
 class ScriptedPythonInterface : virtual public ScriptedInterface {
 public:
   ScriptedPythonInterface(ScriptInterpreterPythonImpl &interpreter);
   ~ScriptedPythonInterface() override = default;
 
 protected:
   template <typename T = StructuredData::ObjectSP>
   T ExtractValueFromPythonObject(python::PythonObject &p, Status &error) {
     return p.CreateStructuredObject();
   }
 
   template <typename T = StructuredData::ObjectSP, typename... Args>
-  T Dispatch(llvm::StringRef method_name, Status &error, Args... args) {
+  T Dispatch(llvm::StringRef method_name, Status &error, Args &&...args) {
     using namespace python;
     using Locker = ScriptInterpreterPythonImpl::Locker;
 
     std::string caller_signature =
         llvm::Twine(LLVM_PRETTY_FUNCTION + llvm::Twine(" (") +
                     llvm::Twine(method_name) + llvm::Twine(")"))
             .str();
     if (!m_object_instance_sp)
       return ErrorWithMessage<T>(caller_signature, "Python object ill-formed",
                                  error);
 
     Locker py_lock(&m_interpreter, Locker::AcquireLock | Locker::NoSTDIN,
                    Locker::FreeLock);
 
     PythonObject implementor(PyRefType::Borrowed,
                              (PyObject *)m_object_instance_sp->GetValue());
 
     if (!implementor.IsAllocated())
       return ErrorWithMessage<T>(caller_signature,
                                  "Python implementor not allocated.", error);
 
-    PythonObject pmeth(
-        PyRefType::Owned,
-        PyObject_GetAttrString(implementor.get(), method_name.str().c_str()));
+    std::tuple<Args...> original_args = std::forward_as_tuple(args...);
+    auto transformed_args = TransformArgs(original_args);
 
-    if (PyErr_Occurred())
-      PyErr_Clear();
+    llvm::Expected<PythonObject> expected_return_object =
+        llvm::make_error<llvm::StringError>("Not initialized.",
+                                            llvm::inconvertibleErrorCode());
+    std::apply(
+        [&implementor, &method_name, &expected_return_object](auto &&...args) {
+          llvm::consumeError(expected_return_object.takeError());
+          expected_return_object =
+              implementor.CallMethod(method_name.data(), args...);
+        },
+        transformed_args);
 
-    if (!pmeth.IsAllocated())
+    if (llvm::Error e = expected_return_object.takeError()) {
+      error.SetErrorString(llvm::toString(std::move(e)).c_str());
       return ErrorWithMessage<T>(caller_signature,
-                                 "Python method not allocated.", error);
+                                 "Python method could not be called.", error);
+    }
 
-    if (PyCallable_Check(pmeth.get()) == 0) {
-      if (PyErr_Occurred())
-        PyErr_Clear();
-      return ErrorWithMessage<T>(caller_signature,
-                                 "Python method not callable.", error);
-    }
+    PythonObject py_return = expected_return_object.get();
+
+    if (!py_return.IsAllocated())
+      return ErrorWithMessage<T>(caller_signature, "Returned object is null.",
+                                 error);
+
+    // Now that we called the python method with the transformed arguments,
+    // we need to interate again over both the original and transformed
+    // parameter pack, and transform back the parameter that were passed in
+    // the original parameter pack as references or pointers.
+    if (sizeof...(Args) > 0)
+      if (!ReassignPtrsOrRefsArgs(original_args, transformed_args))
+        return ErrorWithMessage<T>(
+            caller_signature,
+            "Couldn't re-assign reference and pointer arguments.", error);
+
+    return ExtractValueFromPythonObject<T>(py_return, error);
 
-    if (PyErr_Occurred())
-      PyErr_Clear();
+  Status GetStatusFromMethod(llvm::StringRef method_name);
 
-    // TODO: make `const char *` when removing support for Python 2.
-    char *format = nullptr;
-    std::string format_buffer;
+  template <typename T> struct transformation { using type = T; };
+  template <typename T, typename U> struct reverse_transformation {
+    static void Apply(ScriptedPythonInterface &obj, T &original_arg,
+                      U transformed_arg, Status &error) {
+      // If U is not a PythonObject, don't touch it!
+      return;
+    }
+  };
 
-    if (sizeof...(Args) > 0) {
-      FormatArgs(format_buffer, args...);
-      // TODO: make `const char *` when removing support for Python 2.
-      format = const_cast<char *>(format_buffer.c_str());
+  template <> struct transformation<Status> {
+    using type = python::PythonObject;
+  };
+  template <typename T> struct reverse_transformation<T, python::PythonObject> {
+    static void Apply(ScriptedPythonInterface &obj, T &original_arg,
+                      python::PythonObject transformed_arg, Status &error) {
+      original_arg =
+          obj.ExtractValueFromPythonObject<T>(transformed_arg, error);
     }
+  };
 
-    // TODO: make `const char *` when removing support for Python 2.
-    PythonObject py_return(
-        PyRefType::Owned,
-        PyObject_CallMethod(implementor.get(),
-                            const_cast<char *>(method_name.data()), format,
-                            args...));
+  template <typename T> typename transformation<T>::type Transform(T object) {
+    // No Transformation for generic usage
+    return {object};
+  }
 
-    if (PyErr_Occurred()) {
-      PyErr_Print();
-      PyErr_Clear();
-      return ErrorWithMessage<T>(caller_signature,
-                                 "Python method could not be called.", error);
-    }
+  template <> typename transformation<Status>::type Transform(Status arg) {
+    // Call SWIG Wrapper function
+    return python::ToSWIGWrapper(arg);
 
-    if (!py_return.IsAllocated())
-      return ErrorWithMessage<T>(caller_signature, "Returned object is null.",
-                                 error);
+  template <std::size_t... I, typename... Args>
+  auto TransformTuple(const std::tuple<Args...> &args,
+                      std::index_sequence<I...>) {
+    return std::make_tuple(Transform(std::get<I>(args))...);
+  }
 
-    return ExtractValueFromPythonObject<T>(py_return, error);
+  // This will iterate over the Dispatch parameter pack and replace in-place
+  // every `lldb_private` argument that has a SB counterpart.
+  template <typename... Args>
+  auto TransformArgs(const std::tuple<Args...> &args) {
+    return TransformTuple(args, std::make_index_sequence<sizeof...(Args)>());
   }
 
-  Status GetStatusFromMethod(llvm::StringRef method_name);
+  template <typename T, typename U>
+  void TransformBack(T &original_arg, U transformed_arg, Status &error) {
+    reverse_transformation<T, U>::Apply(*this, original_arg, transformed_arg,
+                                        error);
+  }
+
+  template <std::size_t... I, typename... Ts, typename... Us>
+  bool ReassignPtrsOrRefsArgs(std::tuple<Ts...> &original_args,
+                              std::tuple<Us...> &transformed_args,
+                              std::index_sequence<I...>) {
+    Status error;
+    (TransformBack(std::get<I>(original_args), std::get<I>(transformed_args),
+                   error),
+     ...);
+    return error.Success();
+  }
+
+  template <typename... Ts, typename... Us>
+  bool ReassignPtrsOrRefsArgs(std::tuple<Ts...> &original_args,
+                              std::tuple<Us...> &transformed_args) {
+    if (sizeof...(Ts) != sizeof...(Us))
+      return false;
+
+    return ReassignPtrsOrRefsArgs(original_args, transformed_args,
+                                  std::make_index_sequence<sizeof...(Ts)>());
+  }
 
   template <typename T, typename... Args>
   void FormatArgs(std::string &fmt, T arg, Args... args) const {
     FormatArgs(fmt, arg);
     FormatArgs(fmt, args...);
   }
 
   template <typename T> void FormatArgs(std::string &fmt, T arg) const {
-    fmt += GetPythonValueFormatString(arg);
+    fmt += python::PythonFormat<T>::format;
   }
 
   void FormatArgs(std::string &fmt) const {}
 
   // The lifetime is managed by the ScriptInterpreter
   ScriptInterpreterPythonImpl &m_interpreter;
 };
 
 template <>
 StructuredData::ArraySP
 ScriptedPythonInterface::ExtractValueFromPythonObject<StructuredData::ArraySP>(

lldb/source/Plugins/ScriptInterpreter/Python/ScriptedPythonInterface.cpp

ScriptedPythonInterface::ExtractValueFromPythonObject<
   return result_dict.CreateStructuredDictionary();
 }
 
 template <>
 Status ScriptedPythonInterface::ExtractValueFromPythonObject<Status>(
     python::PythonObject &p, Status &error) {
   if (lldb::SBError *sb_error = reinterpret_cast<lldb::SBError *>(
           LLDBSWIGPython_CastPyObjectToSBError(p.get())))
-    error = m_interpreter.GetStatusFromSBError(*sb_error);
+    return m_interpreter.GetStatusFromSBError(*sb_error);
   else
     error.SetErrorString("Couldn't cast lldb::SBError to lldb::Status.");
 
-  return error;
+  return {};
 }
 
 template <>
 lldb::DataExtractorSP
 ScriptedPythonInterface::ExtractValueFromPythonObject<lldb::DataExtractorSP>(
     python::PythonObject &p, Status &error) {
   lldb::SBData *sb_data = reinterpret_cast<lldb::SBData *>(
       LLDBSWIGPython_CastPyObjectToSBData(p.get()));

lldb/test/API/functionalities/scripted_process/TestScriptedProcess.py

def test_invalid_scripted_register_context(self):
 
         process = target.Launch(launch_info, error)
 
         self.assertSuccess(error)
         self.assertTrue(process, PROCESS_IS_VALID)
         self.assertEqual(process.GetProcessID(), 666)
         self.assertEqual(process.GetNumThreads(), 0)
 
+        addr = 0x500000000
+        buff = process.ReadMemory(addr, 4, error)
+        self.assertEqual(buff, None)
+        self.assertTrue(error.Fail())
+        self.assertEqual(error.GetCString(), "This is an invalid scripted process!")
+
         with open(log_file, 'r') as f:
             log = f.read()
 
         self.assertIn("Failed to get scripted thread registers data.", log)
 
     @skipUnlessDarwin
     def test_scripted_process_and_scripted_thread(self):
         """Test that we can launch an lldb scripted process using the SBAPI,
def test_scripted_process_and_scripted_thread(self):
         launch_info = lldb.SBLaunchInfo(None)
         launch_info.SetProcessPluginName("ScriptedProcess")
         launch_info.SetScriptedProcessClassName("dummy_scripted_process.DummyScriptedProcess")
 
         error = lldb.SBError()
         process = target.Launch(launch_info, error)
         self.assertTrue(process and process.IsValid(), PROCESS_IS_VALID)
         self.assertEqual(process.GetProcessID(), 42)
-
         self.assertEqual(process.GetNumThreads(), 1)
 
+        addr = 0x500000000
+        message = "Hello, world!"
+        buff = process.ReadCStringFromMemory(addr, len(message) + 1, error)
+        self.assertSuccess(error)
+        self.assertEqual(buff, message)
+
         thread = process.GetSelectedThread()
         self.assertTrue(thread, "Invalid thread.")
         self.assertEqual(thread.GetThreadID(), 0x19)
         self.assertEqual(thread.GetName(), "DummyScriptedThread.thread-1")
         self.assertStopReason(thread.GetStopReason(), lldb.eStopReasonSignal)
 
         self.assertGreater(thread.GetNumFrames(), 0)
 

lldb/test/API/functionalities/scripted_process/dummy_scripted_process.py

def get_memory_region_containing_address(self, addr: int) -> lldb.SBMemoryRegionInfo:
         return None
 
     def get_thread_with_id(self, tid: int):
         return {}
 
     def get_registers_for_thread(self, tid: int):
         return {}
 
-    def read_memory_at_address(self, addr: int, size: int) -> lldb.SBData:
+    def read_memory_at_address(self, addr: int, size: int, error: lldb.SBError) -> lldb.SBData:
         data = lldb.SBData().CreateDataFromCString(
                                     self.target.GetByteOrder(),
                                     self.target.GetCodeByteSize(),
                                     "Hello, world!")
+
         return data
 
     def get_loaded_images(self):
         return self.loaded_images
 
     def get_process_id(self) -> int:
         return 42
 

lldb/test/API/functionalities/scripted_process/invalid_scripted_process.py

def get_memory_region_containing_address(self, addr: int) -> lldb.SBMemoryRegionInfo:
         return None
 
     def get_thread_with_id(self, tid: int):
         return {}
 
     def get_registers_for_thread(self, tid: int):
         return {}
 
-    def read_memory_at_address(self, addr: int, size: int) -> lldb.SBData:
-        return None
+    def read_memory_at_address(self, addr: int, size: int, error: lldb.SBError) -> lldb.SBData:
+        error.SetErrorString("This is an invalid scripted process!")
+        return lldb.SBData()
 
     def get_loaded_images(self):
         return self.loaded_images
 
     def get_process_id(self) -> int:
         return 666
 
     def should_stop(self) -> bool:

lldb/test/API/functionalities/scripted_process/stack_core_scripted_process.py

def get_memory_region_containing_address(self, addr: int) -> lldb.SBMemoryRegionInfo:
         return mem_region
 
     def get_thread_with_id(self, tid: int):
         return {}
 
     def get_registers_for_thread(self, tid: int):
         return {}
 
-    def read_memory_at_address(self, addr: int, size: int) -> lldb.SBData:
+    def read_memory_at_address(self, addr: int, size: int, error: lldb.SBError) -> lldb.SBData:
         data = lldb.SBData()
-        error = lldb.SBError()
         bytes_read = self.corefile_process.ReadMemory(addr, size, error)
 
         if error.Fail():
             return data
 
         data.SetDataWithOwnership(error, bytes_read,
                                   self.corefile_target.GetByteOrder(),
                                   self.corefile_target.GetAddressByteSize())

lldb/unittests/ScriptInterpreter/Python/PythonTestSuite.cpp

python::PythonObject lldb_private::LLDBSwigPythonCreateScriptedStopHook(
   return python::PythonObject();
 }
 
 bool lldb_private::LLDBSwigPythonStopHookCallHandleStop(
     void *implementor, lldb::ExecutionContextRefSP exc_ctx_sp,
     lldb::StreamSP stream) {
   return false;
 }
+
+python::PythonObject lldb_private::python::ToSWIGWrapper(const Status &status) {
+  return python::PythonObject();
+}