[lldb/Lua] Generate Lua Bindings and Make Them Available to the Script Interpreter

[JDevlieghere]

Summary

This patch uses SWIG to generate the Lua bindings for the SB API. It covers most of the API, but some methods require a type map similar to Python.

Discussion on the mailing list: http://lists.llvm.org/pipermail/lldb-dev/2019-December/015812.html

[labath]

lldb/scripts/CMakeLists.txt

42–64

This looks like it could/should be shared between lua and python.

lldb/scripts/lldb_lua.swig

11–171

Can we figure out a way to share this stuff? I guess it could be just put into a common header and %included from the language-specific files..

174

This shouldn't be needed if lua is only ever used from within lldb (mode a).

lldb/source/Plugins/ScriptInterpreter/Lua/CMakeLists.txt

6

I'd like to discuss this bit in more detail, as you've chosen to do things differently than python. While it does seem strange to have python (&lua) code built in the API folder, there is a reason for it -- this stuff depends on the lldb public api. Having it here creates a loop between the API folder and this script interpreter. One of the effects of that will be that it will be impossible to create a unit test binary for testing the lower level lua code (which I think it would be useful, just as equivalent python unit tests are shown themselves to be useful).

Unfortunately, moving this stuff to the API folder does not completely remove the cycle (though it makes it much easier to work around it), because there will still be some API code which you'll need to call from within the script interpreter. Right now, that's just the luaopen_lldb but there will be others in the future.

I'd like to take this opportunity to create a solid story for the separation of responsibilities and layering between these two entities. If we can figure out a good solution here, then the we could also retrofit python to follow the same pattern. They way I would define "good layering" is that if "A is implemented on top of B" then it should be possible to define the purpose of "B" as well as implement it without referencing any concepts which are only defined in "A".

The way I would try to define the relationship of lua/python script interpreters and the API code is that the script interpreter operates on the lower level lldb_private debugger objects -- i.e., it sits "on top of" pretty much everything else *except* the API folder. It implements all of the interesting stuff about interacting with python/lua *except* how to actually represent the lldb_private entities in python. This last bit a customization point, which needs to be provided by the thing which it uses it -- in our case the API folder. This could be done by having funtcions which take the lldb_private objects, wrap them in the appropriate lldb object, and then wrap *that* via swig. And unwrapping could proceed similarly. This way, the python/lua code would never see any lldb::SB*** classes -- it would just see lldb_private classes, and their fully lua wrapped versions.

And the API folder would be left with the job of wrapping lldb_private objects into lua form and back, which isn't totally unreasonable -- the main job of the API folder is to provide a stable interface to lldb, and that is something we want to have in lua too. It does mean that there will have to be some language specific code in API folder, but I currently don't see a way to avoid that (though I'd like to see someone try to do that).

[JDevlieghere]

• Rebase
• Move wrapper to API
• Add test case

[clayborg]

Very cool BTW!

[JDevlieghere]

Rebase

[labath]

This looks fine. Things will start to get interesting once you start wanting to access the current session from inside the lua script (not just creating a new one),,

[mstorsjo]

lldb/CMakeLists.txt

55

This change here (unconditionally including the scripts subdir) made SWIG a hard dependency, even if both lua and python are disabled.

I pushed a commit to fix that, I hope that's ok.

[JDevlieghere]

lldb/CMakeLists.txt

55

Totally, thanks for fixing this!

lldb/CMakeLists.txt

execute_process(
     COMMAND ${PYTHON_EXECUTABLE}
         -c "import distutils.sysconfig; print(distutils.sysconfig.get_python_lib(True, False, ''))"
     OUTPUT_VARIABLE LLDB_PYTHON_DEFAULT_RELATIVE_PATH
     OUTPUT_STRIP_TRAILING_WHITESPACE)
 
   file(TO_CMAKE_PATH ${LLDB_PYTHON_DEFAULT_RELATIVE_PATH} LLDB_PYTHON_DEFAULT_RELATIVE_PATH)
   set(LLDB_PYTHON_RELATIVE_PATH ${LLDB_PYTHON_DEFAULT_RELATIVE_PATH}
     CACHE STRING "Path where Python modules are installed, relative to install prefix")
+endif ()
 
-  add_subdirectory(scripts)
-endif ()
 
 # We need the headers generated by instrinsics_gen before we can compile
 # any source file in LLDB as the imported Clang modules might include
 # some of these generated headers. This approach is copied from Clang's main
 # CMakeLists.txt, so it should kept in sync the code in Clang which was added
 # in llvm-svn 308844.
 if(LLVM_ENABLE_MODULES AND NOT LLDB_BUILT_STANDALONE)
   list(APPEND LLVM_COMMON_DEPENDS intrinsics_gen)

lldb/scripts/CMakeLists.txt

 endif()
 
 if(APPLE)
   set(DARWIN_EXTRAS "-D__APPLE__")
 else()
   set(DARWIN_EXTRAS "")
 endif()
 
+set(SWIG_COMMON_FLAGS
+  -c++
+  -features autodoc
+  -I${LLDB_SOURCE_DIR}/include
+  -I${CMAKE_CURRENT_SOURCE_DIR}
+  -D__STDC_LIMIT_MACROS
+  -D__STDC_CONSTANT_MACROS
+  ${DARWIN_EXTRAS}
+  -outdir ${CMAKE_CURRENT_BINARY_DIR}
+)
+
+
+if (LLDB_ENABLE_PYTHON)
   add_custom_command(
     OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/LLDBWrapPython.cpp
     OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/lldb.py
     DEPENDS ${SWIG_SOURCES}
     DEPENDS ${SWIG_INTERFACES}
     DEPENDS ${SWIG_HEADERS}
-  COMMAND ${SWIG_EXECUTABLE} 
+    COMMAND ${SWIG_EXECUTABLE}
+        ${SWIG_COMMON_FLAGS}
         -c++
         -shadow
         -python
-      -features autodoc
         -threads
-      -I${LLDB_SOURCE_DIR}/include
-      -I${CMAKE_CURRENT_SOURCE_DIR}
-      -D__STDC_LIMIT_MACROS
-      -D__STDC_CONSTANT_MACROS
-      ${DARWIN_EXTRAS}
-      -outdir ${CMAKE_CURRENT_BINARY_DIR}
         -o ${CMAKE_CURRENT_BINARY_DIR}/LLDBWrapPython.cpp
         ${LLDB_SOURCE_DIR}/scripts/lldb.swig
     VERBATIM
     COMMENT "Builds LLDB Python wrapper")
 
   add_custom_target(swig_wrapper ALL DEPENDS
     ${CMAKE_CURRENT_BINARY_DIR}/LLDBWrapPython.cpp
     ${CMAKE_CURRENT_BINARY_DIR}/lldb.py
   )
+endif()
+
+if (LLDB_ENABLE_LUA)
+  add_custom_command(
+    OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/LLDBWrapLua.cpp
+    DEPENDS ${SWIG_SOURCES}
+    DEPENDS ${SWIG_INTERFACES}
+    DEPENDS ${SWIG_HEADERS}
+    COMMAND ${SWIG_EXECUTABLE}
+        ${SWIG_COMMON_FLAGS}
+        -lua
+        -w503
+        -o ${CMAKE_CURRENT_BINARY_DIR}/LLDBWrapLua.cpp
+        ${LLDB_SOURCE_DIR}/scripts/lldb_lua.swig
+    VERBATIM
+    COMMENT "Builds LLDB Lua wrapper")
 
+  add_custom_target(swig_wrapper_lua ALL DEPENDS
+    ${CMAKE_CURRENT_BINARY_DIR}/LLDBWrapLua.cpp
+  )
+endif()

lldb/scripts/lldb_lua.swig

+/*
+   lldb.swig
+
+   This is the input file for SWIG, to create the appropriate C++ wrappers and
+   functions for various scripting languages, to enable them to call the
+   liblldb Script Bridge functions.
+*/
+
+%module lldb
+
+%include "./headers.swig"
+
+%{
+using namespace lldb_private;
+using namespace lldb;
+%}
+
+%include "./interfaces.swig"

lldb/source/API/CMakeLists.txt

 if ( CMAKE_SYSTEM_NAME MATCHES "Windows" )
   add_definitions( -DEXPORT_LIBLLDB )
 endif()
 
 get_property(LLDB_ALL_PLUGINS GLOBAL PROPERTY LLDB_PLUGINS)
 
 if(LLDB_ENABLE_PYTHON)
   get_target_property(lldb_scripts_dir swig_wrapper BINARY_DIR)
   set(lldb_python_wrapper ${lldb_scripts_dir}/LLDBWrapPython.cpp)
 endif()
 
+if(LLDB_ENABLE_LUA)
+  get_target_property(lldb_scripts_dir swig_wrapper_lua BINARY_DIR)
+  set(lldb_lua_wrapper ${lldb_scripts_dir}/LLDBWrapLua.cpp)
+endif()
+
 if(LLDB_BUILD_FRAMEWORK)
   set(option_install_prefix INSTALL_PREFIX ${LLDB_FRAMEWORK_INSTALL_DIR})
   set(option_framework FRAMEWORK)
 endif()
 
 add_lldb_library(liblldb SHARED ${option_framework}
   SBAddress.cpp
   SBAttachInfo.cpp
add_lldb_library(liblldb SHARED ${option_framework}
   SBTypeSynthetic.cpp
   SBValue.cpp
   SBValueList.cpp
   SBVariablesOptions.cpp
   SBWatchpoint.cpp
   SBUnixSignals.cpp
   SystemInitializerFull.cpp
   ${lldb_python_wrapper}
+  ${lldb_lua_wrapper}
 
   LINK_LIBS
     lldbBase
     lldbBreakpoint
     lldbCore
     lldbDataFormatters
     lldbExpression
     lldbHost
if(lldb_python_wrapper)
   endif()
   if (LLVM_COMPILER_IS_GCC_COMPATIBLE AND
       NOT "${CMAKE_SYSTEM_NAME}" MATCHES "Darwin")
     set_property(SOURCE ${lldb_python_wrapper} APPEND_STRING
       PROPERTY COMPILE_FLAGS " -Wno-sequence-point -Wno-cast-qual")
   endif ()
 endif()
 
+if(lldb_lua_wrapper)
+  add_dependencies(liblldb swig_wrapper_lua)
+  target_include_directories(liblldb PRIVATE ${LUA_INCLUDE_DIR})
+
+  if (MSVC)
+    set_property(SOURCE ${lldb_lua_wrapper} APPEND_STRING PROPERTY COMPILE_FLAGS " /W0")
+  else()
+    set_property(SOURCE ${lldb_lua_wrapper} APPEND_STRING PROPERTY COMPILE_FLAGS " -w")
+  endif()
+
+  set_source_files_properties(${lldb_lua_wrapper} PROPERTIES GENERATED ON)
+endif()
+
 set_target_properties(liblldb
   PROPERTIES
   VERSION ${LLDB_VERSION}
 )
 
 if (NOT CMAKE_SYSTEM_NAME MATCHES "Windows")
   if (NOT LLDB_EXPORT_ALL_SYMBOLS)
     # If we're not exporting all symbols, we'll want to explicitly set

lldb/source/Plugins/ScriptInterpreter/Lua/CMakeLists.txt

+find_package(Lua REQUIRED)
+
 add_lldb_library(lldbPluginScriptInterpreterLua PLUGIN
   Lua.cpp
   ScriptInterpreterLua.cpp
 
   LINK_LIBS
     lldbCore
     lldbInterpreter
   )
 
 target_include_directories(lldbPluginScriptInterpreterLua PUBLIC ${LUA_INCLUDE_DIR})
 target_link_libraries(lldbPluginScriptInterpreterLua INTERFACE ${LUA_LIBRARIES})

lldb/source/Plugins/ScriptInterpreter/Lua/Lua.h

 
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Error.h"
 
 #include "lua.hpp"
 
 namespace lldb_private {
 
+extern "C" {
+int luaopen_lldb(lua_State *L);
+}
+
 class Lua {
 public:
   Lua() : m_lua_state(luaL_newstate()) {
     assert(m_lua_state);
     luaL_openlibs(m_lua_state);
+    luaopen_lldb(m_lua_state);
   }
 
   ~Lua() {
     assert(m_lua_state);
     luaL_openlibs(m_lua_state);
   }
 
   llvm::Error Run(llvm::StringRef buffer);
 
 private:
   lua_State *m_lua_state;
 };
 
 } // namespace lldb_private
 
 #endif // liblldb_Lua_h_

lldb/test/Shell/ScriptInterpreter/Lua/bindings.test

+# REQUIRES: lua
+# RUN: cat %s | %lldb --script-language lua 2>&1 | FileCheck %s
+script
+debugger = lldb.SBDebugger.Create()
+print(string.format("debugger is valid: %s", debugger:IsValid()))
+# CHECK: debugger is valid: true

lldb/unittests/ScriptInterpreter/Lua/LuaTests.cpp

 //===-- LuaTests.cpp ------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Plugins/ScriptInterpreter/Lua/Lua.h"
 #include "gtest/gtest.h"
 
 using namespace lldb_private;
 
+extern "C" int luaopen_lldb(lua_State *L) { return 0; }
+
 TEST(LuaTest, RunValid) {
   Lua lua;
   llvm::Error error = lua.Run("foo = 1");
   EXPECT_FALSE(static_cast<bool>(error));
 }
 
 TEST(LuaTest, RunInvalid) {
   Lua lua;
   llvm::Error error = lua.Run("nil = foo");
   EXPECT_TRUE(static_cast<bool>(error));
   EXPECT_EQ(llvm::toString(std::move(error)),
             "[string \"buffer\"]:1: unexpected symbol near 'nil'\n");
 }