[lldb/Plugins] Introduce Scripted Platform Plugin

[mib]

Summary

This patch introduces Scripted Platform, a new platform plugin that can
be customized with a python script.

For now this can list processes described in the python script file but
eventually, it will be used to spawn scripted processes and act as an
interface between them.

Signed-off-by: Med Ismail Bennani <[e-mail]>

[mib]

Update ScriptedMetadata header location

[bulbazord]

There's some overlap in implementation between ScriptedPlatform::GetProcessInfo and ScriptedPlatform::FindProcesses. If you don't anticipate these diverging dramatically, it might make sense to extract out common functionality into something like ScriptedPlatform::ExtractProcessInfoFromDict (or something to that effect).

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.cpp

2

This file needs a header.

67–68

This was already checked above on line 60/61.

144

This should be unconditionally true because you set host_os to llvm::Triple::MacOSX right above this.

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.h

2

PlatformPOSIX.h -> ScriptedPlatform.h

[mib]

Address @bulbazord comments

[JDevlieghere]

lldb/source/Plugins/Platform/CMakeLists.txt

9

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.cpp

209

What's your intention by calling this function? In a release build this is still going to crash. Is there a way to initialize the platform with an invalid m_script_object_sp or m_interpreter? If so then we should have proper error handling. If not then these can be regular asserts.

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.h

50–53

This should be a Doxygen comment.

71–75

This is hard to parse, do they really need to be grouped together? I'd add a newline between them like the public functions.

77

Useless comment

81

Missing group opener (@{), but you can just remove it if you do the same for line 76

[mib]

Address @JDevlieghere comments.

[JDevlieghere]

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

25–28 ↗

(On Diff #482413)

I guess this can be inlined now?

[mib]

lldb/source/Plugins/Process/scripted/ScriptedThread.cpp

25–28 ↗

(On Diff #482413)

Right, may be I should even move this to the ScriptedInterface base, so every scripted class can benefit from it.

[labath]

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.cpp

76

Use a shared_ptr directly here. This currently leaks platform object in the error.Fail() case.

262

When I saw the amount of effort put into the error messages in the ParseProcessFunction, I assumed you are going to give those messages to the user somehow, but now I see they are just thrown away. Do you plan to change that? Maybe you could at least log (LLDB_LOG_ERROR) them?

[mib]

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.cpp

262

I wish there was a way to surface these error but the current implementation of the platform plugin doesn't allow it. I guess I'll just log them for now.

[mib]

Log the error message instead of consuming it

[mib]

Add support for process attach to ScriptedPlatform

lldb/examples/python/scripted_process/scripted_platform.py

def __init__(self, args):
         Args:
             args (lldb.SBStructuredData): A Dictionary holding arbitrary
                 key/value pairs used by the scripted platform.
         """
         processes = []
 
     @abstractmethod
     def list_processes(self):
-        """ Get a list of processes that can be ran on the platform.
+        """ Get a list of processes that can be ran on/attached to the platform.
 
         processes = {
             420: {
                     name: a.out,
                     arch: aarch64,
                     pid: 420,
                     parent_pid: 42 (optional),
                     uid: 0 (optional),

lldb/source/Plugins/Platform/CMakeLists.txt

 add_subdirectory(Android)
 add_subdirectory(FreeBSD)
 add_subdirectory(gdb-server)
 add_subdirectory(Linux)
 add_subdirectory(MacOSX)
 add_subdirectory(NetBSD)
 add_subdirectory(OpenBSD)
 add_subdirectory(POSIX)
+add_subdirectory(scripted)
 add_subdirectory(QemuUser)
 add_subdirectory(Windows)

lldb/source/Plugins/Platform/scripted/CMakeLists.txt

+add_lldb_library(lldbPluginScriptedPlatform PLUGIN
+  ScriptedPlatform.cpp
+
+  LINK_LIBS
+    lldbCore
+    lldbTarget
+    lldbUtility
+  )

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.h

+//===-- ScriptedPlatform.h --------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_SOURCE_PLUGINS_SCRIPTED_PLATFORM_H
+#define LLDB_SOURCE_PLUGINS_SCRIPTED_PLATFORM_H
+
+#include "lldb/Interpreter/ScriptedMetadata.h"
+#include "lldb/Target/Platform.h"
+
+namespace lldb_private {
+
+class ScriptedPlatform : public Platform {
+public:
+  ScriptedPlatform(Debugger *debugger,
+                   const ScriptedMetadata *scripted_metadata, Status &error);
+
+  ~ScriptedPlatform() override;
+
+  static lldb::PlatformSP CreateInstance(bool force, const ArchSpec *arch,
+                                         const Debugger *debugger,
+                                         const ScriptedMetadata *metadata);
+
+  static void Initialize();
+
+  static void Terminate();
+
+  static llvm::StringRef GetPluginNameStatic() { return "scripted-platform"; }
+
+  static llvm::StringRef GetDescriptionStatic() {
+    return "Scripted Platform plug-in.";
+  }
+
+  llvm::StringRef GetDescription() override { return GetDescriptionStatic(); }
+
+  llvm::StringRef GetPluginName() override { return GetPluginNameStatic(); }
+
+  std::vector<ArchSpec>
+  GetSupportedArchitectures(const ArchSpec &process_host_arch) override;
+
+  bool IsConnected() const override { return true; }
+
+  lldb::ProcessSP Attach(lldb_private::ProcessAttachInfo &attach_info,
+                         lldb_private::Debugger &debugger,
+                         lldb_private::Target *target, // Can be nullptr, if
+                         // nullptr create a new
+                         // target, else use
+                         // existing one
+                         lldb_private::Status &error) override;
+
+  uint32_t FindProcesses(const ProcessInstanceInfoMatch &match_info,
+                         ProcessInstanceInfoList &proc_infos) override;
+
+  bool GetProcessInfo(lldb::pid_t pid, ProcessInstanceInfo &proc_info) override;
+
+  Status LaunchProcess(ProcessLaunchInfo &launch_info) override;
+
+  Status KillProcess(const lldb::pid_t pid) override;
+
+  void CalculateTrapHandlerSymbolNames() override {}
+
+private:
+  ScriptedPlatform(const ScriptedPlatform &) = delete;
+  const ScriptedPlatform &operator=(const ScriptedPlatform &) = delete;
+
+  void CheckInterpreterAndScriptObject() const;
+  ScriptedPlatformInterface &GetInterface() const;
+  llvm::Expected<ProcessInstanceInfo>
+  ParseProcessInfo(StructuredData::Dictionary &dict, lldb::pid_t pid) const;
+  static bool IsScriptLanguageSupported(lldb::ScriptLanguage language);
+
+  // Member variables.
+  const ScriptedMetadata *m_scripted_metadata = nullptr;
+  lldb_private::ScriptInterpreter *m_interpreter = nullptr;
+  lldb_private::StructuredData::ObjectSP m_script_object_sp = nullptr;
+  //@}
+};
+
+} // namespace lldb_private
+
+#endif // LLDB_SOURCE_PLUGINS_SCRIPTED_PLATFORM_H

lldb/source/Plugins/Platform/scripted/ScriptedPlatform.cpp

+//===-- ScriptedPlatform.cpp ----------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "ScriptedPlatform.h"
+
+#include "lldb/Core/Debugger.h"
+#include "lldb/Core/PluginManager.h"
+#include "lldb/Host/HostInfo.h"
+#include "lldb/Interpreter/ScriptInterpreter.h"
+#include "lldb/Target/ExecutionContext.h"
+#include "lldb/Utility/LLDBLog.h"
+
+using namespace lldb;
+using namespace lldb_private;
+
+LLDB_PLUGIN_DEFINE(ScriptedPlatform)
+
+static uint32_t g_initialize_count = 0;
+
+static constexpr lldb::ScriptLanguage g_supported_script_languages[] = {
+    ScriptLanguage::eScriptLanguagePython,
+};
+
+bool ScriptedPlatform::IsScriptLanguageSupported(
+    lldb::ScriptLanguage language) {
+  llvm::ArrayRef<lldb::ScriptLanguage> supported_languages =
+      llvm::makeArrayRef(g_supported_script_languages);
+
+  return llvm::is_contained(supported_languages, language);
+}
+
+ScriptedPlatformInterface &ScriptedPlatform::GetInterface() const {
+  return m_interpreter->GetScriptedPlatformInterface();
+}
+
+void ScriptedPlatform::CheckInterpreterAndScriptObject() const {
+  lldbassert(m_interpreter && "Invalid Script Interpreter.");
+  lldbassert(m_script_object_sp && "Invalid Script Object.");
+}
+
+lldb::PlatformSP
+ScriptedPlatform::CreateInstance(bool force, const ArchSpec *arch,
+                                 const Debugger *debugger,
+                                 const ScriptedMetadata *metadata) {
+  Log *log = GetLog(LLDBLog::Platform);
+  if (log) {
+    const char *arch_name;
+    if (arch && arch->GetArchitectureName())
+      arch_name = arch->GetArchitectureName();
+    else
+      arch_name = "<null>";
+
+    const char *triple_cstr =
+        arch ? arch->GetTriple().getTriple().c_str() : "<null>";
+
+    LLDB_LOGF(log,
+              "ScriptedPlatform::%s(force=%s, arch={%s,%s}, debugger=%" PRIxPTR
+              ")",
+              __PRETTY_FUNCTION__, force ? "true" : "false", arch_name,
+              triple_cstr, (uintptr_t)debugger);
+  }
+
+  if (!debugger || !IsScriptLanguageSupported(debugger->GetScriptLanguage()))
+    return {};
+
+  if (!metadata)
+    return {};
+
+  Status error;
+  ScriptedPlatform *scripted_platform =
+      new ScriptedPlatform(const_cast<Debugger *>(debugger), metadata, error);
+
+  if (error.Success())
+    return PlatformSP(scripted_platform);
+
+  LLDB_LOGF(log, "ScriptedPlatform::%s() aborting creation of platform",
+            __PRETTY_FUNCTION__);
+
+  return {};
+}
+
+ScriptedPlatform::ScriptedPlatform(Debugger *debugger,
+                                   const ScriptedMetadata *scripted_metadata,
+                                   Status &error)
+    : Platform(false), m_scripted_metadata(scripted_metadata) {
+  if (!debugger) {
+    error.SetErrorStringWithFormat("ScriptedPlatform::%s () - ERROR: %s",
+                                   __FUNCTION__, "Invalid debugger");
+    return;
+  }
+
+  m_interpreter = debugger->GetScriptInterpreter();
+
+  if (!m_interpreter) {
+    error.SetErrorStringWithFormat("ScriptedPlatform::%s () - ERROR: %s",
+                                   __FUNCTION__,
+                                   "Debugger has no Script Interpreter");
+    return;
+  }
+
+  ExecutionContext e;
+
+  StructuredData::GenericSP object_sp = GetInterface().CreatePluginObject(
+      m_scripted_metadata->GetClassName(), e, m_scripted_metadata->GetArgsSP());
+
+  if (!object_sp || !object_sp->IsValid()) {
+    error.SetErrorStringWithFormat("ScriptedPlatform::%s () - ERROR: %s",
+                                   __FUNCTION__,
+                                   "Failed to create valid script object");
+    return;
+  }
+
+  m_hostname = GetHostPlatform()->GetHostname();
+  m_script_object_sp = object_sp;
+}
+
+ScriptedPlatform::~ScriptedPlatform() {}
+
+void ScriptedPlatform::Initialize() {
+  if (g_initialize_count++ == 0) {
+    // NOTE: This should probably be using the driving process platform
+    PluginManager::RegisterPlugin(ScriptedPlatform::GetPluginNameStatic(),
+                                  ScriptedPlatform::GetDescriptionStatic(),
+                                  ScriptedPlatform::CreateInstance);
+  }
+}
+
+void ScriptedPlatform::Terminate() {
+  if (g_initialize_count > 0) {
+    if (--g_initialize_count == 0) {
+      PluginManager::UnregisterPlugin(ScriptedPlatform::CreateInstance);
+    }
+  }
+}
+
+std::vector<ArchSpec>
+ScriptedPlatform::GetSupportedArchitectures(const ArchSpec &process_host_arch) {
+  std::vector<ArchSpec> result;
+  result.push_back(process_host_arch);
+  return result;
+}
+
+lldb::ProcessSP
+ScriptedPlatform::Attach(lldb_private::ProcessAttachInfo &attach_info,
+                         lldb_private::Debugger &debugger,
+                         lldb_private::Target *target, // Can be nullptr, if
+                                                       // nullptr create a new
+                                                       // target, else use
+                                                       // existing one
+                         lldb_private::Status &error) {
+  return nullptr;
+}
+
+llvm::Expected<ProcessInstanceInfo>
+ScriptedPlatform::ParseProcessInfo(StructuredData::Dictionary &dict,
+                                   lldb::pid_t pid) const {
+  if (!dict.HasKey("name"))
+    return llvm::make_error<llvm::StringError>(
+        "No 'arch' key in process info dictionary.",
+        llvm::inconvertibleErrorCode());
+  if (!dict.HasKey("arch"))
+    return llvm::make_error<llvm::StringError>(
+        "No 'arch' key in process info dictionary.",
+        llvm::inconvertibleErrorCode());
+
+  llvm::StringRef result;
+  if (!dict.GetValueForKeyAsString("name", result))
+    return llvm::make_error<llvm::StringError>(
+        "Couldn't extract 'name' key from process info dictionary.",
+        llvm::inconvertibleErrorCode());
+  std::string name = result.str();
+
+  if (!dict.GetValueForKeyAsString("arch", result))
+    return llvm::make_error<llvm::StringError>(
+        "Couldn't extract 'arch' key from process info dictionary.",
+        llvm::inconvertibleErrorCode());
+  const ArchSpec arch(result.data());
+  if (!arch.IsValid())
+    return llvm::make_error<llvm::StringError>(
+        "Invalid 'arch' key in process info dictionary.",
+        llvm::inconvertibleErrorCode());
+
+  ProcessInstanceInfo proc_info = ProcessInstanceInfo(name.c_str(), arch, pid);
+
+  lldb::pid_t parent = LLDB_INVALID_PROCESS_ID;
+  if (dict.GetValueForKeyAsInteger("parent", parent))
+    proc_info.SetParentProcessID(parent);
+
+  uint32_t uid = UINT32_MAX;
+  if (dict.GetValueForKeyAsInteger("uid", uid))
+    proc_info.SetEffectiveUserID(uid);
+
+  uint32_t gid = UINT32_MAX;
+  if (dict.GetValueForKeyAsInteger("gid", gid))
+    proc_info.SetEffectiveGroupID(gid);
+
+  return proc_info;
+}
+
+uint32_t
+ScriptedPlatform::FindProcesses(const ProcessInstanceInfoMatch &match_info,
+                                ProcessInstanceInfoList &proc_infos) {
+  CheckInterpreterAndScriptObject();
+  StructuredData::DictionarySP dict_sp = GetInterface().ListProcesses();
+
+  Status error;
+  if (!dict_sp)
+    return ScriptedInterface::ErrorWithMessage<uint32_t>(
+        LLVM_PRETTY_FUNCTION, "Failed to get scripted platform processes.",
+        error, LLDBLog::Platform);
+
+  auto parse_process_info = [this, &proc_infos](ConstString key,
+                                                StructuredData::Object *val) {
+    if (!val)
+      return false;
+
+    StructuredData::Dictionary *dict = val->GetAsDictionary();
+
+    if (!dict || !dict->IsValid())
+      return false;
+
+    lldb::pid_t pid = LLDB_INVALID_PROCESS_ID;
+    if (!llvm::to_integer(key.GetStringRef(), pid))
+      return false;
+
+    auto proc_info_or_error = ParseProcessInfo(*dict, pid);
+
+    if (!proc_info_or_error) {
+      llvm::consumeError(proc_info_or_error.takeError());
+      return false;
+    }
+
+    proc_infos.push_back(*proc_info_or_error);
+    return true;
+  };
+
+  dict_sp->ForEach(parse_process_info);
+
+  // TODO: Use match_info to filter through processes
+  return proc_infos.size();
+}
+
+bool ScriptedPlatform::GetProcessInfo(lldb::pid_t pid,
+                                      ProcessInstanceInfo &proc_info) {
+  if (pid == LLDB_INVALID_PROCESS_ID)
+    return false;
+
+  StructuredData::DictionarySP dict_sp = GetInterface().GetProcessInfo(pid);
+
+  if (!dict_sp || !dict_sp->IsValid())
+    return false;
+
+  auto proc_info_or_error = ParseProcessInfo(*dict_sp.get(), pid);
+
+  if (!proc_info_or_error) {
+    llvm::consumeError(proc_info_or_error.takeError());
+    return false;
+  }
+
+  proc_info = *proc_info_or_error;
+  return true;
+}
+
+Status ScriptedPlatform::LaunchProcess(ProcessLaunchInfo &launch_info) {
+  return GetInterface().LaunchProcess();
+}
+
+Status ScriptedPlatform::KillProcess(const lldb::pid_t pid) {
+  return GetInterface().KillProcess(pid);
+}