[WIP] Add command interpreter to lldb-vscode

[lanza]

Summary

Add a check for a supportsCommandInterpreter option in the
initialize packet for the debug adapter protocol. If this is
supported then we can attack the CommandInterpreter to stdin/stdout
and have the typical lldb command line interface.

[clayborg]

It is very dangerous to play with any real STDIN, STDOUT, or STDERR in the lldb-vscode world since this is what is usually the file descriptors used for all the packets.

I would suggest an alternate form for this:

• in the response to the "initialize" packet, we return a new key/value pair like: "supportsCLI":true
• if "supportsCLI" was true, then VSCode will create a new terminal window using the master side of a pty (pseudo terminal) and send the slave path down using a new packet:

{"command":"cli","arguments":{"path":"/path/to/slavepty"}}

• then in lldb-vscode.cpp, we open this file once for read only and once for write only and call:

FILE* in = fopen(pty_slave_path, ...);
FILE* out = fopen(pty_slave_path, ...);
g_vsc.debugger.SetInputFileHandle(in, true);
g_vsc.debugger.SetOutputFileHandle(out, true);
g_vsc.debugger.SetErrorFileHandle(out, false);

And it will be all hooked up without needing to launch with socket? This would only work locally of course, but I believe that the currently proposed solution would only work locally as well right?

[clayborg]

actually lldb-vscode is always running locally so my previous concern is no longer valid.

[lanza]

So I was looking at something like this, but I don't see a way to access PTYs & VSCode terminal emulators at that level of granularity. The current api (vscode.window.createTerminal) only let's you launch a single process which is automatically attached to the terminal emulator within vscode. There seems to be a proposed API that would offer this control, but it seems that this has been stuck as a proposed API (e.g. not usable for public plugins) for over a year now.

[xiaobai]

I like the idea of this commit, it seems like we can have lldb-vscode be more useful than just a DAP frontend to lldb.

One thing I noticed is that you are using termios.h a lot, which is not available on windows. I know this is a WIP, but it's something to consider for this patch.

tools/lldb-vscode/lldb-vscode.cpp

476

Could you insert a link to the bug report if it's public?

[rmaz]

tools/lldb-vscode/lldb-vscode.cpp

810

shouldn't this be g_vsc.command_interpreter_thread.join()?

tools/lldb-vscode/VSCode.h

struct VSCode {
   lldb::SBAttachInfo attach_info;
   lldb::SBLaunchInfo launch_info;
   lldb::SBValueList variables;
   lldb::SBBroadcaster broadcaster;
   int64_t num_regs;
   int64_t num_locals;
   int64_t num_globals;
   std::thread event_thread;
+  std::thread command_interpreter_thread;
   std::unique_ptr<std::ofstream> log;
   llvm::DenseMap<lldb::addr_t, int64_t> addr_to_source_ref;
   llvm::DenseMap<int64_t, SourceReference> source_map;
   llvm::StringMap<SourceBreakpointMap> source_breakpoints;
   FunctionBreakpointMap function_breakpoints;
   std::vector<ExceptionBreakpoint> exception_breakpoints;
   std::vector<std::string> init_commands;
   std::vector<std::string> pre_run_commands;

tools/lldb-vscode/lldb-vscode.cpp

 //===-- lldb-vscode.cpp -----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include <assert.h>
 #include <limits.h>
 #include <stdarg.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <sys/stat.h>
 #include <sys/types.h>
+#include <termios.h>
 #if defined(_WIN32)
 // We need to #define NOMINMAX in order to skip `min()` and `max()` macro
 // definitions that conflict with other system headers.
 // We also need to #undef GetObject (which is defined to GetObjectW) because
 // the JSON code we use also has methods named `GetObject()` and we conflict
 // against these.
 #define NOMINMAX
 #include <windows.h>

 }
 
 // Grab any STDOUT and STDERR from the process and send it up to VS Code
 // via an "output" event to the "stdout" and "stderr" categories.
 void SendStdOutStdErr(lldb::SBProcess &process) {
   char buffer[1024];
   size_t count;
   while ((count = process.GetSTDOUT(buffer, sizeof(buffer))) > 0)
     g_vsc.SendOutput(OutputType::Stdout, llvm::StringRef(buffer, count));
   while ((count = process.GetSTDERR(buffer, sizeof(buffer))) > 0)
     g_vsc.SendOutput(OutputType::Stderr, llvm::StringRef(buffer, count));
 }
 
 // All events from the debugger, target, process, thread and frames are
 // received in this function that runs in its own thread. We are using a
 // "FILE *" to output packets back to VS Code and they have mutexes in them
 // them prevent multiple threads from writing simultaneously so no locking
if (sourceMap) {
     strm << "\".\" \"" << sourcePath << "\"";
   }
   strm.flush();
   if (!sourceMapCommand.empty()) {
     g_vsc.RunLLDBCommands("Setting source map:", {sourceMapCommand});
   }
 }
 
+static void reset_stdin_termios();
+static bool g_old_stdin_termios_is_valid = false;
+static struct termios g_old_stdin_termios;
+
+// In the Driver::MainLoop, we change the terminal settings.  This function is
+// added as an atexit handler to make sure we clean them up.
+static void reset_stdin_termios() {
+  if (g_old_stdin_termios_is_valid) {
+    g_old_stdin_termios_is_valid = false;
+    ::tcsetattr(STDIN_FILENO, TCSANOW, &g_old_stdin_termios);
+  }
+}
+
+void CommandInterpreterThreadFunction() {
+  if (::tcgetattr(STDIN_FILENO, &g_old_stdin_termios) == 0) {
+    g_old_stdin_termios_is_valid = true;
+    atexit(reset_stdin_termios);
+  }
+#ifndef _MSC_VER
+  // Disabling stdin buffering with MSVC's 2015 CRT exposes a bug in fgets
+  // which causes it to miss newlines depending on whether there have been an
+  // odd or even number of characters.  Bug has been reported to MS via Connect.
+  ::setbuf(stdin, nullptr);
+#endif
+  ::setbuf(stdout, nullptr);
+  g_vsc.debugger.RunCommandInterpreter(false, false);
+}
+
 // "AttachRequest": {
 //   "allOf": [ { "$ref": "#/definitions/Request" }, {
 //     "type": "object",
 //     "description": "Attach request; value of command field is 'attach'.",
 //     "properties": {
 //       "command": {
 //         "type": "string",
 //         "enum": [ "attach" ]
void request_attach(const llvm::json::Object &request) {
   // Run any pre run LLDB commands the user specified in the launch.json
   g_vsc.RunPreRunCommands();
 
   if (pid == LLDB_INVALID_PROCESS_ID && wait_for) {
     char attach_info[256];
     auto attach_info_len =
         snprintf(attach_info, sizeof(attach_info),
                  "Waiting to attach to \"%s\"...", program.data());
-    g_vsc.SendOutput(OutputType::Console, llvm::StringRef(attach_info,
-                                                          attach_info_len));
+    g_vsc.SendOutput(OutputType::Console,
+                     llvm::StringRef(attach_info, attach_info_len));
   }
   if (attachCommands.empty()) {
     // No "attachCommands", just attach normally.
     // Disable async events so the attach will be successful when we return from
     // the launch call and the launch will happen synchronously
     g_vsc.debugger.SetAsync(false);
     g_vsc.target.Attach(g_vsc.attach_info, error);
     // Reenable async events
case lldb::eStateRunning:
     break;
   }
   g_vsc.SendJSON(llvm::json::Value(std::move(response)));
   SendTerminatedEvent();
   if (g_vsc.event_thread.joinable()) {
     g_vsc.broadcaster.BroadcastEventByType(eBroadcastBitStopEventThread);
     g_vsc.event_thread.join();
   }
+  if (g_vsc.command_interpreter_thread.joinable()) {
+    g_vsc.event_thread.join();
+  }
 }
 
 void request_exceptionInfo(const llvm::json::Object &request) {
   llvm::json::Object response;
   FillResponse(request, response);
   auto arguments = request.getObject("arguments");
   llvm::json::Object body;
   lldb::SBThread thread = g_vsc.GetLLDBThread(*arguments);

 //     },
 //     "supportsVariablePaging": {
 //       "type": "boolean",
 //       "description": "Client supports the paging of variables."
 //     },
 //     "supportsRunInTerminalRequest": {
 //       "type": "boolean",
 //       "description": "Client supports the runInTerminal request."
+//     },
+//     "supportsCommandInterpreter": }
+//       "type": "boolean",
+//       "description": "Client launches lldb-vscode in a terminal and thus
+//                       supports attaching the CommandInterpreter."
 //     }
 //   },
 //   "required": [ "adapterID" ]
 // },
 // "InitializeResponse": {
 //   "allOf": [ { "$ref": "#/definitions/Response" }, {
 //     "type": "object",
 //     "description": "Response to 'initialize' request.",
 //     "properties": {
 //       "body": {
 //         "$ref": "#/definitions/Capabilities",
 //         "description": "The capabilities of this debug adapter."
 //       }
 //     }
 //   }]
 // }
 void request_initialize(const llvm::json::Object &request) {
   g_vsc.debugger = lldb::SBDebugger::Create(true /*source_init_files*/);
   // Create an empty target right away since we might get breakpoint requests
   // before we are given an executable to launch in a "launch" request, or a
   // executable when attaching to a process by process ID in a "attach"
   // request.
-  FILE *out = llvm::sys::RetryAfterSignal(nullptr, fopen, dev_null_path, "w");
-  if (out) {
-    // Set the output and error file handles to redirect into nothing otherwise
-    // if any code in LLDB prints to the debugger file handles, the output and
-    // error file handles are initialized to STDOUT and STDERR and any output
-    // will kill our debug session.
-    g_vsc.debugger.SetOutputFileHandle(out, true);
-    g_vsc.debugger.SetErrorFileHandle(out, false);
-  }
 
   g_vsc.target = g_vsc.debugger.CreateTarget(nullptr);
   lldb::SBListener listener = g_vsc.debugger.GetListener();
   listener.StartListeningForEvents(
       g_vsc.target.GetBroadcaster(),
       lldb::SBTarget::eBroadcastBitBreakpointChanged);
   listener.StartListeningForEvents(g_vsc.broadcaster,
                                    eBroadcastBitStopEventThread);
   // Start our event thread so we can receive events from the debugger, target,
   // process and more.
   g_vsc.event_thread = std::thread(EventThreadFunction);
 
+  // Set the output and error file handles to redirect into nothing iff the
+  // input and output are set to stdin and stdout. If we are receiving DAP
+  // packets over a socket then keep them up. Otherwise if any code in LLDB
+  // prints to the debugger file handles, the output and error file handles are
+  // initialized to STDOUT and STDERR and any output will kill our debug
+  // session.
+  if (!g_vsc.input.descriptor.m_is_socket) {
+    FILE *out = llvm::sys::RetryAfterSignal(nullptr, fopen, dev_null_path, "w");
+    if (out) {
+      g_vsc.debugger.SetOutputFileHandle(out, true);
+      g_vsc.debugger.SetErrorFileHandle(out, false);
+    }
+  }
+
+  // We can't attach the command interpreter if the DAP packets are being sent
+  // via stdin/stdout. We would require extra sockets. So only attach the CI iff
+  // we are communicating via a socket.
+  if (g_vsc.input.descriptor.m_is_socket) {
+    auto arguments = request.getObject("arguments");
+    auto supports_ci =
+        GetBoolean(arguments, "supportsCommandInterpreter", false);
+    if (supports_ci) {
+      g_vsc.command_interpreter_thread =
+          std::thread(CommandInterpreterThreadFunction);
+    }
+  }
+
   llvm::json::Object response;
   FillResponse(request, response);
   llvm::json::Object body;
   // The debug adapter supports the configurationDoneRequest.
   body.try_emplace("supportsConfigurationDoneRequest", true);
   // The debug adapter supports function breakpoints.
   body.try_emplace("supportsFunctionBreakpoints", true);
   // The debug adapter supports conditional breakpoints.