[lldb-vscode] Only close the debuggers in/out when DAP is over stdin/out

[lanza]

Summary

The DAP is usually communicated over STDIN and STDOUT and thus lldb's
Debugger instance printing and reading from these files can cause
conflicts. However, if the DAP communication was set up to be done via
a socket then we can leave these files open as they can provide
valueable logging and feedback to the user.

[clayborg]

lldb/tools/lldb-vscode/lldb-vscode.cpp

1201–1214

Test if this is a socket first so we don't open dev null is we don't need to:

if (!g_vsc.input.descriptor.m_is_socket) {
  FILE *out = llvm::sys::RetryAfterSignal(nullptr, fopen, dev_null_path, "w");
    if (out) {
      // If the input and output descriptors are STDIN and STDOUT then we need to
      // set the output and error file handles to redirect into nothing otherwise
      // if any code in LLDB prints to the debugger file handles, the output and
      // error file handles are initialized to STDOUT and STDERR and any output
      // will kill our debug session. However, if the communication is via sockets
      // then we can leave these open.
      g_vsc.debugger.SetOutputFileHandle(out, true);
      g_vsc.debugger.SetErrorFileHandle(out, false);
    }
}

[clayborg]

minor nit and this is good

[lanza]

fixup according to Greg's requests

[lanza]

Great point! Fixed.

[clayborg]

It would be nice if we could test this works as well. We would need to spawn the lldb-vscode manually and specify a socket, connect with a socket with the vscode.py test stuff, and get able to get some output from it once the session is up and running. Something like sending:

`script print("stdout outut")

to the debugger console after we stop (note the leading backtick so it will not evaluate an expression, but it will cause STDOUT to happen). Make sure the session doesn't die due to the output, and make sure we get the output from the process we spawned in the test suite

lldb/tools/lldb-vscode/lldb-vscode.cpp

 //     }
 //   }]
 // }
 void request_initialize(const llvm::json::Object &request) {
   g_vsc.debugger = lldb::SBDebugger::Create(true /*source_init_files*/);
   // Create an empty target right away since we might get breakpoint requests
   // before we are given an executable to launch in a "launch" request, or a
   // executable when attaching to a process by process ID in a "attach"
   // request.
+  if (!g_vsc.input.descriptor.m_is_socket) {
     FILE *out = llvm::sys::RetryAfterSignal(nullptr, fopen, dev_null_path, "w");
     if (out) {
-    // Set the output and error file handles to redirect into nothing otherwise
-    // if any code in LLDB prints to the debugger file handles, the output and
-    // error file handles are initialized to STDOUT and STDERR and any output
-    // will kill our debug session.
+      // If the input and output descriptors are STDIN and STDOUT then we need
+      // to set the output and error file handles to redirect into nothing
+      // otherwise if any code in LLDB prints to the debugger file handles, the
+      // output and error file handles are initialized to STDOUT and STDERR and
+      // any output will kill our debug session. However, if the communication
+      // is via sockets then we can leave these open.
       g_vsc.debugger.SetOutputFileHandle(out, true);
       g_vsc.debugger.SetErrorFileHandle(out, false);
     }
+  }
 
   g_vsc.target = g_vsc.debugger.CreateTarget(nullptr);
   lldb::SBListener listener = g_vsc.debugger.GetListener();
   listener.StartListeningForEvents(
       g_vsc.target.GetBroadcaster(),
       lldb::SBTarget::eBroadcastBitBreakpointChanged);
   listener.StartListeningForEvents(g_vsc.broadcaster,
                                    eBroadcastBitStopEventThread);