[lldb-vscode] support the completion request

[wallace]

Summary

The DAP has a completion request that has been unimplemented. It allows showing autocompletion tokens inside the Debug Console.
I implemented it in a very simple fashion mimicking what the user would see when autocompleting an expression inside the CLI.
There are two cases: normal variables and commands. The latter occurs when a text is prepepended with ` in the Debug Console.
These two cases work well and have tests.

[teemperor]

lldb/tools/lldb-vscode/lldb-vscode.cpp

948

max_return_elements isn't implemented, so you can just pass '-1' here. This also means you probably want to manually resize the results to no be longer than 20.

955

I am not sure how VSCode works, but you probably should be aware that the results from HandleCompletionWithDescriptions are expected to replace the current token. Token is in LLDB is just a string as a list of unescaped non-quote non-whitespace characters, where the quotes are not part of the token (and will not be included in the match string you get back). You can also quote every argument in LLDB, so completions in quotes can happen everywhere.

Some test that might bug out VSCode are for example

"com[tab]
'com[tab]
settings "se[tab]
settings 'se[tab]

Also the completions you get back from LLDB are *usually* complete tokens (e.g, you can safely add a space character behind for the user), but the exception to this are file paths (where we only complete the current directory). This isn't really exposed in `HandleCompletionWithDescriptions, you can't really know if adding a space is correct or not. Anyway, just be aware that if VSCode adds a trailing space to the completions, this might give wrong results for file paths.

Also you can get completions in the print/expression command (where users can freely use quotes and whitespaces) which means you can get really weird situations like this: expr "";typed[tab] -> "";typedef (at least, I believe that's what we return). I think this is currently also broken in LLDB, so it might not be a big issue as long as VSCode does anything reasonable when it gets weird matches like that.

[wallace]

removed the results limit of 20 because it didn't work and I verified that returning a long list is displayed well on vscode. There's no need for this limit at this point.
I also tested the cases that @teemperor mentioned and all of them work on vscode. I also added them as test cases.

Fortunately, the way completions are represented inside LLDB are exactly the same as what vscode expects

[wallace]

nit

[clayborg]

Maybe limit the matches if posssible if that works. If you type "target variable <tab>" you can complete a list of all global variables everywhere which might be quite a few.

[wallace]

set a return list limit of 50

lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/completions/Makefile

+CXX_SOURCES := main.cpp
+
+include Makefile.rules

lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/completions/TestVSCode_completions.py

+"""
+Test lldb-vscode completions request
+"""
+
+from __future__ import print_function
+
+import lldbvscode_testcase
+import unittest2
+import vscode
+from lldbsuite.test import lldbutil
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+
+
+class TestVSCode_variables(lldbvscode_testcase.VSCodeTestCaseBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    def verify_completions(self, actual_list, expected_list, not_expected_list=[]):
+        print(actual_list)
+        for expected_item in expected_list:
+            self.assertTrue(expected_item in actual_list)
+
+        for not_expected_item in not_expected_list:
+            self.assertFalse(not_expected_item in actual_list)
+
+    @skipIfWindows
+    @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @no_debug_info_test
+    def test_completions(self):
+        """
+            Tests the completion request at different breakpoints
+        """
+        program = self.getBuildArtifact("a.out")
+        self.build_and_launch(program)
+        source = "main.cpp"
+        breakpoint1_line = line_number(source, "// breakpoint 1")
+        breakpoint2_line = line_number(source, "// breakpoint 2")
+        breakpoint_ids = self.set_source_breakpoints(
+            source, [breakpoint1_line, breakpoint2_line]
+        )
+        self.continue_to_next_stop()
+
+        # shouldn't see variables inside main
+        self.verify_completions(
+            self.vscode.get_completions("var"),
+            [
+                {
+                    "text": "var",
+                    "label": "var -- vector<basic_string<char, char_traits<char>, allocator<char> >, allocator<basic_string<char, char_traits<char>, allocator<char> > > > &",
+                }
+            ],
+            [{"text": "var1", "label": "var1 -- int &"}],
+        )
+
+        # should see global keywords but not variables inside main
+        self.verify_completions(
+            self.vscode.get_completions("str"),
+            [{"text": "struct", "label": "struct"}],
+            [{"text": "str1", "label": "str1 -- std::string &"}],
+        )
+
+        self.continue_to_next_stop()
+
+        # should see variables from main but not from the other function
+        self.verify_completions(
+            self.vscode.get_completions("var"),
+            [
+                {"text": "var1", "label": "var1 -- int &"},
+                {"text": "var2", "label": "var2 -- int &"},
+            ],
+            [
+                {
+                    "text": "var",
+                    "label": "var -- vector<basic_string<char, char_traits<char>, allocator<char> >, allocator<basic_string<char, char_traits<char>, allocator<char> > > > &",
+                }
+            ],
+        )
+
+        self.verify_completions(
+            self.vscode.get_completions("str"),
+            [
+                {"text": "struct", "label": "struct"},
+                {"text": "str1", "label": "str1 -- string &"},
+            ],
+        )
+
+        # should complete arbitrary commands including word starts
+        self.verify_completions(
+            self.vscode.get_completions("`log enable  "),
+            [{"text": "gdb-remote", "label": "gdb-remote"}],
+        )
+
+        # should complete expressions with quotes inside
+        self.verify_completions(
+            self.vscode.get_completions('`expr " "; typed'),
+            [{"text": "typedef", "label": "typedef"}],
+        )
+
+        # should complete an incomplete quoted token
+        self.verify_completions(
+            self.vscode.get_completions('`setting "se'),
+            [
+                {
+                    "text": "set",
+                    "label": "set -- Set the value of the specified debugger setting.",
+                }
+            ],
+        )
+        self.verify_completions(
+            self.vscode.get_completions("`'comm"),
+            [
+                {
+                    "text": "command",
+                    "label": "command -- Commands for managing custom LLDB commands.",
+                }
+            ],
+        )

lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/completions/main.cpp

+#include <string>
+#include <vector>
+
+int fun(std::vector<std::string> var) {
+  return var.size(); // breakpoint 1
+}
+
+int main(int argc, char const *argv[]) {
+  int var1 = 0;
+  int var2 = 1;
+  std::string str1 = "a";
+  std::string str2 = "b";
+  std::vector<std::string> vec;
+  fun(vec);
+  return 0; // breakpoint 2
+}

lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/lldbvscode_testcase.py

def launch(self, program=None, args=None, cwd=None, env=None,
                stopOnEntry=False, disableASLR=True,
                disableSTDIO=False, shellExpandArguments=False,
                trace=False, initCommands=None, preRunCommands=None,
                stopCommands=None, exitCommands=None,sourcePath= None,
                debuggerRoot=None, launchCommands=None):
         '''Sending launch request to vscode
         '''
 
-        # Make sure we disconnet and terminate the VSCode debug adaptor,
+        # Make sure we disconnect and terminate the VSCode debug adapter,
         # if we throw an exception during the test case
         def cleanup():
             self.vscode.request_disconnect(terminateDebuggee=True)
             self.vscode.terminate()
 
         # Execute the cleanup function during test case tear down.
         self.addTearDownHook(cleanup)
 

lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/variables/TestVSCode_variables.py

def test_scopes_variables_setVariable_evaluate(self):
             Tests the "scopes", "variables", "setVariable", and "evaluate"
             packets.
         '''
         program = self.getBuildArtifact("a.out")
         self.build_and_launch(program)
         source = 'main.cpp'
         breakpoint1_line = line_number(source, '// breakpoint 1')
         lines = [breakpoint1_line]
-        # Set breakoint in the thread function so we can step the threads
+        # Set breakpoint in the thread function so we can step the threads
         breakpoint_ids = self.set_source_breakpoints(source, lines)
         self.assertTrue(len(breakpoint_ids) == len(lines),
                         "expect correct number of breakpoints")
         self.continue_to_breakpoints(breakpoint_ids)
         locals = self.vscode.get_local_variables()
         globals = self.vscode.get_global_variables()
         buffer_children = make_buffer_verify_dict(0, 32)
         verify_locals = {

lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/vscode.py

def get_stackFrame(self, frameIndex=0, threadId=None):
             return None
         response = self.request_stackTrace(threadId, startFrame=frameIndex,
                                            levels=1)
         if response:
             return response['body']['stackFrames'][0]
         print('invalid response')
         return None
 
+    def get_completions(self, text):
+        response = self.request_completions(text)
+        return response['body']['targets']
+
     def get_scope_variables(self, scope_name, frameIndex=0, threadId=None):
         stackFrame = self.get_stackFrame(frameIndex=frameIndex,
                                          threadId=threadId)
         if stackFrame is None:
             return []
         frameId = stackFrame['id']
         if frameId in self.frame_scopes:
             frame_scopes = self.frame_scopes[frameId]
def request_setFunctionBreakpoints(self, names, condition=None,
         args_dict = {'breakpoints': breakpoints}
         command_dict = {
             'command': 'setFunctionBreakpoints',
             'type': 'request',
             'arguments': args_dict
         }
         return self.send_recv(command_dict)
 
+    def request_completions(self, text):
+        args_dict = {
+            'text': text,
+            'column': len(text)
+        }
+        command_dict = {
+            'command': 'completions',
+            'type': 'request',
+            'arguments': args_dict
+        }
+        return self.send_recv(command_dict)
+
     def request_stackTrace(self, threadId=None, startFrame=None, levels=None,
                            dump=False):
         if threadId is None:
             threadId = self.get_thread_id()
         args_dict = {'threadId': threadId}
         if startFrame is not None:
             args_dict['startFrame'] = startFrame
         if levels is not None:

lldb/tools/lldb-vscode/lldb-vscode.cpp

if (thread.IsValid()) {
     // }
   } else {
     response["success"] = llvm::json::Value(false);
   }
   response.try_emplace("body", std::move(body));
   g_vsc.SendJSON(llvm::json::Value(std::move(response)));
 }
 
+// "CompletionsRequest": {
+//   "allOf": [ { "$ref": "#/definitions/Request" }, {
+//     "type": "object",
+//     "description": "Returns a list of possible completions for a given caret position and text.\nThe CompletionsRequest may only be called if the 'supportsCompletionsRequest' capability exists and is true.",
+//     "properties": {
+//       "command": {
+//         "type": "string",
+//         "enum": [ "completions" ]
+//       },
+//       "arguments": {
+//         "$ref": "#/definitions/CompletionsArguments"
+//       }
+//     },
+//     "required": [ "command", "arguments"  ]
+//   }]
+// },
+// "CompletionsArguments": {
+//   "type": "object",
+//   "description": "Arguments for 'completions' request.",
+//   "properties": {
+//     "frameId": {
+//       "type": "integer",
+//       "description": "Returns completions in the scope of this stack frame. If not specified, the completions are returned for the global scope."
+//     },
+//     "text": {
+//       "type": "string",
+//       "description": "One or more source lines. Typically this is the text a user has typed into the debug console before he asked for completion."
+//     },
+//     "column": {
+//       "type": "integer",
+//       "description": "The character position for which to determine the completion proposals."
+//     },
+//     "line": {
+//       "type": "integer",
+//       "description": "An optional line for which to determine the completion proposals. If missing the first line of the text is assumed."
+//     }
+//   },
+//   "required": [ "text", "column" ]
+// },
+// "CompletionsResponse": {
+//   "allOf": [ { "$ref": "#/definitions/Response" }, {
+//     "type": "object",
+//     "description": "Response to 'completions' request.",
+//     "properties": {
+//       "body": {
+//         "type": "object",
+//         "properties": {
+//           "targets": {
+//             "type": "array",
+//             "items": {
+//               "$ref": "#/definitions/CompletionItem"
+//             },
+//             "description": "The possible completions for ."
+//           }
+//         },
+//         "required": [ "targets" ]
+//       }
+//     },
+//     "required": [ "body" ]
+//   }]
+// },
+// "CompletionItem": {
+//   "type": "object",
+//   "description": "CompletionItems are the suggestions returned from the CompletionsRequest.",
+//   "properties": {
+//     "label": {
+//       "type": "string",
+//       "description": "The label of this completion item. By default this is also the text that is inserted when selecting this completion."
+//     },
+//     "text": {
+//       "type": "string",
+//       "description": "If text is not falsy then it is inserted instead of the label."
+//     },
+//     "sortText": {
+//       "type": "string",
+//       "description": "A string that should be used when comparing this item with other items. When `falsy` the label is used."
+//     },
+//     "type": {
+//       "$ref": "#/definitions/CompletionItemType",
+//       "description": "The item's type. Typically the client uses this information to render the item in the UI with an icon."
+//     },
+//     "start": {
+//       "type": "integer",
+//       "description": "This value determines the location (in the CompletionsRequest's 'text' attribute) where the completion text is added.\nIf missing the text is added at the location specified by the CompletionsRequest's 'column' attribute."
+//     },
+//     "length": {
+//       "type": "integer",
+//       "description": "This value determines how many characters are overwritten by the completion text.\nIf missing the value 0 is assumed which results in the completion text being inserted."
+//     }
+//   },
+//   "required": [ "label" ]
+// },
+// "CompletionItemType": {
+//   "type": "string",
+//   "description": "Some predefined types for the CompletionItem. Please note that not all clients have specific icons for all of them.",
+//   "enum": [ "method", "function", "constructor", "field", "variable", "class", "interface", "module", "property", "unit", "value", "enum", "keyword", "snippet", "text", "color", "file", "reference", "customcolor" ]
+// }
+void request_completions(const llvm::json::Object &request) {
+  llvm::json::Object response;
+  FillResponse(request, response);
+  llvm::json::Object body;
+  auto arguments = request.getObject("arguments");
+  std::string text = GetString(arguments, "text");
+  auto original_column = GetSigned(arguments, "column", text.size());
+  auto actual_column = original_column - 1;
+  llvm::json::Array targets;
+  // NOTE: the 'line' argument is not needed, as multiline expressions
+  // work well already
+  // TODO: support frameID. Currently
+  // g_vsc.debugger.GetCommandInterpreter().HandleCompletionWithDescriptions
+  // is frame-unaware.
+
+  if (!text.empty() && text[0] == '`') {
+    text = text.substr(1);
+    actual_column--;
+  } else {
+    text = "p " + text;
+    actual_column += 2;
+  }
+  lldb::SBStringList matches;
+  lldb::SBStringList descriptions;
+  g_vsc.debugger.GetCommandInterpreter().HandleCompletionWithDescriptions(
+    text.c_str(),
+    actual_column,
+    0, -1, matches, descriptions);
+  targets.reserve(matches.GetSize());
+  for (size_t i = 0; i < matches.GetSize(); i++) {
+    std::string match = matches.GetStringAtIndex(i);
+    std::string description = descriptions.GetStringAtIndex(i);
+
+    llvm::json::Object item;
+    EmplaceSafeString(item, "text", match);
+    if (description.empty())
+      EmplaceSafeString(item, "label", match);
+    else
+      EmplaceSafeString(item, "label", match + " -- " + description);
+
+    targets.emplace_back(std::move(item));
+  }
+
+  body.try_emplace("targets", std::move(targets));
+  response.try_emplace("body", std::move(body));
+  g_vsc.SendJSON(llvm::json::Value(std::move(response)));
+}
+
 //  "EvaluateRequest": {
 //    "allOf": [ { "$ref": "#/definitions/Request" }, {
 //      "type": "object",
 //      "description": "Evaluate request; value of command field is 'evaluate'.
 //                      Evaluates the given expression in the context of the
 //                      top most stack frame. The expression has access to any
 //                      variables and arguments that are in scope.",
 //      "properties": {
void request_initialize(const llvm::json::Object &request) {
   body.try_emplace("supportsSetVariable", true);
   // The debug adapter supports restarting a frame.
   body.try_emplace("supportsRestartFrame", false);
   // The debug adapter supports the gotoTargetsRequest.
   body.try_emplace("supportsGotoTargetsRequest", false);
   // The debug adapter supports the stepInTargetsRequest.
   body.try_emplace("supportsStepInTargetsRequest", false);
   // The debug adapter supports the completionsRequest.
-  body.try_emplace("supportsCompletionsRequest", false);
+  body.try_emplace("supportsCompletionsRequest", true);
   // The debug adapter supports the modules request.
   body.try_emplace("supportsModulesRequest", false);
   // The set of additional module information exposed by the debug adapter.
   //   body.try_emplace("additionalModuleColumns"] = ColumnDescriptor
   // Checksum algorithms supported by the debug adapter.
   //   body.try_emplace("supportedChecksumAlgorithms"] = ChecksumAlgorithm
   // The debug adapter supports the RestartRequest. In this case a client
   // should not implement 'restart' by terminating and relaunching the adapter

 }
 
 const std::map<std::string, RequestCallback> &GetRequestHandlers() {
 #define REQUEST_CALLBACK(name)                                                 \
   { #name, request_##name }
   static std::map<std::string, RequestCallback> g_request_handlers = {
       // VSCode Debug Adaptor requests
       REQUEST_CALLBACK(attach),
+      REQUEST_CALLBACK(completions),
       REQUEST_CALLBACK(continue),
       REQUEST_CALLBACK(configurationDone),
       REQUEST_CALLBACK(disconnect),
       REQUEST_CALLBACK(evaluate),
       REQUEST_CALLBACK(exceptionInfo),
       REQUEST_CALLBACK(initialize),
       REQUEST_CALLBACK(launch),
       REQUEST_CALLBACK(next),