[clangd] Add a flag for setting isIncomplete flag true in all responses

[kbobyrev]

Summary

To simplify re-ranking the code completion results, some clients might
want to mimic receiving the completion lists which are always incomplete.

[kadircet]

some drive-by comments:

if this is a client-specific option, wouldn't it be equally hard for client to just set this in the LSP layer, instead of telling clangd to do that?

by looking at the current patch, it rather seems like a user-specific option, which will help users who want to prevent their editors from re-ranking results.

clang-tools-extra/clangd/ClangdLSPServer.cpp

986

nit: ClangdServerOpts.AlwaysIncomplete || List->HasMore

clang-tools-extra/clangd/tool/ClangdMain.cpp

405 ↗

(On Diff #244424)

if we want clients to set it wouldn't it be better to make it an initialization option instead?

[kbobyrev]

Address the review comments.

[sammccall]

The idea is this is an easy option for the user to work-around editors that do client-side filtering badly (everyone lets you add flags).
But as mentioned offline, I don't think we actually have an example of an editor this fixes (I think it fixes neither vscode nor coc.nvim). So in that case we shouldn't land it.

[kbobyrev]

Okay, makes sense!

clang-tools-extra/clangd/ClangdLSPServer.cpp

CDB.emplace(BaseCDB.get(), Params.initializationOptions.fallbackFlags,
     if (NegotiatedOffsetEncoding)
       WithOffsetEncoding.emplace(kCurrentOffsetEncoding,
                                  *NegotiatedOffsetEncoding);
     Server.emplace(*CDB, FSProvider, ClangdServerOpts,
                    static_cast<ClangdServer::Callbacks *>(this));
   }
   applyConfiguration(Params.initializationOptions.ConfigSettings);
 
+  ClangdServerOpts.AlwaysIncomplete =
+      Params.initializationOptions.AlwaysIncomplete;
+
   CCOpts.EnableSnippets = Params.capabilities.CompletionSnippets;
   CCOpts.IncludeFixIts = Params.capabilities.CompletionFixes;
   if (!CCOpts.BundleOverloads.hasValue())
     CCOpts.BundleOverloads = Params.capabilities.HasSignatureHelp;
   DiagOpts.EmbedFixesInDiagnostics = Params.capabilities.DiagnosticFixes;
   DiagOpts.SendDiagnosticCategory = Params.capabilities.DiagnosticCategory;
   DiagOpts.EmitRelatedLocations =
       Params.capabilities.DiagnosticRelatedInformation;
if (!shouldRunCompletion(Params)) {
     return Reply(CompletionList());
   }
   Server->codeComplete(Params.textDocument.uri.file(), Params.position, CCOpts,
                        [Reply = std::move(Reply),
                         this](llvm::Expected<CodeCompleteResult> List) mutable {
                          if (!List)
                            return Reply(List.takeError());
                          CompletionList LSPList;
-                         LSPList.isIncomplete = List->HasMore;
+                         // Set isIncomplete to true regardless of the actual
+                         // results if the corresponding flag was added to
+                         // Clangd invocation.
+                         LSPList.isIncomplete =
+                             ClangdServerOpts.AlwaysIncomplete || List->HasMore;
                          for (const auto &R : List->Completions) {
                            CompletionItem C = R.render(CCOpts);
                            C.kind = adjustKindToCapability(
                                C.kind, SupportedCompletionItemKinds);
                            LSPList.items.push_back(std::move(C));
                          }
                          return Reply(std::move(LSPList));
                        });

clang-tools-extra/clangd/ClangdServer.h

struct Options {
     std::vector<std::string> QueryDriverGlobs;
 
     /// Enable semantic highlighting features.
     bool SemanticHighlighting = false;
 
     /// Enable cross-file rename feature.
     bool CrossFileRename = false;
 
+    /// Always set isIncomplete to true when returning CompletionList.
+    bool AlwaysIncomplete = false;
+
     /// Returns true if the tweak should be enabled.
     std::function<bool(const Tweak &)> TweakFilter = [](const Tweak &T) {
       return !T.hidden(); // only enable non-hidden tweaks.
     };
 
     explicit operator TUScheduler::Options() const;
   };
   // Sensible default options for use in tests.
void symbolInfo(PathRef File, Position Pos,
                   Callback<std::vector<SymbolDetails>> CB);
 
   /// Get semantic ranges around a specified position in a file.
   void semanticRanges(PathRef File, Position Pos,
                       Callback<std::vector<Range>> CB);
 
   /// Get all document links in a file.
   void documentLinks(PathRef File, Callback<std::vector<DocumentLink>> CB);
- 
   /// Returns estimated memory usage for each of the currently open files.
   /// The order of results is unspecified.
   /// Overall memory usage of clangd may be significantly more than reported
   /// here, as this metric does not account (at least) for:
   ///   - memory occupied by static and dynamic index,
   ///   - memory required for in-flight requests,
   /// FIXME: those metrics might be useful too, we should add them.
   std::vector<std::pair<Path, std::size_t>> getUsedBytesPerFile() const;

clang-tools-extra/clangd/Protocol.h

struct InitializationOptions {
   llvm::Optional<std::string> compilationDatabasePath;
   // Additional flags to be included in the "fallback command" used when
   // the compilation database doesn't describe an opened file.
   // The command used will be approximately `clang $FILE $fallbackFlags`.
   std::vector<std::string> fallbackFlags;
 
   /// Clients supports show file status for textDocument/clangd.fileStatus.
   bool FileStatus = false;
+
+  /// Always sets code completion list's isIncomplete to true. This helps to
+  /// utilize Clangd code completion ranking to a higher degree.
+  bool AlwaysIncomplete = false;
 };
 bool fromJSON(const llvm::json::Value &, InitializationOptions &);
 
 struct InitializeParams {
   /// The process Id of the parent process that started
   /// the server. Is null if the process has not been started by another
   /// process. If the parent process is not alive then the server should exit
   /// (see exit notification) its process.

clang-tools-extra/clangd/Protocol.cpp

bool fromJSON(const llvm::json::Value &Params, InitializationOptions &Opts) {
   llvm::json::ObjectMapper O(Params);
   if (!O)
     return true; // 'any' type in LSP.
 
   fromJSON(Params, Opts.ConfigSettings);
   O.map("compilationDatabasePath", Opts.compilationDatabasePath);
   O.map("fallbackFlags", Opts.fallbackFlags);
   O.map("clangdFileStatus", Opts.FileStatus);
+  O.map("alwaysIncomplete", Opts.AlwaysIncomplete);
   return true;
 }
 
 bool fromJSON(const llvm::json::Value &E, TypeHierarchyDirection &Out) {
   auto T = E.getAsInteger();
   if (!T)
     return false;
   if (*T < static_cast<int>(TypeHierarchyDirection::Children) ||

clang-tools-extra/clangd/test/always-incomplete.test

+# RUN: clangd -lit-test < %s | FileCheck -strict-whitespace %s
+{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"initializationOptions":{"alwaysIncomplete": true},"trace":"off"}}
+---
+{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"test:///main.cpp","languageId":"cpp","version":1,"text":"struct S { int a; };\nint main() {\nS().\n}"}}}
+---
+{"jsonrpc":"2.0","id":1,"method":"textDocument/completion","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":2,"character":4}}}
+#      CHECK:  "id": 1
+# CHECK-NEXT:  "jsonrpc": "2.0",
+# CHECK-NEXT:  "result": {
+# CHECK-NEXT:    "isIncomplete": true,
+# CHECK-NEXT:    "items": [
+# CHECK-NEXT:    {
+# CHECK-NEXT:      "detail": "int",
+# CHECK-NEXT:      "filterText": "a",
+# CHECK-NEXT:      "insertText": "a",
+# CHECK-NEXT:      "insertTextFormat": 1,
+# CHECK-NEXT:      "kind": 5,
+# CHECK-NEXT:      "label": " a",
+# CHECK-NEXT:      "score": {{[0-9]+.[0-9]+}},
+# CHECK-NEXT:      "sortText": "{{.*}}a"
+# CHECK-NEXT:      "textEdit": {
+# CHECK-NEXT:        "newText": "a",
+# CHECK-NEXT:        "range": {
+# CHECK-NEXT:          "end": {
+# CHECK-NEXT:            "character": 4,
+# CHECK-NEXT:            "line": 2
+# CHECK-NEXT:          }
+# CHECK-NEXT:          "start": {
+# CHECK-NEXT:            "character": 4,
+# CHECK-NEXT:            "line": 2
+# CHECK-NEXT:          }
+# CHECK-NEXT:        }
+# CHECK-NEXT:      }
+# CHECK-NEXT:    }
+# CHECK-NEXT:  ]
+---
+{"jsonrpc":"2.0","id":4,"method":"shutdown"}
+---
+{"jsonrpc":"2.0","method":"exit"}