[clangd] Add xrefs LSP boilerplate implementation.

[ilya-biryukov]

clangd/XRefs.cpp

663

Are we going to support the IncludeDeclaration option?
What should its semantics be?

I'm trying to figure out a better name for it, can't get what should it do by looking at the code at the moment :-)

[hokein]

clangd/XRefs.cpp

663

I think so. It is a term defined in LSP (although vscode always sets it true). I think it aligns with the clang::index::SymbolRole::Declaration.

[ilya-biryukov]

This change LG, but I would not commit it before we have an actual implementation.
As soon as we have the references function in ClangdUnit.cpp implemented, the merge of this change should be trivial.

Is there any value in committing empty stubs before an actual implementation is ready?

[hokein]

In D50896#1204310, @ilya-biryukov wrote:

This change LG, but I would not commit it before we have an actual implementation.
As soon as we have the references function in ClangdUnit.cpp implemented, the merge of this change should be trivial.

Is there any value in committing empty stubs before an actual implementation is ready?

Actually, I'd prefer to get it submitted before we have the actual implementation -- because it would make use easier to experience xrefs features in LSP clients when adding xrefs actual implemenation (without patching the whole patch locally).

Is there any concern about it? Since we disable it by default, it would not affect any users.

[ilya-biryukov]

Having unimplemented stubs in the codebase creates some confusion, but that's my personal opinion. Not terribly opposed to that if others feel it's the right way to go.

For experiments, we could simply patch this in locally without committing upstream. With git it's incredibly easy.

[ioeric]

Agreed with Ilya. I'd probably also make this depend on the ongoing implementation, as exposing LSP endpoints without proper implementation might be confusing to clangd users who only look at the the LSP endpoints. Users need to dig two levels of abstraction to find out that it's not implemented.

clangd/ClangdServer.h

161

I think the C++ API can return SymbolOccurrence in the callback, which would allow C++ API users to get richer information about the occurrences e.g. kind, relationship, code snippet.

161

nit: s/includeDeclaration/IncludeDeclaration/

[sammccall]

(Taking this as @hokein is on leave and the dependency has landed in D50958)

[sammccall]

Updated based on findReferences implementation which has now landed.
Removed ReferenceContext support for now, implementation has no options.
references->findReferences in ClangdServer (more consistent with other methods)
Added lit test for this feature.
Set this feature to true in LSP capabilities.
Made initialize-params-invalid less brittle instead of updating it.

[sammccall]

PTAL

clangd/ClangdServer.h

161

I hadn't seen this comment, I agree, but the implementation that landed just returns Locations. We can extend the C++ API but it doesn't belong in this patch.

clangd/XRefs.cpp

663

It's not supported in the underlying implementation today, partly because we weren't sure
about semantics (and usefulness).

FWIW I think it should control whether the roles Declaration *and* Definition are included, because I guess it's meant to distinguish between usages that introduce a name vs usages that reference an existing name.

For now, we don't support any options and don't parse them out of the LSP.

[hokein]

Thanks, LGTM, ship it!

clangd/Protocol.h

881

Maybe add a comment saying we don't support includeDeclaration?

clangd/ClangdLSPServer.h

private:
   void
   onDocumentRangeFormatting(DocumentRangeFormattingParams &Params) override;
   void onDocumentFormatting(DocumentFormattingParams &Params) override;
   void onDocumentSymbol(DocumentSymbolParams &Params) override;
   void onCodeAction(CodeActionParams &Params) override;
   void onCompletion(TextDocumentPositionParams &Params) override;
   void onSignatureHelp(TextDocumentPositionParams &Params) override;
   void onGoToDefinition(TextDocumentPositionParams &Params) override;
+  void onReference(ReferenceParams &Params) override;
   void onSwitchSourceHeader(TextDocumentIdentifier &Params) override;
   void onDocumentHighlight(TextDocumentPositionParams &Params) override;
   void onFileEvent(DidChangeWatchedFilesParams &Params) override;
   void onCommand(ExecuteCommandParams &Params) override;
   void onWorkspaceSymbol(WorkspaceSymbolParams &Params) override;
   void onRename(RenameParams &Parames) override;
   void onHover(TextDocumentPositionParams &Params) override;
   void onChangeConfiguration(DidChangeConfigurationParams &Params) override;

clangd/ClangdLSPServer.cpp

reply(json::Object{
                  {"triggerCharacters", {"(", ","}},
              }},
             {"definitionProvider", true},
             {"documentHighlightProvider", true},
             {"hoverProvider", true},
             {"renameProvider", true},
             {"documentSymbolProvider", true},
             {"workspaceSymbolProvider", true},
+            {"referencesProvider", true},
             {"executeCommandProvider",
              json::Object{
                  {"commands", {ExecuteCommandParams::CLANGD_APPLY_FIX_COMMAND}},
              }},
         }}}});
 }
 
 void ClangdLSPServer::onShutdown(ShutdownParams &Params) {

 }
 
 // FIXME: This function needs to be properly tested.
 void ClangdLSPServer::onChangeConfiguration(
     DidChangeConfigurationParams &Params) {
   applyConfiguration(Params.settings);
 }
 
+void ClangdLSPServer::onReference(ReferenceParams &Params) {
+  Server.findReferences(Params.textDocument.uri.file(), Params.position,
+                        [](llvm::Expected<std::vector<Location>> Locations) {
+                          if (!Locations)
+                            return replyError(
+                                ErrorCode::InternalError,
+                                llvm::toString(Locations.takeError()));
+                          reply(llvm::json::Array(*Locations));
+                        });
+}
+
 ClangdLSPServer::ClangdLSPServer(JSONOutput &Out,
                                  const clangd::CodeCompleteOptions &CCOpts,
                                  llvm::Optional<Path> CompileCommandsDir,
                                  bool ShouldUseInMemoryCDB,
                                  const ClangdServer::Options &Opts)
     : Out(Out), CDB(ShouldUseInMemoryCDB ? CompilationDB::makeInMemory()
                                          : CompilationDB::makeDirectoryBased(
                                                std::move(CompileCommandsDir))),

clangd/ClangdServer.h

public:
   /// Retrieve the top symbols from the workspace matching a query.
   void workspaceSymbols(StringRef Query, int Limit,
                         Callback<std::vector<SymbolInformation>> CB);
 
   /// Retrieve the symbols within the specified file.
   void documentSymbols(StringRef File,
                        Callback<std::vector<SymbolInformation>> CB);
 
+  /// Retrieve locations for symbol references.
+  void findReferences(PathRef File, Position Pos,
+                      Callback<std::vector<Location>> CB);
+
   /// Run formatting for \p Rng inside \p File with content \p Code.
   llvm::Expected<tooling::Replacements> formatRange(StringRef Code,
                                                     PathRef File, Range Rng);
 
   /// Run formatting for the whole \p File with content \p Code.
   llvm::Expected<tooling::Replacements> formatFile(StringRef Code,
                                                    PathRef File);
 

clangd/ClangdServer.cpp

auto Action = [](Callback<std::vector<SymbolInformation>> CB,
     if (!InpAST)
       return CB(InpAST.takeError());
     CB(clangd::getDocumentSymbols(InpAST->AST));
   };
   WorkScheduler.runWithAST("documentSymbols", File,
                            Bind(Action, std::move(CB)));
 }
 
+void ClangdServer::findReferences(PathRef File, Position Pos,
+                                  Callback<std::vector<Location>> CB) {
+  auto Action = [Pos, this](Callback<std::vector<Location>> CB,
+                            llvm::Expected<InputsAndAST> InpAST) {
+    if (!InpAST)
+      return CB(InpAST.takeError());
+    CB(clangd::findReferences(InpAST->AST, Pos, Index));
+  };
+
+  WorkScheduler.runWithAST("References", File, Bind(Action, std::move(CB)));
+}
+
 std::vector<std::pair<Path, std::size_t>>
 ClangdServer::getUsedBytesPerFile() const {
   return WorkScheduler.getUsedBytesPerFile();
 }
 
 LLVM_NODISCARD bool
 ClangdServer::blockUntilIdleForTest(llvm::Optional<double> TimeoutSeconds) {
   return WorkScheduler.blockUntilIdle(timeoutSeconds(TimeoutSeconds));
 }

clangd/Protocol.h

struct DocumentHighlight {
   friend bool operator==(const DocumentHighlight &LHS,
                          const DocumentHighlight &RHS) {
     return LHS.kind == RHS.kind && LHS.range == RHS.range;
   }
 };
 llvm::json::Value toJSON(const DocumentHighlight &DH);
 llvm::raw_ostream &operator<<(llvm::raw_ostream &, const DocumentHighlight &);
 
+struct ReferenceParams : public TextDocumentPositionParams {
+  // For now, no options like context.includeDeclaration are supported.
+};
+bool fromJSON(const llvm::json::Value &, ReferenceParams &);
+
 struct CancelParams {
   /// The request id to cancel.
   /// This can be either a number or string, if it is a number simply print it
   /// out and always use a string.
   std::string ID;
 };
 llvm::json::Value toJSON(const CancelParams &);
 llvm::raw_ostream &operator<<(llvm::raw_ostream &, const CancelParams &);

clangd/Protocol.cpp

 
 bool fromJSON(const json::Value &Params,
               ClangdConfigurationParamsChange &CCPC) {
   json::ObjectMapper O(Params);
   return O && O.map("compilationDatabasePath", CCPC.compilationDatabasePath) &&
          O.map("compilationDatabaseChanges", CCPC.compilationDatabaseChanges);
 }
 
+bool fromJSON(const json::Value &Params, ReferenceParams &R) {
+  TextDocumentPositionParams &Base = R;
+  return fromJSON(Params, Base);
+}
+
 json::Value toJSON(const CancelParams &CP) {
   return json::Object{{"id", CP.ID}};
 }
 
 llvm::raw_ostream &operator<<(llvm::raw_ostream &O, const CancelParams &CP) {
   O << toJSON(CP);
   return O;
 }

clangd/ProtocolHandlers.h

public:
   virtual void
   onDocumentOnTypeFormatting(DocumentOnTypeFormattingParams &Params) = 0;
   virtual void
   onDocumentRangeFormatting(DocumentRangeFormattingParams &Params) = 0;
   virtual void onCodeAction(CodeActionParams &Params) = 0;
   virtual void onCompletion(TextDocumentPositionParams &Params) = 0;
   virtual void onSignatureHelp(TextDocumentPositionParams &Params) = 0;
   virtual void onGoToDefinition(TextDocumentPositionParams &Params) = 0;
+  virtual void onReference(ReferenceParams &Params) = 0;
   virtual void onSwitchSourceHeader(TextDocumentIdentifier &Params) = 0;
   virtual void onFileEvent(DidChangeWatchedFilesParams &Params) = 0;
   virtual void onCommand(ExecuteCommandParams &Params) = 0;
   virtual void onWorkspaceSymbol(WorkspaceSymbolParams &Params) = 0;
   virtual void onRename(RenameParams &Parames) = 0;
   virtual void onDocumentHighlight(TextDocumentPositionParams &Params) = 0;
   virtual void onHover(TextDocumentPositionParams &Params) = 0;
   virtual void onChangeConfiguration(DidChangeConfigurationParams &Params) = 0;

clangd/ProtocolHandlers.cpp

Register("textDocument/rangeFormatting",
            &ProtocolCallbacks::onDocumentRangeFormatting);
   Register("textDocument/onTypeFormatting",
            &ProtocolCallbacks::onDocumentOnTypeFormatting);
   Register("textDocument/formatting", &ProtocolCallbacks::onDocumentFormatting);
   Register("textDocument/codeAction", &ProtocolCallbacks::onCodeAction);
   Register("textDocument/completion", &ProtocolCallbacks::onCompletion);
   Register("textDocument/signatureHelp", &ProtocolCallbacks::onSignatureHelp);
   Register("textDocument/definition", &ProtocolCallbacks::onGoToDefinition);
+  Register("textDocument/references", &ProtocolCallbacks::onReference);
   Register("textDocument/switchSourceHeader",
            &ProtocolCallbacks::onSwitchSourceHeader);
   Register("textDocument/rename", &ProtocolCallbacks::onRename);
   Register("textDocument/hover", &ProtocolCallbacks::onHover);
   Register("textDocument/documentSymbol", &ProtocolCallbacks::onDocumentSymbol);
   Register("workspace/didChangeWatchedFiles", &ProtocolCallbacks::onFileEvent);
   Register("workspace/executeCommand", &ProtocolCallbacks::onCommand);
   Register("textDocument/documentHighlight",
            &ProtocolCallbacks::onDocumentHighlight);
   Register("workspace/didChangeConfiguration",
            &ProtocolCallbacks::onChangeConfiguration);
   Register("workspace/symbol", &ProtocolCallbacks::onWorkspaceSymbol);
   Register("$/cancelRequest", &ProtocolCallbacks::onCancelRequest);
 }

clangd/XRefs.cpp

 #include "SourceCode.h"
 #include "URI.h"
 #include "clang/AST/DeclTemplate.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexingAction.h"
 #include "clang/Index/USRGeneration.h"
 #include "llvm/Support/Path.h"
+
 namespace clang {
 namespace clangd {
 using namespace llvm;
 namespace {
 
 // Get the definition from a given declaration `D`.
 // Return nullptr if no definition is found, or the declaration type of `D` is
 // not supported.
Optional<Hover> getHover(ParsedAST &AST, Position Pos) {
   auto DeducedType = getDeducedType(AST, SourceLocationBeg);
   if (DeducedType && !DeducedType->isNull())
     return getHoverContents(*DeducedType, AST.getASTContext());
 
   return None;
 }
 
 std::vector<Location> findReferences(ParsedAST &AST, Position Pos,
                                      const SymbolIndex *Index) {
   std::vector<Location> Results;
   const SourceManager &SM = AST.getASTContext().getSourceManager();
   auto MainFilePath = getRealPath(SM.getFileEntryForID(SM.getMainFileID()), SM);
   if (!MainFilePath) {
     elog("Failed to get a path for the main file, so no references");
     return Results;
   }
   auto Loc = getBeginningOfIdentifier(AST, Pos, SM.getMainFileID());

test/clangd/initialize-params-invalid.test

 # RUN: clangd -lit-test < %s | FileCheck -strict-whitespace %s
 # Test with invalid initialize request parameters
 {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":"","rootUri":"test:///workspace","capabilities":{},"trace":"off"}}
 #      CHECK:  "id": 0,
 # CHECK-NEXT:  "jsonrpc": "2.0",
 # CHECK-NEXT:  "result": {
 # CHECK-NEXT:    "capabilities": {
-# CHECK-NEXT:      "codeActionProvider": true,
-# CHECK-NEXT:      "completionProvider": {
-# CHECK-NEXT:        "resolveProvider": false,
-# CHECK-NEXT:        "triggerCharacters": [
-# CHECK-NEXT:          ".",
-# CHECK-NEXT:          ">",
-# CHECK-NEXT:          ":"
-# CHECK-NEXT:        ]
-# CHECK-NEXT:      },
-# CHECK-NEXT:      "definitionProvider": true,
-# CHECK-NEXT:      "documentFormattingProvider": true,
-# CHECK-NEXT:      "documentHighlightProvider": true,
-# CHECK-NEXT:      "documentOnTypeFormattingProvider": {
-# CHECK-NEXT:        "firstTriggerCharacter": "}",
-# CHECK-NEXT:        "moreTriggerCharacter": []
-# CHECK-NEXT:      },
-# CHECK-NEXT:      "documentRangeFormattingProvider": true,
-# CHECK-NEXT:      "documentSymbolProvider": true,
-# CHECK-NEXT:      "executeCommandProvider": {
-# CHECK-NEXT:        "commands": [
-# CHECK-NEXT:          "clangd.applyFix"
-# CHECK-NEXT:        ]
-# CHECK-NEXT:      },
-# CHECK-NEXT:      "hoverProvider": true,
-# CHECK-NEXT:      "renameProvider": true,
-# CHECK-NEXT:      "signatureHelpProvider": {
-# CHECK-NEXT:        "triggerCharacters": [
-# CHECK-NEXT:          "(",
-# CHECK-NEXT:          ","
-# CHECK-NEXT:        ]
-# CHECK-NEXT:      },
-# CHECK-NEXT:      "textDocumentSync": 2,
-# CHECK-NEXT:      "workspaceSymbolProvider": true
-# CHECK-NEXT:    }
-# CHECK-NEXT:  }
+# ...
 ---
 {"jsonrpc":"2.0","id":3,"method":"shutdown"}
 ---
 {"jsonrpc":"2.0","method":"exit"}

test/clangd/initialize-params.test

 # CHECK-NEXT:      "documentRangeFormattingProvider": true,
 # CHECK-NEXT:      "documentSymbolProvider": true,
 # CHECK-NEXT:      "executeCommandProvider": {
 # CHECK-NEXT:        "commands": [
 # CHECK-NEXT:          "clangd.applyFix"
 # CHECK-NEXT:        ]
 # CHECK-NEXT:      },
 # CHECK-NEXT:      "hoverProvider": true,
+# CHECK-NEXT:      "referencesProvider": true,
 # CHECK-NEXT:      "renameProvider": true,
 # CHECK-NEXT:      "signatureHelpProvider": {
 # CHECK-NEXT:        "triggerCharacters": [
 # CHECK-NEXT:          "(",
 # CHECK-NEXT:          ","
 # CHECK-NEXT:        ]
 # CHECK-NEXT:      },
 # CHECK-NEXT:      "textDocumentSync": 2,

test/clangd/references.test

+# RUN: clangd -lit-test < %s | FileCheck -strict-whitespace %s
+{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
+---
+{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"test:///main.cpp","languageId":"cpp","version":1,"text":"int x; int y = x;"}}}
+---
+{"jsonrpc":"2.0","id":1,"method":"textDocument/references","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":0,"character":4}}}
+#      CHECK:  "id": 1
+# CHECK-NEXT:  "jsonrpc": "2.0",
+# CHECK-NEXT:  "result": [
+# CHECK-NEXT:    {
+# CHECK-NEXT:      "range": {
+# CHECK-NEXT:        "end": {
+# CHECK-NEXT:          "character": 5,
+# CHECK-NEXT:          "line": 0
+# CHECK-NEXT:        },
+# CHECK-NEXT:        "start": {
+# CHECK-NEXT:          "character": 4,
+# CHECK-NEXT:          "line": 0
+# CHECK-NEXT:        }
+# CHECK-NEXT:      },
+# CHECK-NEXT:      "uri": "test:///main.cpp"
+# CHECK-NEXT:    },
+# CHECK-NEXT:    {
+# CHECK-NEXT:      "range": {
+# CHECK-NEXT:        "end": {
+# CHECK-NEXT:          "character": 16,
+# CHECK-NEXT:          "line": 0
+# CHECK-NEXT:        },
+# CHECK-NEXT:        "start": {
+# CHECK-NEXT:          "character": 15,
+# CHECK-NEXT:          "line": 0
+# CHECK-NEXT:        }
+# CHECK-NEXT:      },
+# CHECK-NEXT:      "uri": "test:///main.cpp"
+# CHECK-NEXT:    },
+# CHECK-NEXT:  ]
+---
+{"jsonrpc":"2.0","id":3,"method":"shutdown"}
+---
+{"jsonrpc":"2.0","method":"exit"}