[clangd] Add support for hierarchical documentSymbol

[ilya-biryukov]

Posted to make sure it's visible that I've started doing this.
Still need to update the tests and check for the capability from the client (and fallback to SymbolInformation if client does not support the new implementation).

Nevertheless, it works and looks really good in VSCode.

Will update the thread when the code is ready for review.

[simark]

Ohh awesome, I didn't know the LSP supported that. I'll try it it Theia when I have time.

[simark]

I just tried this, this looks very promising! It should help build our outline view in a much more robust way than we do currently.

A nit for the final patch, I would suggest omitting the fields that are optional, like children (when the list is empty) and deprecated.

In vscode, is there a way to get a tree representation of this data? When I look at "Go to symbol in File..." (ctrl-shift-o) or the outline view at the bottom of the file explorer, they are both a flat list. What difference does this patch make in how vscode shows the data?

[ilya-biryukov]

In D52311#1255876, @simark wrote:

I just tried this, this looks very promising! It should help build our outline view in a much more robust way than we do currently.
A nit for the final patch, I would suggest omitting the fields that are optional, like children (when the list is empty) and deprecated.

SG, will do.

In vscode, is there a way to get a tree representation of this data? When I look at "Go to symbol in File..." (ctrl-shift-o) or the outline view at the bottom of the file explorer, they are both a flat list. What difference does this patch make in how vscode shows the data?

There's an outline view that shows the tree after this patch.
IIRC, the "Go to symbol in File..." also shows the tree view now without the lack of functionality like filtering by name, etc.
Overall, the experience with a tree seemed strictly better in all scenarios for me.

[ilya-biryukov]

• Improve traversal of the AST.
• Update the tests.
• Add a fallback to SymbolInformation (flat structure) if the client does not support the hierarhical reponse.

Still some work left to do:

• Do not drop the qualifiers for out-of-line declarations, i.e. 'X::foo' is presented as 'foo' now, which is a bit confusing.
• See why some names pop up at the global scope (GTest generates macros like that).

[ilya-biryukov]

Another annoyance to fix:

• 'using namespace any_ns' is shown as '<using-directive>'

[sammccall]

Very nice! Mostly just a few style/structure nits.

clangd/AST.cpp

69 ↗

(On Diff #171067)

just use ND?

clangd/ClangdLSPServer.cpp

28 ↗

(On Diff #171067)

move this nearer the call site?

29 ↗

(On Diff #171067)

or flattenSymbolHierarchy?

31 ↗

(On Diff #171067)

auto-typed lambdas can't be recursive, but std::function shouldn't be too slow here? And clearer I think:

vector<SymbolInformation> Results;
std::function<void(const DocumentSymbol&, StringRef)> Process = [&](const DocumentSymbol &Sym, StringRef ParentName) {
  ...
};
for (const auto& TopLevel : Symbols)
  Process(TopLevel);
return Results;

failing that, I found this pattern confusing - I'd suggest either making it a recursive plain function, or a standard functor object as if it were a lambda (with the recursive call being operator())

44 ↗

(On Diff #171067)

nit: optional<StringRef> for parent name seems slightly neater

clangd/FindSymbols.cpp

190 ↗

(On Diff #171067)

may want to add a FIXME here: per the tests, it's not classifying constructor/destructor/operator correctly (they're all "methods").

cons/dest is just indexSymbolKindToSymbolKind needing an update, but operator isn't represented in index::SymbolKind.
Maybe we should stop using index::SymbolKind entirely, it doesn't appear to be great.

208 ↗

(On Diff #171067)

this class seems maybe too long to live inside this function. (I wouldn't worry about it being visible to other stuff, the file is small and focused)

208 ↗

(On Diff #171067)

(this looks more like a class than a struct?)

208 ↗

(On Diff #171067)

should this be a RecursiveASTVisitor?
I guess not, but please explain why (there's no documentation on the implementation strategy, and it's complicated)

211 ↗

(On Diff #171067)

again, doing the work in the constructor is a bit odd - own the data and expose it?

253 ↗

(On Diff #171067)

we only consider visiting named decls, maybe reflect that here?
(not needed now but may allow future refinement)

264 ↗

(On Diff #171067)

isn't this covered by D->isImplicit?

276 ↗

(On Diff #171067)

isn't this covered by D->isImplicit() above?

clangd/Protocol.h

43 ↗

(On Diff #171067)

why?

154 ↗

(On Diff #171067)

just an overload of contains()?

unittests/clangd/FindSymbolsTests.cpp

48 ↗

(On Diff #171067)

ChildrenAre() (with no args) and NoChildren() are the same matcher I think.

I'd suggest flattening Children() into ChildrenAre() and renaming it to Children() - less things for the reader to understand.

442 ↗

(On Diff #171067)

this one is to be fixed, right?

521 ↗

(On Diff #171067)

hmm, this seems pretty confusing - I think Tmpl<float> would be a clearer name for a specialization, even if we just have Tmpl for the primary template.
Partial specializations are confusing, though :-/

523 ↗

(On Diff #171067)

why the change in policy with this patch? (one of these previously was deliberately not indexed, now is)

524 ↗

(On Diff #171067)

these assertions are confusing as it's not clear which expected belongs with which piece of actual code. Obviously they all need to have the same name though.
Best workaround I can think of is putting "int a", "int b" etc between the confusing decls, and including them in the expected top-level symbols (since you assert in order)

571 ↗

(On Diff #171067)

hmm, our handling of anonymous enums seems different than anonymous namespaces - why?

[ilya-biryukov]

• Address comments

clangd/FindSymbols.cpp

190 ↗

(On Diff #171067)

Added a FIXME.
From what I can tell, writing our own classification function shouldn't be too much work and would clearly fix those cases. We should do this!

208 ↗

(On Diff #171067)

Added a comment.

264 ↗

(On Diff #171067)

Nope, IIUC things are only marked "implicit" in clangd if they were generated by the compiler from the start.
For this case, the initial template code was written by the user, so isImplicit() returns false for both the template and its instantiations.

276 ↗

(On Diff #171067)

See the other comment.

clangd/Protocol.h

43 ↗

(On Diff #171067)

Sorry, accidental change. Reverted.

unittests/clangd/FindSymbolsTests.cpp

442 ↗

(On Diff #171067)

Why? The outline view gives both the declaration and the definition, since both were written in the code.
That seems to be in line with what I'd expect from the outline view.

521 ↗

(On Diff #171067)

Done. Now prints as Tmpl<float>.
This may also include some arguments not written by the users (e.g. some default args), but added a FIXME to fix this, it's not entirely trivial

523 ↗

(On Diff #171067)

We might have different goals in mind here.
From my perspective, the purpose of the outline is to cover all things written in the code: there are 4 decls that the user wrote: one primary template, one template specialization and two template instantiations.

What is your model here?

571 ↗

(On Diff #171067)

No rigorous reasons. The namespaces tend to group more things together, so it's arguably more useful to have an option of folding their subitems in the tree.
Anonymous enums are typically used only in the classes (and on the top-level in C to define constants?) and I feel having an extra node there merely produces noise.
Happy to reconsider, what's your preferred behavior?

[ilya-biryukov]

• Remove accidentally added qualifiers

[sammccall]

unittests/clangd/FindSymbolsTests.cpp

442 ↗

(On Diff #171067)

Sorry, I meant the name should be Foo::def instead of def, which you mentioned in a previous comment.

OK to land without this, but I think it deserves a fixme.
(If you think this is the *right* behavior, let's discuss further!)

521 ↗

(On Diff #171067)

Nice! I thought this was going to be really hard.

(You can specialize without naming the defaulted args? That sounds... obscure. Definitely fine to leave for later)

523 ↗

(On Diff #171067)

I don't have a particular opinion, just wondering if this was a deliberate change or fell out of the implementation.

(The old behavior was tested, so I guess Marc-Andre had a reason, but it didn't come up in the review. I don't think this is common enough to worry much about either way)

571 ↗

(On Diff #171067)

I *think* I might prefer the opposite - namespaces flat and enums grouped :-)
I'm not certain nor adamant about either, so consider my thoughts below, and pick something.

Namespaces: anonymous namespaces are mostly (entirely?) used to control visibility. I don't think they *group* in a meaningful way, they just hide things like static does. In a perfect world I think there'd be a "private" bit on outline elements that was shown in the UI, and we'd put it on both things-in-anon-namespaces and static-as-in-private decls. The main counterargument I can see: because we typically don't indent inside namespaces, the enclosing anonymous namespace can be hard to recognize and jump to.

Enums: conversely, I think these _typically_ do group their elements, and there's often a strong semantic boundary between the last entry in the enum and the first decl after it. When people use a namespacing prefix (C-style VK_Foo, VK_Bar) then it's visually noticeable, but this is far from universal in C++ (particularly at class scope).

Consistency: It's not *really* confusing if these are different, just *slightly* confusing. Anonymous structs etc must certainly be grouped, so this is a (weak) argument for grouping everything.

[ilya-biryukov]

unittests/clangd/FindSymbolsTests.cpp

442 ↗

(On Diff #171067)

Oh, yeah, that I agree with! Will look into fixing this in a follow-up change.

521 ↗

(On Diff #171067)

Yeah, you can. It's especially common with function where specializations don't name any arguments and those are inferred from the function type...

523 ↗

(On Diff #171067)

Yeah, this fell out of the implementation of walking over the "user-written" part of the AST and it looked sensible so I updated the test.

571 ↗

(On Diff #171067)

I'm sold on the consistency argument. Besides, the anonymous enums are not too frequent anyway, so treating them in a special way might be an overkill.
We can always tweak this in the future.

[ilya-biryukov]

• Do not drop anonymous enums

[ilya-biryukov]

• Handle 'using namespace' in printName
• Do not mix-in symbols with locations from other files.

clang-tools-extra/trunk/clangd/AST.h

 
 /// Returns the qualified name of ND. The scope doesn't contain unwritten scopes
 /// like inline namespaces.
 std::string printQualifiedName(const NamedDecl &ND);
 
 /// Returns the first enclosing namespace scope starting from \p DC.
 std::string printNamespaceScope(const DeclContext &DC);
 
+/// Prints unqualified name of the decl for the purpose of displaying it to the
+/// user. Anonymous decls return names of the form "(anonymous {kind})", e.g.
+/// "(anonymous struct)" or "(anonymous namespace)".
+std::string printName(const ASTContext &Ctx, const NamedDecl &ND);
+
 /// Gets the symbol ID for a declaration, if possible.
 llvm::Optional<SymbolID> getSymbolID(const Decl *D);
 
 /// Gets the symbol ID for a macro, if possible.
 /// Currently, this is an encoded USR of the macro, which incorporates macro
 /// locations (e.g. file name, offset in file).
 /// FIXME: the USR semantics might not be stable enough as the ID for index
 /// macro (e.g. a change in definition offset can result in a different USR). We

clang-tools-extra/trunk/clangd/AST.cpp

 //===--- AST.cpp - Utility AST functions  -----------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "AST.h"
 
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/DeclTemplate.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Index/USRGeneration.h"
+#include "llvm/Support/Casting.h"
+#include "llvm/Support/ScopedPrinter.h"
 
 using namespace llvm;
 namespace clang {
 namespace clangd {
 
 // Returns true if the complete name of decl \p D is spelled in the source code.
 // This is not the case for symbols formed via macro concatenation.
 // (We used to attempt to treat names spelled on the command-line this way too,
std::string printQualifiedName(const NamedDecl &ND) {
   // namespaces to query: the preamble doesn't have a dedicated list.
   Policy.SuppressUnwrittenScope = true;
   ND.printQualifiedName(OS, Policy);
   OS.flush();
   assert(!StringRef(QName).startswith("::"));
   return QName;
 }
 
+static const TemplateArgumentList *
+getTemplateSpecializationArgs(const NamedDecl &ND) {
+  if (auto *Func = llvm::dyn_cast<FunctionDecl>(&ND))
+    return Func->getTemplateSpecializationArgs();
+  if (auto *Cls = llvm::dyn_cast<ClassTemplateSpecializationDecl>(&ND))
+    return &Cls->getTemplateInstantiationArgs();
+  if (auto *Var = llvm::dyn_cast<VarTemplateSpecializationDecl>(&ND))
+    return &Var->getTemplateInstantiationArgs();
+  return nullptr;
+}
+
+std::string printName(const ASTContext &Ctx, const NamedDecl &ND) {
+  std::string Name;
+  llvm::raw_string_ostream Out(Name);
+  PrintingPolicy PP(Ctx.getLangOpts());
+  // Handle 'using namespace'. They all have the same name - <using-directive>.
+  if (auto *UD = llvm::dyn_cast<UsingDirectiveDecl>(&ND)) {
+    Out << "using namespace ";
+    if (auto *Qual = UD->getQualifier())
+      Qual->print(Out, PP);
+    UD->getNominatedNamespaceAsWritten()->printName(Out);
+    return Out.str();
+  }
+  ND.getDeclName().print(Out, PP);
+  if (!Out.str().empty()) {
+    // FIXME(ibiryukov): do not show args not explicitly written by the user.
+    if (auto *ArgList = getTemplateSpecializationArgs(ND))
+      printTemplateArgumentList(Out, ArgList->asArray(), PP);
+    return Out.str();
+  }
+  // The name was empty, so present an anonymous entity.
+  if (auto *NS = llvm::dyn_cast<NamespaceDecl>(&ND))
+    return "(anonymous namespace)";
+  if (auto *Cls = llvm::dyn_cast<RecordDecl>(&ND))
+    return ("(anonymous " + Cls->getKindName() + ")").str();
+  if (auto *En = llvm::dyn_cast<EnumDecl>(&ND))
+    return "(anonymous enum)";
+  return "(anonymous)";
+}
+
 std::string printNamespaceScope(const DeclContext &DC) {
   for (const auto *Ctx = &DC; Ctx != nullptr; Ctx = Ctx->getParent())
     if (const auto *NS = dyn_cast<NamespaceDecl>(Ctx))
       if (!NS->isAnonymousNamespace() && !NS->isInlineNamespace())
         return printQualifiedName(*NS) + "::";
   return "";
 }
 

clang-tools-extra/trunk/clangd/ClangdLSPServer.h

private:
   void onDocumentDidChange(const DidChangeTextDocumentParams &);
   void onDocumentDidClose(const DidCloseTextDocumentParams &);
   void onDocumentOnTypeFormatting(const DocumentOnTypeFormattingParams &,
                                   Callback<std::vector<TextEdit>>);
   void onDocumentRangeFormatting(const DocumentRangeFormattingParams &,
                                  Callback<std::vector<TextEdit>>);
   void onDocumentFormatting(const DocumentFormattingParams &,
                             Callback<std::vector<TextEdit>>);
+  // The results are serialized 'vector<DocumentSymbol>' if
+  // SupportsHierarchicalDocumentSymbol is true and 'vector<SymbolInformation>'
+  // otherwise.
   void onDocumentSymbol(const DocumentSymbolParams &,
-                        Callback<std::vector<SymbolInformation>>);
+                        Callback<llvm::json::Value>);
   void onCodeAction(const CodeActionParams &, Callback<llvm::json::Value>);
   void onCompletion(const TextDocumentPositionParams &,
                     Callback<CompletionList>);
   void onSignatureHelp(const TextDocumentPositionParams &,
                        Callback<SignatureHelp>);
   void onGoToDefinition(const TextDocumentPositionParams &,
                         Callback<std::vector<Location>>);
   void onReference(const ReferenceParams &, Callback<std::vector<Location>>);
private:
   /// Options used for diagnostics.
   ClangdDiagnosticOptions DiagOpts;
   /// The supported kinds of the client.
   SymbolKindBitset SupportedSymbolKinds;
   /// The supported completion item kinds of the client.
   CompletionItemKindBitset SupportedCompletionItemKinds;
   // Whether the client supports CodeAction response objects.
   bool SupportsCodeAction = false;
+  /// From capabilities of textDocument/documentSymbol.
+  bool SupportsHierarchicalDocumentSymbol = false;
 
   // Store of the current versions of the open documents.
   DraftStore DraftMgr;
 
   // The CDB is created by the "initialize" LSP method.
   bool UseDirBasedCDB;                     // FIXME: make this a capability.
   llvm::Optional<Path> CompileCommandsDir; // FIXME: merge with capability?
   std::unique_ptr<GlobalCompilationDatabase> BaseCDB;

clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp

 #include "llvm/Support/Path.h"
 #include "llvm/Support/ScopedPrinter.h"
 
 using namespace llvm;
 namespace clang {
 namespace clangd {
 namespace {
 
+void adjustSymbolKinds(llvm::MutableArrayRef<DocumentSymbol> Syms,
+                       SymbolKindBitset Kinds) {
+  for (auto &S : Syms) {
+    S.kind = adjustKindToCapability(S.kind, Kinds);
+    adjustSymbolKinds(S.children, Kinds);
+  }
+}
+
 SymbolKindBitset defaultSymbolKinds() {
   SymbolKindBitset Defaults;
   for (size_t I = SymbolKindMin; I <= static_cast<size_t>(SymbolKind::Array);
        ++I)
     Defaults.set(I);
   return Defaults;
 }
 
void ClangdLSPServer::onInitialize(const InitializeParams &Params,
   CCOpts.EnableSnippets = Params.capabilities.CompletionSnippets;
   DiagOpts.EmbedFixesInDiagnostics = Params.capabilities.DiagnosticFixes;
   DiagOpts.SendDiagnosticCategory = Params.capabilities.DiagnosticCategory;
   if (Params.capabilities.WorkspaceSymbolKinds)
     SupportedSymbolKinds |= *Params.capabilities.WorkspaceSymbolKinds;
   if (Params.capabilities.CompletionItemKinds)
     SupportedCompletionItemKinds |= *Params.capabilities.CompletionItemKinds;
   SupportsCodeAction = Params.capabilities.CodeActionStructure;
+  SupportsHierarchicalDocumentSymbol =
+      Params.capabilities.HierarchicalDocumentSymbol;
 
   Reply(json::Object{
       {{"capabilities",
         json::Object{
             {"textDocumentSync", (int)TextDocumentSyncKind::Incremental},
             {"documentFormattingProvider", true},
             {"documentRangeFormattingProvider", true},
             {"documentOnTypeFormattingProvider",
void ClangdLSPServer::onDocumentFormatting(
 
   auto ReplacementsOrError = Server->formatFile(*Code, File);
   if (ReplacementsOrError)
     Reply(replacementsToEdits(*Code, ReplacementsOrError.get()));
   else
     Reply(ReplacementsOrError.takeError());
 }
 
-void ClangdLSPServer::onDocumentSymbol(
-    const DocumentSymbolParams &Params,
-    Callback<std::vector<SymbolInformation>> Reply) {
+/// The functions constructs a flattened view of the DocumentSymbol hierarchy.
+/// Used by the clients that do not support the hierarchical view.
+static std::vector<SymbolInformation>
+flattenSymbolHierarchy(llvm::ArrayRef<DocumentSymbol> Symbols,
+                       const URIForFile &FileURI) {
+
+  std::vector<SymbolInformation> Results;
+  std::function<void(const DocumentSymbol &, StringRef)> Process =
+      [&](const DocumentSymbol &S, Optional<StringRef> ParentName) {
+        SymbolInformation SI;
+        SI.containerName = ParentName ? "" : *ParentName;
+        SI.name = S.name;
+        SI.kind = S.kind;
+        SI.location.range = S.range;
+        SI.location.uri = FileURI;
+
+        Results.push_back(std::move(SI));
+        std::string FullName =
+            !ParentName ? S.name : (ParentName->str() + "::" + S.name);
+        for (auto &C : S.children)
+          Process(C, /*ParentName=*/FullName);
+      };
+  for (auto &S : Symbols)
+    Process(S, /*ParentName=*/"");
+  return Results;
+}
+
+void ClangdLSPServer::onDocumentSymbol(const DocumentSymbolParams &Params,
+                                       Callback<json::Value> Reply) {
+  URIForFile FileURI = Params.textDocument.uri;
   Server->documentSymbols(
       Params.textDocument.uri.file(),
       Bind(
-          [this](decltype(Reply) Reply,
-                 Expected<std::vector<SymbolInformation>> Items) {
+          [this, FileURI](decltype(Reply) Reply,
+                          Expected<std::vector<DocumentSymbol>> Items) {
             if (!Items)
               return Reply(Items.takeError());
-            for (auto &Sym : *Items)
-              Sym.kind = adjustKindToCapability(Sym.kind, SupportedSymbolKinds);
-            Reply(std::move(*Items));
+            adjustSymbolKinds(*Items, SupportedSymbolKinds);
+            if (SupportsHierarchicalDocumentSymbol)
+              return Reply(std::move(*Items));
+            else
+              return Reply(flattenSymbolHierarchy(*Items, FileURI));
           },
           std::move(Reply)));
 }
 
 static Optional<Command> asCommand(const CodeAction &Action) {
   Command Cmd;
   if (Action.command && Action.edit)
     return None; // Not representable. (We never emit these anyway).

clang-tools-extra/trunk/clangd/ClangdServer.h

void findHover(PathRef File, Position Pos,
                  Callback<llvm::Optional<Hover>> CB);
 
   /// Retrieve the top symbols from the workspace matching a query.
   void workspaceSymbols(StringRef Query, int Limit,
                         Callback<std::vector<SymbolInformation>> CB);
 
   /// Retrieve the symbols within the specified file.
   void documentSymbols(StringRef File,
-                       Callback<std::vector<SymbolInformation>> CB);
+                       Callback<std::vector<DocumentSymbol>> CB);
 
   /// Retrieve locations for symbol references.
   void findReferences(PathRef File, Position Pos,
                       Callback<std::vector<Location>> CB);
 
   /// Run formatting for \p Rng inside \p File with content \p Code.
   llvm::Expected<tooling::Replacements> formatRange(StringRef Code,
                                                     PathRef File, Range Rng);

clang-tools-extra/trunk/clangd/ClangdServer.cpp

WorkScheduler.run(
       Bind(
           [QueryCopy, Limit, this](decltype(CB) CB) {
             CB(clangd::getWorkspaceSymbols(QueryCopy, Limit, Index,
                                            WorkspaceRoot.getValueOr("")));
           },
           std::move(CB)));
 }
 
-void ClangdServer::documentSymbols(
-    StringRef File, Callback<std::vector<SymbolInformation>> CB) {
-  auto Action = [](Callback<std::vector<SymbolInformation>> CB,
-                   Expected<InputsAndAST> InpAST) {
+void ClangdServer::documentSymbols(StringRef File,
+                                   Callback<std::vector<DocumentSymbol>> CB) {
+  auto Action = [](Callback<std::vector<DocumentSymbol>> CB,
+                   llvm::Expected<InputsAndAST> InpAST) {
     if (!InpAST)
       return CB(InpAST.takeError());
     CB(clangd::getDocumentSymbols(InpAST->AST));
   };
   WorkScheduler.runWithAST("documentSymbols", File,
                            Bind(Action, std::move(CB)));
 }
 

clang-tools-extra/trunk/clangd/FindSymbols.h

 /// \p HintPath This is used when resolving URIs. If empty, URI resolution can
 /// fail if a hint path is required for the scheme of a specific URI.
 llvm::Expected<std::vector<SymbolInformation>>
 getWorkspaceSymbols(llvm::StringRef Query, int Limit,
                     const SymbolIndex *const Index, llvm::StringRef HintPath);
 
 /// Retrieves the symbols contained in the "main file" section of an AST in the
 /// same order that they appear.
-llvm::Expected<std::vector<SymbolInformation>>
-getDocumentSymbols(ParsedAST &AST);
+llvm::Expected<std::vector<DocumentSymbol>> getDocumentSymbols(ParsedAST &AST);
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clang-tools-extra/trunk/clangd/FindSymbols.cpp

 
 #include "AST.h"
 #include "ClangdUnit.h"
 #include "FuzzyMatch.h"
 #include "Logger.h"
 #include "Quality.h"
 #include "SourceCode.h"
 #include "index/Index.h"
+#include "clang/AST/DeclTemplate.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexSymbol.h"
 #include "clang/Index/IndexingAction.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/Path.h"
+#include "llvm/Support/ScopedPrinter.h"
 
 #define DEBUG_TYPE "FindSymbols"
 
 using namespace llvm;
 namespace clang {
 namespace clangd {
 namespace {
 
Index->fuzzyFind(Req, [HintPath, &Top, &Filter](const Symbol &Sym) {
     Top.push({Score, std::move(Info)});
   });
   for (auto &R : std::move(Top).items())
     Result.push_back(std::move(R.second));
   return Result;
 }
 
 namespace {
-/// Finds document symbols in the main file of the AST.
-class DocumentSymbolsConsumer : public index::IndexDataConsumer {
-  ASTContext *
-  std::vector<SymbolInformation> Symbols;
-  // We are always list document for the same file, so cache the value.
-  Optional<URIForFile> MainFileUri;
+llvm::Optional<DocumentSymbol> declToSym(ASTContext &Ctx, const NamedDecl &ND) {
+  auto &SM = Ctx.getSourceManager();
 
+  SourceLocation NameLoc = findNameLoc(&ND);
+  // getFileLoc is a good choice for us, but we also need to make sure
+  // sourceLocToPosition won't switch files, so we call getSpellingLoc on top of
+  // that to make sure it does not switch files.
+  // FIXME: sourceLocToPosition should not switch files!
+  SourceLocation BeginLoc =  SM.getSpellingLoc(SM.getFileLoc(ND.getBeginLoc()));
+  SourceLocation EndLoc = SM.getSpellingLoc(SM.getFileLoc(ND.getEndLoc()));
+  if (NameLoc.isInvalid() || BeginLoc.isInvalid() || EndLoc.isInvalid())
+    return llvm::None;
+
+  if (!SM.isWrittenInMainFile(NameLoc) || !SM.isWrittenInMainFile(BeginLoc) ||
+      !SM.isWrittenInMainFile(EndLoc))
+    return llvm::None;
+
+  Position NameBegin = sourceLocToPosition(SM, NameLoc);
+  Position NameEnd = sourceLocToPosition(
+      SM, Lexer::getLocForEndOfToken(NameLoc, 0, SM, Ctx.getLangOpts()));
+
+  index::SymbolInfo SymInfo = index::getSymbolInfo(&ND);
+  // FIXME: this is not classifying constructors, destructors and operators
+  //        correctly (they're all "methods").
+  SymbolKind SK = indexSymbolKindToSymbolKind(SymInfo.Kind);
+
+  DocumentSymbol SI;
+  SI.name = printName(Ctx, ND);
+  SI.kind = SK;
+  SI.deprecated = ND.isDeprecated();
+  SI.range =
+      Range{sourceLocToPosition(SM, BeginLoc), sourceLocToPosition(SM, EndLoc)};
+  SI.selectionRange = Range{NameBegin, NameEnd};
+  if (!SI.range.contains(SI.selectionRange)) {
+    // 'selectionRange' must be contained in 'range', so in cases where clang
+    // reports unrelated ranges we need to reconcile somehow.
+    SI.range = SI.selectionRange;
+  }
+  return SI;
+}
+
+/// A helper class to build an outline for the parse AST. It traverse the AST
+/// directly instead of using RecursiveASTVisitor (RAV) for three main reasons:
+///    - there is no way to keep RAV from traversing subtrees we're not
+///      interested in. E.g. not traversing function locals or implicit template
+///      instantiations.
+///    - it's easier to combine results of recursive passes, e.g.
+///    - visiting decls is actually simple, so we don't hit the complicated
+///      cases that RAV mostly helps with (types and expressions, etc.)
+class DocumentOutline {
 public:
-  DocumentSymbolsConsumer(ASTContext &AST) : AST(AST) {}
-  std::vector<SymbolInformation> takeSymbols() { return std::move(Symbols); }
+  DocumentOutline(ParsedAST &AST) : AST(AST) {}
 
-  void initialize(ASTContext &Ctx) override {
-    // Compute the absolute path of the main file which we will use for all
-    // results.
-    const SourceManager &SM = AST.getSourceManager();
-    const FileEntry *F = SM.getFileEntryForID(SM.getMainFileID());
-    if (!F)
+  /// Builds the document outline for the generated AST.
+  std::vector<DocumentSymbol> build() {
+    std::vector<DocumentSymbol> Results;
+    for (auto &TopLevel : AST.getLocalTopLevelDecls())
+      traverseDecl(TopLevel, Results);
+    return Results;
+  }
+
+private:
+  enum class VisitKind { No, OnlyDecl, DeclAndChildren };
+
+  void traverseDecl(Decl *D, std::vector<DocumentSymbol> &Results) {
+    if (auto *Templ = llvm::dyn_cast<TemplateDecl>(D))
+      D = Templ->getTemplatedDecl();
+    auto *ND = llvm::dyn_cast<NamedDecl>(D);
+    if (!ND)
       return;
-    auto FilePath = getRealPath(F, SM);
-    if (FilePath)
-      MainFileUri = URIForFile(*FilePath);
-  }
-
-  bool shouldIncludeSymbol(const NamedDecl *ND) {
-    if (!ND || ND->isImplicit())
-      return false;
-    // Skip anonymous declarations, e.g (anonymous enum/class/struct).
-    if (ND->getDeclName().isEmpty())
-      return false;
-    return true;
-  }
-
-  bool
-  handleDeclOccurence(const Decl *, index::SymbolRoleSet Roles,
-                      ArrayRef<index::SymbolRelation> Relations,
-                      SourceLocation Loc,
-                      index::IndexDataConsumer::ASTNodeInfo ASTNode) override {
-    assert(ASTNode.OrigD);
-    // No point in continuing the index consumer if we could not get the
-    // absolute path of the main file.
-    if (!MainFileUri)
-      return false;
-    // We only want declarations and definitions, i.e. no references.
-    if (!(Roles & static_cast<unsigned>(index::SymbolRole::Declaration) ||
-          Roles & static_cast<unsigned>(index::SymbolRole::Definition)))
-      return true;
-    SourceLocation NameLoc = findNameLoc(ASTNode.OrigD);
-    const SourceManager &SourceMgr = AST.getSourceManager();
-    // We should be only be looking at "local" decls in the main file.
-    if (!SourceMgr.isWrittenInMainFile(NameLoc)) {
-      // Even thought we are visiting only local (non-preamble) decls,
-      // we can get here when in the presence of "extern" decls.
-      return true;
-    }
-    const NamedDecl *ND = dyn_cast<NamedDecl>(ASTNode.OrigD);
-    if (!shouldIncludeSymbol(ND))
-      return true;
-
-    SourceLocation EndLoc =
-        Lexer::getLocForEndOfToken(NameLoc, 0, SourceMgr, AST.getLangOpts());
-    Position Begin = sourceLocToPosition(SourceMgr, NameLoc);
-    Position End = sourceLocToPosition(SourceMgr, EndLoc);
-    Range R = {Begin, End};
-    Location L;
-    L.uri = *MainFileUri;
-    L.range = R;
-
-    std::string QName = printQualifiedName(*ND);
-    StringRef Scope, Name;
-    std::tie(Scope, Name) = splitQualifiedName(QName);
-    Scope.consume_back("::");
+    VisitKind Visit = shouldVisit(ND);
+    if (Visit == VisitKind::No)
+      return;
+    llvm::Optional<DocumentSymbol> Sym = declToSym(AST.getASTContext(), *ND);
+    if (!Sym)
+      return;
+    if (Visit == VisitKind::DeclAndChildren)
+      traverseChildren(D, Sym->children);
+    Results.push_back(std::move(*Sym));
+  }
 
-    index::SymbolInfo SymInfo = index::getSymbolInfo(ND);
-    SymbolKind SK = indexSymbolKindToSymbolKind(SymInfo.Kind);
+  void traverseChildren(Decl *D, std::vector<DocumentSymbol> &Results) {
+    auto *Scope = llvm::dyn_cast<DeclContext>(D);
+    if (!Scope)
+      return;
+    for (auto *C : Scope->decls())
+      traverseDecl(C, Results);
+  }
 
-    SymbolInformation SI;
-    SI.name = Name;
-    SI.kind = SK;
-    SI.location = L;
-    SI.containerName = Scope;
-    Symbols.push_back(std::move(SI));
-    return true;
+  VisitKind shouldVisit(NamedDecl *D) {
+    if (D->isImplicit())
+      return VisitKind::No;
+
+    if (auto Func = llvm::dyn_cast<FunctionDecl>(D)) {
+      // Some functions are implicit template instantiations, those should be
+      // ignored.
+      if (auto *Info = Func->getTemplateSpecializationInfo()) {
+        if (!Info->isExplicitInstantiationOrSpecialization())
+          return VisitKind::No;
+      }
+      // Only visit the function itself, do not visit the children (i.e.
+      // function parameters, etc.)
+      return VisitKind::OnlyDecl;
+    }
+    // Handle template instantiations. We have three cases to consider:
+    //   - explicit instantiations, e.g. 'template class std::vector<int>;'
+    //     Visit the decl itself (it's present in the code), but not the
+    //     children.
+    //   - implicit instantiations, i.e. not written by the user.
+    //     Do not visit at all, they are not present in the code.
+    //   - explicit specialization, e.g. 'template <> class vector<bool> {};'
+    //     Visit both the decl and its children, both are written in the code.
+    if (auto *TemplSpec = llvm::dyn_cast<ClassTemplateSpecializationDecl>(D)) {
+      if (TemplSpec->isExplicitInstantiationOrSpecialization())
+        return TemplSpec->isExplicitSpecialization()
+                   ? VisitKind::DeclAndChildren
+                   : VisitKind::OnlyDecl;
+      return VisitKind::No;
+    }
+    if (auto *TemplSpec = llvm::dyn_cast<VarTemplateSpecializationDecl>(D)) {
+      if (TemplSpec->isExplicitInstantiationOrSpecialization())
+        return TemplSpec->isExplicitSpecialization()
+                   ? VisitKind::DeclAndChildren
+                   : VisitKind::OnlyDecl;
+      return VisitKind::No;
+    }
+    // For all other cases, visit both the children and the decl.
+    return VisitKind::DeclAndChildren;
   }
-};
-} // namespace
 
-Expected<std::vector<SymbolInformation>> getDocumentSymbols(ParsedAST &AST) {
-  DocumentSymbolsConsumer DocumentSymbolsCons(AST.getASTContext());
+  ParsedAST &AST;
+};
 
-  index::IndexingOptions IndexOpts;
-  IndexOpts.SystemSymbolFilter =
-      index::IndexingOptions::SystemSymbolFilterKind::DeclarationsOnly;
-  IndexOpts.IndexFunctionLocals = false;
-  indexTopLevelDecls(AST.getASTContext(), AST.getPreprocessor(),
-                     AST.getLocalTopLevelDecls(), DocumentSymbolsCons,
-                     IndexOpts);
+std::vector<DocumentSymbol> collectDocSymbols(ParsedAST &AST) {
+  return DocumentOutline(AST).build();
+}
+} // namespace
 
-  return DocumentSymbolsCons.takeSymbols();
+llvm::Expected<std::vector<DocumentSymbol>> getDocumentSymbols(ParsedAST &AST) {
+  return collectDocSymbols(AST);
 }
 
 } // namespace clangd
 } // namespace clang

clang-tools-extra/trunk/clangd/Protocol.h

struct Range {
   friend bool operator!=(const Range &LHS, const Range &RHS) {
     return !(LHS == RHS);
   }
   friend bool operator<(const Range &LHS, const Range &RHS) {
     return std::tie(LHS.start, LHS.end) < std::tie(RHS.start, RHS.end);
   }
 
   bool contains(Position Pos) const { return start <= Pos && Pos < end; }
+  bool contains(Range Rng) const {
+    return start <= Rng.start && Rng.end <= end;
+  }
 };
 bool fromJSON(const llvm::json::Value &, Range &);
 llvm::json::Value toJSON(const Range &);
 llvm::raw_ostream &operator<<(llvm::raw_ostream &, const Range &);
 
 struct Location {
   /// The text document's URI.
   URIForFile uri;
struct ClientCapabilities {
   /// using the "category" extension.
   /// textDocument.publishDiagnostics.categorySupport
   bool DiagnosticCategory = false;
 
   /// Client supports snippets as insert text.
   /// textDocument.completion.completionItem.snippetSupport
   bool CompletionSnippets = false;
 
+  /// Client supports hierarchical document symbols.
+  bool HierarchicalDocumentSymbol = false;
+
   /// The supported set of CompletionItemKinds for textDocument/completion.
   /// textDocument.completion.completionItemKind.valueSet
   llvm::Optional<CompletionItemKindBitset> CompletionItemKinds;
 
   /// Client supports CodeAction return value for textDocument/codeAction.
   /// textDocument.codeAction.codeActionLiteralSupport.
   bool CodeActionStructure = false;
 };
struct CodeAction {
   llvm::Optional<WorkspaceEdit> edit;
 
   /// A command this code action executes. If a code action provides an edit
   /// and a command, first the edit is executed and then the command.
   llvm::Optional<Command> command;
 };
 llvm::json::Value toJSON(const CodeAction &);
 
+/// Represents programming constructs like variables, classes, interfaces etc.
+/// that appear in a document. Document symbols can be hierarchical and they
+/// have two ranges: one that encloses its definition and one that points to its
+/// most interesting range, e.g. the range of an identifier.
+struct DocumentSymbol {
+  /// The name of this symbol.
+  std::string name;
+
+  /// More detail for this symbol, e.g the signature of a function.
+  std::string detail;
+
+  /// The kind of this symbol.
+  SymbolKind kind;
+
+  /// Indicates if this symbol is deprecated.
+  bool deprecated;
+
+  /// The range enclosing this symbol not including leading/trailing whitespace
+  /// but everything else like comments. This information is typically used to
+  /// determine if the clients cursor is inside the symbol to reveal in the
+  /// symbol in the UI.
+  Range range;
+
+  /// The range that should be selected and revealed when this symbol is being
+  /// picked, e.g the name of a function. Must be contained by the `range`.
+  Range selectionRange;
+
+  /// Children of this symbol, e.g. properties of a class.
+  std::vector<DocumentSymbol> children;
+};
+llvm::raw_ostream &operator<<(llvm::raw_ostream &O, const DocumentSymbol &S);
+llvm::json::Value toJSON(const DocumentSymbol &S);
+
 /// Represents information about programming constructs like variables, classes,
 /// interfaces etc.
 struct SymbolInformation {
   /// The name of this symbol.
   std::string name;
 
   /// The kind of this symbol.
   SymbolKind kind;

clang-tools-extra/trunk/clangd/Protocol.cpp

 
 #include "Protocol.h"
 #include "Logger.h"
 #include "URI.h"
 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 namespace clang {
 namespace clangd {
 
 char LSPError::ID;
if (auto *Completion = TextDocument->getObject("completion")) {
             return false;
         }
       }
     }
     if (auto *CodeAction = TextDocument->getObject("codeAction")) {
       if (CodeAction->getObject("codeActionLiteralSupport"))
         R.CodeActionStructure = true;
     }
+    if (auto *DocumentSymbol = TextDocument->getObject("documentSymbol")) {
+      if (auto HierarchicalSupport =
+              DocumentSymbol->getBoolean("hierarchicalDocumentSymbolSupport"))
+        R.HierarchicalDocumentSymbol = *HierarchicalSupport;
+    }
   }
   if (auto *Workspace = O->getObject("workspace")) {
     if (auto *Symbol = Workspace->getObject("symbol")) {
       if (auto *SymbolKind = Symbol->getObject("symbolKind")) {
         if (auto *ValueSet = SymbolKind->get("valueSet")) {
           R.WorkspaceSymbolKinds.emplace();
           if (!fromJSON(*ValueSet, *R.WorkspaceSymbolKinds))
             return false;
if (CA.diagnostics)
     CodeAction["diagnostics"] = json::Array(*CA.diagnostics);
   if (CA.edit)
     CodeAction["edit"] = *CA.edit;
   if (CA.command)
     CodeAction["command"] = *CA.command;
   return std::move(CodeAction);
 }
 
+llvm::raw_ostream &operator<<(llvm::raw_ostream &O, const DocumentSymbol &S) {
+  return O << S.name << " - " << toJSON(S);
+}
+
+llvm::json::Value toJSON(const DocumentSymbol &S) {
+  json::Object Result{{"name", S.name},
+                      {"kind", static_cast<int>(S.kind)},
+                      {"range", S.range},
+                      {"selectionRange", S.selectionRange}};
+
+  if (!S.detail.empty())
+    Result["detail"] = S.detail;
+  if (!S.children.empty())
+    Result["children"] = S.children;
+  if (S.deprecated)
+    Result["deprecated"] = true;
+  return Result;
+}
+
 json::Value toJSON(const WorkspaceEdit &WE) {
   if (!WE.changes)
     return json::Object{};
   json::Object FileChanges;
   for (auto &Change : *WE.changes)
     FileChanges[Change.first] = json::Array(Change.second);
   return json::Object{{"changes", std::move(FileChanges)}};
 }

clang-tools-extra/trunk/clangd/clients/clangd-vscode/package.json

 {
     "name": "vscode-clangd",
     "displayName": "vscode-clangd",
     "description": "Clang Language Server",
     "version": "0.0.6",
     "publisher": "llvm-vs-code-extensions",
     "homepage": "https://clang.llvm.org/extra/clangd.html",
     "engines": {
-        "vscode": "^1.18.0"
+        "vscode": "^1.27.0"
     },
     "categories": [
         "Programming Languages",
         "Linters",
         "Snippets"
     ],
     "keywords": [
         "C",

     "main": "./out/src/extension",
     "scripts": {
         "vscode:prepublish": "tsc -p ./",
         "compile": "tsc -watch -p ./",
         "postinstall": "node ./node_modules/vscode/bin/install",
         "test": "node ./node_modules/vscode/bin/test"
     },
     "dependencies": {
-        "vscode-languageclient": "^4.0.0",
-        "vscode-languageserver": "^4.0.0"
+        "vscode-languageclient": "^5.1.0",
+        "vscode-languageserver": "^5.1.0"
     },
     "devDependencies": {
         "typescript": "^2.0.3",
         "vscode": "^1.1.0",
         "mocha": "^2.3.3",
         "@types/node": "^6.0.40",
         "@types/mocha": "^2.2.32"
     },

clang-tools-extra/trunk/unittests/clangd/FindSymbolsTests.cpp

 //===-- FindSymbolsTests.cpp -------------------------*- C++ -*------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 #include "Annotations.h"
 #include "ClangdServer.h"
 #include "FindSymbols.h"
 #include "SyncAPI.h"
 #include "TestFS.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
+using namespace llvm;
+
 namespace clang {
 namespace clangd {
 
 namespace {
 
 using ::testing::AllOf;
 using ::testing::AnyOf;
 using ::testing::ElementsAre;
 using ::testing::ElementsAreArray;
+using ::testing::Field;
 using ::testing::IsEmpty;
 using ::testing::UnorderedElementsAre;
 
 class IgnoreDiagnostics : public DiagnosticsConsumer {
   void onDiagnosticsReady(PathRef File,
                           std::vector<Diag> Diagnostics) override {}
 };
 
 // GMock helpers for matching SymbolInfos items.
 MATCHER_P(QName, Name, "") {
   if (arg.containerName.empty())
     return arg.name == Name;
   return (arg.containerName + "::" + arg.name) == Name;
 }
+MATCHER_P(WithName, N, "") { return arg.name == N; }
 MATCHER_P(WithKind, Kind, "") { return arg.kind == Kind; }
 MATCHER_P(SymRange, Range, "") { return arg.location.range == Range; }
 
+// GMock helpers for matching DocumentSymbol.
+MATCHER_P(SymNameRange, Range, "") { return arg.selectionRange == Range; }
+template <class... ChildMatchers>
+testing::Matcher<DocumentSymbol> Children(ChildMatchers... ChildrenM) {
+  return Field(&DocumentSymbol::children, ElementsAre(ChildrenM...));
+}
+
 ClangdServer::Options optsForTests() {
   auto ServerOpts = ClangdServer::optsForTest();
   ServerOpts.WorkspaceRoot = testRoot();
   ServerOpts.BuildDynamicSymbolIndex = true;
   return ServerOpts;
 }
 
 class WorkspaceSymbolsTest : public ::testing::Test {
DocumentSymbolsTest()
       : Server(CDB, FSProvider, DiagConsumer, optsForTests()) {}
 
 protected:
   MockFSProvider FSProvider;
   MockCompilationDatabase CDB;
   IgnoreDiagnostics DiagConsumer;
   ClangdServer Server;
 
-  std::vector<SymbolInformation> getSymbols(PathRef File) {
+  std::vector<DocumentSymbol> getSymbols(PathRef File) {
     EXPECT_TRUE(Server.blockUntilIdleForTest()) << "Waiting for preamble";
     auto SymbolInfos = runDocumentSymbols(Server, File);
     EXPECT_TRUE(bool(SymbolInfos)) << "documentSymbols returned an error";
     return *SymbolInfos;
   }
 
   void addFile(StringRef FilePath, StringRef Contents) {
     FSProvider.Files[FilePath] = Contents;
Annotations Main(R"(
       // FIXME: using declaration is not supported as the IndexAction will ignore
       // implicit declarations (the implicit using shadow declaration) by default,
       // and there is no way to customize this behavior at the moment.
       using bar::v2;
       } // namespace foo
     )");
 
   addFile(FilePath, Main.code());
-  EXPECT_THAT(getSymbols(FilePath),
-              ElementsAreArray(
-                  {AllOf(QName("Foo"), WithKind(SymbolKind::Class)),
-                   AllOf(QName("Foo"), WithKind(SymbolKind::Class)),
-                   AllOf(QName("Foo::Foo"), WithKind(SymbolKind::Method)),
-                   AllOf(QName("Foo::Foo"), WithKind(SymbolKind::Method)),
-                   AllOf(QName("Foo::f"), WithKind(SymbolKind::Method)),
-                   AllOf(QName("f1"), WithKind(SymbolKind::Function)),
-                   AllOf(QName("Foo::operator="), WithKind(SymbolKind::Method)),
-                   AllOf(QName("Foo::~Foo"), WithKind(SymbolKind::Method)),
-                   AllOf(QName("Foo::Nested"), WithKind(SymbolKind::Class)),
-                   AllOf(QName("Foo::Nested::f"), WithKind(SymbolKind::Method)),
-                   AllOf(QName("Friend"), WithKind(SymbolKind::Class)),
-                   AllOf(QName("f1"), WithKind(SymbolKind::Function)),
-                   AllOf(QName("f2"), WithKind(SymbolKind::Function)),
-                   AllOf(QName("KInt"), WithKind(SymbolKind::Variable)),
-                   AllOf(QName("kStr"), WithKind(SymbolKind::Variable)),
-                   AllOf(QName("f1"), WithKind(SymbolKind::Function)),
-                   AllOf(QName("foo"), WithKind(SymbolKind::Namespace)),
-                   AllOf(QName("foo::int32"), WithKind(SymbolKind::Class)),
-                   AllOf(QName("foo::int32_t"), WithKind(SymbolKind::Class)),
-                   AllOf(QName("foo::v1"), WithKind(SymbolKind::Variable)),
-                   AllOf(QName("foo::bar"), WithKind(SymbolKind::Namespace)),
-                   AllOf(QName("foo::bar::v2"), WithKind(SymbolKind::Variable)),
-                   AllOf(QName("foo::baz"), WithKind(SymbolKind::Namespace))}));
+  EXPECT_THAT(
+      getSymbols(FilePath),
+          {AllOf(WithName("Foo"), WithKind(SymbolKind::Class), Children()),
+           AllOf(WithName("Foo"), WithKind(SymbolKind::Class),
+                 Children(AllOf(WithName("Foo"), WithKind(SymbolKind::Method),
+                                Children()),
+                          AllOf(WithName("Foo"), WithKind(SymbolKind::Method),
+                                Children()),
+                          AllOf(WithName("f"), WithKind(SymbolKind::Method),
+                                Children()),
+                          AllOf(WithName("operator="),
+                                WithKind(SymbolKind::Method), Children()),
+                          AllOf(WithName("~Foo"), WithKind(SymbolKind::Method),
+                                Children()),
+                          AllOf(WithName("Nested"), WithKind(SymbolKind::Class),
+                                Children(AllOf(WithName("f"),
+                                               WithKind(SymbolKind::Method),
+                                               Children()))))),
+           AllOf(WithName("Friend"), WithKind(SymbolKind::Class), Children()),
+           AllOf(WithName("f1"), WithKind(SymbolKind::Function), Children()),
+           AllOf(WithName("f2"), WithKind(SymbolKind::Function), Children()),
+           AllOf(WithName("KInt"), WithKind(SymbolKind::Variable), Children()),
+           AllOf(WithName("kStr"), WithKind(SymbolKind::Variable), Children()),
+           AllOf(WithName("f1"), WithKind(SymbolKind::Function), Children()),
+           AllOf(WithName("foo"), WithKind(SymbolKind::Namespace),
+                 Children(
+                     AllOf(WithName("int32"), WithKind(SymbolKind::Class),
+                           Children()),
+                     AllOf(WithName("int32_t"), WithKind(SymbolKind::Class),
+                           Children()),
+                     AllOf(WithName("v1"), WithKind(SymbolKind::Variable),
+                           Children()),
+                     AllOf(WithName("bar"), WithKind(SymbolKind::Namespace),
+                           Children(AllOf(WithName("v2"),
+                                          WithKind(SymbolKind::Variable),
+                                          Children()))),
+                     AllOf(WithName("baz"), WithKind(SymbolKind::Namespace),
+                           Children()),
+                     AllOf(WithName("v2"), WithKind(SymbolKind::Variable))))}));
 }
 
 TEST_F(DocumentSymbolsTest, DeclarationDefinition) {
   std::string FilePath = testPath("foo.cpp");
   Annotations Main(R"(
       class Foo {
         void $decl[[f]]();
       };
       void Foo::$def[[f]]() {
       }
     )");
 
   addFile(FilePath, Main.code());
   EXPECT_THAT(getSymbols(FilePath),
-              ElementsAre(AllOf(QName("Foo"), WithKind(SymbolKind::Class)),
-                          AllOf(QName("Foo::f"), WithKind(SymbolKind::Method),
-                                SymRange(Main.range("decl"))),
-                          AllOf(QName("Foo::f"), WithKind(SymbolKind::Method),
-                                SymRange(Main.range("def")))));
+              ElementsAre(AllOf(WithName("Foo"), WithKind(SymbolKind::Class),
+                                Children(AllOf(
+                                    WithName("f"), WithKind(SymbolKind::Method),
+                                    SymNameRange(Main.range("decl"))))),
+                          AllOf(WithName("f"), WithKind(SymbolKind::Method),
+                                SymNameRange(Main.range("def")))));
 }
 
 TEST_F(DocumentSymbolsTest, ExternSymbol) {
   std::string FilePath = testPath("foo.cpp");
   addFile(testPath("foo.h"), R"cpp(
       extern int var = 2;
       )cpp");
   addFile(FilePath, R"cpp(
       #include "foo.h"
       )cpp");
 
   EXPECT_THAT(getSymbols(FilePath), IsEmpty());
 }
 
 TEST_F(DocumentSymbolsTest, NoLocals) {
   std::string FilePath = testPath("foo.cpp");
   addFile(FilePath,
           R"cpp(
       void test(int FirstParam, int SecondParam) {
         struct LocalClass {};
         int local_var;
       })cpp");
-  EXPECT_THAT(getSymbols(FilePath), ElementsAre(QName("test")));
+  EXPECT_THAT(getSymbols(FilePath), ElementsAre(WithName("test")));
 }
 
 TEST_F(DocumentSymbolsTest, Unnamed) {
   std::string FilePath = testPath("foo.h");
   addFile(FilePath,
           R"cpp(
       struct {
         int InUnnamed;
       } UnnamedStruct;
       )cpp");
   EXPECT_THAT(
       getSymbols(FilePath),
-      ElementsAre(AllOf(QName("UnnamedStruct"), WithKind(SymbolKind::Variable)),
-                  AllOf(QName("(anonymous struct)::InUnnamed"),
-                        WithKind(SymbolKind::Field))));
+      ElementsAre(
+          AllOf(WithName("(anonymous struct)"), WithKind(SymbolKind::Struct),
+                Children(AllOf(WithName("InUnnamed"),
+                               WithKind(SymbolKind::Field), Children()))),
+          AllOf(WithName("UnnamedStruct"), WithKind(SymbolKind::Variable),
+                Children())));
 }
 
 TEST_F(DocumentSymbolsTest, InHeaderFile) {
   addFile("bar.h", R"cpp(
       int foo() {
       }
       )cpp");
   std::string FilePath = testPath("foo.h");
   addFile(FilePath, R"cpp(
       #include "bar.h"
       int test() {
       }
       )cpp");
   addFile("foo.cpp", R"cpp(
       #include "foo.h"
       )cpp");
-  EXPECT_THAT(getSymbols(FilePath), ElementsAre(QName("test")));
+  EXPECT_THAT(getSymbols(FilePath), ElementsAre(WithName("test")));
 }
 
 TEST_F(DocumentSymbolsTest, Template) {
   std::string FilePath = testPath("foo.cpp");
   addFile(FilePath, R"(
-    // Primary templates and specializations are included but instantiations
-    // are not.
     template <class T> struct Tmpl {T x = 0;};
-    template <> struct Tmpl<int> {};
+    template <> struct Tmpl<int> {
+      int y = 0;
+    };
     extern template struct Tmpl<float>;
     template struct Tmpl<double>;
+
+    template <class T, class U, class Z = float>
+    int funcTmpl(U a);
+    template <>
+    int funcTmpl<int>(double a);
+
+    template <class T, class U = double>
+    int varTmpl = T();
+    template <>
+    double varTmpl<int> = 10.0;
   )");
-  EXPECT_THAT(getSymbols(FilePath),
-              ElementsAre(AllOf(QName("Tmpl"), WithKind(SymbolKind::Struct)),
-                          AllOf(QName("Tmpl::x"), WithKind(SymbolKind::Field)),
-                          AllOf(QName("Tmpl"), WithKind(SymbolKind::Struct))));
+  EXPECT_THAT(
+      getSymbols(FilePath),
+      ElementsAre(
+          AllOf(WithName("Tmpl"), WithKind(SymbolKind::Struct),
+                Children(AllOf(WithName("x"), WithKind(SymbolKind::Field)))),
+          AllOf(WithName("Tmpl<int>"), WithKind(SymbolKind::Struct),
+                Children(WithName("y"))),
+          AllOf(WithName("Tmpl<float>"), WithKind(SymbolKind::Struct),
+                Children()),
+          AllOf(WithName("Tmpl<double>"), WithKind(SymbolKind::Struct),
+                Children()),
+          AllOf(WithName("funcTmpl"), Children()),
+          // FIXME(ibiryukov): template args should be <int> to match the code.
+          AllOf(WithName("funcTmpl<int, double, float>"), Children()),
+          AllOf(WithName("varTmpl"), Children()),
+          // FIXME(ibiryukov): template args should be <int> to match the code.
+          AllOf(WithName("varTmpl<int, double>"), Children())));
 }
 
 TEST_F(DocumentSymbolsTest, Namespaces) {
   std::string FilePath = testPath("foo.cpp");
   addFile(FilePath, R"cpp(
       namespace ans1 {
         int ai1;
       namespace ans2 {
addFile(FilePath, R"cpp(
       // This is still inlined.
       namespace nb {
       class Bar {};
       }
       }
       )cpp");
   EXPECT_THAT(
       getSymbols(FilePath),
-      ElementsAreArray({QName("ans1"), QName("ans1::ai1"), QName("ans1::ans2"),
-                        QName("ans1::ans2::ai2"), QName("test"), QName("na"),
-                        QName("na::nb"), QName("na::Foo"), QName("na"),
-                        QName("na::nb"), QName("na::Bar")}));
+      ElementsAreArray<testing::Matcher<DocumentSymbol>>(
+          {AllOf(WithName("ans1"),
+                 Children(AllOf(WithName("ai1"), Children()),
+                          AllOf(WithName("ans2"), Children(WithName("ai2"))))),
+           AllOf(WithName("(anonymous namespace)"), Children(WithName("test"))),
+           AllOf(WithName("na"),
+                 Children(AllOf(WithName("nb"), Children(WithName("Foo"))))),
+           AllOf(WithName("na"),
+                 Children(AllOf(WithName("nb"), Children(WithName("Bar")))))}));
 }
 
 TEST_F(DocumentSymbolsTest, Enums) {
   std::string FilePath = testPath("foo.cpp");
   addFile(FilePath, R"(
       enum {
         Red
       };
       enum Color {
         Green
       };
       enum class Color2 {
         Yellow
       };
       namespace ns {
       enum {
         Black
       };
       }
     )");
-  EXPECT_THAT(getSymbols(FilePath),
-              ElementsAre(QName("Red"), QName("Color"), QName("Green"),
-                          QName("Color2"), QName("Color2::Yellow"), QName("ns"),
-                          QName("ns::Black")));
+  EXPECT_THAT(
+      getSymbols(FilePath),
+      ElementsAre(
+          AllOf(WithName("(anonymous enum)"), Children(WithName("Red"))),
+          AllOf(WithName("Color"), Children(WithName("Green"))),
+          AllOf(WithName("Color2"), Children(WithName("Yellow"))),
+          AllOf(WithName("ns"), Children(AllOf(WithName("(anonymous enum)"),
+                                               Children(WithName("Black")))))));
 }
 
 TEST_F(DocumentSymbolsTest, FromMacro) {
   std::string FilePath = testPath("foo.cpp");
   Annotations Main(R"(
     #define FF(name) \
       class name##_Test {};
 
     $expansion[[FF]](abc);
 
     #define FF2() \
       class $spelling[[Test]] {};
 
     FF2();
   )");
   addFile(FilePath, Main.code());
   EXPECT_THAT(
       getSymbols(FilePath),
-      ElementsAre(AllOf(QName("abc_Test"), SymRange(Main.range("expansion"))),
-                  AllOf(QName("Test"), SymRange(Main.range("spelling")))));
+      ElementsAre(
+          AllOf(WithName("abc_Test"), SymNameRange(Main.range("expansion"))),
+          AllOf(WithName("Test"), SymNameRange(Main.range("spelling")))));
+}
+
+TEST_F(DocumentSymbolsTest, FuncTemplates) {
+  std::string FilePath = testPath("foo.cpp");
+  Annotations Source(R"cpp(
+    template <class T>
+    T foo() {}
+
+    auto x = foo<int>();
+    auto y = foo<double>()
+  )cpp");
+  addFile(FilePath, Source.code());
+  // Make sure we only see the template declaration, not instantiations.
+  EXPECT_THAT(getSymbols(FilePath),
+              ElementsAre(WithName("foo"), WithName("x"), WithName("y")));
+}
+
+TEST_F(DocumentSymbolsTest, UsingDirectives) {
+  std::string FilePath = testPath("foo.cpp");
+  Annotations Source(R"cpp(
+    namespace ns {
+      int foo;
+    }
+
+    namespace ns_alias = ns;
+
+    using namespace ::ns;     // check we don't loose qualifiers.
+    using namespace ns_alias; // and namespace aliases.
+  )cpp");
+  addFile(FilePath, Source.code());
+  EXPECT_THAT(getSymbols(FilePath),
+              ElementsAre(WithName("ns"), WithName("ns_alias"),
+                          WithName("using namespace ::ns"),
+                          WithName("using namespace ns_alias")));
 }
 
 } // namespace clangd
 } // namespace clang

clang-tools-extra/trunk/unittests/clangd/SyncAPI.h

 llvm::Expected<std::vector<tooling::Replacement>>
 runRename(ClangdServer &Server, PathRef File, Position Pos, StringRef NewName);
 
 std::string runDumpAST(ClangdServer &Server, PathRef File);
 
 llvm::Expected<std::vector<SymbolInformation>>
 runWorkspaceSymbols(ClangdServer &Server, StringRef Query, int Limit);
 
-llvm::Expected<std::vector<SymbolInformation>>
-runDocumentSymbols(ClangdServer &Server, PathRef File);
+Expected<std::vector<DocumentSymbol>> runDocumentSymbols(ClangdServer &Server,
+                                                         PathRef File);
 
 SymbolSlab runFuzzyFind(const SymbolIndex &Index, StringRef Query);
 SymbolSlab runFuzzyFind(const SymbolIndex &Index, const FuzzyFindRequest &Req);
 RefSlab getRefs(const SymbolIndex &Index, SymbolID ID);
 
 } // namespace clangd
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_UNITTESTS_CLANGD_SYNCAPI_H

clang-tools-extra/trunk/unittests/clangd/SyncAPI.cpp

 
 Expected<std::vector<SymbolInformation>>
 runWorkspaceSymbols(ClangdServer &Server, StringRef Query, int Limit) {
   Optional<Expected<std::vector<SymbolInformation>>> Result;
   Server.workspaceSymbols(Query, Limit, capture(Result));
   return std::move(*Result);
 }
 
-Expected<std::vector<SymbolInformation>>
-runDocumentSymbols(ClangdServer &Server, PathRef File) {
-  Optional<Expected<std::vector<SymbolInformation>>> Result;
+Expected<std::vector<DocumentSymbol>> runDocumentSymbols(ClangdServer &Server,
+                                                         PathRef File) {
+  Optional<Expected<std::vector<DocumentSymbol>>> Result;
   Server.documentSymbols(File, capture(Result));
   return std::move(*Result);
 }
 
 SymbolSlab runFuzzyFind(const SymbolIndex &Index, StringRef Query) {
   FuzzyFindRequest Req;
   Req.Query = Query;
   Req.AnyScope = true;