[clangd] Add "member" symbols to the index

[malaperle]

Summary

This adds more symbols to the index:

• member variables and functions
• enum constants in scoped enums

The code completion behavior should remain intact but workspace symbols should
now provide much more useful symbols.
Other symbols should be considered such as the ones in "main files" (files not
being included) but this can be done separately as this introduces its fair
share of problems.

Signed-off-by: Marc-Andre Laperle <[e-mail]>

[malaperle]

clangd/index/Index.h

142 ↗

(On Diff #139997)

I'm thinking of replacing this with something like:
Decl::Kind DeclContextKind;
bool IsScoped (enum);
bool IsInMainFile;

255 ↗

(On Diff #139997)

Would be removed when "ForCompletion" is replaced.

clangd/index/MemIndex.cpp

48 ↗

(On Diff #139997)

Would be removed when "ForCompletion" is replaced.

clangd/index/SymbolCollector.cpp

124 ↗

(On Diff #139997)

This code would be replaced with setting individual Symbol fields instead.

342 ↗

(On Diff #139997)

Here we would set fields that will replace "ForCompletion"

[malaperle]

Rework to include member vars/functions and scoped enumerators.

[malaperle]

clangd/CodeComplete.cpp

932 ↗

(On Diff #146391)

I want to add a comment here, but I want to make sure I understand why in the first place we were not indexing symbols outside these contexts for the purpose of code completion. Is it because those will be available by Sema code completion anyway?

[malaperle]

@ioeric You mentioned in D46751 that it would make sense to add a flag to disable indexing members. Could you comment on that? What kind of granularity were you thinking? Would a "member" flag cover both class members (member vars and functions) and enum class enumerators for example? I think that would be reasonable. But I will also add symbols in main files too, so another flag for that? Hmmm.

[ilya-biryukov]

A few questions regarding class members. To pinpoint some interesting cases and agree on how we want those to behave in the long run.

How do we handle template specializations? What will the qualified names of those instantiations be?
I.e. how do I query for push_back inside a vector? Which of the following queries should produce a result?

• vector::push_back. Should it match both vector<T>::push_back and vector<bool>::push_back or only the first one?
• vector<bool>::push_back
• vector<int>::push_back

What scopes will non-scoped enum members have?
E.g. if I have enum En { A,B,C}, which of the following queries will and won't find enumerators?

• En::A
• ::A
• A

clangd/CodeComplete.cpp

932 ↗

(On Diff #146391)

C++ lookup rules inside classes are way more complicated than in namespaces and we can't possibly hope to give a decent approximation for those.
Moreover, completion inside classes does not require any non-local information, so there does not seem to be a win when using the index anyway.
So we'd rather rely on clang to do completion there, it will give way more useful results than any index implementation.

clangd/index/Index.h

160 ↗

(On Diff #146391)

How do we use DeclContextKind?
Why did we decide to not go with a bool ForCompletion instead? (I'm probably missing a conversation in the workspaceSymbol review, could you point me to those instead?)

I'm asking because clang enums are very detailed and designed for use in the compiler, using them in the index seems to complicate things.
It feels we don't need this level of detail in the symbols. Similar to how we don't store the whole structural type, but rely on string representation of completion label instead.

[ioeric]

In D44954#1101922, @malaperle wrote:

@ioeric You mentioned in D46751 that it would make sense to add a flag to disable indexing members. Could you comment on that? What kind of granularity were you thinking? Would a "member" flag cover both class members (member vars and functions) and enum class enumerators for example? I think that would be reasonable. But I will also add symbols in main files too, so another flag for that? Hmmm.

Sam convinced me that members would still be interesting for our internal index service (e.g. locations of members would be useful for go-to-def). I'll investigate how much impact that would be by running the indexer with your change patched in, but I don't want to block you on that, so I'm fine with checking this in without any filter. We could revisit the filter design when needed.

We actually had an option for indexing symbols in main files :) But it was removed as it turned out to be unused for the features clangd had at that point. I think it would be reasonable to add it back if we start supporting collecting main file symbols again. Maybe out of the scope of this patch, but I am interested in the use cases you have for symbols in main files, besides in workspaceSymbols?

clangd/index/Index.h

160 ↗

(On Diff #146391)

+1

ForCompletion sounds reasonable as the current design of index-based code completion relies on assumptions about contexts.

162 ↗

(On Diff #146391)

In case we do need these fields, I think they should go into index::SymbolInfo (i.e. SymInfo above)? As Ilya said, Symbol might be the wrong level to put these. And we would want them in index-while-build anyway.

[ioeric]

clangd/index/SymbolCollector.cpp

113 ↗

(On Diff #146391)

We should be careful when removing the InTopLevelScope filter. According to clang/AST/DeclBase.h, DeclContext can be any of the following. For example, indexing symbols in functions seems to be out of the scope of this patch. I think we should keep a whitelist instead of turning them all at once. It's unfortunately we don't have tests to catch those...

///   TranslationUnitDecl
///   NamespaceDecl
///   FunctionDecl
///   TagDecl
///   ObjCMethodDecl
///   ObjCContainerDecl
///   LinkageSpecDecl
///   ExportDecl
///   BlockDecl
///   OMPDeclareReductionDecl

For class members, I would be good to add more tests for symbol collector e.g. more realistic class layouts with constructor, friends etc.

[malaperle]

In D44954#1102807, @ilya-biryukov wrote:

A few questions regarding class members. To pinpoint some interesting cases and agree on how we want those to behave in the long run.

How do we handle template specializations? What will the qualified names of those instantiations be?
I.e. how do I query for push_back inside a vector? Which of the following queries should produce a result?

• vector::push_back. Should it match both vector<T>::push_back and vector<bool>::push_back or only the first one?
• vector<bool>::push_back
• vector<int>::push_back

It's probably better to consider this in a future patch. Maybe something like the first suggestion: vector::push_back and match both. Otherwise, I would think it might be a bit too verbose to have to spell out all of the specialization. Maybe we could allow it too. So... all of the above? :)

What scopes will non-scoped enum members have?
E.g. if I have enum En { A,B,C}, which of the following queries will and won't find enumerators?

• En::A
• ::A
• A

Hmm. I think all of them, since you can refer them like that in code too. Case #1 doesn't work but that was the case before this patch so it can probably be addressed separately. I'll add some tests though!

In D44954#1103026, @ioeric wrote:

In D44954#1101922, @malaperle wrote:

@ioeric You mentioned in D46751 that it would make sense to add a flag to disable indexing members. Could you comment on that? What kind of granularity were you thinking? Would a "member" flag cover both class members (member vars and functions) and enum class enumerators for example? I think that would be reasonable. But I will also add symbols in main files too, so another flag for that? Hmmm.

Sam convinced me that members would still be interesting for our internal index service (e.g. locations of members would be useful for go-to-def). I'll investigate how much impact that would be by running the indexer with your change patched in, but I don't want to block you on that, so I'm fine with checking this in without any filter. We could revisit the filter design when needed.

We actually had an option for indexing symbols in main files :) But it was removed as it turned out to be unused for the features clangd had at that point. I think it would be reasonable to add it back if we start supporting collecting main file symbols again. Maybe out of the scope of this patch, but I am interested in the use cases you have for symbols in main files, besides in workspaceSymbols?

It's also for textDocument/documentSymbol. For this, we technically don't need them in the static index since we could collect symbols when the document is opened, but we also want them for workspaceSymbols so we might as well use the same symbol collector, etc. There should be more things that would also use symbol in main files, like Type Hierarchy.

clangd/CodeComplete.cpp

932 ↗

(On Diff #146391)

Makes sense. Thanks! I'll be able to document this better now with the full picture.

clangd/index/Index.h

160 ↗

(On Diff #146391)

My thinking was that the "ForCompletion" boolean was too specific and tailored for one client use. I thought the Symbol information should not have that much hard-coded knowledge on how it would be used. It would be odd to have "ForWorkspaceSymbol", "ForDocumentSymbol", etc. That is why I was going for a slightly more detailed symbol information that opened the door for more arbitrary queries for symbol clients. But it complicates things a bit more and I'd be happy to bring back the "ForCompletion" if it makes more sense for now.

clangd/index/SymbolCollector.cpp

113 ↗

(On Diff #146391)

For symbol in functions, they are not collected because "IndexOpts.IndexFunctionLocals = false" for the symbol collector. I'll investigate the other ones and add more tests.

[ioeric]

It's also for textDocument/documentSymbol. For this, we technically don't need them in the static index since we could collect symbols when the document is opened, but we also want them for workspaceSymbols so we might as well use the same symbol collector, etc. There should be more things that would also use symbol in main files, like Type Hierarchy.

I see. Thanks! Looks like we would need to collect symbols in main file at some point. We could probably filter out main file symbols in dynamic index for code completion, but we would probably need a flag to turn off main file symbols in case this leads to a huge size increase. I would also expect different indexers (dynamic and static ) to share SymbolCollector, but it should be easily configured to behave differently via SymbolCollector::Options. Anyhow, let's discuss more about this when we are turning on main file symbols :)

clangd/index/Index.h

160 ↗

(On Diff #146391)

I think code completion, with the most complicated use of the index so far, probably deserves a flag :P I would expect/hope other features to be less "picky" about symbols. A high-level flag like ForCompletion would help keep knowledge about filtering for code completion (e.g. enums ...) inside symbol collector, which I think could be a win.

FWIW, ForCompletion makes it sound like a symbols is collected specifically for code completion. Maybe something like bool SupportGlobalCompletion would be better?

[ilya-biryukov]

In D44954#1103664, @malaperle wrote:

It's probably better to consider this in a future patch. Maybe something like the first suggestion: vector::push_back and match both. Otherwise, I would think it might be a bit too verbose to have to spell out all of the specialization. Maybe we could allow it too. So... all of the above? :)

This certainly does not have to be addressed in this patch. Just wanted to collect opinions on what the behavior we want in the long term.
My thoughts would be towards allowing only vector::push_back and make it match both push_backs: in vector<bool> and vector<T>.
Other cases might work too, but I wouldn't try implementing something that matches specializations. It's just too complicated in the general case. This will only be used in workspaceSymbol and it's fine to give a few more results there...

What scopes will non-scoped enum members have?

Hmm. I think all of them, since you can refer them like that in code too. Case #1 doesn't work but that was the case before this patch so it can probably be addressed separately. I'll add some tests though!

I would vote for making queries En::A and A match the enumerator, but not ::A. The reasoning is: yes, you can reference it this way in a C++ file, but workspaceSymbol is not a real C++ resolve and I think it should match the outline of the code rather than the actual C++ lookup rules.
E.g. I wouldn't expect it to match symbols from base classes, etc. This should also simplify implementation too.

[malaperle]

In D44954#1104178, @ilya-biryukov wrote:

In D44954#1103664, @malaperle wrote:

It's probably better to consider this in a future patch. Maybe something like the first suggestion: vector::push_back and match both. Otherwise, I would think it might be a bit too verbose to have to spell out all of the specialization. Maybe we could allow it too. So... all of the above? :)

This certainly does not have to be addressed in this patch. Just wanted to collect opinions on what the behavior we want in the long term.
My thoughts would be towards allowing only vector::push_back and make it match both push_backs: in vector<bool> and vector<T>.
Other cases might work too, but I wouldn't try implementing something that matches specializations. It's just too complicated in the general case. This will only be used in workspaceSymbol and it's fine to give a few more results there...

Sounds very reasonable.

What scopes will non-scoped enum members have?

Hmm. I think all of them, since you can refer them like that in code too. Case #1 doesn't work but that was the case before this patch so it can probably be addressed separately. I'll add some tests though!

I would vote for making queries En::A and A match the enumerator, but not ::A. The reasoning is: yes, you can reference it this way in a C++ file, but workspaceSymbol is not a real C++ resolve and I think it should match the outline of the code rather than the actual C++ lookup rules.
E.g. I wouldn't expect it to match symbols from base classes, etc. This should also simplify implementation too.

I don't have a strong opinion, so I can try this suggestion!

clangd/index/Index.h

160 ↗

(On Diff #146391)

Maybe something like bool SupportGlobalCompletion would be better?

Sounds good!

[malaperle]

In D44954#1104497, @malaperle wrote:

What scopes will non-scoped enum members have?

Hmm. I think all of them, since you can refer them like that in code too. Case #1 doesn't work but that was the case before this patch so it can probably be addressed separately. I'll add some tests though!

I would vote for making queries En::A and A match the enumerator, but not ::A. The reasoning is: yes, you can reference it this way in a C++ file, but workspaceSymbol is not a real C++ resolve and I think it should match the outline of the code rather than the actual C++ lookup rules.
E.g. I wouldn't expect it to match symbols from base classes, etc. This should also simplify implementation too.

I don't have a strong opinion, so I can try this suggestion!

I changed the behavior of non-scoped enums as suggested here: D47223

[malaperle]

Use "SupportGlobalCompletion", white-list decl contexts, add more tests

[ioeric]

The change looks mostly good. Some nits and questions about the testing.

clangd/index/Index.h

158 ↗

(On Diff #148329)

s/this is symbol/this symbol/?

clangd/index/SymbolCollector.cpp

158 ↗

(On Diff #148329)

(Disclaimer: I don't know obj-c...)

It seems that some objc contexts here are good for global code completion. If we want to support objc symbols, it might also make sense to properly set SupportGlobalCompletion for them.

359 ↗

(On Diff #148329)

Could we pull this into a helper function? Something like S.SupportGlobalCompletion = DeclSupportsGlobalCompletion(...)?

unittests/clangd/CodeCompleteTests.cpp

657 ↗

(On Diff #148329)

IIUC, this tests the SupportGlobalCompletion filtering logic for index completion? If so, I think it would be more straightforward to do something like:

Sym NoGlobal(...);
NoGlobal.SupportGlobalCompletion = false;
...
completions (Code, {NoGlobal, func(...), cls(...)})
// Expect only global symbols in completion results.

This avoid setting up the ClangdServer manually.

741 ↗

(On Diff #148329)

It's not clear to me what the following tests (Enums, AnonymousNamespace, InMainFile) are testing. Do they test code completion or symbol collector? If these test symbol collection, they should be moved int SymbolCollectorTest.cpp

785 ↗

(On Diff #148329)

I think the behaviors above are the same before this change, so I'm not quite sure what they are testing. Note that symbols in main files are not collected into dynamic index, and by default the tests do not use dynamic index.

846 ↗

(On Diff #148329)

I think this is expected to be empty even for symbols that are not in anonymous namespace because symbols in main files are not indexed.

unittests/clangd/SymbolCollectorTests.cpp

70 ↗

(On Diff #148329)

nit: I'd probably call this Global for convenience.

216 ↗

(On Diff #148329)

Could you also add checkers for the SupportGlobalCompletion field?

404 ↗

(On Diff #148329)

Could you please also add checkers for GlobalCodeCompletion for these enums?

[malaperle]

Address comments.

clangd/index/SymbolCollector.cpp

158 ↗

(On Diff #148329)

Sounds good. Maybe it would be OK to do this in another (small) patch? I also know next to nothing about obj-c :)

unittests/clangd/CodeCompleteTests.cpp

741 ↗

(On Diff #148329)

They are testing that code completion works as intended regardless of how symbol collector is implemented. It's similar to our previous discussion in D44882 about "black box testing". I can remove them but it seems odd to me to not have code completion level tests for all cases because we assume that it will behave a certain way at the symbol collection and querying levels.

[malaperle]

unittests/clangd/SymbolCollectorTests.cpp

141 ↗

(On Diff #148834)

This allows to override the "-xc++" with something else, i.e. -xobjective-c++

[sammccall]

clangd/index/Index.h

279 ↗

(On Diff #148834)

please also avoid "global" here, e.g. RestrictForCodeCompletion

158 ↗

(On Diff #148329)

Sorry, I hadn't seen this patch until now.
When it was part of the workspace/symbol patch, I was the other person concerned this was too coupled to existing use cases and preferred something more descriptive.

I dug up an analysis @hokein and I worked on. One concluseion we came to was that we thought results needed by completion were a subset of what workspace/symbol needs, so a boolean would work. It seemed a bit ad-hoc and fragile though.

The cases we looked at were:

	private	member	local	primary template	template specialization	nested in template
code complete	N	N	N	Y	N	N
workspace/symbol	Y	Y	N	Y	Y	Y
go to defn	Y	Y	?	?	?	?

(Y = index should return it, N = index should not return it, ? = don't care)

So the most relevant information seems to be:

• is this private (private member, internal linkage, no decl outside cpp file, etc)
• is this nested in a class type (or template)
• is this a template specialization

I could live with bundling these into a single property (though they seem like good ranking signals, and we'd lose them for that purpose), it will certainly make a posting-list based index more efficient.

In that case I think we should have canonical documentation *somewhere* about exactly what this subset is, and this field should reference that.
e.g. isIndexedForCodeCompletion() in CodeComplete.h with docs and IsIndexedForCodeCompletion here. (Please avoid "global" as it has too many different meanings - here we mean "index-based").

[malaperle]

clangd/index/Index.h

158 ↗

(On Diff #148329)

OK, I added documentation on the SymbolCollector (which was outdated) about what kinds of symbols are collected, with a reference to shouldFilterDecl. The subset is documented on isIndexedForCodeCompletion(), also referenced from the Symbol field. Was that more or less what you meant?

[malaperle]

Update with suggestions.

[ioeric]

lgtm

clangd/index/Index.h

158 ↗

(On Diff #148329)

I could live with bundling these into a single property (though they seem like good ranking signals, and we'd lose them for that purpose), it will certainly make a posting-list based index more efficient.

FWIW, I think we could still add those signals when we need them in the future. It seems reasonable to me for the code completion flag to co-exist with ranking signals that could potentially overlap.

unittests/clangd/CodeCompleteTests.cpp

741 ↗

(On Diff #148329)

FWIW, I am not against those tests at all; increasing coverage is always a nice thing to do IMO. I just thought it would make more sense to add them in a separate patch if they are not related to changes in this patch; I found unrelated tests a bit confusing otherwise.

unittests/clangd/SymbolCollectorTests.cpp

141 ↗

(On Diff #148834)

I think this could also be a comment in the code :)

[sammccall]

Thanks, LG!

clangd/CodeComplete.h

86 ↗

(On Diff #149206)

nit: want -> consider?

94 ↗

(On Diff #149206)

A little more context:
"// Other symbols still appear in the index for other purposes, like workspace/symbols or textDocument/definition, but are not used for code completion"

clangd/index/SymbolCollector.h

22 ↗

(On Diff #149206)

nit: can you change "symbols" here to "declarations"?
currently we rely a bit too heavily on the user knowing what "symbol" means

[malaperle]

Address comments.

unittests/clangd/CodeCompleteTests.cpp

741 ↗

(On Diff #148329)

I found unrelated tests a bit confusing otherwise.

Fair point!

clang-tools-extra/trunk/clangd/CodeComplete.h

 #include "Path.h"
 #include "Protocol.h"
 #include "index/Index.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Sema/CodeCompleteOptions.h"
 #include "clang/Tooling/CompilationDatabase.h"
 
 namespace clang {
+class NamedDecl;
 class PCHContainerOperations;
 namespace clangd {
 
 struct CodeCompleteOptions {
   /// Returns options that can be passed to clang's completion engine.
   clang::CodeCompleteOptions getClangCompleteOpts() const;
 
   /// When true, completion items will contain expandable code snippets in

 /// Get signature help at a specified \p Pos in \p FileName.
 SignatureHelp signatureHelp(PathRef FileName,
                             const tooling::CompileCommand &Command,
                             PrecompiledPreamble const *Preamble,
                             StringRef Contents, Position Pos,
                             IntrusiveRefCntPtr<vfs::FileSystem> VFS,
                             std::shared_ptr<PCHContainerOperations> PCHs);
 
+// For index-based completion, we only consider:
+//   * symbols in namespaces or translation unit scopes (e.g. no class
+//     members, no locals)
+//   * enum constants in unscoped enum decl (e.g. "red" in "enum {red};")
+//   * primary templates (no specializations)
+// For the other cases, we let Clang do the completion because it does not
+// need any non-local information and it will be much better at following
+// lookup rules. Other symbols still appear in the index for other purposes,
+// like workspace/symbols or textDocument/definition, but are not used for code
+// completion.
+bool isIndexedForCodeCompletion(const NamedDecl &ND, ASTContext &ASTCtx);
 } // namespace clangd
 } // namespace clang
 
 #endif

clang-tools-extra/trunk/clangd/CodeComplete.cpp

 #include "FuzzyMatch.h"
 #include "Headers.h"
 #include "Logger.h"
 #include "Quality.h"
 #include "SourceCode.h"
 #include "Trace.h"
 #include "URI.h"
 #include "index/Index.h"
+#include "clang/ASTMatchers/ASTMatchFinder.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/Index/USRGeneration.h"
 #include "clang/Sema/CodeCompleteConsumer.h"
 #include "clang/Sema/Sema.h"
 #include "clang/Tooling/Core/Replacement.h"
SymbolSlab queryIndex() {
     SPAN_ATTACH(Tracer, "limit", Opts.Limit);
 
     SymbolSlab::Builder ResultsBuilder;
     // Build the query.
     FuzzyFindRequest Req;
     if (Opts.Limit)
       Req.MaxCandidateCount = Opts.Limit;
     Req.Query = Filter->pattern();
+    Req.RestrictForCodeCompletion = true;
     Req.Scopes = getQueryScopes(Recorder->CCContext,
                                 Recorder->CCSema->getSourceManager());
     log(llvm::formatv("Code complete: fuzzyFind(\"{0}\", scopes=[{1}])",
                       Req.Query,
                       llvm::join(Req.Scopes.begin(), Req.Scopes.end(), ",")));
     // Run the query against the index.
     if (Opts.Index->fuzzyFind(
             Req, [&](const Symbol &Sym) { ResultsBuilder.insert(Sym); }))
SignatureHelp signatureHelp(PathRef FileName,
   std::vector<Inclusion> PreambleInclusions = {}; // Unused for signatureHelp
   semaCodeComplete(llvm::make_unique<SignatureHelpCollector>(Options, Result),
                    Options,
                    {FileName, Command, Preamble, PreambleInclusions, Contents,
                     Pos, std::move(VFS), std::move(PCHs)});
   return Result;
 }
 
+bool isIndexedForCodeCompletion(const NamedDecl &ND, ASTContext &ASTCtx) {
+  using namespace clang::ast_matchers;
+  auto InTopLevelScope = hasDeclContext(
+      anyOf(namespaceDecl(), translationUnitDecl(), linkageSpecDecl()));
+  return !match(decl(anyOf(InTopLevelScope,
+                           hasDeclContext(
+                               enumDecl(InTopLevelScope, unless(isScoped()))))),
+                ND, ASTCtx)
+              .empty();
+}
+
 } // namespace clangd
 } // namespace clang

clang-tools-extra/trunk/clangd/index/Index.h

struct Symbol {
   // A C++ symbol may have multiple declarations, and we pick one to prefer.
   //   * For classes, the canonical declaration should be the definition.
   //   * For non-inline functions, the canonical declaration typically appears
   //     in the ".h" file corresponding to the definition.
   SymbolLocation CanonicalDeclaration;
   // The number of translation units that reference this symbol from their main
   // file. This number is only meaningful if aggregated in an index.
   unsigned References = 0;
-
+  /// Whether or not this symbol is meant to be used for the code completion.
+  /// See also isIndexedForCodeCompletion().
+  bool IsIndexedForCodeCompletion = false;
   /// A brief description of the symbol that can be displayed in the completion
-  /// candidate list. For example, "Foo(X x, Y y) const" is a labal for a
+  /// candidate list. For example, "Foo(X x, Y y) const" is a label for a
   /// function.
   llvm::StringRef CompletionLabel;
   /// The piece of text that the user is expected to type to match the
   /// code-completion string, typically a keyword or the name of a declarator or
   /// macro.
   llvm::StringRef CompletionFilterText;
   /// What to insert when completing this symbol (plain text version).
   llvm::StringRef CompletionPlainInsertText;
struct FuzzyFindRequest {
   /// is provided, the matched symbols must be defined in namespace xyz but not
   /// namespace xyz::abc.
   ///
   /// The global scope is "", a top level scope is "foo::", etc.
   std::vector<std::string> Scopes;
   /// \brief The number of top candidates to return. The index may choose to
   /// return more than this, e.g. if it doesn't know which candidates are best.
   size_t MaxCandidateCount = UINT_MAX;
+  /// If set to true, only symbols for completion support will be considered.
+  bool RestrictForCodeCompletion = false;
 };
 
 struct LookupRequest {
   llvm::DenseSet<SymbolID> IDs;
 };
 
 /// \brief Interface for symbol indexes that can be used for searching or
 /// matching symbols among a set of symbols based on names or unique IDs.

clang-tools-extra/trunk/clangd/index/MemIndex.cpp

bool MemIndex::fuzzyFind(
   {
     std::lock_guard<std::mutex> Lock(Mutex);
     for (const auto Pair : Index) {
       const Symbol *Sym = Pair.second;
 
       // Exact match against all possible scopes.
       if (!Req.Scopes.empty() && !llvm::is_contained(Req.Scopes, Sym->Scope))
         continue;
+      if (Req.RestrictForCodeCompletion && !Sym->IsIndexedForCodeCompletion)
+        continue;
 
       if (auto Score = Filter.match(Sym->Name)) {
         Top.emplace(-*Score * quality(*Sym), Sym);
         if (Top.size() > Req.MaxCandidateCount) {
           More = true;
           Top.pop();
         }
       }

clang-tools-extra/trunk/clangd/index/SymbolCollector.h

 #include "clang/AST/Decl.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexSymbol.h"
 #include "clang/Sema/CodeCompleteConsumer.h"
 
 namespace clang {
 namespace clangd {
 
-/// \brief Collect top-level symbols from an AST. These are symbols defined
-/// immediately inside a namespace or a translation unit scope. For example,
-/// symbols in classes or functions are not collected. Note that this only
-/// collects symbols that declared in at least one file that is not a main
-/// file (i.e. the source file corresponding to a TU). These are symbols that
-/// can be imported by other files by including the file where symbols are
-/// declared.
+/// \brief Collect declarations (symbols) from an AST.
+/// It collects most declarations except:
+/// - Implicit declarations
+/// - Anonymous declarations (anonymous enum/class/struct, etc)
+/// - Declarations in anonymous namespaces
+/// - Local declarations (in function bodies, blocks, etc)
+/// - Declarations in main files
+/// - Template specializations
+/// - Library-specific private declarations (e.g. private declaration generated
+/// by protobuf compiler)
+///
+/// See also shouldFilterDecl().
 ///
 /// Clients (e.g. clangd) can use SymbolCollector together with
 /// index::indexTopLevelDecls to retrieve all symbols when the source file is
 /// changed.
 class SymbolCollector : public index::IndexDataConsumer {
 public:
   struct Options {
     /// When symbol paths cannot be resolved to absolute paths (e.g. files in

clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp

 //===--- SymbolCollector.cpp -------------------------------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "SymbolCollector.h"
 #include "../AST.h"
+#include "../CodeComplete.h"
 #include "../CodeCompletionStrings.h"
 #include "../Logger.h"
 #include "../SourceCode.h"
 #include "../URI.h"
 #include "CanonicalIncludes.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/DeclTemplate.h"
 #include "clang/ASTMatchers/ASTMatchFinder.h"
bool shouldFilterDecl(const NamedDecl *ND, ASTContext *ASTCtx,
   // `foo::<anonymous>::bar`, which need a special handling.
   // In real world projects, we have a relatively large set of header files
   // that define static variables (like "static const int A = 1;"), we still
   // want to collect these symbols, although they cause potential ODR
   // violations.
   if (ND->isInAnonymousNamespace())
     return true;
 
-  // We only want:
-  //   * symbols in namespaces or translation unit scopes (e.g. no class
-  //     members)
-  //   * enum constants in unscoped enum decl (e.g. "red" in "enum {red};")
-  auto InTopLevelScope = hasDeclContext(
-      anyOf(namespaceDecl(), translationUnitDecl(), linkageSpecDecl()));
-  // Don't index template specializations.
+  // We want most things but not "local" symbols such as symbols inside
+  // FunctionDecl, BlockDecl, ObjCMethodDecl and OMPDeclareReductionDecl.
+  // FIXME: Need a matcher for ExportDecl in order to include symbols declared
+  // within an export.
+  auto InNonLocalContext = hasDeclContext(anyOf(
+      translationUnitDecl(), namespaceDecl(), linkageSpecDecl(), recordDecl(),
+      enumDecl(), objcProtocolDecl(), objcInterfaceDecl(), objcCategoryDecl(),
+      objcCategoryImplDecl(), objcImplementationDecl()));
+  // Don't index template specializations and expansions in main files.
   auto IsSpecialization =
       anyOf(functionDecl(isExplicitTemplateSpecialization()),
             cxxRecordDecl(isExplicitTemplateSpecialization()),
             varDecl(isExplicitTemplateSpecialization()));
-  if (match(decl(allOf(unless(isExpansionInMainFile()),
-                       anyOf(InTopLevelScope,
-                             hasDeclContext(enumDecl(InTopLevelScope,
-                                                     unless(isScoped())))),
+  if (match(decl(allOf(unless(isExpansionInMainFile()), InNonLocalContext,
                        unless(IsSpecialization))),
             *ND, *ASTCtx)
           .empty())
     return true;
 
   // Avoid indexing internal symbols in protobuf generated headers.
   if (isPrivateProtoDecl(*ND))
     return true;
const Symbol *SymbolCollector::addDeclaration(const NamedDecl &ND,
   Policy.SuppressUnwrittenScope = true;
   ND.printQualifiedName(OS, Policy);
   OS.flush();
   assert(!StringRef(QName).startswith("::"));
 
   Symbol S;
   S.ID = std::move(ID);
   std::tie(S.Scope, S.Name) = splitQualifiedName(QName);
+
+  S.IsIndexedForCodeCompletion = isIndexedForCodeCompletion(ND, Ctx);
   S.SymInfo = index::getSymbolInfo(&ND);
   std::string FileURI;
   if (auto DeclLoc =
           getSymbolLocation(ND, SM, Opts, ASTCtx->getLangOpts(), FileURI))
     S.CanonicalDeclaration = *DeclLoc;
 
   // Add completion info.
   // FIXME: we may want to choose a different redecl, or combine from several.

clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp

static void mapping(IO &IO, Symbol &Sym) {
     IO.mapRequired("ID", NSymbolID->HexString);
     IO.mapRequired("Name", Sym.Name);
     IO.mapRequired("Scope", Sym.Scope);
     IO.mapRequired("SymInfo", Sym.SymInfo);
     IO.mapOptional("CanonicalDeclaration", Sym.CanonicalDeclaration,
                    SymbolLocation());
     IO.mapOptional("Definition", Sym.Definition, SymbolLocation());
     IO.mapOptional("References", Sym.References, 0u);
+    IO.mapOptional("IsIndexedForCodeCompletion", Sym.IsIndexedForCodeCompletion,
+                   false);
     IO.mapRequired("CompletionLabel", Sym.CompletionLabel);
     IO.mapRequired("CompletionFilterText", Sym.CompletionFilterText);
     IO.mapRequired("CompletionPlainInsertText", Sym.CompletionPlainInsertText);
 
     IO.mapOptional("CompletionSnippetInsertText",
                    Sym.CompletionSnippetInsertText);
     IO.mapOptional("Detail", NDetail->Opt);
   }

clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp

 
 namespace {
 using namespace llvm;
 using ::testing::AllOf;
 using ::testing::Contains;
 using ::testing::Each;
 using ::testing::ElementsAre;
 using ::testing::Field;
+using ::testing::IsEmpty;
 using ::testing::Not;
 using ::testing::UnorderedElementsAre;
 
 class IgnoreDiagnostics : public DiagnosticsConsumer {
   void onDiagnosticsReady(PathRef File,
                           std::vector<Diag> Diagnostics) override {}
 };
 
if (Pos == llvm::StringRef::npos) {
     USR += "@N@" + replace(QName.substr(0, Pos), "::", "@N@"); // ns:: -> @N@ns
   }
   USR += Regex("^.*$").sub(USRFormat, Sym.Name); // e.g. func -> @F@func#
   Sym.ID = SymbolID(USR);
   Sym.CompletionPlainInsertText = Sym.Name;
   Sym.CompletionSnippetInsertText = Sym.Name;
   Sym.CompletionLabel = Sym.Name;
   Sym.SymInfo.Kind = Kind;
+  Sym.IsIndexedForCodeCompletion = true;
   return Sym;
 }
 Symbol func(StringRef Name) { // Assumes the function has no args.
   return sym(Name, index::SymbolKind::Function, "@F@\\0#"); // no args
 }
 Symbol cls(StringRef Name) {
   return sym(Name, index::SymbolKind::Class, "@S@\\0");
 }
EXPECT_THAT(Results.items,
               Contains(AllOf(Named("foo"), Doc("Non-doxygen comment."))));
   EXPECT_THAT(
       Results.items,
       Contains(AllOf(Named("bar"), Doc("Doxygen comment.\n\\param int a"))));
   EXPECT_THAT(Results.items,
               Contains(AllOf(Named("baz"), Doc("Multi-line\nblock comment"))));
 }
 
+TEST(CompletionTest, GlobalCompletionFiltering) {
+
+  Symbol Class = cls("XYZ");
+  Class.IsIndexedForCodeCompletion = false;
+  Symbol Func = func("XYZ::foooo");
+  Func.IsIndexedForCodeCompletion = false;
+
+  auto Results = completions(R"(//      void f() {
+      XYZ::foooo^
+      })",
+                             {Class, Func});
+  EXPECT_THAT(Results.items, IsEmpty());
+}
+
 TEST(CodeCompleteTest, DisableTypoCorrection) {
   auto Results = completions(R"cpp(
      namespace clang { int v; }
      void f() { clangd::^
   )cpp");
   EXPECT_TRUE(Results.items.empty());
 }
 

clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp

 }
 
 TEST(FileIndexTest, RemoveNonExisting) {
   FileIndex M;
   M.update("no.cpp", nullptr, nullptr);
   EXPECT_THAT(match(M, FuzzyFindRequest()), UnorderedElementsAre());
 }
 
-TEST(FileIndexTest, IgnoreClassMembers) {
+TEST(FileIndexTest, ClassMembers) {
   FileIndex M;
   update(M, "f1", "class X { static int m1; int m2; static void f(); };");
 
   FuzzyFindRequest Req;
   Req.Query = "";
-  EXPECT_THAT(match(M, Req), UnorderedElementsAre("X"));
+  EXPECT_THAT(match(M, Req),
+              UnorderedElementsAre("X", "X::m1", "X::m2", "X::f"));
 }
 
 TEST(FileIndexTest, NoIncludeCollected) {
   FileIndex M;
   update(M, "f", "class string {};");
 
   FuzzyFindRequest Req;
   Req.Query = "";

clang-tools-extra/trunk/unittests/clangd/FindSymbolsTests.cpp

addFile("foo.h", R"cpp(
         int InUnnamed;
       } UnnamedStruct;)cpp");
   addFile("foo.cpp", R"cpp(
       #include "foo.h"
       )cpp");
   EXPECT_THAT(getSymbols("UnnamedStruct"),
               ElementsAre(AllOf(Named("UnnamedStruct"),
                                 WithKind(SymbolKind::Variable))));
-  EXPECT_THAT(getSymbols("InUnnamed"), IsEmpty());
+  EXPECT_THAT(
+      getSymbols("InUnnamed"),
+      ElementsAre(AllOf(Named("InUnnamed"), InContainer("(anonymous struct)"),
+                        WithKind(SymbolKind::Field))));
 }
 
 TEST_F(WorkspaceSymbolsTest, InMainFile) {
   addFile("foo.cpp", R"cpp(
       int test() {
       }
       )cpp");
   EXPECT_THAT(getSymbols("test"), IsEmpty());
EXPECT_THAT(
           AllOf(Named("Foo"), InContainer(""), WithKind(SymbolKind::Class)),
           AllOf(Named("foo"), InContainer(""), WithKind(SymbolKind::Function)),
           AllOf(Named("ns"), InContainer(""),
                 WithKind(SymbolKind::Namespace))));
   EXPECT_THAT(getSymbols(":"), IsEmpty());
   EXPECT_THAT(getSymbols(""), IsEmpty());
 }
 
+TEST_F(WorkspaceSymbolsTest, Enums) {
+  addFile("foo.h", R"cpp(
+    enum {
+      Red
+    };
+    enum Color {
+      Green
+    };
+    enum class Color2 {
+      Yellow
+    };
+    namespace ns {
+      enum {
+        Black
+      };
+      enum Color3 {
+        Blue
+      };
+      enum class Color4 {
+        White
+      };
+    }
+      )cpp");
+  addFile("foo.cpp", R"cpp(
+      #include "foo.h"
+      )cpp");
+  EXPECT_THAT(getSymbols("Red"), ElementsAre(Named("Red")));
+  EXPECT_THAT(getSymbols("::Red"), ElementsAre(Named("Red")));
+  EXPECT_THAT(getSymbols("Green"), ElementsAre(Named("Green")));
+  EXPECT_THAT(getSymbols("Green"), ElementsAre(Named("Green")));
+  EXPECT_THAT(getSymbols("Color2::Yellow"), ElementsAre(Named("Yellow")));
+  EXPECT_THAT(getSymbols("Yellow"), ElementsAre(Named("Yellow")));
+
+  EXPECT_THAT(getSymbols("ns::Black"), ElementsAre(Named("Black")));
+  EXPECT_THAT(getSymbols("ns::Blue"), ElementsAre(Named("Blue")));
+  EXPECT_THAT(getSymbols("ns::Color4::White"), ElementsAre(Named("White")));
+}
+
 TEST_F(WorkspaceSymbolsTest, WithLimit) {
   addFile("foo.h", R"cpp(
       int foo;
       int foo2;
       )cpp");
   addFile("foo.cpp", R"cpp(
       #include "foo.h"
       )cpp");

clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

 MATCHER_P(DefRange, Pos, "") {
   return std::tie(arg.Definition.Start.Line,
                   arg.Definition.Start.Column, arg.Definition.End.Line,
                   arg.Definition.End.Column) ==
          std::tie(Pos.start.line, Pos.start.character, Pos.end.line,
                   Pos.end.character);
 }
 MATCHER_P(Refs, R, "") { return int(arg.References) == R; }
+MATCHER_P(ForCodeCompletion, IsIndexedForCodeCompletion, "") {
+  return arg.IsIndexedForCodeCompletion == IsIndexedForCodeCompletion;
+}
 
 namespace clang {
 namespace clangd {
 
 namespace {
 class SymbolIndexActionFactory : public tooling::FrontendActionFactory {
 public:
   SymbolIndexActionFactory(SymbolCollector::Options COpts,
bool runSymbolCollector(StringRef HeaderCode, StringRef MainCode,
                           const std::vector<std::string> &ExtraArgs = {}) {
     llvm::IntrusiveRefCntPtr<FileManager> Files(
         new FileManager(FileSystemOptions(), InMemoryFileSystem));
 
     auto Factory = llvm::make_unique<SymbolIndexActionFactory>(
         CollectorOpts, PragmaHandler.get());
 
     std::vector<std::string> Args = {
-        "symbol_collector", "-fsyntax-only", "-xc++",     "-std=c++11",
-        "-include",         TestHeaderName,  TestFileName};
+        "symbol_collector", "-fsyntax-only", "-xc++",
+        "-std=c++11",       "-include",      TestHeaderName};
     Args.insert(Args.end(), ExtraArgs.begin(), ExtraArgs.end());
+    // This allows to override the "-xc++" with something else, i.e.
+    // -xobjective-c++.
+    Args.push_back(TestFileName);
+
     tooling::ToolInvocation Invocation(
         Args,
         Factory->create(), Files.get(),
         std::make_shared<PCHContainerOperations>());
 
     InMemoryFileSystem->addFile(TestHeaderName, 0,
                                 llvm::MemoryBuffer::getMemBuffer(HeaderCode));
     InMemoryFileSystem->addFile(TestFileName, 0,
protected:
   SymbolSlab Symbols;
   SymbolCollector::Options CollectorOpts;
   std::unique_ptr<CommentHandler> PragmaHandler;
 };
 
 TEST_F(SymbolCollectorTest, CollectSymbols) {
   const std::string Header = R"(
     class Foo {
+      Foo() {}
+      Foo(int a) {}
       void f();
+      friend void f1();
+      friend class Friend;
+      Foo& operator=(const Foo&);
+      ~Foo();
+      class Nested {
+      void f();
+      };
     };
+    class Friend {
+    };
+
     void f1();
     inline void f2() {}
     static const int KInt = 2;
     const char* kStr = "123";
 
     namespace {
     void ff() {} // ignore
     }
const std::string Header = R"(
     // implicit declarations (the implicit using shadow declaration) by default,
     // and there is no way to customize this behavior at the moment.
     using bar::v2;
     } // namespace foo
   )";
   runSymbolCollector(Header, /*Main=*/"");
   EXPECT_THAT(Symbols,
               UnorderedElementsAreArray(
-                  {QName("Foo"), QName("f1"), QName("f2"), QName("KInt"),
-                   QName("kStr"), QName("foo"), QName("foo::bar"),
-                   QName("foo::int32"), QName("foo::int32_t"), QName("foo::v1"),
-                   QName("foo::bar::v2"), QName("foo::baz")}));
+                  {AllOf(QName("Foo"), ForCodeCompletion(true)),
+                   AllOf(QName("Foo::Foo"), ForCodeCompletion(false)),
+                   AllOf(QName("Foo::Foo"), ForCodeCompletion(false)),
+                   AllOf(QName("Foo::f"), ForCodeCompletion(false)),
+                   AllOf(QName("Foo::~Foo"), ForCodeCompletion(false)),
+                   AllOf(QName("Foo::operator="), ForCodeCompletion(false)),
+                   AllOf(QName("Foo::Nested"), ForCodeCompletion(false)),
+                   AllOf(QName("Foo::Nested::f"), ForCodeCompletion(false)),
+
+                   AllOf(QName("Friend"), ForCodeCompletion(true)),
+                   AllOf(QName("f1"), ForCodeCompletion(true)),
+                   AllOf(QName("f2"), ForCodeCompletion(true)),
+                   AllOf(QName("KInt"), ForCodeCompletion(true)),
+                   AllOf(QName("kStr"), ForCodeCompletion(true)),
+                   AllOf(QName("foo"), ForCodeCompletion(true)),
+                   AllOf(QName("foo::bar"), ForCodeCompletion(true)),
+                   AllOf(QName("foo::int32"), ForCodeCompletion(true)),
+                   AllOf(QName("foo::int32_t"), ForCodeCompletion(true)),
+                   AllOf(QName("foo::v1"), ForCodeCompletion(true)),
+                   AllOf(QName("foo::bar::v2"), ForCodeCompletion(true)),
+                   AllOf(QName("foo::baz"), ForCodeCompletion(true))}));
 }
 
 TEST_F(SymbolCollectorTest, Template) {
   Annotations Header(R"(
     // Template is indexed, specialization and instantiation is not.
-    template <class T> struct [[Tmpl]] {T x = 0;};
+    template <class T> struct [[Tmpl]] {T $xdecl[[x]] = 0;};
     template <> struct Tmpl<int> {};
     extern template struct Tmpl<float>;
     template struct Tmpl<double>;
   )");
   runSymbolCollector(Header.code(), /*Main=*/"");
-  EXPECT_THAT(Symbols, UnorderedElementsAreArray({AllOf(
-                           QName("Tmpl"), DeclRange(Header.range()))}));
+  EXPECT_THAT(Symbols,
+              UnorderedElementsAreArray(
+                  {AllOf(QName("Tmpl"), DeclRange(Header.range())),
+                   AllOf(QName("Tmpl::x"), DeclRange(Header.range("xdecl")))}));
+}
+
+TEST_F(SymbolCollectorTest, ObjCSymbols) {
+  const std::string Header = R"(
+    @interface Person
+    - (void)someMethodName:(void*)name1 lastName:(void*)lName;
+    @end
+
+    @implementation Person
+    - (void)someMethodName:(void*)name1 lastName:(void*)lName{
+      int foo;
+      ^(int param){ int bar; };
+    }
+    @end
+
+    @interface Person (MyCategory)
+    - (void)someMethodName2:(void*)name2;
+    @end
+
+    @implementation Person (MyCategory)
+    - (void)someMethodName2:(void*)name2 {
+      int foo2;
+    }
+    @end
+
+    @protocol MyProtocol
+    - (void)someMethodName3:(void*)name3;
+    @end
+  )";
+  TestFileName = "test.m";
+  runSymbolCollector(Header, /*Main=*/"", {"-fblocks", "-xobjective-c++"});
+  EXPECT_THAT(Symbols,
+              UnorderedElementsAre(
+                  QName("Person"), QName("Person::someMethodName:lastName:"),
+                  QName("MyCategory"), QName("Person::someMethodName2:"),
+                  QName("MyProtocol"), QName("MyProtocol::someMethodName3:")));
 }
 
 TEST_F(SymbolCollectorTest, Locations) {
   Annotations Header(R"cpp(
     // Declared in header, defined in main.
     extern int $xdecl[[X]];
     class $clsdecl[[Cls]];
     void $printdecl[[print]]();
TEST_F(SymbolCollectorTest, IncludeEnums) {
   const std::string Header = R"(
     enum {
       Red
     };
     enum Color {
       Green
     };
     enum class Color2 {
-      Yellow // ignore
+      Yellow
     };
     namespace ns {
     enum {
       Black
     };
     }
   )";
   runSymbolCollector(Header, /*Main=*/"");
-  EXPECT_THAT(Symbols, UnorderedElementsAre(QName("Red"), QName("Color"),
-                                            QName("Green"), QName("Color2"),
-                                            QName("ns"), QName("ns::Black")));
+  EXPECT_THAT(Symbols,
+              UnorderedElementsAre(
+                  AllOf(QName("Red"), ForCodeCompletion(true)),
+                  AllOf(QName("Color"), ForCodeCompletion(true)),
+                  AllOf(QName("Green"), ForCodeCompletion(true)),
+                  AllOf(QName("Color2"), ForCodeCompletion(true)),
+                  AllOf(QName("Color2::Yellow"), ForCodeCompletion(false)),
+                  AllOf(QName("ns"), ForCodeCompletion(true)),
+                  AllOf(QName("ns::Black"), ForCodeCompletion(true))));
 }
 
-TEST_F(SymbolCollectorTest, IgnoreNamelessSymbols) {
+TEST_F(SymbolCollectorTest, NamelessSymbols) {
   const std::string Header = R"(
     struct {
       int a;
     } Foo;
   )";
   runSymbolCollector(Header, /*Main=*/"");
-  EXPECT_THAT(Symbols,
-              UnorderedElementsAre(QName("Foo")));
+  EXPECT_THAT(Symbols, UnorderedElementsAre(QName("Foo"),
+                                            QName("(anonymous struct)::a")));
 }
 
 TEST_F(SymbolCollectorTest, SymbolFormedFromMacro) {
 
   Annotations Header(R"(
     #define FF(name) \
       class name##_Test {};
 
const std::string Main = R"(
     void main_f() {} // ignore
     void f1() {}
   )";
   runSymbolCollector(Header, Main);
   EXPECT_THAT(Symbols,
               UnorderedElementsAre(QName("Foo"), QName("f1"), QName("f2")));
 }
 
-TEST_F(SymbolCollectorTest, IgnoreClassMembers) {
+TEST_F(SymbolCollectorTest, ClassMembers) {
   const std::string Header = R"(
     class Foo {
       void f() {}
       void g();
       static void sf() {}
       static void ssf();
       static int x;
     };
   )";
   const std::string Main = R"(
     void Foo::g() {}
     void Foo::ssf() {}
   )";
   runSymbolCollector(Header, Main);
-  EXPECT_THAT(Symbols, UnorderedElementsAre(QName("Foo")));
+  EXPECT_THAT(Symbols,
+              UnorderedElementsAre(QName("Foo"), QName("Foo::f"),
+                                   QName("Foo::g"), QName("Foo::sf"),
+                                   QName("Foo::ssf"), QName("Foo::x")));
 }
 
 TEST_F(SymbolCollectorTest, Scopes) {
   const std::string Header = R"(
     namespace na {
     class Foo {};
     namespace nb {
     class Bar {};

 CanonicalDeclaration:
   FileURI:        file:///path/foo.h
   Start:
     Line: 1
     Column: 0
   End:
     Line: 1
     Column: 1
+IsIndexedForCodeCompletion:    true
 CompletionLabel:    'Foo1-label'
 CompletionFilterText:    'filter'
 CompletionPlainInsertText:    'plain'
 Detail:
   Documentation:    'Foo doc'
   CompletionDetail:    'int'
 ...
 )";
   const std::string YAML2 = R"(
 ---
 ID: 057557CEBF6E6B2DD437FBF60CC58F352D1DF858
 Name:   'Foo2'
 Scope:   'clang::'
 SymInfo:
   Kind:            Function
   Lang:            Cpp
 CanonicalDeclaration:
   FileURI:        file:///path/bar.h
   Start:
     Line: 1
     Column: 0
   End:
     Line: 1
     Column: 1
+IsIndexedForCodeCompletion:    false
 CompletionLabel:    'Foo2-label'
 CompletionFilterText:    'filter'
 CompletionPlainInsertText:    'plain'
 CompletionSnippetInsertText:    'snippet'
 ...
 )";
 
   auto Symbols1 = SymbolsFromYAML(YAML1);
 
   EXPECT_THAT(Symbols1,
               UnorderedElementsAre(AllOf(
                   QName("clang::Foo1"), Labeled("Foo1-label"), Doc("Foo doc"),
-                  Detail("int"), DeclURI("file:///path/foo.h"))));
+                  Detail("int"), DeclURI("file:///path/foo.h"),
+                  ForCodeCompletion(true))));
   auto Symbols2 = SymbolsFromYAML(YAML2);
-  EXPECT_THAT(Symbols2, UnorderedElementsAre(AllOf(
-                            QName("clang::Foo2"), Labeled("Foo2-label"),
-                            Not(HasDetail()), DeclURI("file:///path/bar.h"))));
+  EXPECT_THAT(Symbols2,
+              UnorderedElementsAre(AllOf(
+                  QName("clang::Foo2"), Labeled("Foo2-label"), Not(HasDetail()),
+                  DeclURI("file:///path/bar.h"), ForCodeCompletion(false))));
 
   std::string ConcatenatedYAML;
   {
     llvm::raw_string_ostream OS(ConcatenatedYAML);
     SymbolsToYAML(Symbols1, OS);
     SymbolsToYAML(Symbols2, OS);
   }
   auto ConcatenatedSymbols = SymbolsFromYAML(ConcatenatedYAML);
Annotations Header(R"(
     // Forward declarations of TagDecls.
     class C;
     struct S;
     union U;
 
     // Canonical declarations.
     class $cdecl[[C]] {};
     struct $sdecl[[S]] {};
-    union $udecl[[U]] {int x; bool y;};
+    union $udecl[[U]] {int $xdecl[[x]]; bool $ydecl[[y]];};
   )");
   runSymbolCollector(Header.code(), /*Main=*/"");
-  EXPECT_THAT(Symbols,
-              UnorderedElementsAre(
-                  AllOf(QName("C"), DeclURI(TestHeaderURI),
-                        DeclRange(Header.range("cdecl")),
-                        IncludeHeader(TestHeaderURI), DefURI(TestHeaderURI),
-                        DefRange(Header.range("cdecl"))),
-                  AllOf(QName("S"), DeclURI(TestHeaderURI),
-                        DeclRange(Header.range("sdecl")),
-                        IncludeHeader(TestHeaderURI), DefURI(TestHeaderURI),
-                        DefRange(Header.range("sdecl"))),
-                  AllOf(QName("U"), DeclURI(TestHeaderURI),
-                        DeclRange(Header.range("udecl")),
-                        IncludeHeader(TestHeaderURI), DefURI(TestHeaderURI),
-                        DefRange(Header.range("udecl")))));
+  EXPECT_THAT(
+      Symbols,
+                DeclRange(Header.range("cdecl")), IncludeHeader(TestHeaderURI),
+                DefURI(TestHeaderURI), DefRange(Header.range("cdecl"))),
+                DeclRange(Header.range("sdecl")), IncludeHeader(TestHeaderURI),
+                DefURI(TestHeaderURI), DefRange(Header.range("sdecl"))),
+                DeclRange(Header.range("udecl")), IncludeHeader(TestHeaderURI),
+                DefURI(TestHeaderURI), DefRange(Header.range("udecl"))),
+          AllOf(QName("U::x"), DeclURI(TestHeaderURI),
+                DeclRange(Header.range("xdecl")), DefURI(TestHeaderURI),
+                DefRange(Header.range("xdecl"))),
+          AllOf(QName("U::y"), DeclURI(TestHeaderURI),
+                DeclRange(Header.range("ydecl")), DefURI(TestHeaderURI),
+                DefRange(Header.range("ydecl")))));
 }
 
 TEST_F(SymbolCollectorTest, ClassForwardDeclarationIsCanonical) {
   CollectorOpts.CollectIncludePath = true;
   runSymbolCollector(/*Header=*/"class X;", /*Main=*/"class X {};");
   EXPECT_THAT(Symbols, UnorderedElementsAre(AllOf(
                            QName("X"), DeclURI(TestHeaderURI),
                            IncludeHeader(TestHeaderURI), DefURI(TestFileURI))));