[clangd] Suggest adding missing includes for incomplete type diagnostics.

[ioeric]

Summary

This enables clangd to intercept compiler diagnostics and attach fixes (e.g. by
querying index). This patch adds missing includes for incomplete types e.g.
member access into class with only forward declaration. This would allow adding
missing includes for user-typed symbol names that are missing declarations
(e.g. typos) in the future.

[ioeric]

• add comments

[sammccall]

This looks pretty good! My main concern is latency (and variability of latency) in practice.
Particularly:

• we should avoid paying for fuzzyfind and fetching hundreds of results when we want exact-match
• limit the damage in degenerate cases: should we limit to e.g. 5 index queries per TU?

Actually, now that I think about it - if we can see a forward declaration, can't we do a lookup() instead of a fuzzyFind?
Is the problem that this doesn't generalize to the no-declaration case (where it looks like a typo)?

If we had an operation that looked like lookup() but worked by qname, would we design the feature the same way?
In particular, would we want to batch the requests to the index (lookup takes a set of IDs)? This would affect the code structure a bit. But it would make the feature cost basically O(1) instead of O(errs)...

clangd/ClangdUnit.cpp

291

Probably worth a two-line comment explaining at a high level what this is doing.
Somewhat redundant with IncludeFixer.h but probably worth it anyway since this containing function is a complicated mess.

clangd/ClangdUnit.h

64

(nit: can we keep this as a plain struct? Once the number of members get large, initializing them one-by-one probably reads better anyway...)

74

Index is a reasonable (if surprising) param here, but I think we need to explicitly enable the include-fixing behavior. Even the very minimal hard-coded list of clang-tidy checks caused issues :-( And this is going to result in new network RPCs in some configs.

I'd suggest keeping the Index param, and wrapping the "use include fixer?" policy along with the clang-tidy options in D55256 as some sort of "diagnostic options" struct. (Though be wary of name clash with the one in Diagnostics.h, sigh).

clangd/Diagnostics.h

103

This seems like a suspicious dependency.
Can we indirect here e.g. void contributeFixes(std::function<vector<Fix>(const clang::Diagnostic&)>)

clangd/IncludeFixer.cpp

2

new copyright notice

40

can you add a trace to this function so we know what the impact on latency is?

67

limit?

75

so issuing a bunch of fuzzy finds in sequence is clearly not ideal from a performance standpoint.
Any ideas on what we might do better, or how we can limit the worst case?
(not sure we need to do any better in this patch, but might be worth comments)

113

should be no need for toString here

clangd/IncludeFixer.h

2

new copyright notice

40

nit: incomplete is one word

unittests/clangd/IncludeFixerTests.cpp

1 ↗

(On Diff #182796)

new copyright notice; wrong filename

26 ↗

(On Diff #182796)

because the helpers and public APIs tested are common, I wonder if we'd be better off creating DiagnosticsTests.cpp and moving the relevant tests from both ClangdUnit and here to there.

(Testing the fixes directly through the IncludeFixer class would make sense in this file, but it's awkward - I think testing through the diagnostics API is actually the right thing)

[ilya-biryukov]

A drop-by comment.

clangd/IncludeFixer.cpp

67

Why do we use fuzzyFind and not lookup here?
Are there cases when we can't construct SymbolID for the TagDecl?

[ioeric]

• Address review comments

[ioeric]

In D56903#1365487, @sammccall wrote:

This looks pretty good! My main concern is latency (and variability of latency) in practice.
Particularly:

• we should avoid paying for fuzzyfind and fetching hundreds of results when we want exact-match
• limit the damage in degenerate cases: should we limit to e.g. 5 index queries per TU?

Actually, now that I think about it - if we can see a forward declaration, can't we do a lookup() instead of a fuzzyFind?
Is the problem that this doesn't generalize to the no-declaration case (where it looks like a typo)?

I switched to use lookup requests for incomplete type diagnostics. And yes, the typo errors need to use FuzzyFind or lookup by names as no USR is available.

If we had an operation that looked like lookup() but worked by qname, would we design the feature the same way?
In particular, would we want to batch the requests to the index (lookup takes a set of IDs)? This would affect the code structure a bit. But it would make the feature cost basically O(1) instead of O(errs)...

As chatted offline, we decided to leave the batching optimization as a TODO in this patch. See inline comment for more detailed response regarding performance.

clangd/ClangdUnit.h

74

How about ParseOptions?

clangd/IncludeFixer.cpp

40

Added a tracer in fixInCompleteType to avoid noise from unsupported diagnostics.

67

Switched to use lookup. The typo diagnostics (i.e. undefined symbol) can't use lookup as there is no declaration, but we can definitely use lookup for incomplete types in this patch.

75

As you suggested, batching requests from all diagnostics would definitely help. SymbolIndex already supports batch lookup by IDs, but we would need to extend SymbolIndex to support lookup by Names for the typo errors that we are handling in D57021. In order to batch index requests from all diagnostics, the existing logic around handling and storing diagnostics might also need refactoring. I added a TODO for this. We can revisit if the performance turned out to be a big problem.

Another thing we could do is adding cache for index results. For example, for the code copy-paste case, one incomplete/undefined symbol can cause multiple errors. Caching should save us some unnecessary index queries.

For this patch, I added a limit on the number of index requests in IncludeFixer (5 for now), which would prevent us from sending too many index requests when building AST.

[ioeric]

Friendly ping.

[ioeric]

• minor touch on documentation.

[ioeric]

• Rebase correctly

[sammccall]

Just a bunch of nits, up to you to work out how/whether to address.

One substantive issue around classes defined outside headers - will chat offline.

clangd/ClangdServer.h

118

Can we call this option SuggestMissingIncludes or so? include-fixer is the name of a separate tool.

clangd/ClangdUnit.cpp

291

nit: can you hoist this above the line declaring FixIncludes?
That way (I think) it reads more naturally as "this is what the following block of code does" in a big method, doesn't fold away with the if, etc

305

nit: I'm not sure I agree with the IncludeFixer class taking ownership of the include inserter.
It reads it but never writes to it, so full ownership isn't *needed*. And if we had other features that wanted to compute include insertions, we'd want them to share an inserter.

clangd/Compiler.h

48

isn't an optional pointer just a pointer?

clangd/IncludeFixer.cpp

40

consider moving this 5 to a constructor param, so this file can be entirely mechanism and the policy is spelled out at the construction site.
(No need to actually make it configurable, I think)

42

(You could also just write this as just a switch on Info.getID(), I'd find that a little direct/easier to read, up to you)

62

TypeName?

64

this seems an odd thing to check and way to phrase - there *was* a type in the message, but its name happens to be empty. Am I missing something? (I'd suggest just removing this)

81

fixme is obsolete

82

no need to check name

87

There's a case I'm worried about:

Foo.h declares class X;, Foo.cpp defines class X{}.
Now Bar.cpp tries to do sizeof(X), so we get an incomplete type diagnostic.

• suggesting #include Foo.cpp is bad - can't include impl file
• suggesting #include Foo.h is also bad - doesn't help
• providing no suggestion is correct.

I'm not sure the index really has enough structured information to detect this case, but we could check e.g. that definition header == canonical declaration header? Or that basename(IncludeHeaders) == basename(definition header)?

If possible, can you add a test for this case?

clangd/IncludeFixer.h

40

not done - method is still "fixInCompleteType"

clangd/tool/ClangdMain.cpp

210

similarly, I'd call this -suggest-missing-includes or just -suggest-includes

214

why hidden?

[ioeric]

• address review comments

clangd/CMakeLists.txt

add_clang_library(clangDaemon
   ExpectedTypes.cpp
   FindSymbols.cpp
   FileDistance.cpp
   FS.cpp
   FSProvider.cpp
   FuzzyMatch.cpp
   GlobalCompilationDatabase.cpp
   Headers.cpp
+  IncludeFixer.cpp
   JSONTransport.cpp
   Logger.cpp
   Protocol.cpp
   Quality.cpp
   RIFF.cpp
   SourceCode.cpp
   Threading.cpp
   Trace.cpp

clangd/ClangdServer.h

struct Options {
     /// defaults and -resource-dir compiler flag).
     /// If None, ClangdServer calls CompilerInvocation::GetResourcePath() to
     /// obtain the standard resource directory.
     llvm::Optional<std::string> ResourceDir = llvm::None;
 
     /// Time to wait after a new file version before computing diagnostics.
     std::chrono::steady_clock::duration UpdateDebounce =
         std::chrono::milliseconds(500);
+
+    bool EnableIncludeFixer = false;
   };
   // Sensible default options for use in tests.
   // Features like indexing must be enabled if desired.
   static Options optsForTest();
 
   /// Creates a new ClangdServer instance.
   ///
   /// ClangdServer uses \p CDB to obtain compilation arguments for parsing. Note
private:
   // If present, the new "auto-index" maintained in background threads.
   std::unique_ptr<BackgroundIndex> BackgroundIdx;
   // Storage for merged views of the various indexes.
   std::vector<std::unique_ptr<SymbolIndex>> MergedIdx;
 
   // The provider used to provide a clang-tidy option for a specific file.
   tidy::ClangTidyOptionsProvider *ClangTidyOptProvider = nullptr;
 
+  // If this is true, add missing includes for errors like incomplete types.
+  bool EnableIncludeFixer = false;
+
   // GUARDED_BY(CachedCompletionFuzzyFindRequestMutex)
   llvm::StringMap<llvm::Optional<FuzzyFindRequest>>
       CachedCompletionFuzzyFindRequestByFile;
   mutable std::mutex CachedCompletionFuzzyFindRequestMutex;
 
   llvm::Optional<std::string> WorkspaceRoot;
   std::shared_ptr<PCHContainerOperations> PCHs;
   // WorkScheduler has to be the last member, because its destructor has to be

clangd/ClangdServer.cpp

ClangdServer::ClangdServer(const GlobalCompilationDatabase &CDB,
                            const FileSystemProvider &FSProvider,
                            DiagnosticsConsumer &DiagConsumer,
                            const Options &Opts)
     : CDB(CDB), FSProvider(FSProvider),
       DynamicIdx(Opts.BuildDynamicSymbolIndex
                      ? new FileIndex(Opts.HeavyweightDynamicSymbolIndex)
                      : nullptr),
       ClangTidyOptProvider(Opts.ClangTidyOptProvider),
+      EnableIncludeFixer(Opts.EnableIncludeFixer),
       WorkspaceRoot(Opts.WorkspaceRoot),
       PCHs(std::make_shared<PCHContainerOperations>()),
       // Pass a callback into `WorkScheduler` to extract symbols from a newly
       // parsed file and rebuild the file index synchronously each time an AST
       // is parsed.
       // FIXME(ioeric): this can be slow and we may be able to index on less
       // critical paths.
       WorkScheduler(Opts.AsyncThreadsCount, Opts.StorePreamblesInMemory,
if (Opts.BackgroundIndex) {
     AddIndex(BackgroundIdx.get());
   }
   if (DynamicIdx)
     AddIndex(DynamicIdx.get());
 }
 
 void ClangdServer::addDocument(PathRef File, llvm::StringRef Contents,
                                WantDiagnostics WantDiags) {
-  tidy::ClangTidyOptions Options = tidy::ClangTidyOptions::getDefaults();
+  ParseOptions Opts;
+  Opts.ClangTidyOpts = tidy::ClangTidyOptions::getDefaults();
   if (ClangTidyOptProvider)
-    Options = ClangTidyOptProvider->getOptions(File);
+    Opts.ClangTidyOpts = ClangTidyOptProvider->getOptions(File);
+  Opts.EnableIncludeFixer = EnableIncludeFixer;
   // FIXME: some build systems like Bazel will take time to preparing
   // environment to build the file, it would be nice if we could emit a
   // "PreparingBuild" status to inform users, it is non-trivial given the
   // current implementation.
-  WorkScheduler.update(File, ParseInputs{getCompileCommand(File),
-                                         FSProvider.getFileSystem(),
-                                         Contents.str(), Options},
+  WorkScheduler.update(File,
+                       ParseInputs{getCompileCommand(File),
+                                   FSProvider.getFileSystem(), Contents.str(),
+                                   Index, std::move(Opts)},
                        WantDiags);
 }
 
 void ClangdServer::removeDocument(PathRef File) { WorkScheduler.remove(File); }
 
 void ClangdServer::codeComplete(PathRef File, Position Pos,
                                 const clangd::CodeCompleteOptions &Opts,
                                 Callback<CodeCompleteResult> CB) {

clangd/ClangdUnit.h

 
 #include "Compiler.h"
 #include "Diagnostics.h"
 #include "FS.h"
 #include "Function.h"
 #include "Headers.h"
 #include "Path.h"
 #include "Protocol.h"
+#include "index/Index.h"
 #include "clang/Frontend/FrontendAction.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Serialization/ASTBitCodes.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "clang/Tooling/Core/Replacement.h"
 #include <memory>
 #include <string>
struct PreambleData {
   // Processes like code completions and go-to-definitions will need #include
   // information, and their compile action skips preamble range.
   IncludeStructure Includes;
   // Cache of FS operations performed when building the preamble.
   // When reusing a preamble, this cache can be consumed to save IO.
   std::unique_ptr<PreambleFileStatusCache> StatCache;
 };
 
 /// Stores and provides access to parsed AST.
 class ParsedAST {
 public:
   /// Attempts to run Clang and store parsed AST. If \p Preamble is non-null
   /// it is reused during parsing.
   static llvm::Optional<ParsedAST>
   build(std::unique_ptr<clang::CompilerInvocation> CI,
         std::shared_ptr<const PreambleData> Preamble,
         std::unique_ptr<llvm::MemoryBuffer> Buffer,
         std::shared_ptr<PCHContainerOperations> PCHs,
-        IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS,
-        const tidy::ClangTidyOptions &ClangTidyOpts);
+        IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS, const SymbolIndex *Index,
+        const ParseOptions &Opts);
 
   ParsedAST(ParsedAST &&Other);
   ParsedAST &operator=(ParsedAST &&Other);
 
   ~ParsedAST();
 
   /// Note that the returned ast will not contain decls from the preamble that
   /// were not deserialized during parsing. Clients should expect only decls

clangd/ClangdUnit.cpp

 //===--- ClangdUnit.cpp ------------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "ClangdUnit.h"
 #include "../clang-tidy/ClangTidyDiagnosticConsumer.h"
 #include "../clang-tidy/ClangTidyModuleRegistry.h"
 #include "Compiler.h"
 #include "Diagnostics.h"
+#include "Headers.h"
+#include "IncludeFixer.h"
 #include "Logger.h"
 #include "SourceCode.h"
 #include "Trace.h"
+#include "index/Index.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/Frontend/Utils.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexingAction.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Lex/MacroInfo.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Sema/Sema.h"
 #include "clang/Serialization/ASTWriter.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 
 namespace clang {
 namespace clangd {
 namespace {

 }
 
 llvm::Optional<ParsedAST>
 ParsedAST::build(std::unique_ptr<CompilerInvocation> CI,
                  std::shared_ptr<const PreambleData> Preamble,
                  std::unique_ptr<llvm::MemoryBuffer> Buffer,
                  std::shared_ptr<PCHContainerOperations> PCHs,
                  llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS,
-                 const tidy::ClangTidyOptions &ClangTidyOpts) {
+                 const SymbolIndex *Index, const ParseOptions &Opts) {
   assert(CI);
   // Command-line parsing sets DisableFree to true by default, but we don't want
   // to leak memory in clangd.
   CI->getFrontendOpts().DisableFree = false;
   const PrecompiledPreamble *PreamblePCH =
       Preamble ? &Preamble->Preamble : nullptr;
 
   StoreDiags ASTDiags;
+  std::string Content = Buffer->getBuffer();
+
   auto Clang =
       prepareCompilerInstance(std::move(CI), PreamblePCH, std::move(Buffer),
-                              std::move(PCHs), std::move(VFS), ASTDiags);
+                              std::move(PCHs), VFS, ASTDiags);
   if (!Clang)
     return None;
 
   auto Action = llvm::make_unique<ClangdFrontendAction>();
   const FrontendInputFile &MainInput = Clang->getFrontendOpts().Inputs[0];
   if (!Action->BeginSourceFile(*Clang, MainInput)) {
     log("BeginSourceFile() failed when building AST for {0}",
         MainInput.getFile());
ParsedAST::build(std::unique_ptr<CompilerInvocation> CI,
   ast_matchers::MatchFinder CTFinder;
   llvm::Optional<tidy::ClangTidyContext> CTContext;
   {
     trace::Span Tracer("ClangTidyInit");
     tidy::ClangTidyCheckFactories CTFactories;
     for (const auto &E : tidy::ClangTidyModuleRegistry::entries())
       E.instantiate()->addCheckFactories(CTFactories);
     CTContext.emplace(llvm::make_unique<tidy::DefaultOptionsProvider>(
-        tidy::ClangTidyGlobalOptions(), ClangTidyOpts));
+        tidy::ClangTidyGlobalOptions(), Opts.ClangTidyOpts));
     CTContext->setDiagnosticsEngine(&Clang->getDiagnostics());
     CTContext->setASTContext(&Clang->getASTContext());
     CTContext->setCurrentFile(MainInput.getFile());
     CTFactories.createChecks(CTContext.getPointer(), CTChecks);
     for (const auto &Check : CTChecks) {
       // FIXME: the PP callbacks skip the entire preamble.
       // Checks that want to see #includes in the main file do not see them.
       Check->registerPPCallbacks(*Clang);
       Check->registerMatchers(&CTFinder);
     }
   }
 
+  llvm::Optional<IncludeFixer> FixIncludes;
+  auto BuildDir = VFS->getCurrentWorkingDirectory();
+  if (Opts.EnableIncludeFixer && Index && !BuildDir.getError()) {
+    // Add IncludeFixer which can recorver diagnostics caused by missing
+    // includes (e.g. incomplete type) and attach include insertion fixes to
+    // diagnostics.
+    auto Style = getFormatStyleForFile(MainInput.getFile(), Content, VFS.get());
+    auto Inserter = llvm::make_unique<IncludeInserter>(
+        MainInput.getFile(), Content, Style, BuildDir.get(),
+        Clang->getPreprocessor().getHeaderSearchInfo());
+    if (Preamble) {
+      for (const auto &Inc : Preamble->Includes.MainFileIncludes)
+        Inserter->addExisting(Inc);
+    }
+    FixIncludes.emplace(MainInput.getFile(), std::move(Inserter), *Index);
+    ASTDiags.contributeFixes([&FixIncludes](DiagnosticsEngine::Level DiagLevl,
+                                            const clang::Diagnostic &Info) {
+      return FixIncludes->fix(DiagLevl, Info);
+    });
+  }
+
   // Copy over the includes from the preamble, then combine with the
   // non-preamble includes below.
   auto Includes = Preamble ? Preamble->Includes : IncludeStructure{};
   // Replay the preamble includes so that clang-tidy checks can see them.
   if (Preamble)
     ReplayPreamble::attach(Includes, *Clang);
   // Important: collectIncludeStructure is registered *after* ReplayPreamble!
   // Otherwise we would collect the replayed includes again...
buildAST(PathRef FileName, std::unique_ptr<CompilerInvocation> Invocation,
   if (Preamble && Preamble->StatCache)
     VFS = Preamble->StatCache->getConsumingFS(std::move(VFS));
   if (VFS->setCurrentWorkingDirectory(Inputs.CompileCommand.Directory)) {
     log("Couldn't set working directory when building the preamble.");
     // We proceed anyway, our lit-tests rely on results for non-existing working
     // dirs.
   }
 
-  return ParsedAST::build(llvm::make_unique<CompilerInvocation>(*Invocation),
-                          Preamble,
-                          llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents),
-                          PCHs, std::move(VFS), Inputs.ClangTidyOpts);
+  return ParsedAST::build(
+      llvm::make_unique<CompilerInvocation>(*Invocation), Preamble,
+      llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents), PCHs,
+      std::move(VFS), Inputs.Index ? *Inputs.Index : nullptr, Inputs.Opts);
 }
 
 SourceLocation getBeginningOfIdentifier(ParsedAST &Unit, const Position &Pos,
                                         const FileID FID) {
   const ASTContext &AST = Unit.getASTContext();
   const SourceManager &SourceMgr = AST.getSourceManager();
   auto Offset = positionToOffset(SourceMgr.getBufferData(FID), Pos);
   if (!Offset) {

clangd/CodeComplete.cpp

for (const auto &Chunk : CCS) {
     default:
       Result += Chunk.Text;
       break;
     }
   }
   return Result;
 }
 
-/// Creates a `HeaderFile` from \p Header which can be either a URI or a literal
-/// include.
-static llvm::Expected<HeaderFile> toHeaderFile(llvm::StringRef Header,
-                                               llvm::StringRef HintPath) {
-  if (isLiteralInclude(Header))
-    return HeaderFile{Header.str(), /*Verbatim=*/true};
-  auto U = URI::parse(Header);
-  if (!U)
-    return U.takeError();
-
-  auto IncludePath = URI::includeSpelling(*U);
-  if (!IncludePath)
-    return IncludePath.takeError();
-  if (!IncludePath->empty())
-    return HeaderFile{std::move(*IncludePath), /*Verbatim=*/true};
-
-  auto Resolved = URI::resolve(*U, HintPath);
-  if (!Resolved)
-    return Resolved.takeError();
-  return HeaderFile{std::move(*Resolved), /*Verbatim=*/false};
-}
-
 /// A code completion result, in clang-native form.
 /// It may be promoted to a CompletionItem if it's among the top-ranked results.
 struct CompletionCandidate {
   llvm::StringRef Name; // Used for filtering and sorting.
   // We may have a result from Sema, from the index, or both.
   const CodeCompletionResult *SemaResult = nullptr;
   const Symbol *IndexResult = nullptr;
   llvm::SmallVector<llvm::StringRef, 1> RankedIncludeHeaders;

 bool semaCodeComplete(std::unique_ptr<CodeCompleteConsumer> Consumer,
                       const clang::CodeCompleteOptions &Options,
                       const SemaCompleteInput &Input,
                       IncludeStructure *Includes = nullptr) {
   trace::Span Tracer("Sema completion");
   llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS = Input.VFS;
   if (Input.Preamble && Input.Preamble->StatCache)
     VFS = Input.Preamble->StatCache->getConsumingFS(std::move(VFS));
-  auto CI = buildCompilerInvocation(
-      ParseInputs{Input.Command, VFS, Input.Contents,
-                  tidy::ClangTidyOptions::getDefaults()});
+  auto CI =
+      buildCompilerInvocation(ParseInputs{Input.Command, VFS, Input.Contents,
+                                          /*Index=*/nullptr, ParseOptions()});
   if (!CI) {
     elog("Couldn't create CompilerInvocation");
     return false;
   }
   auto &FrontendOpts = CI->getFrontendOpts();
   FrontendOpts.SkipFunctionBodies = true;
   // Disable typo correction in Sema.
   CI->getLangOpts()->SpellChecking = false;
elog("Failed to speculate filter text for code completion at Pos "
          "{0}:{1}: {2}",
          Pos.line, Pos.character, Filter.takeError());
     return None;
   }
   CachedReq.Query = *Filter;
   return CachedReq;
 }
 
-// Returns the most popular include header for \p Sym. If two headers are
-// equally popular, prefer the shorter one. Returns empty string if \p Sym has
-// no include header.
-llvm::SmallVector<llvm::StringRef, 1> getRankedIncludes(const Symbol &Sym) {
-  auto Includes = Sym.IncludeHeaders;
-  // Sort in descending order by reference count and header length.
-  llvm::sort(Includes, [](const Symbol::IncludeHeaderWithReferences &LHS,
-                          const Symbol::IncludeHeaderWithReferences &RHS) {
-    if (LHS.References == RHS.References)
-      return LHS.IncludeHeader.size() < RHS.IncludeHeader.size();
-    return LHS.References > RHS.References;
-  });
-  llvm::SmallVector<llvm::StringRef, 1> Headers;
-  for (const auto &Include : Includes)
-    Headers.push_back(Include.IncludeHeader);
-  return Headers;
-}
-
 // Runs Sema-based (AST) and Index-based completion, returns merged results.
 //
 // There are a few tricky considerations:
 //   - the AST provides information needed for the index query (e.g. which
 //     namespaces to search in). So Sema must start first.
 //   - we only want to return the top results (Opts.Limit).
 //     Building CompletionItems for everything else is wasteful, so we want to
 //     preserve the "native" format until we're done with scoring.
CodeCompleteResult run(const SemaCompleteInput &SemaCCInput) && {
     }
 
     // We run Sema code completion first. It builds an AST and calculates:
     //   - completion results based on the AST.
     //   - partial identifier and context. We need these for the index query.
     CodeCompleteResult Output;
     auto RecorderOwner = llvm::make_unique<CompletionRecorder>(Opts, [&]() {
       assert(Recorder && "Recorder is not set");
-      auto Style =
-          format::getStyle(format::DefaultFormatStyle, SemaCCInput.FileName,
-                           format::DefaultFallbackStyle, SemaCCInput.Contents,
-                           SemaCCInput.VFS.get());
-      if (!Style) {
-        log("getStyle() failed for file {0}: {1}. Fallback is LLVM style.",
-            SemaCCInput.FileName, Style.takeError());
-        Style = format::getLLVMStyle();
-      }
+      auto Style = getFormatStyleForFile(
+          SemaCCInput.FileName, SemaCCInput.Contents, SemaCCInput.VFS.get());
       // If preprocessor was run, inclusions from preprocessor callback should
       // already be added to Includes.
       Inserter.emplace(
-          SemaCCInput.FileName, SemaCCInput.Contents, *Style,
+          SemaCCInput.FileName, SemaCCInput.Contents, Style,
           SemaCCInput.Command.Directory,
           Recorder->CCSema->getPreprocessor().getHeaderSearchInfo());
       for (const auto &Inc : Includes.MainFileIncludes)
         Inserter->addExisting(Inc);
 
       // Most of the cost of file proximity is in initializing the FileDistance
       // structures based on the observed includes, once per query. Conceptually
       // that happens here (though the per-URI-scheme initialization is lazy).

clangd/Compiler.h

 // run their own compile actions that share logic.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_COMPILER_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_COMPILER_H
 
 #include "../clang-tidy/ClangTidyOptions.h"
+#include "index/Index.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Tooling/CompilationDatabase.h"
 
 namespace clang {
 namespace clangd {
 
 class IgnoreDiagnostics : public DiagnosticConsumer {
 public:
   static void log(DiagnosticsEngine::Level DiagLevel,
                   const clang::Diagnostic &Info);
 
   void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
                         const clang::Diagnostic &Info) override;
 };
 
+struct ParseOptions {
+  tidy::ClangTidyOptions ClangTidyOpts;
+  bool EnableIncludeFixer = false;
+};
+
 /// Information required to run clang, e.g. to parse AST or do code completion.
 struct ParseInputs {
   tooling::CompileCommand CompileCommand;
   IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS;
   std::string Contents;
-  tidy::ClangTidyOptions ClangTidyOpts;
+  // Used to recover from diagnostics (e.g. find missing includes for symbol).
+  llvm::Optional<const SymbolIndex *> Index;
+  ParseOptions Opts;
 };
 
 /// Builds compiler invocation that could be used to build AST or preamble.
 std::unique_ptr<CompilerInvocation>
 buildCompilerInvocation(const ParseInputs &Inputs);
 
 /// Creates a compiler instance, configured so that:
 ///   - Contents of the parsed file are remapped to \p MainFile.

clangd/Diagnostics.h

 public:
   std::vector<Diag> take();
 
   void BeginSourceFile(const LangOptions &Opts, const Preprocessor *) override;
   void EndSourceFile() override;
   void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
                         const clang::Diagnostic &Info) override;
 
+  using DiagFixer = std::function<std::vector<Fix>(DiagnosticsEngine::Level,
+                                                   const clang::Diagnostic &)>;
+  /// If set, possibly adds fixes for diagnostics using \p Fixer.
+  void contributeFixes(DiagFixer Fixer) { this->Fixer = Fixer; }
+
 private:
   void flushLastDiag();
 
+  DiagFixer Fixer = nullptr;
   std::vector<Diag> Output;
   llvm::Optional<LangOptions> LangOpts;
   llvm::Optional<Diag> LastDiag;
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_DIAGNOSTICS_H

clangd/Diagnostics.cpp

if (!isNote(DiagLevel)) {
     // Handle the new main diagnostic.
     flushLastDiag();
 
     LastDiag = Diag();
     FillDiagBase(*LastDiag);
 
     if (!Info.getFixItHints().empty())
       AddFix(true /* try to invent a message instead of repeating the diag */);
+    if (Fixer) {
+      auto ExtraFixes = Fixer(DiagLevel, Info);
+      LastDiag->Fixes.insert(LastDiag->Fixes.end(), ExtraFixes.begin(),
+                             ExtraFixes.end());
+    }
   } else {
     // Handle a note to an existing diagnostic.
     if (!LastDiag) {
       assert(false && "Adding a note without main diagnostic");
       IgnoreDiagnostics::log(DiagLevel, Info);
       return;
     }
 

clangd/Headers.h

 //===--- Headers.h - Include headers -----------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_HEADERS_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_HEADERS_H
 
 #include "Path.h"
 #include "Protocol.h"
 #include "SourceCode.h"
+#include "index/Index.h"
 #include "clang/Format/Format.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/PPCallbacks.h"
 #include "clang/Tooling/Inclusions/HeaderIncludes.h"
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/StringSet.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/VirtualFileSystem.h"
 
 namespace clang {
 namespace clangd {
 
 /// Returns true if \p Include is literal include like "path" or <path>.
 bool isLiteralInclude(llvm::StringRef Include);
 
 /// Represents a header file to be #include'd.
 struct HeaderFile {
   std::string File;
   /// If this is true, `File` is a literal string quoted with <> or "" that
   /// can be #included directly; otherwise, `File` is an absolute file path.
   bool Verbatim;
 
   bool valid() const;
 };
 
+/// Creates a `HeaderFile` from \p Header which can be either a URI or a literal
+/// include.
+llvm::Expected<HeaderFile> toHeaderFile(llvm::StringRef Header,
+                                        llvm::StringRef HintPath);
+
+// Returns include headers for \p Sym sorted by popularity. If two headers are
+// equally popular, prefer the shorter one.
+llvm::SmallVector<llvm::StringRef, 1> getRankedIncludes(const Symbol &Sym);
+
 // An #include directive that we found in the main file.
 struct Inclusion {
   Range R;             // Inclusion range.
   std::string Written; // Inclusion name as written e.g. <vector>.
   Path Resolved;       // Resolved path of included file. Empty if not resolved.
   unsigned HashOffset = 0; // Byte offset from start of file to #.
   SrcMgr::CharacteristicKind FileKind = SrcMgr::C_User;
 };

clangd/Headers.cpp

bool isLiteralInclude(llvm::StringRef Include) {
   return Include.startswith("<") || Include.startswith("\"");
 }
 
 bool HeaderFile::valid() const {
   return (Verbatim && isLiteralInclude(File)) ||
          (!Verbatim && llvm::sys::path::is_absolute(File));
 }
 
+llvm::Expected<HeaderFile> toHeaderFile(llvm::StringRef Header,
+                                        llvm::StringRef HintPath) {
+  if (isLiteralInclude(Header))
+    return HeaderFile{Header.str(), /*Verbatim=*/true};
+  auto U = URI::parse(Header);
+  if (!U)
+    return U.takeError();
+
+  auto IncludePath = URI::includeSpelling(*U);
+  if (!IncludePath)
+    return IncludePath.takeError();
+  if (!IncludePath->empty())
+    return HeaderFile{std::move(*IncludePath), /*Verbatim=*/true};
+
+  auto Resolved = URI::resolve(*U, HintPath);
+  if (!Resolved)
+    return Resolved.takeError();
+  return HeaderFile{std::move(*Resolved), /*Verbatim=*/false};
+}
+
+llvm::SmallVector<llvm::StringRef, 1> getRankedIncludes(const Symbol &Sym) {
+  auto Includes = Sym.IncludeHeaders;
+  // Sort in descending order by reference count and header length.
+  llvm::sort(Includes, [](const Symbol::IncludeHeaderWithReferences &LHS,
+                          const Symbol::IncludeHeaderWithReferences &RHS) {
+    if (LHS.References == RHS.References)
+      return LHS.IncludeHeader.size() < RHS.IncludeHeader.size();
+    return LHS.References > RHS.References;
+  });
+  llvm::SmallVector<llvm::StringRef, 1> Headers;
+  for (const auto &Include : Includes)
+    Headers.push_back(Include.IncludeHeader);
+  return Headers;
+}
+
 std::unique_ptr<PPCallbacks>
 collectIncludeStructureCallback(const SourceManager &SM,
                                 IncludeStructure *Out) {
   return llvm::make_unique<RecordHeaders>(SM, Out);
 }
 
 void IncludeStructure::recordInclude(llvm::StringRef IncludingName,
                                      llvm::StringRef IncludedName,

clangd/IncludeFixer.h

+//===--- IncludeFixer.h ------------------------------------------*- C++-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_INCLUDE_FIXER_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_INCLUDE_FIXER_H
+
+#include "Diagnostics.h"
+#include "Headers.h"
+#include "index/Index.h"
+#include "clang/AST/Type.h"
+#include "clang/Basic/Diagnostic.h"
+#include "llvm/ADT/StringRef.h"
+#include <memory>
+
+namespace clang {
+namespace clangd {
+
+/// Attempts to recover from error diagnostics by suggesting include insertion
+/// fixes. For example, member access into incomplete type can be fixes by
+/// include headers with the definition.
+class IncludeFixer {
+public:
+  IncludeFixer(llvm::StringRef File, std::unique_ptr<IncludeInserter> Inserter,
+               const SymbolIndex &Index)
+      : File(File), Inserter(std::move(Inserter)), Index(Index) {}
+
+  /// Returns include insertions that can potentially recover the diagnostic.
+  std::vector<Fix> fix(DiagnosticsEngine::Level DiagLevel,
+                       const clang::Diagnostic &Info) const;
+
+private:
+  /// Attempts to recover diagnostic caused by an incomplete type \p T.
+  std::vector<Fix> fixInCompleteType(const Type &T) const;
+
+  /// Generates header insertion fixes for \p Sym.
+  std::vector<Fix> fixesForSymbol(const Symbol &Sym) const;
+
+  std::string File;
+  std::unique_ptr<IncludeInserter> Inserter;
+  const SymbolIndex &Index;
+  mutable unsigned IndexRequestCount = 0;
+};
+
+} // namespace clangd
+} // namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_INCLUDE_FIXER_H

clangd/IncludeFixer.cpp

+//===--- IncludeFixer.cpp ----------------------------------------*- C++-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "IncludeFixer.h"
+#include "AST.h"
+#include "Diagnostics.h"
+#include "Logger.h"
+#include "SourceCode.h"
+#include "Trace.h"
+#include "index/Index.h"
+#include "clang/AST/Decl.h"
+#include "clang/AST/Type.h"
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticSema.h"
+#include "llvm/ADT/None.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FormatVariadic.h"
+
+namespace clang {
+namespace clangd {
+
+namespace {
+
+bool isIncompleteTypeDiag(unsigned int DiagID) {
+  return DiagID == diag::err_incomplete_type ||
+         DiagID == diag::err_incomplete_member_access ||
+         DiagID == diag::err_incomplete_base_class;
+}
+
+} // namespace
+
+std::vector<Fix> IncludeFixer::fix(DiagnosticsEngine::Level DiagLevel,
+                                   const clang::Diagnostic &Info) const {
+  if (IndexRequestCount >= 5)
+    return {}; // Avoid querying index too many times in a single parse.
+  if (isIncompleteTypeDiag(Info.getID())) {
+    // Incomplete type diagnostics should have a QualType argument for the
+    // incomplete type.
+    for (unsigned i = 0; i < Info.getNumArgs(); ++i) {
+      if (Info.getArgKind(i) == DiagnosticsEngine::ak_qualtype) {
+        auto QT = QualType::getFromOpaquePtr((void *)Info.getRawArg(i));
+        if (const Type *T = QT.getTypePtrOrNull())
+          if (T->isIncompleteType())
+            return fixInCompleteType(*T);
+      }
+    }
+  }
+  return {};
+}
+
+std::vector<Fix> IncludeFixer::fixInCompleteType(const Type &T) const {
+  // Only handle incomplete TagDecl type.
+  const TagDecl *TD = T.getAsTagDecl();
+  if (!TD)
+    return {};
+  std::string IncompleteType = printQualifiedName(*TD);
+  if (IncompleteType.empty()) {
+    vlog("No incomplete type name is found in diagnostic. Ignore.");
+    return {};
+  }
+  trace::Span Tracer("Fix include for incomplete type");
+  SPAN_ATTACH(Tracer, "type", IncompleteType);
+  vlog("Trying to fix include for incomplete type {0}", IncompleteType);
+
+  auto ID = getSymbolID(TD);
+  if (!ID)
+    return {};
+  ++IndexRequestCount;
+  // FIXME: consider batching the requests for all diagnostics.
+  // FIXME: consider caching the lookup results.
+  LookupRequest Req;
+  Req.IDs.insert(*ID);
+  llvm::Optional<Symbol> Matched;
+  Index.lookup(Req, [&](const Symbol &Sym) {
+    // FIXME: support multiple matched symbols.
+    if (Matched || (Sym.Scope + Sym.Name).str() != IncompleteType)
+      return;
+    Matched = Sym;
+  });
+
+  if (!Matched || Matched->IncludeHeaders.empty())
+    return {};
+  return fixesForSymbol(*Matched);
+}
+
+std::vector<Fix> IncludeFixer::fixesForSymbol(const Symbol &Sym) const {
+  auto Inserted = [&](llvm::StringRef Header)
+      -> llvm::Expected<std::pair<std::string, bool>> {
+    auto ResolvedDeclaring =
+        toHeaderFile(Sym.CanonicalDeclaration.FileURI, File);
+    if (!ResolvedDeclaring)
+      return ResolvedDeclaring.takeError();
+    auto ResolvedInserted = toHeaderFile(Header, File);
+    if (!ResolvedInserted)
+      return ResolvedInserted.takeError();
+    return std::make_pair(
+        Inserter->calculateIncludePath(*ResolvedDeclaring, *ResolvedInserted),
+        Inserter->shouldInsertInclude(*ResolvedDeclaring, *ResolvedInserted));
+  };
+
+  std::vector<Fix> Fixes;
+  for (const auto &Inc : getRankedIncludes(Sym)) {
+    if (auto ToInclude = Inserted(Inc)) {
+      if (ToInclude->second)
+        if (auto Edit = Inserter->insert(ToInclude->first))
+          Fixes.push_back(
+              Fix{llvm::formatv("Add include {0} for symbol {1}{2}",
+                                ToInclude->first, Sym.Scope, Sym.Name),
+                  {std::move(*Edit)}});
+    } else {
+      vlog("Failed to calculate include insertion for {0} into {1}: {2}", File,
+           Inc, ToInclude.takeError());
+    }
+  }
+  return Fixes;
+}
+
+} // namespace clangd
+} // namespace clang

clangd/SourceCode.h

 //
 //===----------------------------------------------------------------------===//
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_SOURCECODE_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_SOURCECODE_H
 #include "Protocol.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/SourceManager.h"
+#include "clang/Format/Format.h"
 #include "clang/Tooling/Core/Replacement.h"
+#include "llvm/ADT/StringRef.h"
 #include "llvm/Support/SHA1.h"
 
 namespace clang {
 class SourceManager;
 
 namespace clangd {
 
 // We tend to generate digests for source codes in a lot of different places.

 ///
 /// This function should be used when paths needs to be used outside the
 /// component that generate it, so that paths are normalized as much as
 /// possible.
 llvm::Optional<std::string> getCanonicalPath(const FileEntry *F,
                                              const SourceManager &SourceMgr);
 
 bool IsRangeConsecutive(const Range &Left, const Range &Right);
+
+format::FormatStyle getFormatStyleForFile(llvm::StringRef File,
+                                          llvm::StringRef Content,
+                                          llvm::vfs::FileSystem *FS);
+
 } // namespace clangd
 } // namespace clang
 #endif

clangd/SourceCode.cpp

 llvm::Optional<FileDigest> digestFile(const SourceManager &SM, FileID FID) {
   bool Invalid = false;
   llvm::StringRef Content = SM.getBufferData(FID, &Invalid);
   if (Invalid)
     return None;
   return digest(Content);
 }
 
+format::FormatStyle getFormatStyleForFile(llvm::StringRef File,
+                                          llvm::StringRef Content,
+                                          llvm::vfs::FileSystem *FS) {
+  auto Style = format::getStyle(format::DefaultFormatStyle, File,
+                                format::DefaultFallbackStyle, Content, FS);
+  if (!Style) {
+    log("getStyle() failed for file {0}: {1}. Fallback is LLVM style.", File,
+        Style.takeError());
+    Style = format::getLLVMStyle();
+  }
+  return *Style;
+}
+
 } // namespace clangd
 } // namespace clang

clangd/tool/ClangdMain.cpp

static llvm::cl::opt<bool> EnableFunctionArgSnippets(
     llvm::cl::init(CodeCompleteOptions().EnableFunctionArgSnippets));
 
 static llvm::cl::opt<std::string> ClangTidyChecks(
     "clang-tidy-checks",
     llvm::cl::desc("List of clang-tidy checks to run (this will overrides "
                    ".clang-tidy files)"),
     llvm::cl::init(""), llvm::cl::Hidden);
 
+static llvm::cl::opt<bool> EnableIncludeFixer(
+    "include-fixer",
+    llvm::cl::desc("Attempts to fix diagnostic errors caused by missing "
+                   "includes using index."),
+    llvm::cl::init(false), llvm::cl::Hidden);
+
 namespace {
 
 /// \brief Supports a test URI scheme with relaxed constraints for lit tests.
 /// The path in a test URI will be combined with a platform-specific fake
 /// directory to form an absolute path. For example, test:///a.cpp is resolved
 /// C:\clangd-test\a.cpp on Windows and /clangd-test/a.cpp on Unix.
 class TestScheme : public URIScheme {
 public:
#endif
   // Create an empty clang-tidy option.
   auto OverrideClangTidyOptions = tidy::ClangTidyOptions::getDefaults();
   OverrideClangTidyOptions.Checks = ClangTidyChecks;
   tidy::FileOptionsProvider ClangTidyOptProvider(
       tidy::ClangTidyGlobalOptions(),
       /* Default */ tidy::ClangTidyOptions::getDefaults(),
       /* Override */ OverrideClangTidyOptions, FSProvider.getFileSystem());
   Opts.ClangTidyOptProvider = &ClangTidyOptProvider;
+  Opts.EnableIncludeFixer = EnableIncludeFixer;
   ClangdLSPServer LSPServer(
       *TransportLayer, FSProvider, CCOpts, CompileCommandsDirPath,
       /*UseDirBasedCDB=*/CompileArgsFrom == FilesystemCompileArgs, Opts);
   llvm::set_thread_name("clangd.main");
   return LSPServer.run() ? 0
                          : static_cast<int>(ErrorResultCode::NoShutdownRequest);
 }

unittests/clangd/CMakeLists.txt

add_extra_unittest(ClangdTests
   BackgroundIndexTests.cpp
   CancellationTests.cpp
   ClangdTests.cpp
   ClangdUnitTests.cpp
   CodeCompleteTests.cpp
   CodeCompletionStringsTests.cpp
   ContextTests.cpp
   DexTests.cpp
+  DiagnosticsTests.cpp
   DraftStoreTests.cpp
   ExpectedTypeTest.cpp
   FileDistanceTests.cpp
   FileIndexTests.cpp
   FindSymbolsTests.cpp
   FSTests.cpp
   FunctionTests.cpp
   FuzzyMatchTests.cpp

unittests/clangd/ClangdUnitTests.cpp

 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace clang {
 namespace clangd {
 namespace {
 
 using testing::ElementsAre;
-using testing::Field;
-using testing::IsEmpty;
-using testing::Pair;
-using testing::UnorderedElementsAre;
-
-testing::Matcher<const Diag &> WithFix(testing::Matcher<Fix> FixMatcher) {
-  return Field(&Diag::Fixes, ElementsAre(FixMatcher));
-}
-
-testing::Matcher<const Diag &> WithNote(testing::Matcher<Note> NoteMatcher) {
-  return Field(&Diag::Notes, ElementsAre(NoteMatcher));
-}
-
-MATCHER_P2(Diag, Range, Message,
-           "Diag at " + llvm::to_string(Range) + " = [" + Message + "]") {
-  return arg.Range == Range && arg.Message == Message;
-}
-
-MATCHER_P3(Fix, Range, Replacement, Message,
-           "Fix " + llvm::to_string(Range) + " => " +
-               testing::PrintToString(Replacement) + " = [" + Message + "]") {
-  return arg.Message == Message && arg.Edits.size() == 1 &&
-         arg.Edits[0].range == Range && arg.Edits[0].newText == Replacement;
-}
-
-MATCHER_P(EqualToLSPDiag, LSPDiag,
-          "LSP diagnostic " + llvm::to_string(LSPDiag)) {
-  return std::tie(arg.range, arg.severity, arg.message) ==
-         std::tie(LSPDiag.range, LSPDiag.severity, LSPDiag.message);
-}
-
-MATCHER_P(EqualToFix, Fix, "LSP fix " + llvm::to_string(Fix)) {
-  if (arg.Message != Fix.Message)
-    return false;
-  if (arg.Edits.size() != Fix.Edits.size())
-    return false;
-  for (std::size_t I = 0; I < arg.Edits.size(); ++I) {
-    if (arg.Edits[I].range != Fix.Edits[I].range ||
-        arg.Edits[I].newText != Fix.Edits[I].newText)
-      return false;
-  }
-  return true;
-}
-
-// Helper function to make tests shorter.
-Position pos(int line, int character) {
-  Position Res;
-  Res.line = line;
-  Res.character = character;
-  return Res;
-}
-
-TEST(DiagnosticsTest, DiagnosticRanges) {
-  // Check we report correct ranges, including various edge-cases.
-  Annotations Test(R"cpp(
-    namespace test{};
-    void $decl[[foo]]();
-    int main() {
-      $typo[[go\
-o]]();
-      foo()$semicolon[[]]//with comments
-      $unk[[unknown]]();
-      double $type[[bar]] = "foo";
-      struct Foo { int x; }; Foo a;
-      a.$nomember[[y]];
-      test::$nomembernamespace[[test]];
-    }
-  )cpp");
-  EXPECT_THAT(
-      TestTU::withCode(Test.code()).build().getDiagnostics(),
-      ElementsAre(
-          // This range spans lines.
-          AllOf(Diag(Test.range("typo"),
-                     "use of undeclared identifier 'goo'; did you mean 'foo'?"),
-                WithFix(
-                    Fix(Test.range("typo"), "foo", "change 'go\\ o' to 'foo'")),
-                // This is a pretty normal range.
-                WithNote(Diag(Test.range("decl"), "'foo' declared here"))),
-          // This range is zero-width and insertion. Therefore make sure we are
-          // not expanding it into other tokens. Since we are not going to
-          // replace those.
-          AllOf(Diag(Test.range("semicolon"), "expected ';' after expression"),
-                WithFix(Fix(Test.range("semicolon"), ";", "insert ';'"))),
-          // This range isn't provided by clang, we expand to the token.
-          Diag(Test.range("unk"), "use of undeclared identifier 'unknown'"),
-          Diag(Test.range("type"),
-               "cannot initialize a variable of type 'double' with an lvalue "
-               "of type 'const char [4]'"),
-          Diag(Test.range("nomember"), "no member named 'y' in 'Foo'"),
-          Diag(Test.range("nomembernamespace"),
-               "no member named 'test' in namespace 'test'")));
-}
-
-TEST(DiagnosticsTest, FlagsMatter) {
-  Annotations Test("[[void]] main() {}");
-  auto TU = TestTU::withCode(Test.code());
-  EXPECT_THAT(TU.build().getDiagnostics(),
-              ElementsAre(AllOf(Diag(Test.range(), "'main' must return 'int'"),
-                                WithFix(Fix(Test.range(), "int",
-                                            "change 'void' to 'int'")))));
-  // Same code built as C gets different diagnostics.
-  TU.Filename = "Plain.c";
-  EXPECT_THAT(
-      TU.build().getDiagnostics(),
-      ElementsAre(AllOf(
-          Diag(Test.range(), "return type of 'main' is not 'int'"),
-          WithFix(Fix(Test.range(), "int", "change return type to 'int'")))));
-}
-
-TEST(DiagnosticsTest, ClangTidy) {
-  Annotations Test(R"cpp(
-    #include $deprecated[["assert.h"]]
-
-    #define $macrodef[[SQUARE]](X) (X)*(X)
-    int main() {
-      return $doubled[[sizeof]](sizeof(int));
-      int y = 4;
-      return SQUARE($macroarg[[++]]y);
-    }
-  )cpp");
-  auto TU = TestTU::withCode(Test.code());
-  TU.HeaderFilename = "assert.h"; // Suppress "not found" error.
-  TU.ClangTidyChecks =
-      "-*, bugprone-sizeof-expression, bugprone-macro-repeated-side-effects, "
-      "modernize-deprecated-headers";
-  EXPECT_THAT(
-      TU.build().getDiagnostics(),
-      UnorderedElementsAre(
-          AllOf(Diag(Test.range("deprecated"),
-                     "inclusion of deprecated C++ header 'assert.h'; consider "
-                     "using 'cassert' instead [modernize-deprecated-headers]"),
-                WithFix(Fix(Test.range("deprecated"), "<cassert>",
-                            "change '\"assert.h\"' to '<cassert>'"))),
-          Diag(Test.range("doubled"),
-               "suspicious usage of 'sizeof(sizeof(...))' "
-               "[bugprone-sizeof-expression]"),
-          AllOf(
-              Diag(Test.range("macroarg"),
-                   "side effects in the 1st macro argument 'X' are repeated in "
-                   "macro expansion [bugprone-macro-repeated-side-effects]"),
-              WithNote(Diag(Test.range("macrodef"),
-                            "macro 'SQUARE' defined here "
-                            "[bugprone-macro-repeated-side-effects]"))),
-          Diag(Test.range("macroarg"),
-               "multiple unsequenced modifications to 'y'")));
-}
-
-TEST(DiagnosticsTest, Preprocessor) {
-  // This looks like a preamble, but there's an #else in the middle!
-  // Check that:
-  //  - the #else doesn't generate diagnostics (we had this bug)
-  //  - we get diagnostics from the taken branch
-  //  - we get no diagnostics from the not taken branch
-  Annotations Test(R"cpp(
-    #ifndef FOO
-    #define FOO
-      int a = [[b]];
-    #else
-      int x = y;
-    #endif
-    )cpp");
-  EXPECT_THAT(
-      TestTU::withCode(Test.code()).build().getDiagnostics(),
-      ElementsAre(Diag(Test.range(), "use of undeclared identifier 'b'")));
-}
-
-TEST(DiagnosticsTest, InsideMacros) {
-  Annotations Test(R"cpp(
-    #define TEN 10
-    #define RET(x) return x + 10
-
-    int* foo() {
-      RET($foo[[0]]);
-    }
-    int* bar() {
-      return $bar[[TEN]];
-    }
-    )cpp");
-  EXPECT_THAT(TestTU::withCode(Test.code()).build().getDiagnostics(),
-              ElementsAre(Diag(Test.range("foo"),
-                               "cannot initialize return object of type "
-                               "'int *' with an rvalue of type 'int'"),
-                          Diag(Test.range("bar"),
-                               "cannot initialize return object of type "
-                               "'int *' with an rvalue of type 'int'")));
-}
-
-TEST(DiagnosticsTest, ToLSP) {
-  clangd::Diag D;
-  D.Message = "something terrible happened";
-  D.Range = {pos(1, 2), pos(3, 4)};
-  D.InsideMainFile = true;
-  D.Severity = DiagnosticsEngine::Error;
-  D.File = "foo/bar/main.cpp";
-
-  clangd::Note NoteInMain;
-  NoteInMain.Message = "declared somewhere in the main file";
-  NoteInMain.Range = {pos(5, 6), pos(7, 8)};
-  NoteInMain.Severity = DiagnosticsEngine::Remark;
-  NoteInMain.File = "../foo/bar/main.cpp";
-  NoteInMain.InsideMainFile = true;
-  D.Notes.push_back(NoteInMain);
-
-  clangd::Note NoteInHeader;
-  NoteInHeader.Message = "declared somewhere in the header file";
-  NoteInHeader.Range = {pos(9, 10), pos(11, 12)};
-  NoteInHeader.Severity = DiagnosticsEngine::Note;
-  NoteInHeader.File = "../foo/baz/header.h";
-  NoteInHeader.InsideMainFile = false;
-  D.Notes.push_back(NoteInHeader);
-
-  clangd::Fix F;
-  F.Message = "do something";
-  D.Fixes.push_back(F);
-
-  auto MatchingLSP = [](const DiagBase &D, StringRef Message) {
-    clangd::Diagnostic Res;
-    Res.range = D.Range;
-    Res.severity = getSeverity(D.Severity);
-    Res.message = Message;
-    return Res;
-  };
-
-  // Diagnostics should turn into these:
-  clangd::Diagnostic MainLSP = MatchingLSP(D, R"(Something terrible happened
-
-main.cpp:6:7: remark: declared somewhere in the main file
-
-../foo/baz/header.h:10:11:
-note: declared somewhere in the header file)");
-
-  clangd::Diagnostic NoteInMainLSP =
-      MatchingLSP(NoteInMain, R"(Declared somewhere in the main file
-
-main.cpp:2:3: error: something terrible happened)");
-
-  // Transform dianostics and check the results.
-  std::vector<std::pair<clangd::Diagnostic, std::vector<clangd::Fix>>> LSPDiags;
-  toLSPDiags(D,
-#ifdef _WIN32
-             URIForFile::canonicalize("c:\\path\\to\\foo\\bar\\main.cpp",
-                                      /*TUPath=*/""),
-#else
-      URIForFile::canonicalize("/path/to/foo/bar/main.cpp", /*TUPath=*/""),
-#endif
-             ClangdDiagnosticOptions(),
-             [&](clangd::Diagnostic LSPDiag, ArrayRef<clangd::Fix> Fixes) {
-               LSPDiags.push_back(
-                   {std::move(LSPDiag),
-                    std::vector<clangd::Fix>(Fixes.begin(), Fixes.end())});
-             });
-
-  EXPECT_THAT(
-      LSPDiags,
-      ElementsAre(Pair(EqualToLSPDiag(MainLSP), ElementsAre(EqualToFix(F))),
-                  Pair(EqualToLSPDiag(NoteInMainLSP), IsEmpty())));
-}
 
 TEST(ClangdUnitTest, GetBeginningOfIdentifier) {
   std::string Preamble = R"cpp(
 struct Bar { int func(); };
 #define MACRO(X) void f() { X; }
 Bar* bar;
   )cpp";
   // First ^ is the expected beginning, last is the search position.

unittests/clangd/CodeCompleteTests.cpp

 #include "CodeComplete.h"
 #include "Compiler.h"
 #include "Matchers.h"
 #include "Protocol.h"
 #include "Quality.h"
 #include "SourceCode.h"
 #include "SyncAPI.h"
 #include "TestFS.h"
+#include "TestIndex.h"
 #include "index/MemIndex.h"
 #include "clang/Sema/CodeCompleteConsumer.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Testing/Support/Error.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace clang {
CodeCompleteResult completions(llvm::StringRef Text,
   MockFSProvider FS;
   MockCompilationDatabase CDB;
   IgnoreDiagnostics DiagConsumer;
   ClangdServer Server(CDB, FS, DiagConsumer, ClangdServer::optsForTest());
   return completions(Server, Text, std::move(IndexSymbols), std::move(Opts),
                      FilePath);
 }
 
-std::string replace(llvm::StringRef Haystack, llvm::StringRef Needle,
-                    llvm::StringRef Repl) {
-  std::string Result;
-  llvm::raw_string_ostream OS(Result);
-  std::pair<llvm::StringRef, llvm::StringRef> Split;
-  for (Split = Haystack.split(Needle); !Split.second.empty();
-       Split = Split.first.split(Needle))
-    OS << Split.first << Repl;
-  Result += Split.first;
-  OS.flush();
-  return Result;
-}
-
-// Helpers to produce fake index symbols for memIndex() or completions().
-// USRFormat is a regex replacement string for the unqualified part of the USR.
-Symbol sym(llvm::StringRef QName, index::SymbolKind Kind,
-           llvm::StringRef USRFormat) {
-  Symbol Sym;
-  std::string USR = "c:"; // We synthesize a few simple cases of USRs by hand!
-  size_t Pos = QName.rfind("::");
-  if (Pos == llvm::StringRef::npos) {
-    Sym.Name = QName;
-    Sym.Scope = "";
-  } else {
-    Sym.Name = QName.substr(Pos + 2);
-    Sym.Scope = QName.substr(0, Pos + 2);
-    USR += "@N@" + replace(QName.substr(0, Pos), "::", "@N@"); // ns:: -> @N@ns
-  }
-  USR += llvm::Regex("^.*$").sub(USRFormat, Sym.Name); // e.g. func -> @F@func#
-  Sym.ID = SymbolID(USR);
-  Sym.SymInfo.Kind = Kind;
-  Sym.Flags |= Symbol::IndexedForCodeCompletion;
-  Sym.Origin = SymbolOrigin::Static;
-  return Sym;
-}
-Symbol func(llvm::StringRef Name) { // Assumes the function has no args.
-  return sym(Name, index::SymbolKind::Function, "@F@\\0#"); // no args
-}
-Symbol cls(llvm::StringRef Name) {
-  return sym(Name, index::SymbolKind::Class, "@S@\\0");
-}
-Symbol var(llvm::StringRef Name) {
-  return sym(Name, index::SymbolKind::Variable, "@\\0");
-}
-Symbol ns(llvm::StringRef Name) {
-  return sym(Name, index::SymbolKind::Namespace, "@N@\\0");
-}
 Symbol withReferences(int N, Symbol S) {
   S.References = N;
   return S;
 }
 
 TEST(CompletionTest, Limit) {
   clangd::CodeCompleteOptions Opts;
   Opts.Limit = 2;

unittests/clangd/DiagnosticsTests.cpp

-//===-- ClangdUnitTests.cpp - ClangdUnit tests ------------------*- C++ -*-===//
+//===--- DiagnosticsTests.cpp ------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Annotations.h"
 #include "ClangdUnit.h"
 #include "SourceCode.h"
+#include "TestIndex.h"
 #include "TestTU.h"
+#include "index/MemIndex.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace clang {
 namespace clangd {
 namespace {
 
MATCHER_P(EqualToFix, Fix, "LSP fix " + llvm::to_string(Fix)) {
   for (std::size_t I = 0; I < arg.Edits.size(); ++I) {
     if (arg.Edits[I].range != Fix.Edits[I].range ||
         arg.Edits[I].newText != Fix.Edits[I].newText)
       return false;
   }
   return true;
 }
 
+
 // Helper function to make tests shorter.
 Position pos(int line, int character) {
   Position Res;
   Res.line = line;
   Res.character = character;
   return Res;
 }
 
#endif
              });
 
   EXPECT_THAT(
       LSPDiags,
       ElementsAre(Pair(EqualToLSPDiag(MainLSP), ElementsAre(EqualToFix(F))),
                   Pair(EqualToLSPDiag(NoteInMainLSP), IsEmpty())));
 }
 
-TEST(ClangdUnitTest, GetBeginningOfIdentifier) {
-  std::string Preamble = R"cpp(
-struct Bar { int func(); };
-#define MACRO(X) void f() { X; }
-Bar* bar;
-  )cpp";
-  // First ^ is the expected beginning, last is the search position.
-  for (std::string Text : std::vector<std::string>{
-           "int ^f^oo();", // inside identifier
-           "int ^foo();",  // beginning of identifier
-           "int ^foo^();", // end of identifier
-           "int foo(^);",  // non-identifier
-           "^int foo();",  // beginning of file (can't back up)
-           "int ^f0^0();", // after a digit (lexing at N-1 is wrong)
-           "int ^λλ^λ();", // UTF-8 handled properly when backing up
-
-           // identifier in macro arg
-           "MACRO(bar->^func())",  // beginning of identifier
-           "MACRO(bar->^fun^c())", // inside identifier
-           "MACRO(bar->^func^())", // end of identifier
-           "MACRO(^bar->func())",  // begin identifier
-           "MACRO(^bar^->func())", // end identifier
-           "^MACRO(bar->func())",  // beginning of macro name
-           "^MAC^RO(bar->func())", // inside macro name
-           "^MACRO^(bar->func())", // end of macro name
-       }) {
-    std::string WithPreamble = Preamble + Text;
-    Annotations TestCase(WithPreamble);
-    auto AST = TestTU::withCode(TestCase.code()).build();
-    const auto &SourceMgr = AST.getASTContext().getSourceManager();
-    SourceLocation Actual = getBeginningOfIdentifier(
-        AST, TestCase.points().back(), SourceMgr.getMainFileID());
-    Position ActualPos = offsetToPosition(
-        TestCase.code(),
-        SourceMgr.getFileOffset(SourceMgr.getSpellingLoc(Actual)));
-    EXPECT_EQ(TestCase.points().front(), ActualPos) << Text;
-  }
+TEST(IncludeFixerTest, IncompleteType) {
+  Annotations Test(R"cpp(
+$insert[[]]namespace ns {
+  class X;
+class Y : $base[[public ns::X]] {};
+int main() {
+  ns::X *x;
+  x$access[[->]]f();
 }
+  )cpp");
+  auto TU = TestTU::withCode(Test.code());
+  Symbol Sym = cls("ns::X");
+  Sym.Flags |= Symbol::IndexedForCodeCompletion;
+  Sym.CanonicalDeclaration.FileURI = "unittest:///x.h";
+  Sym.IncludeHeaders.emplace_back("\"x.h\"", 1);
+
+  SymbolSlab::Builder Slab;
+  Slab.insert(Sym);
+  auto Index = MemIndex::build(std::move(Slab).build(), RefSlab());
+  TU.ExternalIndex = Index.get();
 
-MATCHER_P(DeclNamed, Name, "") {
-  if (NamedDecl *ND = dyn_cast<NamedDecl>(arg))
-    if (ND->getName() == Name)
-      return true;
-  if (auto *Stream = result_listener->stream()) {
-    llvm::raw_os_ostream OS(*Stream);
-    arg->dump(OS);
-  }
-  return false;
+  EXPECT_THAT(
+      TU.build().getDiagnostics(),
+      UnorderedElementsAre(
+          AllOf(Diag(Test.range("base"), "base class has incomplete type"),
+                WithFix(Fix(Test.range("insert"), "#include \"x.h\"\n",
+                            "Add include \"x.h\" for symbol ns::X"))),
+          AllOf(Diag(Test.range("access"),
+                     "member access into incomplete type 'ns::X'"),
+                WithFix(Fix(Test.range("insert"), "#include \"x.h\"\n",
+                            "Add include \"x.h\" for symbol ns::X")))));
 }
 
-TEST(ClangdUnitTest, TopLevelDecls) {
-  TestTU TU;
-  TU.HeaderCode = R"(
-    int header1();
-    int header2;
-  )";
-  TU.Code = "int main();";
-  auto AST = TU.build();
-  EXPECT_THAT(AST.getLocalTopLevelDecls(), ElementsAre(DeclNamed("main")));
-}
 
 } // namespace
 } // namespace clangd
 } // namespace clang
+

unittests/clangd/FileIndexTests.cpp

auto PreambleData = buildPreamble(
       tooling::CompileCommand(), PI, std::make_shared<PCHContainerOperations>(),
       /*StoreInMemory=*/true,
       [&](ASTContext &Ctx, std::shared_ptr<Preprocessor> PP) {});
   // Build AST for main file with preamble.
   auto AST =
       ParsedAST::build(createInvocationFromCommandLine(Cmd), PreambleData,
                        llvm::MemoryBuffer::getMemBufferCopy(Main.code()),
                        std::make_shared<PCHContainerOperations>(), PI.FS,
-                       tidy::ClangTidyOptions::getDefaults());
+                       /*Index=*/nullptr, ParseOptions());
   ASSERT_TRUE(AST);
   FileIndex Index;
   Index.updateMain(MainFile, *AST);
 
   auto Foo = findSymbol(TestTU::withHeaderCode(Header).headerSymbols(), "Foo");
   RefsRequest Request;
   Request.IDs.insert(Foo.ID);
 
   // Expect to see references in main file, references in headers are excluded
   // because we only index main AST.
   EXPECT_THAT(getRefs(Index, Foo.ID), RefsAre({RefRange(Main.range())}));
 }
 
 } // namespace
 } // namespace clangd
 } // namespace clang

unittests/clangd/TUSchedulerTests.cpp

MATCHER_P2(TUState, State, ActionName, "") {
   return arg.Action.S == State && arg.Action.Name == ActionName;
 }
 
 class TUSchedulerTests : public ::testing::Test {
 protected:
   ParseInputs getInputs(PathRef File, std::string Contents) {
     return ParseInputs{*CDB.getCompileCommand(File),
                        buildTestFS(Files, Timestamps), std::move(Contents),
-                       tidy::ClangTidyOptions::getDefaults()};
+                       /*Index=*/nullptr, ParseOptions()};
   }
 
   void updateWithCallback(TUScheduler &S, PathRef File,
                           llvm::StringRef Contents, WantDiagnostics WD,
                           llvm::unique_function<void()> CB) {
     WithContextValue Ctx(llvm::make_scope_exit(std::move(CB)));
     S.update(File, getInputs(File, Contents), WD);
   }

unittests/clangd/TestIndex.h

 #include "index/Merge.h"
 
 namespace clang {
 namespace clangd {
 
 // Creates Symbol instance and sets SymbolID to given QualifiedName.
 Symbol symbol(llvm::StringRef QName);
 
+// Helpers to produce fake index symbols with proper SymbolID.
+// USRFormat is a regex replacement string for the unqualified part of the USR.
+Symbol sym(llvm::StringRef QName, index::SymbolKind Kind,
+           llvm::StringRef USRFormat);
+// Creats a function symbol assuming no function arg.
+Symbol func(llvm::StringRef Name);
+// Creates a class symbol.
+Symbol cls(llvm::StringRef Name);
+// Creates a variable symbol.
+Symbol var(llvm::StringRef Name);
+// Creates a namespace symbol.
+Symbol ns(llvm::StringRef Name);
+
 // Create a slab of symbols with the given qualified names as IDs and names.
 SymbolSlab generateSymbols(std::vector<std::string> QualifiedNames);
 
 // Create a slab of symbols with IDs and names [Begin, End].
 SymbolSlab generateNumSymbols(int Begin, int End);
 
 // Returns fully-qualified name out of given symbol.
 std::string getQualifiedName(const Symbol &Sym);

unittests/clangd/TestIndex.cpp

 //===-- IndexHelpers.cpp ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "TestIndex.h"
+#include "clang/Index/IndexSymbol.h"
+#include "llvm/Support/Regex.h"
 
 namespace clang {
 namespace clangd {
 
 Symbol symbol(llvm::StringRef QName) {
   Symbol Sym;
   Sym.ID = SymbolID(QName.str());
   size_t Pos = QName.rfind("::");
   if (Pos == llvm::StringRef::npos) {
     Sym.Name = QName;
     Sym.Scope = "";
   } else {
     Sym.Name = QName.substr(Pos + 2);
     Sym.Scope = QName.substr(0, Pos + 2);
   }
   return Sym;
 }
 
+static std::string replace(llvm::StringRef Haystack, llvm::StringRef Needle,
+                           llvm::StringRef Repl) {
+  std::string Result;
+  llvm::raw_string_ostream OS(Result);
+  std::pair<llvm::StringRef, llvm::StringRef> Split;
+  for (Split = Haystack.split(Needle); !Split.second.empty();
+       Split = Split.first.split(Needle))
+    OS << Split.first << Repl;
+  Result += Split.first;
+  OS.flush();
+  return Result;
+}
+
+// Helpers to produce fake index symbols for memIndex() or completions().
+// USRFormat is a regex replacement string for the unqualified part of the USR.
+Symbol sym(llvm::StringRef QName, index::SymbolKind Kind,
+           llvm::StringRef USRFormat) {
+  Symbol Sym;
+  std::string USR = "c:"; // We synthesize a few simple cases of USRs by hand!
+  size_t Pos = QName.rfind("::");
+  if (Pos == llvm::StringRef::npos) {
+    Sym.Name = QName;
+    Sym.Scope = "";
+  } else {
+    Sym.Name = QName.substr(Pos + 2);
+    Sym.Scope = QName.substr(0, Pos + 2);
+    USR += "@N@" + replace(QName.substr(0, Pos), "::", "@N@"); // ns:: -> @N@ns
+  }
+  USR += llvm::Regex("^.*$").sub(USRFormat, Sym.Name); // e.g. func -> @F@func#
+  Sym.ID = SymbolID(USR);
+  Sym.SymInfo.Kind = Kind;
+  Sym.Flags |= Symbol::IndexedForCodeCompletion;
+  Sym.Origin = SymbolOrigin::Static;
+  return Sym;
+}
+
+Symbol func(llvm::StringRef Name) { // Assumes the function has no args.
+  return sym(Name, index::SymbolKind::Function, "@F@\\0#"); // no args
+}
+
+Symbol cls(llvm::StringRef Name) {
+  return sym(Name, index::SymbolKind::Class, "@S@\\0");
+}
+
+Symbol var(llvm::StringRef Name) {
+  return sym(Name, index::SymbolKind::Variable, "@\\0");
+}
+
+Symbol ns(llvm::StringRef Name) {
+  return sym(Name, index::SymbolKind::Namespace, "@N@\\0");
+}
+
 SymbolSlab generateSymbols(std::vector<std::string> QualifiedNames) {
   SymbolSlab::Builder Slab;
   for (llvm::StringRef QName : QualifiedNames)
     Slab.insert(symbol(QName));
   return std::move(Slab).build();
 }
 
 SymbolSlab generateNumSymbols(int Begin, int End) {

unittests/clangd/TestTU.h

struct TestTU {
   // Define contents of a header which will be implicitly included by Code.
   std::string HeaderCode;
   std::string HeaderFilename = "TestTU.h";
 
   // Extra arguments for the compiler invocation.
   std::vector<const char *> ExtraArgs;
 
   llvm::Optional<std::string> ClangTidyChecks;
+  // Index to use when building AST.
+  const SymbolIndex *ExternalIndex = nullptr;
 
   ParsedAST build() const;
   SymbolSlab headerSymbols() const;
   std::unique_ptr<SymbolIndex> index() const;
 };
 
 // Look up an index symbol by qualified name, which must be unique.
 const Symbol &findSymbol(const SymbolSlab &, llvm::StringRef QName);

unittests/clangd/TestTU.cpp

ParsedAST TestTU::build() const {
   }
   Cmd.insert(Cmd.end(), ExtraArgs.begin(), ExtraArgs.end());
   ParseInputs Inputs;
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.CommandLine = {Cmd.begin(), Cmd.end()};
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
   Inputs.FS = buildTestFS({{FullFilename, Code}, {FullHeaderName, HeaderCode}});
-  Inputs.ClangTidyOpts = tidy::ClangTidyOptions::getDefaults();
-  Inputs.ClangTidyOpts.Checks = ClangTidyChecks;
+  Inputs.Opts = ParseOptions();
+  Inputs.Opts.ClangTidyOpts.Checks = ClangTidyChecks;
+  Inputs.Index = ExternalIndex;
+  if (Inputs.Index)
+    Inputs.Opts.EnableIncludeFixer = true;
   auto PCHs = std::make_shared<PCHContainerOperations>();
   auto CI = buildCompilerInvocation(Inputs);
   assert(CI && "Failed to build compilation invocation.");
   auto Preamble =
       buildPreamble(FullFilename, *CI,
                     /*OldPreamble=*/nullptr,
                     /*OldCompileCommand=*/Inputs.CompileCommand, Inputs, PCHs,
                     /*StoreInMemory=*/true, /*PreambleCallback=*/nullptr);