[clangd] Debounce streams of updates.

[sammccall]

Summary

Don't actually start building ASTs for new revisions until either:

• 500ms have passed since the last revision, or
• we actually need the revision for something (or to unblock the queue)

In practice, this avoids the "first keystroke results in diagnostics" problem.
This is kind of awkward to test, and the test is pretty bad.
It can be observed nicely by capturing a trace, though.

[ilya-biryukov]

clangd/TUScheduler.cpp

142

Maybe make it const and put beside RunSync? Both are scheduling options, so it probably makes sense to group them together.

324

It looks like if we unwrap Optional<Deadline> to Deadline, we could replace this code with wait helper from Threading.h.
The tracing code (e.g. if (!Requests.empty) { /*...*/}) could be changed to log only when *Deadline - steady_clock::now() is positive.
Will probably make the code simpler. WDYT?

356

NIT: I tend to find multi-level nested statements easier to read with braces, e.g.

for (const auto &R : Requests) {
  if (Cond)
    return None
}

But this is up to you.

clangd/TUScheduler.h

55

As discussed offline, we want to expose the debounce parameter in ClangdServer, as there are existing clients of the code that already send updates with low frequency and it would be wasteful for them to do any extra waits.

[sammccall]

Make Deadline a real type with 0/finite/inf semantics.
Pull out another wait() overload.
Expose debounce delay option in clangdserver.

clangd/TUScheduler.cpp

324

Made Deadline a real type, and added a wait() overload without a condition.

[sammccall]

rebase

[ilya-biryukov]

clangd/TUScheduler.h

56

Maybe we could remove this default argument now that ClangdServer also has the default debounce?
I guess the key thing that bothers me is having multiple places that have the magical default constant of 500ms, it would be nice to have it in one place.

clangd/Threading.h

76

Maybe prefer grouping all the fields together? It seems we're giving that up to avoid writing one extra instance of enum Type.

enum Type { Zero, Infinite, Finite };
Deadline(enum Type Type) : Type(Type) {}

enum Type Type;
std::chrono::steady_clock::time_point Time;

83

s/Wait for on/Wait on/?

unittests/clangd/TUSchedulerTests.cpp

97

I wonder if the default debounce of 500ms will make other tests (especially those that use ClangdServer) too slow?
Maybe we should consider settings a smaller default (maybe even Deadline::zero()?) and having 500ms set only by ClangdLSPServer?

[sammccall]

Thanks for catching all those things!

unittests/clangd/TUSchedulerTests.cpp

97

Good point.
For now I've changed the ClangdServer default to 20ms, with a comment that this is for tests (to make sure we're testing the "real" codepath).
After pulling out the options struct, I'd like to gave a function that returns default options to be used in tests (in-memory preambles, fixed number of threads, short debounce) and make
500ms the "real" default.

[sammccall]

review comments

[ilya-biryukov]

LGTM

clangd/ClangdLSPServer.h

private:
   void onSignatureHelp(TextDocumentPositionParams &Params) override;
   void onGoToDefinition(TextDocumentPositionParams &Params) override;
   void onSwitchSourceHeader(TextDocumentIdentifier &Params) override;
   void onDocumentHighlight(TextDocumentPositionParams &Params) override;
   void onFileEvent(DidChangeWatchedFilesParams &Params) override;
   void onCommand(ExecuteCommandParams &Params) override;
   void onRename(RenameParams &Parames) override;
   void onHover(TextDocumentPositionParams &Params) override;
+  void onChangeConfiguration(DidChangeConfigurationParams &Params) override;
 
   std::vector<TextEdit> getFixIts(StringRef File, const clangd::Diagnostic &D);
 
   JSONOutput &Out;
   /// Used to indicate that the 'shutdown' request was received from the
   /// Language Server client.
   bool ShutdownRequestReceived = false;
 

clangd/ClangdLSPServer.cpp

 }
 
 void ClangdLSPServer::onExit(ExitParams &Params) { IsDone = true; }
 
 void ClangdLSPServer::onDocumentDidOpen(DidOpenTextDocumentParams &Params) {
   if (Params.metadata && !Params.metadata->extraFlags.empty())
     CDB.setExtraFlagsForFile(Params.textDocument.uri.file(),
                              std::move(Params.metadata->extraFlags));
-  Server.addDocument(Params.textDocument.uri.file(), Params.textDocument.text);
+  Server.addDocument(Params.textDocument.uri.file(), Params.textDocument.text,
+                     WantDiagnostics::Yes);
 }
 
 void ClangdLSPServer::onDocumentDidChange(DidChangeTextDocumentParams &Params) {
   if (Params.contentChanges.size() != 1)
     return replyError(ErrorCode::InvalidParams,
                       "can only apply one change at a time");
   // We only support full syncing right now.
   Server.addDocument(Params.textDocument.uri.file(),
-                     Params.contentChanges[0].text);
+                     Params.contentChanges[0].text, WantDiagnostics::Auto);
 }
 
 void ClangdLSPServer::onFileEvent(DidChangeWatchedFilesParams &Params) {
   Server.onFileEvent(Params);
 }
 
 void ClangdLSPServer::onCommand(ExecuteCommandParams &Params) {
   auto ApplyEdit = [](WorkspaceEdit WE) {
Server.findHover(Params.textDocument.uri.file(), Params.position,
                                   llvm::toString(H.takeError()));
                        return;
                      }
 
                      reply(H->Value);
                    });
 }
 
+// FIXME: This function needs to be properly tested.
+void ClangdLSPServer::onChangeConfiguration(
+    DidChangeConfigurationParams &Params) {
+  ClangdConfigurationParamsChange &Settings = Params.settings;
+
+  // Compilation database change.
+  if (Settings.compilationDatabasePath.hasValue()) {
+    CDB.setCompileCommandsDir(Settings.compilationDatabasePath.getValue());
+    Server.reparseOpenedFiles();
+  }
+}
+
 ClangdLSPServer::ClangdLSPServer(JSONOutput &Out, unsigned AsyncThreadsCount,
                                  bool StorePreamblesInMemory,
                                  const clangd::CodeCompleteOptions &CCOpts,
                                  llvm::Optional<StringRef> ResourceDir,
                                  llvm::Optional<Path> CompileCommandsDir,
                                  bool BuildDynamicSymbolIndex,
                                  SymbolIndex *StaticIdx)
     : Out(Out), CDB(std::move(CompileCommandsDir)), CCOpts(CCOpts),

clangd/ClangdServer.h

public:
   /// added file (i.e., when processing a first call to addDocument) and reuses
   /// those arguments for subsequent reparses. However, ClangdServer will check
   /// if compilation arguments changed on calls to forceReparse().
   ///
   /// After each parsing request finishes, ClangdServer reports diagnostics to
   /// \p DiagConsumer. Note that a callback to \p DiagConsumer happens on a
   /// worker thread. Therefore, instances of \p DiagConsumer must properly
   /// synchronize access to shared state.
+  /// UpdateDebounce determines how long to wait after a new version of the file
+  /// before starting to compute diagnostics.
   ///
   /// \p StorePreamblesInMemory defines whether the Preambles generated by
   /// clangd are stored in-memory or on disk.
   ///
   /// If \p BuildDynamicSymbolIndex is true, ClangdServer builds a dynamic
   /// in-memory index for symbols in all opened files and uses the index to
   /// augment code completion results.
   ///
   /// If \p StaticIdx is set, ClangdServer uses the index for global code
   /// completion.
+  /// FIXME(sammccall): pull out an options struct.
   ClangdServer(GlobalCompilationDatabase &CDB,
                DiagnosticsConsumer &DiagConsumer,
                FileSystemProvider &FSProvider, unsigned AsyncThreadsCount,
                bool StorePreamblesInMemory,
                bool BuildDynamicSymbolIndex = false,
                SymbolIndex *StaticIdx = nullptr,
-               llvm::Optional<StringRef> ResourceDir = llvm::None);
+               llvm::Optional<StringRef> ResourceDir = llvm::None,
+               std::chrono::steady_clock::duration UpdateDebounce =
+                   std::chrono::milliseconds(500));
 
   /// Set the root path of the workspace.
   void setRootPath(PathRef RootPath);
 
   /// Add a \p File to the list of tracked C++ files or update the contents if
   /// \p File is already tracked. Also schedules parsing of the AST for it on a
   /// separate thread. When the parsing is complete, DiagConsumer passed in
   /// constructor will receive onDiagnosticsReady callback.
-  void addDocument(PathRef File, StringRef Contents);
+  void addDocument(PathRef File, StringRef Contents,
+                   WantDiagnostics WD = WantDiagnostics::Auto);
 
   /// Remove \p File from list of tracked files, schedule a request to free
   /// resources associated with it.
   void removeDocument(PathRef File);
 
   /// Force \p File to be reparsed using the latest contents.
   /// Will also check if CompileCommand, provided by GlobalCompilationDatabase
   /// for \p File has changed. If it has, will remove currently stored Preamble
   /// and AST and rebuild them from scratch.
   void forceReparse(PathRef File);
 
+  /// Calls forceReparse() on all currently opened files.
+  /// As a result, this method may be very expensive.
+  /// This method is normally called when the compilation database is changed.
+  void reparseOpenedFiles();
+
   /// Run code completion for \p File at \p Pos.
   /// Request is processed asynchronously.
   ///
   /// If \p OverridenContents is not None, they will used only for code
   /// completion, i.e. no diagnostics update will be scheduled and a draft for
   /// \p File will not be updated. If \p OverridenContents is None, contents of
   /// the current draft for \p File will be used. If \p UsedFS is non-null, it
   /// will be overwritten by vfs::FileSystem used for completion.
private:
   /// FIXME: This stats several files to find a .clang-format file. I/O can be
   /// slow. Think of a way to cache this.
   llvm::Expected<tooling::Replacements>
   formatCode(llvm::StringRef Code, PathRef File,
              ArrayRef<tooling::Range> Ranges);
 
   void
   scheduleReparseAndDiags(PathRef File, VersionedDraft Contents,
+                          WantDiagnostics WD,
                           Tagged<IntrusiveRefCntPtr<vfs::FileSystem>> TaggedFS);
 
   CompileArgsCache CompileArgs;
   DiagnosticsConsumer &DiagConsumer;
   FileSystemProvider &FSProvider;
   DraftStore DraftMgr;
   // The index used to look up symbols. This could be:
   //   - null (all index functionality is optional)

clangd/ClangdServer.cpp

 }
 
 ClangdServer::ClangdServer(GlobalCompilationDatabase &CDB,
                            DiagnosticsConsumer &DiagConsumer,
                            FileSystemProvider &FSProvider,
                            unsigned AsyncThreadsCount,
                            bool StorePreamblesInMemory,
                            bool BuildDynamicSymbolIndex, SymbolIndex *StaticIdx,
-                           llvm::Optional<StringRef> ResourceDir)
+                           llvm::Optional<StringRef> ResourceDir,
+                           std::chrono::steady_clock::duration UpdateDebounce)
     : CompileArgs(CDB,
                   ResourceDir ? ResourceDir->str() : getStandardResourceDir()),
       DiagConsumer(DiagConsumer), FSProvider(FSProvider),
       FileIdx(BuildDynamicSymbolIndex ? new FileIndex() : nullptr),
       PCHs(std::make_shared<PCHContainerOperations>()),
       // Pass a callback into `WorkScheduler` to extract symbols from a newly
       // parsed file and rebuild the file index synchronously each time an AST
       // is parsed.
       // FIXME(ioeric): this can be slow and we may be able to index on less
       // critical paths.
       WorkScheduler(AsyncThreadsCount, StorePreamblesInMemory,
                     FileIdx
                         ? [this](PathRef Path,
                                  ParsedAST *AST) { FileIdx->update(Path, AST); }
-                        : ASTParsedCallback()) {
+                        : ASTParsedCallback(),
+                    UpdateDebounce) {
   if (FileIdx && StaticIdx) {
     MergedIndex = mergeIndex(FileIdx.get(), StaticIdx);
     Index = MergedIndex.get();
   } else if (FileIdx)
     Index = FileIdx.get();
   else if (StaticIdx)
     Index = StaticIdx;
   else
     Index = nullptr;
 }
 
 void ClangdServer::setRootPath(PathRef RootPath) {
   std::string NewRootPath = llvm::sys::path::convert_to_slash(
       RootPath, llvm::sys::path::Style::posix);
   if (llvm::sys::fs::is_directory(NewRootPath))
     this->RootPath = NewRootPath;
 }
 
-void ClangdServer::addDocument(PathRef File, StringRef Contents) {
+void ClangdServer::addDocument(PathRef File, StringRef Contents,
+                               WantDiagnostics WantDiags) {
   DocVersion Version = DraftMgr.updateDraft(File, Contents);
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
   scheduleReparseAndDiags(File, VersionedDraft{Version, Contents.str()},
-                          std::move(TaggedFS));
+                          WantDiags, std::move(TaggedFS));
 }
 
 void ClangdServer::removeDocument(PathRef File) {
   DraftMgr.removeDraft(File);
   CompileArgs.invalidate(File);
   WorkScheduler.remove(File);
 }
 
 void ClangdServer::forceReparse(PathRef File) {
   auto FileContents = DraftMgr.getDraft(File);
   assert(FileContents.Draft &&
          "forceReparse() was called for non-added document");
 
   // forceReparse promises to request new compilation flags from CDB, so we
   // remove any cahced flags.
   CompileArgs.invalidate(File);
 
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
-  scheduleReparseAndDiags(File, std::move(FileContents), std::move(TaggedFS));
+  scheduleReparseAndDiags(File, std::move(FileContents), WantDiagnostics::Yes,
+                          std::move(TaggedFS));
 }
 
 void ClangdServer::codeComplete(
     PathRef File, Position Pos, const clangd::CodeCompleteOptions &Opts,
     UniqueFunction<void(Tagged<CompletionList>)> Callback,
     llvm::Optional<StringRef> OverridenContents,
     IntrusiveRefCntPtr<vfs::FileSystem> *UsedFS) {
   using CallbackType = UniqueFunction<void(Tagged<CompletionList>)>;
auto Action = [Pos, TaggedFS](decltype(Callback) Callback,
     Callback(make_tagged(std::move(Result), TaggedFS.Tag));
   };
 
   WorkScheduler.runWithAST("Hover", File,
                            BindWithForward(Action, std::move(Callback)));
 }
 
 void ClangdServer::scheduleReparseAndDiags(
-    PathRef File, VersionedDraft Contents,
+    PathRef File, VersionedDraft Contents, WantDiagnostics WantDiags,
     Tagged<IntrusiveRefCntPtr<vfs::FileSystem>> TaggedFS) {
   tooling::CompileCommand Command = CompileArgs.getCompileCommand(File);
 
-  using OptDiags = llvm::Optional<std::vector<DiagWithFixIts>>;
-
   DocVersion Version = Contents.Version;
   Path FileStr = File.str();
   VFSTag Tag = std::move(TaggedFS.Tag);
 
-  auto Callback = [this, Version, FileStr, Tag](OptDiags Diags) {
-    if (!Diags)
-      return; // A new reparse was requested before this one completed.
-
+  auto Callback = [this, Version, FileStr,
+                   Tag](std::vector<DiagWithFixIts> Diags) {
     // We need to serialize access to resulting diagnostics to avoid calling
     // `onDiagnosticsReady` in the wrong order.
     std::lock_guard<std::mutex> DiagsLock(DiagnosticsMutex);
     DocVersion &LastReportedDiagsVersion = ReportedDiagnosticVersions[FileStr];
     // FIXME(ibiryukov): get rid of '<' comparison here. In the current
     // implementation diagnostics will not be reported after version counters'
     // overflow. This should not happen in practice, since DocVersion is a
     // 64-bit unsigned integer.
     if (Version < LastReportedDiagsVersion)
       return;
     LastReportedDiagsVersion = Version;
 
     DiagConsumer.onDiagnosticsReady(
-        FileStr, make_tagged(std::move(*Diags), std::move(Tag)));
+        FileStr, make_tagged(std::move(Diags), std::move(Tag)));
   };
 
   WorkScheduler.update(File,
                        ParseInputs{std::move(Command),
                                    std::move(TaggedFS.Value),
                                    std::move(*Contents.Draft)},
-                       std::move(Callback));
+                       WantDiags, std::move(Callback));
+}
+
+void ClangdServer::reparseOpenedFiles() {
+  for (const Path &FilePath : DraftMgr.getActiveFiles())
+    forceReparse(FilePath);
 }
 
 void ClangdServer::onFileEvent(const DidChangeWatchedFilesParams &Params) {
   // FIXME: Do nothing for now. This will be used for indexing and potentially
   // invalidating other caches.
 }
 
 std::vector<std::pair<Path, std::size_t>>
 ClangdServer::getUsedBytesPerFile() const {
   return WorkScheduler.getUsedBytesPerFile();
 }
 
 LLVM_NODISCARD bool
 ClangdServer::blockUntilIdleForTest(llvm::Optional<double> TimeoutSeconds) {
   return WorkScheduler.blockUntilIdle(timeoutSeconds(TimeoutSeconds));
 }

clangd/CodeComplete.cpp

auto Bounds =
         ComputePreambleBounds(*CI->getLangOpts(), ContentsBuffer.get(), 0);
     // FIXME(ibiryukov): Remove this call to CanReuse() after we'll fix
     // clients relying on getting stats for preamble files during code
     // completion.
     // Note that results of CanReuse() are ignored, see the comment above.
     Input.Preamble->CanReuse(*CI, ContentsBuffer.get(), Bounds,
                              Input.VFS.get());
   }
+  // The diagnostic options must be set before creating a CompilerInstance.
+  CI->getDiagnosticOpts().IgnoreWarnings = true;
   auto Clang = prepareCompilerInstance(
       std::move(CI), Input.Preamble, std::move(ContentsBuffer),
       std::move(Input.PCHs), std::move(Input.VFS), DummyDiagsConsumer);
-  auto &DiagOpts = Clang->getDiagnosticOpts();
-  DiagOpts.IgnoreWarnings = true;
 
   // Disable typo correction in Sema.
   Clang->getLangOpts().SpellChecking = false;
 
   auto &FrontendOpts = Clang->getFrontendOpts();
   FrontendOpts.SkipFunctionBodies = true;
   FrontendOpts.CodeCompleteOpts = Options;
   FrontendOpts.CodeCompletionAt.FileName = Input.FileName;

clangd/DraftStore.h

 /// filenames. The contents are owned by the DraftStore. Versions are mantained
 /// for the all added documents, including removed ones. The document version is
 /// incremented on each update and removal of the document.
 class DraftStore {
 public:
   /// \return version and contents of the stored document.
   /// For untracked files, a (0, None) pair is returned.
   VersionedDraft getDraft(PathRef File) const;
+
+  /// \return List of names of active drafts in this store.  Drafts that were
+  /// removed (which still have an entry in the Drafts map) are not returned by
+  /// this function.
+  std::vector<Path> getActiveFiles() const;
+
   /// \return version of the tracked document.
   /// For untracked files, 0 is returned.
   DocVersion getVersion(PathRef File) const;
 
   /// Replace contents of the draft for \p File with \p Contents.
   /// \return The new version of the draft for \p File.
   DocVersion updateDraft(PathRef File, StringRef Contents);
   /// Remove the contents of the draft

clangd/DraftStore.cpp

VersionedDraft DraftStore::getDraft(PathRef File) const {
   std::lock_guard<std::mutex> Lock(Mutex);
 
   auto It = Drafts.find(File);
   if (It == Drafts.end())
     return {0, llvm::None};
   return It->second;
 }
 
+std::vector<Path> DraftStore::getActiveFiles() const {
+  std::lock_guard<std::mutex> Lock(Mutex);
+  std::vector<Path> ResultVector;
+
+  for (auto DraftIt = Drafts.begin(); DraftIt != Drafts.end(); DraftIt++)
+    if (DraftIt->second.Draft)
+      ResultVector.push_back(DraftIt->getKey());
+
+  return ResultVector;
+}
+
 DocVersion DraftStore::getVersion(PathRef File) const {
   std::lock_guard<std::mutex> Lock(Mutex);
 
   auto It = Drafts.find(File);
   if (It == Drafts.end())
     return 0;
   return It->second.Version;
 }

clangd/GlobalCompilationDatabase.h

public:
   /// Scans File's parents looking for compilation databases.
   /// Any extra flags will be added.
   llvm::Optional<tooling::CompileCommand>
   getCompileCommand(PathRef File) const override;
 
   /// Uses the default fallback command, adding any extra flags.
   tooling::CompileCommand getFallbackCommand(PathRef File) const override;
 
+  /// Set the compile commands directory to \p P.
+  void setCompileCommandsDir(Path P);
+
   /// Sets the extra flags that should be added to a file.
   void setExtraFlagsForFile(PathRef File, std::vector<std::string> ExtraFlags);
 
 private:
   tooling::CompilationDatabase *getCDBForFile(PathRef File) const;
   tooling::CompilationDatabase *getCDBInDirLocked(PathRef File) const;
   void addExtraFlags(PathRef File, tooling::CompileCommand &C) const;
 

clangd/GlobalCompilationDatabase.cpp

 tooling::CompileCommand
 DirectoryBasedGlobalCompilationDatabase::getFallbackCommand(
     PathRef File) const {
   auto C = GlobalCompilationDatabase::getFallbackCommand(File);
   addExtraFlags(File, C);
   return C;
 }
 
+void DirectoryBasedGlobalCompilationDatabase::setCompileCommandsDir(Path P) {
+  std::lock_guard<std::mutex> Lock(Mutex);
+  CompileCommandsDir = P;
+  CompilationDatabases.clear();
+}
+
 void DirectoryBasedGlobalCompilationDatabase::setExtraFlagsForFile(
     PathRef File, std::vector<std::string> ExtraFlags) {
   std::lock_guard<std::mutex> Lock(Mutex);
   ExtraFlagsForFile[File] = std::move(ExtraFlags);
 }
 
 void DirectoryBasedGlobalCompilationDatabase::addExtraFlags(
     PathRef File, tooling::CompileCommand &C) const {

clangd/Headers.cpp

return llvm::make_error<llvm::StringError>(
         "Failed to create a compiler instance for " + File,
         llvm::inconvertibleErrorCode());
   CI->getFrontendOpts().DisableFree = false;
   // Parse the main file to get all existing #includes in the file, and then we
   // can make sure the same header (even with different include path) is not
   // added more than once.
   CI->getPreprocessorOpts().SingleFileParseMode = true;
 
+  // The diagnostic options must be set before creating a CompilerInstance.
+  CI->getDiagnosticOpts().IgnoreWarnings = true;
   auto Clang = prepareCompilerInstance(
       std::move(CI), /*Preamble=*/nullptr,
       llvm::MemoryBuffer::getMemBuffer(Code, File),
       std::make_shared<PCHContainerOperations>(), FS, IgnoreDiags);
-  auto &DiagOpts = Clang->getDiagnosticOpts();
-  DiagOpts.IgnoreWarnings = true;
 
   if (Clang->getFrontendOpts().Inputs.empty())
     return llvm::make_error<llvm::StringError>(
         "Empty frontend action inputs empty for file " + File,
         llvm::inconvertibleErrorCode());
   PreprocessOnlyAction Action;
   if (!Action.BeginSourceFile(*Clang, Clang->getFrontendOpts().Inputs[0]))
     return llvm::make_error<llvm::StringError>(

clangd/Protocol.h

 bool fromJSON(const json::Expr &, FileEvent &);
 
 struct DidChangeWatchedFilesParams {
   /// The actual file events.
   std::vector<FileEvent> changes;
 };
 bool fromJSON(const json::Expr &, DidChangeWatchedFilesParams &);
 
+/// Clangd extension to manage a workspace/didChangeConfiguration notification
+/// since the data received is described as 'any' type in LSP.
+struct ClangdConfigurationParamsChange {
+  llvm::Optional<std::string> compilationDatabasePath;
+};
+bool fromJSON(const json::Expr &, ClangdConfigurationParamsChange &);
+
+struct DidChangeConfigurationParams {
+  // We use this predefined struct because it is easier to use
+  // than the protocol specified type of 'any'.
+  ClangdConfigurationParamsChange settings;
+};
+bool fromJSON(const json::Expr &, DidChangeConfigurationParams &);
+
 struct FormattingOptions {
   /// Size of a tab in spaces.
   int tabSize = 0;
 
   /// Prefer spaces over tabs.
   bool insertSpaces = false;
 };
 bool fromJSON(const json::Expr &, FormattingOptions &);

clangd/Protocol.cpp

 
 json::Expr toJSON(const DocumentHighlight &DH) {
   return json::obj{
       {"range", toJSON(DH.range)},
       {"kind", static_cast<int>(DH.kind)},
   };
 }
 
+bool fromJSON(const json::Expr &Params, DidChangeConfigurationParams &CCP) {
+  json::ObjectMapper O(Params);
+  return O && O.map("settings", CCP.settings);
+}
+
+bool fromJSON(const json::Expr &Params, ClangdConfigurationParamsChange &CCPC) {
+  json::ObjectMapper O(Params);
+  return O && O.map("compilationDatabasePath", CCPC.compilationDatabasePath);
+}
+
 } // namespace clangd
 } // namespace clang

clangd/ProtocolHandlers.h

public:
   virtual void onSignatureHelp(TextDocumentPositionParams &Params) = 0;
   virtual void onGoToDefinition(TextDocumentPositionParams &Params) = 0;
   virtual void onSwitchSourceHeader(TextDocumentIdentifier &Params) = 0;
   virtual void onFileEvent(DidChangeWatchedFilesParams &Params) = 0;
   virtual void onCommand(ExecuteCommandParams &Params) = 0;
   virtual void onRename(RenameParams &Parames) = 0;
   virtual void onDocumentHighlight(TextDocumentPositionParams &Params) = 0;
   virtual void onHover(TextDocumentPositionParams &Params) = 0;
+  virtual void onChangeConfiguration(DidChangeConfigurationParams &Params) = 0;
 };
 
 void registerCallbackHandlers(JSONRPCDispatcher &Dispatcher, JSONOutput &Out,
                               ProtocolCallbacks &Callbacks);
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/ProtocolHandlers.cpp

void clangd::registerCallbackHandlers(JSONRPCDispatcher &Dispatcher,
   Register("textDocument/switchSourceHeader",
            &ProtocolCallbacks::onSwitchSourceHeader);
   Register("textDocument/rename", &ProtocolCallbacks::onRename);
   Register("textDocument/hover", &ProtocolCallbacks::onHover);
   Register("workspace/didChangeWatchedFiles", &ProtocolCallbacks::onFileEvent);
   Register("workspace/executeCommand", &ProtocolCallbacks::onCommand);
   Register("textDocument/documentHighlight",
            &ProtocolCallbacks::onDocumentHighlight);
+  Register("workspace/didChangeConfiguration",
+           &ProtocolCallbacks::onChangeConfiguration);
 }

clangd/TUScheduler.h

 
 #include "ClangdUnit.h"
 #include "Function.h"
 #include "Threading.h"
 #include "llvm/ADT/StringMap.h"
 
 namespace clang {
 namespace clangd {
+
 /// Returns a number of a default async threads to use for TUScheduler.
 /// Returned value is always >= 1 (i.e. will not cause requests to be processed
 /// synchronously).
 unsigned getDefaultAsyncThreadsCount();
 
 struct InputsAndAST {
   const ParseInputs &Inputs;
   ParsedAST *
 };
 
 struct InputsAndPreamble {
   const ParseInputs &Inputs;
   const PreambleData *Preamble;
 };
 
+/// Determines whether diagnostics should be generated for a file snapshot.
+enum class WantDiagnostics {
+  Yes,  /// Diagnostics must be generated for this snapshot.
+  No,   /// Diagnostics must not be generated for this snapshot.
+  Auto, /// Diagnostics must be generated for this snapshot or a subsequent one,
+        /// within a bounded amount of time.
+};
+
 /// Handles running tasks for ClangdServer and managing the resources (e.g.,
 /// preambles and ASTs) for opened files.
 /// TUScheduler is not thread-safe, only one thread should be providing updates
 /// and scheduling tasks.
 /// Callbacks are run on a threadpool and it's appropriate to do slow work in
 /// them. Each task has a name, used for tracing (should be UpperCamelCase).
+/// FIXME(sammccall): pull out a scheduler options struct.
 class TUScheduler {
 public:
   TUScheduler(unsigned AsyncThreadsCount, bool StorePreamblesInMemory,
-              ASTParsedCallback ASTCallback);
+              ASTParsedCallback ASTCallback,
+              std::chrono::steady_clock::duration UpdateDebounce =
+                  std::chrono::milliseconds(500));
   ~TUScheduler();
 
   /// Returns estimated memory usage for each of the currently open files.
   /// The order of results is unspecified.
   std::vector<std::pair<Path, std::size_t>> getUsedBytesPerFile() const;
 
   /// Schedule an update for \p File. Adds \p File to a list of tracked files if
   /// \p File was not part of it before.
   /// FIXME(ibiryukov): remove the callback from this function.
-  void update(PathRef File, ParseInputs Inputs,
-              UniqueFunction<void(llvm::Optional<std::vector<DiagWithFixIts>>)>
-                  OnUpdated);
+  void update(PathRef File, ParseInputs Inputs, WantDiagnostics WD,
+              UniqueFunction<void(std::vector<DiagWithFixIts>)> OnUpdated);
 
   /// Remove \p File from the list of tracked files and schedule removal of its
   /// resources.
   void remove(PathRef File);
 
   /// Schedule an async read of the AST. \p Action will be called when AST is
   /// ready. The AST passed to \p Action refers to the version of \p File
   /// tracked at the time of the call, even if new updates are received before
private:
   const std::shared_ptr<PCHContainerOperations> PCHOps;
   const ASTParsedCallback ASTCallback;
   Semaphore Barrier;
   llvm::StringMap<std::unique_ptr<FileData>> Files;
   // None when running tasks synchronously and non-None when running tasks
   // asynchronously.
   llvm::Optional<AsyncTaskRunner> PreambleTasks;
   llvm::Optional<AsyncTaskRunner> WorkerThreads;
+  std::chrono::steady_clock::duration UpdateDebounce;
 };
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/TUScheduler.cpp

 #include "llvm/Support/Errc.h"
 #include "llvm/Support/Path.h"
 #include <memory>
 #include <queue>
 #include <thread>
 
 namespace clang {
 namespace clangd {
+using std::chrono::steady_clock;
 namespace {
 class ASTWorkerHandle;
 
 /// Owns one instance of the AST, schedules updates and reads of it.
 /// Also responsible for building and providing access to the preamble.
 /// Each ASTWorker processes the async requests sent to it on a separate
 /// dedicated thread.
 /// The ASTWorker that manages the AST is shared by both the processing thread
 /// and the TUScheduler. The TUScheduler should discard an ASTWorker when
 /// remove() is called, but its thread may be busy and we don't want to block.
 /// So the workers are accessed via an ASTWorkerHandle. Destroying the handle
 /// signals the worker to exit its run loop and gives up shared ownership of the
 /// worker.
 class ASTWorker {
   friend class ASTWorkerHandle;
-  ASTWorker(llvm::StringRef File, Semaphore &Barrier, CppFile AST,
-            bool RunSync);
+  ASTWorker(llvm::StringRef File, Semaphore &Barrier, CppFile AST, bool RunSync,
+            steady_clock::duration UpdateDebounce);
 
 public:
   /// Create a new ASTWorker and return a handle to it.
   /// The processing thread is spawned using \p Tasks. However, when \p Tasks
   /// is null, all requests will be processed on the calling thread
   /// synchronously instead. \p Barrier is acquired when processing each
   /// request, it is be used to limit the number of actively running threads.
   static ASTWorkerHandle Create(llvm::StringRef File, AsyncTaskRunner *Tasks,
-                                Semaphore &Barrier, CppFile AST);
+                                Semaphore &Barrier, CppFile AST,
+                                steady_clock::duration UpdateDebounce);
   ~ASTWorker();
 
-  void update(ParseInputs Inputs,
-              UniqueFunction<void(llvm::Optional<std::vector<DiagWithFixIts>>)>
-                  OnUpdated);
+  void update(ParseInputs Inputs, WantDiagnostics,
+              UniqueFunction<void(std::vector<DiagWithFixIts>)> OnUpdated);
   void runWithAST(llvm::StringRef Name,
                   UniqueFunction<void(llvm::Expected<InputsAndAST>)> Action);
   bool blockUntilIdle(Deadline Timeout) const;
 
   std::shared_ptr<const PreambleData> getPossiblyStalePreamble() const;
   std::size_t getUsedBytes() const;
 
 private:
   // Must be called exactly once on processing thread. Will return after
   // stop() is called on a separate thread and all pending requests are
   // processed.
   void run();
   /// Signal that run() should finish processing pending requests and exit.
   void stop();
   /// Adds a new task to the end of the request queue.
   void startTask(llvm::StringRef Name, UniqueFunction<void()> Task,
-                 bool isUpdate, llvm::Optional<CancellationFlag> CF);
+                 llvm::Optional<WantDiagnostics> UpdateType);
+  /// Determines the next action to perform.
+  /// All actions that should never run are disarded.
+  /// Returns a deadline for the next action. If it's expired, run now.
+  /// scheduleLocked() is called again at the deadline, or if requests arrive.
+  Deadline scheduleLocked();
+  /// Should the first task in the queue be skipped instead of run?
+  bool shouldSkipHeadLocked() const;
 
   struct Request {
     UniqueFunction<void()> Action;
     std::string Name;
+    steady_clock::time_point AddTime;
     Context Ctx;
+    llvm::Optional<WantDiagnostics> UpdateType;
   };
 
-  std::string File;
+  const std::string File;
   const bool RunSync;
+  // Time to wait after an update to see whether another update obsoletes it.
+  const steady_clock::duration UpdateDebounce;
+
   Semaphore &Barrier;
   // AST and FileInputs are only accessed on the processing thread from run().
   CppFile AST;
   // Inputs, corresponding to the current state of AST.
   ParseInputs FileInputs;
   // Guards members used by both TUScheduler and the worker thread.
   mutable std::mutex Mutex;
   std::shared_ptr<const PreambleData> LastBuiltPreamble; /* GUARDED_BY(Mutex) */
   // Result of getUsedBytes() after the last rebuild or read of AST.
   std::size_t LastASTSize; /* GUARDED_BY(Mutex) */
   // Set to true to signal run() to finish processing.
   bool Done;                           /* GUARDED_BY(Mutex) */
-  std::queue<Request> Requests;        /* GUARDED_BY(Mutex) */
-  // Only set when last request is an update. This allows us to cancel an update
-  // that was never read, if a subsequent update comes in.
-  llvm::Optional<CancellationFlag> LastUpdateCF; /* GUARDED_BY(Mutex) */
+  std::deque<Request> Requests;        /* GUARDED_BY(Mutex) */
   mutable std::condition_variable RequestsCV;
 };
 
 /// A smart-pointer-like class that points to an active ASTWorker.
 /// In destructor, signals to the underlying ASTWorker that no new requests will
 /// be sent and the processing loop may exit (after running all pending
 /// requests).
 class ASTWorkerHandle {
   friend class ASTWorker;
   ASTWorkerHandle(std::shared_ptr<ASTWorker> Worker)
       : Worker(std::move(Worker)) {
public:
   /// are possible.
   std::shared_ptr<const ASTWorker> lock() { return Worker; }
 
 private:
   std::shared_ptr<ASTWorker> Worker;
 };
 
 ASTWorkerHandle ASTWorker::Create(llvm::StringRef File, AsyncTaskRunner *Tasks,
-                                  Semaphore &Barrier, CppFile AST) {
-  std::shared_ptr<ASTWorker> Worker(
-      new ASTWorker(File, Barrier, std::move(AST), /*RunSync=*/!Tasks));
+                                  Semaphore &Barrier, CppFile AST,
+                                  steady_clock::duration UpdateDebounce) {
+  std::shared_ptr<ASTWorker> Worker(new ASTWorker(
+      File, Barrier, std::move(AST), /*RunSync=*/!Tasks, UpdateDebounce));
   if (Tasks)
     Tasks->runAsync("worker:" + llvm::sys::path::filename(File),
                     [Worker]() { Worker->run(); });
 
   return ASTWorkerHandle(std::move(Worker));
 }
 
 ASTWorker::ASTWorker(llvm::StringRef File, Semaphore &Barrier, CppFile AST,
-                     bool RunSync)
-    : File(File), RunSync(RunSync), Barrier(Barrier), AST(std::move(AST)),
-      Done(false) {
+                     bool RunSync, steady_clock::duration UpdateDebounce)
+    : File(File), RunSync(RunSync), UpdateDebounce(UpdateDebounce),
+      Barrier(Barrier), AST(std::move(AST)), Done(false) {
   if (RunSync)
     return;
 }
 
 ASTWorker::~ASTWorker() {
 #ifndef NDEBUG
   std::lock_guard<std::mutex> Lock(Mutex);
   assert(Done && "handle was not destroyed");
   assert(Requests.empty() && "unprocessed requests when destroying ASTWorker");
 #endif
 }
 
 void ASTWorker::update(
-    ParseInputs Inputs,
-    UniqueFunction<void(llvm::Optional<std::vector<DiagWithFixIts>>)>
-        OnUpdated) {
-  auto Task = [=](CancellationFlag CF, decltype(OnUpdated) OnUpdated) mutable {
-    if (CF.isCancelled()) {
-      OnUpdated(llvm::None);
-      return;
-    }
+    ParseInputs Inputs, WantDiagnostics WantDiags,
+    UniqueFunction<void(std::vector<DiagWithFixIts>)> OnUpdated) {
+  auto Task = [=](decltype(OnUpdated) OnUpdated) mutable {
     FileInputs = Inputs;
     auto Diags = AST.rebuild(std::move(Inputs));
 
     {
       std::lock_guard<std::mutex> Lock(Mutex);
       if (AST.getPreamble())
         LastBuiltPreamble = AST.getPreamble();
       LastASTSize = AST.getUsedBytes();
     }
     // We want to report the diagnostics even if this update was cancelled.
     // It seems more useful than making the clients wait indefinitely if they
     // spam us with updates.
-    OnUpdated(std::move(Diags));
+    if (Diags && WantDiags != WantDiagnostics::No)
+      OnUpdated(std::move(*Diags));
   };
 
-  CancellationFlag UpdateCF;
-  startTask("Update", BindWithForward(Task, UpdateCF, std::move(OnUpdated)),
-            /*isUpdate=*/true, UpdateCF);
+  startTask("Update", BindWithForward(Task, std::move(OnUpdated)), WantDiags);
 }
 
 void ASTWorker::runWithAST(
     llvm::StringRef Name,
     UniqueFunction<void(llvm::Expected<InputsAndAST>)> Action) {
   auto Task = [=](decltype(Action) Action) {
     ParsedAST *ActualAST = AST.getAST();
     if (!ActualAST) {
       Action(llvm::make_error<llvm::StringError>("invalid AST",
                                                  llvm::errc::invalid_argument));
       return;
     }
     Action(InputsAndAST{FileInputs, *ActualAST});
 
     // Size of the AST might have changed after reads too, e.g. if some decls
     // were deserialized from preamble.
     std::lock_guard<std::mutex> Lock(Mutex);
     LastASTSize = ActualAST->getUsedBytes();
   };
 
   startTask(Name, BindWithForward(Task, std::move(Action)),
-            /*isUpdate=*/false, llvm::None);
+            /*UpdateType=*/llvm::None);
 }
 
 std::shared_ptr<const PreambleData>
 ASTWorker::getPossiblyStalePreamble() const {
   std::lock_guard<std::mutex> Lock(Mutex);
   return LastBuiltPreamble;
 }
 
 std::size_t ASTWorker::getUsedBytes() const {
   std::lock_guard<std::mutex> Lock(Mutex);
   return LastASTSize;
 }
 
 void ASTWorker::stop() {
   {
     std::lock_guard<std::mutex> Lock(Mutex);
     assert(!Done && "stop() called twice");
     Done = true;
   }
   RequestsCV.notify_all();
 }
 
 void ASTWorker::startTask(llvm::StringRef Name, UniqueFunction<void()> Task,
-                          bool isUpdate, llvm::Optional<CancellationFlag> CF) {
-  assert(isUpdate == CF.hasValue() &&
-         "Only updates are expected to pass CancellationFlag");
-
+                          llvm::Optional<WantDiagnostics> UpdateType) {
   if (RunSync) {
     assert(!Done && "running a task after stop()");
     trace::Span Tracer(Name + ":" + llvm::sys::path::filename(File));
     Task();
     return;
   }
 
   {
     std::lock_guard<std::mutex> Lock(Mutex);
     assert(!Done && "running a task after stop()");
-    if (isUpdate) {
-      if (!Requests.empty() && LastUpdateCF) {
-        // There were no reads for the last unprocessed update, let's cancel it
-        // to not waste time on it.
-        LastUpdateCF->cancel();
-      }
-      LastUpdateCF = std::move(*CF);
-    } else {
-      LastUpdateCF = llvm::None;
-    }
-    Requests.push({std::move(Task), Name, Context::current().clone()});
-  } // unlock Mutex.
+    Requests.push_back({std::move(Task), Name, steady_clock::now(),
+                        Context::current().clone(), UpdateType});
   RequestsCV.notify_all();
 }
 
 void ASTWorker::run() {
   while (true) {
     Request Req;
     {
       std::unique_lock<std::mutex> Lock(Mutex);
-      RequestsCV.wait(Lock, [&]() { return Done || !Requests.empty(); });
-      if (Requests.empty()) {
-        assert(Done);
+      for (auto Wait = scheduleLocked(); !Wait.expired();
+           Wait = scheduleLocked()) {
+        if (Done) {
+          if (Requests.empty())
             return;
+          else     // Even though Done is set, finish pending requests.
+            break; // However, skip delays to shutdown fast.
+        }
+
+        // Tracing: we have a next request, attribute this sleep to it.
+        Optional<WithContext> Ctx;
+        Optional<trace::Span> Tracer;
+        if (!Requests.empty()) {
+          Ctx.emplace(Requests.front().Ctx.clone());
+          Tracer.emplace("Debounce");
+          SPAN_ATTACH(*Tracer, "next_request", Requests.front().Name);
+          if (!(Wait == Deadline::infinity()))
+            SPAN_ATTACH(*Tracer, "sleep_ms",
+                        std::chrono::duration_cast<std::chrono::milliseconds>(
+                            Wait.time() - steady_clock::now())
+                            .count());
         }
-      // Even when Done is true, we finish processing all pending requests
-      // before exiting the processing loop.
 
+        wait(Lock, RequestsCV, Wait);
+      }
       Req = std::move(Requests.front());
       // Leave it on the queue for now, so waiters don't see an empty queue.
     } // unlock Mutex
 
     {
       std::lock_guard<Semaphore> BarrierLock(Barrier);
       WithContext Guard(std::move(Req.Ctx));
       trace::Span Tracer(Req.Name);
       Req.Action();
     }
 
     {
       std::lock_guard<std::mutex> Lock(Mutex);
-      Requests.pop();
+      Requests.pop_front();
     }
     RequestsCV.notify_all();
   }
 }
 
+Deadline ASTWorker::scheduleLocked() {
+  if (Requests.empty())
+    return Deadline::infinity(); // Wait for new requests.
+  while (shouldSkipHeadLocked())
+    Requests.pop_front();
+  assert(!Requests.empty() && "skipped the whole queue");
+  // Some updates aren't dead yet, but never end up being used.
+  // e.g. the first keystroke is live until obsoleted by the second.
+  // We debounce "maybe-unused" writes, sleeping 500ms in case they become dead.
+  // But don't delay reads (including updates where diagnostics are needed).
+  for (const auto &R : Requests)
+    if (R.UpdateType == None || R.UpdateType == WantDiagnostics::Yes)
+      return Deadline::zero();
+  // Front request needs to be debounced, so determine when we're ready.
+  Deadline D(Requests.front().AddTime + UpdateDebounce);
+  return D;
+}
+
+// Returns true if Requests.front() is a dead update that can be skipped.
+bool ASTWorker::shouldSkipHeadLocked() const {
+  assert(!Requests.empty());
+  auto Next = Requests.begin();
+  auto UpdateType = Next->UpdateType;
+  if (!UpdateType) // Only skip updates.
+    return false;
+  ++Next;
+  // An update is live if its AST might still be read.
+  // That is, if it's not immediately followed by another update.
+  if (Next == Requests.end() || !Next->UpdateType)
+    return false;
+  // The other way an update can be live is if its diagnostics might be used.
+  switch (*UpdateType) {
+  case WantDiagnostics::Yes:
+    return false; // Always used.
+  case WantDiagnostics::No:
+    return true; // Always dead.
+  case WantDiagnostics::Auto:
+    // Used unless followed by an update that generates diagnostics.
+    for (; Next != Requests.end(); ++Next)
+      if (Next->UpdateType == WantDiagnostics::Yes ||
+          Next->UpdateType == WantDiagnostics::Auto)
+        return true; // Prefer later diagnostics.
+    return false;
+  }
+}
+
 bool ASTWorker::blockUntilIdle(Deadline Timeout) const {
   std::unique_lock<std::mutex> Lock(Mutex);
   return wait(Lock, RequestsCV, Timeout, [&] { return Requests.empty(); });
 }
 
 } // namespace
 
 unsigned getDefaultAsyncThreadsCount() {

 struct TUScheduler::FileData {
   /// Latest inputs, passed to TUScheduler::update().
   ParseInputs Inputs;
   ASTWorkerHandle Worker;
 };
 
 TUScheduler::TUScheduler(unsigned AsyncThreadsCount,
                          bool StorePreamblesInMemory,
-                         ASTParsedCallback ASTCallback)
+                         ASTParsedCallback ASTCallback,
+                         steady_clock::duration UpdateDebounce)
     : StorePreamblesInMemory(StorePreamblesInMemory),
       PCHOps(std::make_shared<PCHContainerOperations>()),
-      ASTCallback(std::move(ASTCallback)), Barrier(AsyncThreadsCount) {
+      ASTCallback(std::move(ASTCallback)), Barrier(AsyncThreadsCount),
+      UpdateDebounce(UpdateDebounce) {
   if (0 < AsyncThreadsCount) {
     PreambleTasks.emplace();
     WorkerThreads.emplace();
   }
 }
 
 TUScheduler::~TUScheduler() {
   // Notify all workers that they need to stop.
if (!File.getValue()->Worker->blockUntilIdle(D))
       return false;
   if (PreambleTasks)
     if (!PreambleTasks->wait(D))
       return false;
   return true;
 }
 
 void TUScheduler::update(
-    PathRef File, ParseInputs Inputs,
-    UniqueFunction<void(llvm::Optional<std::vector<DiagWithFixIts>>)>
-        OnUpdated) {
+    PathRef File, ParseInputs Inputs, WantDiagnostics WantDiags,
+    UniqueFunction<void(std::vector<DiagWithFixIts>)> OnUpdated) {
   std::unique_ptr<FileData> &FD = Files[File];
   if (!FD) {
     // Create a new worker to process the AST-related tasks.
     ASTWorkerHandle Worker = ASTWorker::Create(
         File, WorkerThreads ? WorkerThreads.getPointer() : nullptr, Barrier,
-        CppFile(File, StorePreamblesInMemory, PCHOps, ASTCallback));
+        CppFile(File, StorePreamblesInMemory, PCHOps, ASTCallback),
+        UpdateDebounce);
     FD = std::unique_ptr<FileData>(new FileData{Inputs, std::move(Worker)});
   } else {
     FD->Inputs = Inputs;
   }
-  FD->Worker->update(std::move(Inputs), std::move(OnUpdated));
+  FD->Worker->update(std::move(Inputs), WantDiags, std::move(OnUpdated));
 }
 
 void TUScheduler::remove(PathRef File) {
   bool Removed = Files.erase(File);
   if (!Removed)
     log("Trying to remove file from TUScheduler that is not tracked. File:" +
         File);
 }

clangd/Threading.h

 //===--- ThreadPool.h --------------------------------------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_THREADING_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_THREADING_H
 
 #include "Context.h"
 #include "Function.h"
 #include "llvm/ADT/Twine.h"
-#include <atomic>
 #include <cassert>
 #include <condition_variable>
 #include <memory>
 #include <mutex>
 #include <vector>
 
 namespace clang {
 namespace clangd {
 
-/// A shared boolean flag indicating if the computation was cancelled.
-/// Once cancelled, cannot be returned to the previous state.
-class CancellationFlag {
+/// A threadsafe flag that is initially clear.
+class Notification {
 public:
-  CancellationFlag();
-
-  void cancel() {
-    assert(WasCancelled && "the object was moved");
-    WasCancelled->store(true);
-  }
-
-  bool isCancelled() const {
-    assert(WasCancelled && "the object was moved");
-    return WasCancelled->load();
-  }
+  // Sets the flag. No-op if already set.
+  void notify();
+  // Blocks until flag is set.
+  void wait() const;
 
 private:
-  std::shared_ptr<std::atomic<bool>> WasCancelled;
+  bool Notified = false;
+  mutable std::condition_variable CV;
+  mutable std::mutex Mu;
 };
 
 /// Limits the number of threads that can acquire the lock at the same time.
 class Semaphore {
 public:
   Semaphore(std::size_t MaxLocks);
 
   void lock();
   void unlock();
 
 private:
   std::mutex Mutex;
   std::condition_variable SlotsChanged;
   std::size_t FreeSlots;
 };
 
-/// A point in time we may wait for, or None to wait forever.
+/// A point in time we can wait for.
+/// Can be zero (don't wait) or infinity (wait forever).
 /// (Not time_point::max(), because many std::chrono implementations overflow).
-using Deadline = llvm::Optional<std::chrono::steady_clock::time_point>;
-/// Makes a deadline from a timeout in seconds.
+class Deadline {
+public:
+  Deadline(std::chrono::steady_clock::time_point Time)
+      : Type(Finite), Time(Time) {}
+  static Deadline zero() { return Deadline(Zero); }
+  static Deadline infinity() { return Deadline(Infinite); }
+
+  std::chrono::steady_clock::time_point time() const {
+    assert(Type == Finite);
+    return Time;
+  }
+  bool expired() const {
+    return (Type == Zero) ||
+           (Type == Finite && Time < std::chrono::steady_clock::now());
+  }
+  bool operator==(const Deadline &Other) const {
+    return (Type == Other.Type) && (Type != Finite || Time == Other.Time);
+  }
+
+private:
+  enum Type { Zero, Infinite, Finite } Type;
+  Deadline(enum Type Type) : Type(Type) {}
+  std::chrono::steady_clock::time_point Time;
+};
+
+/// Makes a deadline from a timeout in seconds. None means wait forever.
 Deadline timeoutSeconds(llvm::Optional<double> Seconds);
+/// Wait once for on CV for the specified duration.
+void wait(std::unique_lock<std::mutex> &Lock, std::condition_variable &CV,
+          Deadline D);
 /// Waits on a condition variable until F() is true or D expires.
 template <typename Func>
 LLVM_NODISCARD bool wait(std::unique_lock<std::mutex> &Lock,
                          std::condition_variable &CV, Deadline D, Func F) {
-  if (D)
-    return CV.wait_until(Lock, *D, F);
-  CV.wait(Lock, F);
+  while (!F()) {
+    if (D.expired())
+      return false;
+    wait(Lock, CV, D);
+  }
   return true;
 }
 
 /// Runs tasks on separate (detached) threads and wait for all tasks to finish.
 /// Objects that need to spawn threads can own an AsyncTaskRunner to ensure they
 /// all complete on destruction.
 class AsyncTaskRunner {
 public:
   /// Destructor waits for all pending tasks to finish.
   ~AsyncTaskRunner();
 
-  void wait() const { (void) wait(llvm::None); }
+  void wait() const { (void)wait(Deadline::infinity()); }
   LLVM_NODISCARD bool wait(Deadline D) const;
   // The name is used for tracing and debugging (e.g. to name a spawned thread).
   void runAsync(llvm::Twine Name, UniqueFunction<void()> Action);
 
 private:
   mutable std::mutex Mutex;
   mutable std::condition_variable TasksReachedZero;
   std::size_t InFlightTasks = 0;
 };
 } // namespace clangd
 } // namespace clang
 #endif

clangd/Threading.cpp

 #include "Threading.h"
 #include "llvm/ADT/ScopeExit.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/Threading.h"
 #include <thread>
 
 namespace clang {
 namespace clangd {
 
-CancellationFlag::CancellationFlag()
-    : WasCancelled(std::make_shared<std::atomic<bool>>(false)) {}
+void Notification::notify() {
+  {
+    std::lock_guard<std::mutex> Lock(Mu);
+    Notified = true;
+  }
+  CV.notify_all();
+}
+
+void Notification::wait() const {
+  std::unique_lock<std::mutex> Lock(Mu);
+  CV.wait(Lock, [this] { return Notified; });
+}
 
 Semaphore::Semaphore(std::size_t MaxLocks) : FreeSlots(MaxLocks) {}
 
 void Semaphore::lock() {
   std::unique_lock<std::mutex> Lock(Mutex);
   SlotsChanged.wait(Lock, [&]() { return FreeSlots > 0; });
   --FreeSlots;
 }
std::thread(
       },
       Name.str(), std::move(Action), std::move(CleanupTask))
       .detach();
 }
 
 Deadline timeoutSeconds(llvm::Optional<double> Seconds) {
   using namespace std::chrono;
   if (!Seconds)
-    return llvm::None;
+    return Deadline::infinity();
   return steady_clock::now() +
          duration_cast<steady_clock::duration>(duration<double>(*Seconds));
 }
 
+void wait(std::unique_lock<std::mutex> &Lock, std::condition_variable &CV,
+          Deadline D) {
+  if (D == Deadline::zero())
+    return;
+  if (D == Deadline::infinity())
+    return CV.wait(Lock);
+  CV.wait_until(Lock, D.time());
+}
+
 } // namespace clangd
 } // namespace clang

unittests/clangd/ClangdTests.cpp

 //===-- ClangdTests.cpp - Clangd unit tests ---------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
+#include "Annotations.h"
 #include "ClangdLSPServer.h"
 #include "ClangdServer.h"
 #include "Matchers.h"
 #include "SyncAPI.h"
 #include "TestFS.h"
 #include "URI.h"
 #include "clang/Config/config.h"
 #include "llvm/ADT/SmallVector.h"
public:
   VFSTag lastVFSTag() { return LastVFSTag; }
 
 private:
   std::mutex Mutex;
   bool HadErrorInLastDiags = false;
   VFSTag LastVFSTag = VFSTag();
 };
 
+/// For each file, record whether the last published diagnostics contained at
+/// least one error.
+class MultipleErrorCheckingDiagConsumer : public DiagnosticsConsumer {
+public:
+  void
+  onDiagnosticsReady(PathRef File,
+                     Tagged<std::vector<DiagWithFixIts>> Diagnostics) override {
+    bool HadError = diagsContainErrors(Diagnostics.Value);
+
+    std::lock_guard<std::mutex> Lock(Mutex);
+    LastDiagsHadError[File] = HadError;
+  }
+
+  /// Exposes all files consumed by onDiagnosticsReady in an unspecified order.
+  /// For each file, a bool value indicates whether the last diagnostics
+  /// contained an error.
+  std::vector<std::pair<Path, bool>> filesWithDiags() const {
+    std::vector<std::pair<Path, bool>> Result;
+    std::lock_guard<std::mutex> Lock(Mutex);
+
+    for (const auto &it : LastDiagsHadError) {
+      Result.emplace_back(it.first(), it.second);
+    }
+
+    return Result;
+  }
+
+  void clear() {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    LastDiagsHadError.clear();
+  }
+
+private:
+  mutable std::mutex Mutex;
+  llvm::StringMap<bool> LastDiagsHadError;
+};
+
 /// Replaces all patterns of the form 0x123abc with spaces
 std::string replacePtrsInDump(std::string const &Dump) {
   llvm::Regex RE("0x[0-9a-fA-F]+");
   llvm::SmallVector<StringRef, 1> Matches;
   llvm::StringRef Pending = Dump;
 
   std::string Result;
   while (RE.match(Pending, &Matches)) {
)cpp";
   // But forceReparse should reparse the file with proper flags.
   Server.forceReparse(FooCpp);
   EXPECT_FALSE(DiagConsumer.hadErrorInLastDiags());
   // Subsequent addDocument call should finish without errors too.
   Server.addDocument(FooCpp, SourceContents);
   EXPECT_FALSE(DiagConsumer.hadErrorInLastDiags());
 }
 
+// Test ClangdServer.reparseOpenedFiles.
+TEST_F(ClangdVFSTest, ReparseOpenedFiles) {
+  Annotations FooSource(R"cpp(
+#ifdef MACRO
+$one[[static void bob() {}]]
+#else
+$two[[static void bob() {}]]
+#endif
+
+int main () { bo^b (); return 0; }
+)cpp");
+
+  Annotations BarSource(R"cpp(
+#ifdef MACRO
+this is an error
+#endif
+)cpp");
+
+  Annotations BazSource(R"cpp(
+int hello;
+)cpp");
+
+  MockFSProvider FS;
+  MockCompilationDatabase CDB;
+  MultipleErrorCheckingDiagConsumer DiagConsumer;
+  ClangdServer Server(CDB, DiagConsumer, FS,
+                      /*AsyncThreadsCount=*/0,
+                      /*StorePreamblesInMemory=*/true);
+
+  auto FooCpp = testPath("foo.cpp");
+  auto BarCpp = testPath("bar.cpp");
+  auto BazCpp = testPath("baz.cpp");
+
+  FS.Files[FooCpp] = "";
+  FS.Files[BarCpp] = "";
+  FS.Files[BazCpp] = "";
+
+  CDB.ExtraClangFlags = {"-DMACRO=1"};
+  Server.addDocument(FooCpp, FooSource.code());
+  Server.addDocument(BarCpp, BarSource.code());
+  Server.addDocument(BazCpp, BazSource.code());
+
+  EXPECT_THAT(DiagConsumer.filesWithDiags(),
+              UnorderedElementsAre(Pair(FooCpp, false), Pair(BarCpp, true),
+                                   Pair(BazCpp, false)));
+
+  auto Locations = runFindDefinitions(Server, FooCpp, FooSource.point());
+  EXPECT_TRUE(bool(Locations));
+  EXPECT_THAT(Locations->Value, ElementsAre(Location{URIForFile{FooCpp},
+                                                     FooSource.range("one")}));
+
+  // Undefine MACRO, close baz.cpp.
+  CDB.ExtraClangFlags.clear();
+  DiagConsumer.clear();
+  Server.removeDocument(BazCpp);
+  Server.reparseOpenedFiles();
+
+  EXPECT_THAT(DiagConsumer.filesWithDiags(),
+              UnorderedElementsAre(Pair(FooCpp, false), Pair(BarCpp, false)));
+
+  Locations = runFindDefinitions(Server, FooCpp, FooSource.point());
+  EXPECT_TRUE(bool(Locations));
+  EXPECT_THAT(Locations->Value, ElementsAre(Location{URIForFile{FooCpp},
+                                                     FooSource.range("two")}));
+}
+
 TEST_F(ClangdVFSTest, MemoryUsage) {
   MockFSProvider FS;
   ErrorCheckingDiagConsumer DiagConsumer;
   MockCompilationDatabase CDB;
   ClangdServer Server(CDB, DiagConsumer, FS,
                       /*AsyncThreadsCount=*/0,
                       /*StorePreamblesInMemory=*/true);
 

unittests/clangd/TUSchedulerTests.cpp

TUScheduler S(getDefaultAsyncThreadsCount(),
                 /*ASTParsedCallback=*/nullptr);
 
   auto Added = testPath("added.cpp");
   Files[Added] = "";
 
   auto Missing = testPath("missing.cpp");
   Files[Missing] = "";
 
-  S.update(Added, getInputs(Added, ""), ignoreUpdate);
+  S.update(Added, getInputs(Added, ""), WantDiagnostics::No, ignoreUpdate);
 
   // Assert each operation for missing file is an error (even if it's available
   // in VFS).
   S.runWithAST("", Missing, [&](llvm::Expected<InputsAndAST> AST) {
     ASSERT_FALSE(bool(AST));
     ignoreError(AST.takeError());
   });
   S.runWithPreamble("", Missing,
TEST_F(TUSchedulerTests, MissingFiles) {
   S.runWithPreamble("", Added, [&](llvm::Expected<InputsAndPreamble> Preamble) {
     ASSERT_FALSE(bool(Preamble));
     ignoreError(Preamble.takeError());
   });
   // remove() shouldn't crash on missing files.
   S.remove(Added);
 }
 
+TEST_F(TUSchedulerTests, WantDiagnostics) {
+  std::atomic<int> CallbackCount(0);
+  {
+    // To avoid a racy test, don't allow tasks to actualy run on the worker
+    // thread until we've scheduled them all.
+    Notification Ready;
+    TUScheduler S(getDefaultAsyncThreadsCount(),
+                  /*StorePreamblesInMemory=*/true,
+                  /*ASTParsedCallback=*/nullptr);
+    auto Path = testPath("foo.cpp");
+    S.update(Path, getInputs(Path, ""), WantDiagnostics::Yes,
+             [&](std::vector<DiagWithFixIts>) { Ready.wait(); });
+
+    S.update(Path, getInputs(Path, "request diags"), WantDiagnostics::Yes,
+             [&](std::vector<DiagWithFixIts> Diags) { ++CallbackCount; });
+    S.update(Path, getInputs(Path, "auto (clobbered)"), WantDiagnostics::Auto,
+             [&](std::vector<DiagWithFixIts> Diags) {
+               ADD_FAILURE() << "auto should have been cancelled by auto";
+             });
+    S.update(Path, getInputs(Path, "request no diags"), WantDiagnostics::No,
+             [&](std::vector<DiagWithFixIts> Diags) {
+               ADD_FAILURE() << "no diags should not be called back";
+             });
+    S.update(Path, getInputs(Path, "auto (produces)"), WantDiagnostics::Auto,
+             [&](std::vector<DiagWithFixIts> Diags) { ++CallbackCount; });
+    Ready.notify();
+  }
+  EXPECT_EQ(2, CallbackCount);
+}
+
+TEST_F(TUSchedulerTests, Debounce) {
+  std::atomic<int> CallbackCount(0);
+  {
+    TUScheduler S(getDefaultAsyncThreadsCount(),
+                  /*StorePreamblesInMemory=*/true,
+                  /*ASTParsedCallback=*/nullptr,
+                  /*UpdateDebounce=*/std::chrono::milliseconds(50));
+    auto Path = testPath("foo.cpp");
+    S.update(Path, getInputs(Path, "auto (debounced)"), WantDiagnostics::Auto,
+             [&](std::vector<DiagWithFixIts> Diags) {
+               ADD_FAILURE() << "auto should have been debounced and canceled";
+             });
+    std::this_thread::sleep_for(std::chrono::milliseconds(10));
+    S.update(Path, getInputs(Path, "auto (timed out)"), WantDiagnostics::Auto,
+             [&](std::vector<DiagWithFixIts> Diags) { ++CallbackCount; });
+    std::this_thread::sleep_for(std::chrono::milliseconds(60));
+    S.update(Path, getInputs(Path, "auto (shut down)"), WantDiagnostics::Auto,
+             [&](std::vector<DiagWithFixIts> Diags) { ++CallbackCount; });
+  }
+  EXPECT_EQ(2, CallbackCount);
+}
+
 TEST_F(TUSchedulerTests, ManyUpdates) {
   const int FilesCount = 3;
   const int UpdatesPerFile = 10;
 
   std::mutex Mut;
   int TotalASTReads = 0;
   int TotalPreambleReads = 0;
   int TotalUpdates = 0;
for (int FileI = 0; FileI < FilesCount; ++FileI) {
       for (int UpdateI = 0; UpdateI < UpdatesPerFile; ++UpdateI) {
         auto Contents = AllContents[(FileI + UpdateI) % AllContentsSize];
 
         auto File = Files[FileI];
         auto Inputs = getInputs(File, Contents.str());
 
         {
           WithContextValue WithNonce(NonceKey, ++Nonce);
-          S.update(File, Inputs,
+          S.update(File, Inputs, WantDiagnostics::Auto,
                    [Nonce, &Mut, &TotalUpdates](
                        llvm::Optional<std::vector<DiagWithFixIts>> Diags) {
                      EXPECT_THAT(Context::current().get(NonceKey),
                                  Pointee(Nonce));
 
                      std::lock_guard<std::mutex> Lock(Mut);
                      ++TotalUpdates;
                    });