[clangd] Refactoring of GlobalCompilationDatabase

[sammccall]

Summary

The interface used is now tooling::CompilationDatabase.
The various behaviors of the existing CDB implementation is decomposed into
several classes responsible for one aspect each.

• The fallback commands are now a FixedCompilationDatabase. For now, this is done by both ClangdLSPServer and ClangdServer (preserving behavior) but I'd like to eliminate the latter, as embedders of ClangdServer are in a better position to know what fallback is appropriate.
• the fallback now uses '.' as the directory, rather than the file's parent.
• The compilation database file scan caches missing as well as present files.
• -fsyntax-only is now applied to all commands, not just the fallback. We also string output flags.
• The final command line used for each file is now logged.
• The -resource-dir flag is managed by ClangdServer rather than being done deeper in the stack. All flag manipulation is done by CDBs set up by ClangdLSPServer and ClangdServer.
• Race in extra-flags fixed: when reading extra flags for a file we now lock the map

[ilya-biryukov]

clangd/ClangdLSPServer.cpp

254

Clangd still changes working directory when running the parsing, so "." might end up being a different dir on multiple runs over the same file.
Maybe use file's parent for now?

clangd/ClangdServer.cpp

178

Maybe move this code out of ClangdServer? Arguably, it should be less confusing for clients of ClangdServer if they get exactly the compile commands returned by CDB.

This would also allow to get rid of ResourceDir parameter, which seems to be handled by CompilationDatabase now.

clangd/ClangdServer.h

208

Are there any mutating methods in CompilataionDatabase? Why do we use const & instead of &?

clangd/GlobalCompilationDatabase.cpp

57

IIRC, Cmd could come directly from compile_commands.json.
We should avoid crashing clangd if contents of compile_commands.json are invalid. Could we log the error instead and continue with other commands instead?

clangd/GlobalCompilationDatabase.h

34

It seems that this const means more trouble for implementations (i.e. using mutable, etc.) without providing any value.
Maybe we should consider removing it from the interface?
Am I missing the upsides of using const here?

61

Maybe we should consider using non-owning reference here?
It gives more flexibility, e.g. allowing to allocate on stack or store CDB directly inside the class fields, albeit it requires a bit more code to setup.
I'd say upsides of non-owning semantics outweigh a bit of added complexity

[malaperle]

@Nebiroth , Will this be compatible with your patch to change CompilationDatabase at runtime? https://reviews.llvm.org/D39571

clangd/ClangdLSPServer.h

private:
   std::mutex FixItsMutex;
   typedef std::map<clangd::Diagnostic, std::vector<clang::tooling::Replacement>>
       DiagnosticToReplacementMap;
   /// Caches FixIts per file and diagnostics
   llvm::StringMap<DiagnosticToReplacementMap> FixItsMap;
 
   // Various ClangdServer parameters go here. It's important they're created
   // before ClangdServer.
-  DirectoryBasedGlobalCompilationDatabase CDB;
+  ExtraFlagsCompilationDatabase CDB;
   RealFileSystemProvider FSProvider;
 
   // Server must be the last member of the class to allow its destructor to exit
   // the worker thread that may otherwise run an async callback on partially
   // destructed instance of ClangdLSPServer.
   ClangdServer Server;
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/ClangdLSPServer.cpp

 
 void ClangdLSPServer::onSwitchSourceHeader(Ctx C,
                                            TextDocumentIdentifier &Params) {
   llvm::Optional<Path> Result = Server.switchSourceHeader(Params.uri.file);
   std::string ResultUri;
   C.reply(Result ? URI::fromFile(*Result).uri : "");
 }
 
+// Creates the base compilation database, which must always return a command.
+// The per-file extra flags are overlaid on top of this.
+static std::unique_ptr<tooling::CompilationDatabase>
+CreateCDB(clangd::Logger &Out, llvm::Optional<Path> CompileCommandsDir) {
+  using namespace tooling;
+  std::unique_ptr<CompilationDatabase> Primary;
+  if (CompileCommandsDir) {
+    std::string Err;
+    Primary = CompilationDatabase::loadFromDirectory(*CompileCommandsDir, Err);
+    if (!Primary)
+      Out.log(llvm::Twine("Failed to load compilation database from ") +
+              *CompileCommandsDir + ", falling back to default: " + Err);
+  }
+  if (!Primary)
+    Primary = llvm::make_unique<GlobalCompilationDatabase>();
+  return llvm::make_unique<FallbackCompilationDatabase>(
+      std::move(Primary), llvm::make_unique<FixedCompilationDatabase>(
+                              ".", ArrayRef<std::string>{}));
+}
+
 ClangdLSPServer::ClangdLSPServer(JSONOutput &Out, unsigned AsyncThreadsCount,
                                  bool StorePreamblesInMemory,
                                  const clangd::CodeCompleteOptions &CCOpts,
                                  llvm::Optional<StringRef> ResourceDir,
                                  llvm::Optional<Path> CompileCommandsDir)
-    : Out(Out), CDB(/*Logger=*/Out, std::move(CompileCommandsDir)),
+    : Out(Out), CDB(CreateCDB(Out, std::move(CompileCommandsDir))),
       Server(CDB, /*DiagConsumer=*/*this, FSProvider, AsyncThreadsCount,
              StorePreamblesInMemory, CCOpts,
              /*Logger=*/Out, ResourceDir) {}
 
 bool ClangdLSPServer::run(std::istream &In) {
   assert(!IsDone && "Run was called before");
 
   // Set up JSONRPCDispatcher.

clangd/ClangdServer.h

 //===--- ClangdServer.h - Main clangd server code ----------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDSERVER_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDSERVER_H
 
 #include "ClangdUnitStore.h"
 #include "DraftStore.h"
-#include "GlobalCompilationDatabase.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "clang/Tooling/Core/Replacement.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/StringRef.h"
 
 #include "ClangdUnit.h"
 #include "Function.h"
public:
   /// \p DiagConsumer. Note that a callback to \p DiagConsumer happens on a
   /// worker thread. Therefore, instances of \p DiagConsumer must properly
   /// synchronize access to shared state.
   ///
   /// \p StorePreamblesInMemory defines whether the Preambles generated by
   /// clangd are stored in-memory or on disk.
   ///
   /// Various messages are logged using \p Logger.
-  ClangdServer(GlobalCompilationDatabase &CDB,
+  ClangdServer(const tooling::CompilationDatabase &CDB,
                DiagnosticsConsumer &DiagConsumer,
                FileSystemProvider &FSProvider, unsigned AsyncThreadsCount,
                bool StorePreamblesInMemory,
                const clangd::CodeCompleteOptions &CodeCompleteOpts,
                clangd::Logger &Logger,
                llvm::Optional<StringRef> ResourceDir = llvm::None);
 
   /// Set the root path of the workspace.
   void setRootPath(PathRef RootPath);
 
   /// Add a \p File to the list of tracked C++ files or update the contents if
   /// \p File is already tracked. Also schedules parsing of the AST for it on a
   /// separate thread. When the parsing is complete, DiagConsumer passed in
   /// constructor will receive onDiagnosticsReady callback.
   /// \return A future that will become ready when the rebuild (including
   /// diagnostics) is finished.
   std::future<void> addDocument(PathRef File, StringRef Contents);
   /// Remove \p File from list of tracked files, schedule a request to free
   /// resources associated with it.
   /// \return A future that will become ready when the file is removed and all
   /// associated resources are freed.
   std::future<void> removeDocument(PathRef File);
   /// Force \p File to be reparsed using the latest contents.
-  /// Will also check if CompileCommand, provided by GlobalCompilationDatabase
+  /// Will also check if CompileCommand, provided by CompilationDatabase
   /// for \p File has changed. If it has, will remove currently stored Preamble
   /// and AST and rebuild them from scratch.
   std::future<void> forceReparse(PathRef File);
 
   /// DEPRECATED. Please use a callback-based version, this API is deprecated
   /// and will soon be removed.
   ///
   /// Run code completion for \p File at \p Pos.
private:
   std::future<void>
   scheduleReparseAndDiags(PathRef File, VersionedDraft Contents,
                           std::shared_ptr<CppFile> Resources,
                           Tagged<IntrusiveRefCntPtr<vfs::FileSystem>> TaggedFS);
 
   std::future<void> scheduleCancelRebuild(std::shared_ptr<CppFile> Resources);
 
   clangd::Logger &Logger;
-  GlobalCompilationDatabase &CDB;
+  std::unique_ptr<tooling::CompilationDatabase> CDB;
   DiagnosticsConsumer &DiagConsumer;
   FileSystemProvider &FSProvider;
   DraftStore DraftMgr;
   CppFileCollection Units;
   std::string ResourceDir;
   // If set, this represents the workspace path.
   llvm::Optional<std::string> RootPath;
   std::shared_ptr<PCHContainerOperations> PCHs;

clangd/ClangdServer.cpp

 //===--- ClangdServer.cpp - Main clangd server code --------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===-------------------------------------------------------------------===//
 
 #include "ClangdServer.h"
+#include "GlobalCompilationDatabase.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
+#include "clang/Tooling/CommonOptionsParser.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "clang/Tooling/Refactoring/RefactoringResultConsumer.h"
 #include "clang/Tooling/Refactoring/Rename/RenamingAction.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/FormatProviders.h"
 #include "llvm/Support/FormatVariadic.h"
if (RunSynchronously)
     Done = true;
   } // unlock Mutex
   RequestCV.notify_all();
 
   for (auto &Worker : Workers)
     Worker.join();
 }
 
-ClangdServer::ClangdServer(GlobalCompilationDatabase &CDB,
+static std::unique_ptr<tooling::CompilationDatabase>
+CreateCDB(const tooling::CompilationDatabase &CDB,
+          llvm::Optional<StringRef> ResourceDir) {
+  // Create an owned version of CDB to use ArgumentsAdjustingCompilations.
+  struct OwnedCDB : public tooling::CompilationDatabase {
+    OwnedCDB(const tooling::CompilationDatabase &Ref) : Ref(Ref) {}
+    const tooling::CompilationDatabase &Ref;
+    std::vector<tooling::CompileCommand>
+    getCompileCommands(StringRef P) const override {
+      return Ref.getCompileCommands(P);
+    }
+  };
+  std::unique_ptr<tooling::CompilationDatabase> Base(new OwnedCDB(CDB));
+  // FIXME: Instead of falling back to fixed arguments, we should refuse to
+  // parse files without commands. The caller can set a fallback policy.
+  Base = llvm::make_unique<FallbackCompilationDatabase>(
+      std::move(Base), llvm::make_unique<tooling::FixedCompilationDatabase>(
+                           ".", ArrayRef<std::string>{}));
+  auto Result = llvm::make_unique<tooling::ArgumentsAdjustingCompilations>(
+      std::move(Base));
+  Result->appendArgumentsAdjuster(tooling::getClangSyntaxOnlyAdjuster());
+  Result->appendArgumentsAdjuster(tooling::getClangStripOutputAdjuster());
+  Result->appendArgumentsAdjuster(tooling::getInsertArgumentAdjuster(
+      {"-resource-dir=" +
+       (ResourceDir ? ResourceDir->str() : getStandardResourceDir())},
+      tooling::ArgumentInsertPosition::BEGIN));
+  return std::move(Result);
+}
+
+ClangdServer::ClangdServer(const tooling::CompilationDatabase &CDB,
                            DiagnosticsConsumer &DiagConsumer,
                            FileSystemProvider &FSProvider,
                            unsigned AsyncThreadsCount,
                            bool StorePreamblesInMemory,
                            const clangd::CodeCompleteOptions &CodeCompleteOpts,
                            clangd::Logger &Logger,
                            llvm::Optional<StringRef> ResourceDir)
-    : Logger(Logger), CDB(CDB), DiagConsumer(DiagConsumer),
-      FSProvider(FSProvider),
-      ResourceDir(ResourceDir ? ResourceDir->str() : getStandardResourceDir()),
+    : Logger(Logger), CDB(CreateCDB(CDB, ResourceDir)),
+      DiagConsumer(DiagConsumer), FSProvider(FSProvider),
       PCHs(std::make_shared<PCHContainerOperations>()),
       StorePreamblesInMemory(StorePreamblesInMemory),
       CodeCompleteOpts(CodeCompleteOpts), WorkScheduler(AsyncThreadsCount) {}
 
 void ClangdServer::setRootPath(PathRef RootPath) {
   std::string NewRootPath = llvm::sys::path::convert_to_slash(
       RootPath, llvm::sys::path::Style::posix);
   if (llvm::sys::fs::is_directory(NewRootPath))
     this->RootPath = NewRootPath;
 }
 
 std::future<void> ClangdServer::addDocument(PathRef File, StringRef Contents) {
   DocVersion Version = DraftMgr.updateDraft(File, Contents);
 
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
-  std::shared_ptr<CppFile> Resources = Units.getOrCreateFile(
-      File, ResourceDir, CDB, StorePreamblesInMemory, PCHs, Logger);
+  std::shared_ptr<CppFile> Resources =
+      Units.getOrCreateFile(File, *CDB, StorePreamblesInMemory, PCHs, Logger);
   return scheduleReparseAndDiags(File, VersionedDraft{Version, Contents.str()},
                                  std::move(Resources), std::move(TaggedFS));
 }
 
 std::future<void> ClangdServer::removeDocument(PathRef File) {
   DraftMgr.removeDraft(File);
   std::shared_ptr<CppFile> Resources = Units.removeIfPresent(File);
   return scheduleCancelRebuild(std::move(Resources));
 }
 
 std::future<void> ClangdServer::forceReparse(PathRef File) {
   auto FileContents = DraftMgr.getDraft(File);
   assert(FileContents.Draft &&
          "forceReparse() was called for non-added document");
 
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
   auto Recreated = Units.recreateFileIfCompileCommandChanged(
-      File, ResourceDir, CDB, StorePreamblesInMemory, PCHs, Logger);
+      File, *CDB, StorePreamblesInMemory, PCHs, Logger);
 
   // Note that std::future from this cleanup action is ignored.
   scheduleCancelRebuild(std::move(Recreated.RemovedFile));
   // Schedule a reparse.
   return scheduleReparseAndDiags(File, std::move(FileContents),
                                  std::move(Recreated.FileInCollection),
                                  std::move(TaggedFS));
 }

clangd/ClangdUnit.cpp

 
 CppFile::CppFile(PathRef FileName, tooling::CompileCommand Command,
                  bool StorePreamblesInMemory,
                  std::shared_ptr<PCHContainerOperations> PCHs,
                  clangd::Logger &Logger)
     : FileName(FileName), Command(std::move(Command)),
       StorePreamblesInMemory(StorePreamblesInMemory), RebuildCounter(0),
       RebuildInProgress(false), PCHs(std::move(PCHs)), Logger(Logger) {
+  Logger.log(llvm::Twine("File ") + FileName +
+             " has command: " + llvm::join(this->Command.CommandLine, " "));
 
   std::lock_guard<std::mutex> Lock(Mutex);
   LatestAvailablePreamble = nullptr;
   PreamblePromise.set_value(nullptr);
   PreambleFuture = PreamblePromise.get_future();
 
   ASTPromise.set_value(std::make_shared<ParsedASTWrapper>(llvm::None));
   ASTFuture = ASTPromise.get_future();

clangd/ClangdUnitStore.h

 //===--- ClangdUnitStore.h - A container of CppFiles -------------*-C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===---------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDUNITSTORE_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDUNITSTORE_H
 
 #include <mutex>
 
 #include "ClangdUnit.h"
-#include "GlobalCompilationDatabase.h"
 #include "Path.h"
 #include "clang/Tooling/CompilationDatabase.h"
 
 namespace clang {
 namespace clangd {
 
 class Logger;
 
 /// Thread-safe mapping from FileNames to CppFile.
 class CppFileCollection {
 public:
   std::shared_ptr<CppFile>
-  getOrCreateFile(PathRef File, PathRef ResourceDir,
-                  GlobalCompilationDatabase &CDB, bool StorePreamblesInMemory,
+  getOrCreateFile(PathRef File, tooling::CompilationDatabase &CDB,
+                  bool StorePreamblesInMemory,
                   std::shared_ptr<PCHContainerOperations> PCHs,
                   clangd::Logger &Logger) {
     std::lock_guard<std::mutex> Lock(Mutex);
 
     auto It = OpenedFiles.find(File);
     if (It == OpenedFiles.end()) {
-      auto Command = getCompileCommand(CDB, File, ResourceDir);
-
+      auto Commands = CDB.getCompileCommands(File);
+      assert(!Commands.empty() && "Expected at least a fallback command!");
       It = OpenedFiles
-               .try_emplace(File, CppFile::Create(File, std::move(Command),
-                                                  StorePreamblesInMemory,
-                                                  std::move(PCHs), Logger))
+               .try_emplace(File,
+                            CppFile::Create(File, std::move(Commands.front()),
                .first;
     }
     return It->second;
   }
 
   struct RecreateResult {
     /// A CppFile, stored in this CppFileCollection for the corresponding
     /// filepath after calling recreateFileIfCompileCommandChanged.
     std::shared_ptr<CppFile> FileInCollection;
     /// If a new CppFile had to be created to account for changed
     /// CompileCommand, a previous CppFile instance will be returned in this
     /// field.
     std::shared_ptr<CppFile> RemovedFile;
   };
 
   /// Similar to getOrCreateFile, but will replace a current CppFile for \p File
   /// with a new one if CompileCommand, provided by \p CDB has changed.
   /// If a currently stored CppFile had to be replaced, the previous instance
   /// will be returned in RecreateResult.RemovedFile.
   RecreateResult recreateFileIfCompileCommandChanged(
-      PathRef File, PathRef ResourceDir, GlobalCompilationDatabase &CDB,
+      PathRef File, tooling::CompilationDatabase &CDB,
       bool StorePreamblesInMemory, std::shared_ptr<PCHContainerOperations> PCHs,
       clangd::Logger &Logger);
 
   std::shared_ptr<CppFile> getFile(PathRef File) {
     std::lock_guard<std::mutex> Lock(Mutex);
 
     auto It = OpenedFiles.find(File);
     if (It == OpenedFiles.end())
       return nullptr;
     return It->second;
   }
 
   /// Removes a CppFile, stored for \p File, if it's inside collection and
   /// returns it.
   std::shared_ptr<CppFile> removeIfPresent(PathRef File);
 
 private:
-  tooling::CompileCommand getCompileCommand(GlobalCompilationDatabase &CDB,
-                                            PathRef File, PathRef ResourceDir);
-
   bool compileCommandsAreEqual(tooling::CompileCommand const &LHS,
                                tooling::CompileCommand const &RHS);
 
   std::mutex Mutex;
   llvm::StringMap<std::shared_ptr<CppFile>> OpenedFiles;
 };
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/ClangdUnitStore.cpp

std::shared_ptr<CppFile> CppFileCollection::removeIfPresent(PathRef File) {
 
   std::shared_ptr<CppFile> Result = It->second;
   OpenedFiles.erase(It);
   return Result;
 }
 
 CppFileCollection::RecreateResult
 CppFileCollection::recreateFileIfCompileCommandChanged(
-    PathRef File, PathRef ResourceDir, GlobalCompilationDatabase &CDB,
+    PathRef File, tooling::CompilationDatabase &CDB,
     bool StorePreamblesInMemory, std::shared_ptr<PCHContainerOperations> PCHs,
     clangd::Logger &Logger) {
-  auto NewCommand = getCompileCommand(CDB, File, ResourceDir);
+  auto Commands = CDB.getCompileCommands(File);
+  assert(!Commands.empty() && "Expected at least a fallback command!");
+  auto &NewCommand = Commands.front();
 
   std::lock_guard<std::mutex> Lock(Mutex);
 
   RecreateResult Result;
 
   auto It = OpenedFiles.find(File);
   if (It == OpenedFiles.end()) {
     It = OpenedFiles
              .try_emplace(File, CppFile::Create(File, std::move(NewCommand),
                                                 StorePreamblesInMemory,
                                                 std::move(PCHs), Logger))
              .first;
   } else if (!compileCommandsAreEqual(It->second->getCompileCommand(),
                                       NewCommand)) {
     Result.RemovedFile = std::move(It->second);
     It->second =
         CppFile::Create(File, std::move(NewCommand), StorePreamblesInMemory,
                         std::move(PCHs), Logger);
   }
   Result.FileInCollection = It->second;
   return Result;
 }
 
-tooling::CompileCommand
-CppFileCollection::getCompileCommand(GlobalCompilationDatabase &CDB,
-                                     PathRef File, PathRef ResourceDir) {
-  std::vector<tooling::CompileCommand> Commands = CDB.getCompileCommands(File);
-  if (Commands.empty())
-    // Add a fake command line if we know nothing.
-    Commands.push_back(getDefaultCompileCommand(File));
-
-  // Inject the resource dir.
-  // FIXME: Don't overwrite it if it's already there.
-  Commands.front().CommandLine.push_back("-resource-dir=" +
-                                         std::string(ResourceDir));
-  return std::move(Commands.front());
-}
-
 bool CppFileCollection::compileCommandsAreEqual(
     tooling::CompileCommand const &LHS, tooling::CompileCommand const &RHS) {
   // tooling::CompileCommand.Output is ignored, it's not relevant for clangd.
   return LHS.Directory == RHS.Directory &&
          LHS.CommandLine.size() == RHS.CommandLine.size() &&
          std::equal(LHS.CommandLine.begin(), LHS.CommandLine.end(),
                     RHS.CommandLine.begin());
 }

clangd/GlobalCompilationDatabase.h

 //===--- GlobalCompilationDatabase.h ----------------------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===---------------------------------------------------------------------===//
+//
+// Defines CompilationDatabase that help clangd determine compile commands.
+// We plug together a few of these, depending on configuration.
+//
+//===---------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_GLOBALCOMPILATIONDATABASE_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_GLOBALCOMPILATIONDATABASE_H
 
 #include "Path.h"
+#include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/StringMap.h"
 #include <memory>
 #include <mutex>
 #include <vector>
 
 namespace clang {
-
-namespace tooling {
-class CompilationDatabase;
-struct CompileCommand;
-} // namespace tooling
-
 namespace clangd {
-
 class Logger;
 
-/// Returns a default compile command to use for \p File.
-tooling::CompileCommand getDefaultCompileCommand(PathRef File);
-
-/// Provides compilation arguments used for parsing C and C++ files.
-class GlobalCompilationDatabase {
+/// Provides compile commands for any file by scanning parent directories
+/// looking for compilation databases. These are cached for efficiency.
+class GlobalCompilationDatabase : public tooling::CompilationDatabase {
 public:
-  virtual ~GlobalCompilationDatabase() = default;
+  std::vector<tooling::CompileCommand>
+  getCompileCommands(PathRef File) const override;
 
-  virtual std::vector<tooling::CompileCommand>
-  getCompileCommands(PathRef File) = 0;
+private:
+  tooling::CompilationDatabase *getCompilationDatabase(PathRef File) const;
 
-  /// FIXME(ibiryukov): add facilities to track changes to compilation flags of
-  /// existing targets.
+  mutable std::mutex Mutex;
+  /// Caches compilation databases from directories. Dir -> DB or null.
+  mutable llvm::StringMap<std::unique_ptr<clang::tooling::CompilationDatabase>>
+      Cache;
 };
 
-/// Gets compile args from tooling::CompilationDatabases built for parent
-/// directories.
-class DirectoryBasedGlobalCompilationDatabase
-    : public GlobalCompilationDatabase {
+/// Adds extra flags into the compile commands of another database.
+/// The extra flags are configured per file.
+class ExtraFlagsCompilationDatabase : public tooling::CompilationDatabase {
 public:
-  DirectoryBasedGlobalCompilationDatabase(
-      clangd::Logger &Logger, llvm::Optional<Path> CompileCommandsDir);
+  ExtraFlagsCompilationDatabase(
+      std::unique_ptr<tooling::CompilationDatabase> Inner)
+      : Inner(std::move(Inner)) {}
 
   std::vector<tooling::CompileCommand>
-  getCompileCommands(PathRef File) override;
+  getCompileCommands(PathRef File) const override;
 
-  void setExtraFlagsForFile(PathRef File, std::vector<std::string> ExtraFlags);
+  void setExtraFlagsForFile(PathRef File, std::vector<std::string> Flags);
 
 private:
-  tooling::CompilationDatabase *getCompilationDatabase(PathRef File);
-  tooling::CompilationDatabase *tryLoadDatabaseFromPath(PathRef File);
+  mutable std::mutex Mutex;
+  llvm::StringMap<std::vector<std::string>> ExtraFlags;
+  std::unique_ptr<tooling::CompilationDatabase> Inner;
+};
+
+/// Attempts to use a primary compilation database, falling back to a second
+/// one if the first returns no commands.
+class FallbackCompilationDatabase : public tooling::CompilationDatabase {
+public:
+  FallbackCompilationDatabase(
+      std::unique_ptr<tooling::CompilationDatabase> Primary,
+      std::unique_ptr<tooling::CompilationDatabase> Fallback)
+      : Primary(std::move(Primary)), Fallback(std::move(Fallback)) {}
 
-  std::mutex Mutex;
-  /// Caches compilation databases loaded from directories(keys are
-  /// directories).
-  llvm::StringMap<std::unique_ptr<clang::tooling::CompilationDatabase>>
-      CompilationDatabases;
-
-  /// Stores extra flags per file.
-  llvm::StringMap<std::vector<std::string>> ExtraFlagsForFile;
-  /// Used for logging.
-  clangd::Logger &Logger;
-  /// Used for command argument pointing to folder where compile_commands.json
-  /// is located.
-  llvm::Optional<Path> CompileCommandsDir;
+  std::vector<tooling::CompileCommand>
+  getCompileCommands(PathRef File) const override;
+
+private:
+  std::unique_ptr<tooling::CompilationDatabase> Primary, Fallback;
 };
+
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/GlobalCompilationDatabase.cpp

 #include "GlobalCompilationDatabase.h"
 #include "Logger.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 
 namespace clang {
 namespace clangd {
+using namespace tooling;
+namespace path = llvm::sys::path;
 
-static void addExtraFlags(tooling::CompileCommand &Command,
-                          const std::vector<std::string> &ExtraFlags) {
-  if (ExtraFlags.empty())
-    return;
-  assert(Command.CommandLine.size() >= 2 &&
-         "Expected a command line containing at least 2 arguments, the "
-         "compiler binary and the output file");
-  // The last argument of CommandLine is the name of the input file.
-  // Add ExtraFlags before it.
-  auto It = Command.CommandLine.end();
-  --It;
-  Command.CommandLine.insert(It, ExtraFlags.begin(), ExtraFlags.end());
-}
-
-tooling::CompileCommand getDefaultCompileCommand(PathRef File) {
-  std::vector<std::string> CommandLine{"clang", "-fsyntax-only", File.str()};
-  return tooling::CompileCommand(llvm::sys::path::parent_path(File),
-                                 llvm::sys::path::filename(File), CommandLine,
-                                 /*Output=*/"");
-}
-
-DirectoryBasedGlobalCompilationDatabase::
-    DirectoryBasedGlobalCompilationDatabase(
-        clangd::Logger &Logger, llvm::Optional<Path> CompileCommandsDir)
-    : Logger(Logger), CompileCommandsDir(std::move(CompileCommandsDir)) {}
-
-std::vector<tooling::CompileCommand>
-DirectoryBasedGlobalCompilationDatabase::getCompileCommands(PathRef File) {
-  std::vector<tooling::CompileCommand> Commands;
-
-  auto CDB = getCompilationDatabase(File);
-  if (CDB)
+std::vector<CompileCommand>
+GlobalCompilationDatabase::getCompileCommands(PathRef File) const {
+  std::vector<CompileCommand> Commands;
+  if (auto CDB = getCompilationDatabase(File))
     Commands = CDB->getCompileCommands(File);
-  if (Commands.empty())
-    Commands.push_back(getDefaultCompileCommand(File));
-
-  auto It = ExtraFlagsForFile.find(File);
-  if (It != ExtraFlagsForFile.end()) {
-    // Append the user-specified flags to the compile commands.
-    for (tooling::CompileCommand &Command : Commands)
-      addExtraFlags(Command, It->second);
-  }
-
   return Commands;
 }
 
-void DirectoryBasedGlobalCompilationDatabase::setExtraFlagsForFile(
-    PathRef File, std::vector<std::string> ExtraFlags) {
-  ExtraFlagsForFile[File] = std::move(ExtraFlags);
-}
-
-tooling::CompilationDatabase *
-DirectoryBasedGlobalCompilationDatabase::tryLoadDatabaseFromPath(PathRef File) {
-
-  namespace path = llvm::sys::path;
-  auto CachedIt = CompilationDatabases.find(File);
-
-  assert((path::is_absolute(File, path::Style::posix) ||
-          path::is_absolute(File, path::Style::windows)) &&
-         "path must be absolute");
-
-  if (CachedIt != CompilationDatabases.end())
-    return CachedIt->second.get();
-  std::string Error = "";
-  auto CDB = tooling::CompilationDatabase::loadFromDirectory(File, Error);
-  if (CDB) {
-    auto Result = CDB.get();
-    CompilationDatabases.insert(std::make_pair(File, std::move(CDB)));
-    return Result;
+CompilationDatabase *
+GlobalCompilationDatabase::getCompilationDatabase(PathRef File) const {
+  std::lock_guard<std::mutex> Lock(Mutex);
+  std::string IgnoredErr;
+  for (auto Dir = path::parent_path(File); !Dir.empty();
+       Dir = path::parent_path(Dir)) {
+    auto Insert = Cache.try_emplace(Dir, nullptr);
+    if (Insert.second) // We actually need to populate the cache.
+      Insert.first->getValue() =
+          CompilationDatabase::loadFromDirectory(Dir, IgnoredErr);
+    if (auto *DirDB = Insert.first->getValue().get())
+      return DirDB;
   }
-
   return nullptr;
 }
 
-tooling::CompilationDatabase *
-DirectoryBasedGlobalCompilationDatabase::getCompilationDatabase(PathRef File) {
+std::vector<tooling::CompileCommand>
+ExtraFlagsCompilationDatabase::getCompileCommands(PathRef File) const {
+  auto Commands = Inner->getCompileCommands(File);
+  std::vector<std::string> Flags;
+  {
     std::lock_guard<std::mutex> Lock(Mutex);
+    auto It = ExtraFlags.find(File);
+    if (It == ExtraFlags.end())
+      return Commands;
+    Flags = It->getValue();
+  }
+  for (auto &Cmd : Commands) {
+    assert(Cmd.CommandLine.size() >= 2 &&
+           "Expected at least the command and the filename");
+    auto It = Cmd.CommandLine.end();
+    Cmd.CommandLine.insert(--It, make_move_iterator(Flags.begin()),
+                           make_move_iterator(Flags.end()));
+  }
+  return Commands;
+}
 
-  namespace path = llvm::sys::path;
-  if (CompileCommandsDir.hasValue()) {
-    tooling::CompilationDatabase *ReturnValue =
-        tryLoadDatabaseFromPath(CompileCommandsDir.getValue());
-    if (ReturnValue == nullptr)
-      Logger.log("Failed to find compilation database for " + Twine(File) +
-                 "in overriden directory " + CompileCommandsDir.getValue() +
-                 "\n");
-    return ReturnValue;
-  }
-
-  for (auto Path = path::parent_path(File); !Path.empty();
-       Path = path::parent_path(Path)) {
-    auto CDB = tryLoadDatabaseFromPath(Path);
-    if (!CDB)
-      continue;
-    // FIXME(ibiryukov): Invalidate cached compilation databases on changes
-    return CDB;
-  }
+void ExtraFlagsCompilationDatabase::setExtraFlagsForFile(
+    PathRef File, std::vector<std::string> Flags) {
+  std::lock_guard<std::mutex> Lock(Mutex);
+  ExtraFlags[File] = Flags;
 
-  Logger.log("Failed to find compilation database for " + Twine(File) + "\n");
-  return nullptr;
+std::vector<tooling::CompileCommand>
+FallbackCompilationDatabase::getCompileCommands(PathRef File) const {
+  auto Result = Primary->getCompileCommands(File);
+  if (Result.empty())
+    Result = Fallback->getCompileCommands(File);
+  return Result;
 }
 
 } // namespace clangd
 } // namespace clang

unittests/clangd/ClangdTests.cpp

public:
   VFSTag lastVFSTag() { return LastVFSTag; }
 
 private:
   std::mutex Mutex;
   bool HadErrorInLastDiags = false;
   VFSTag LastVFSTag = VFSTag();
 };
 
-class MockCompilationDatabase : public GlobalCompilationDatabase {
+class MockCompilationDatabase : public tooling::CompilationDatabase {
 public:
   MockCompilationDatabase(bool AddFreestandingFlag) {
     // We have to add -ffreestanding to VFS-specific tests to avoid errors on
     // implicit includes of stdc-predef.h.
     if (AddFreestandingFlag)
       ExtraClangFlags.push_back("-ffreestanding");
   }
 
   std::vector<tooling::CompileCommand>
-  getCompileCommands(PathRef File) override {
+  getCompileCommands(PathRef File) const override {
     if (ExtraClangFlags.empty())
       return {};
 
     std::vector<std::string> CommandLine;
     CommandLine.reserve(3 + ExtraClangFlags.size());
     CommandLine.insert(CommandLine.end(), {"clang", "-fsyntax-only"});
     CommandLine.insert(CommandLine.end(), ExtraClangFlags.begin(),
                        ExtraClangFlags.end());