[clangd] Auto-index watches global CDB for changes.

[sammccall]

Summary

Instead of receiving compilation commands, auto-index is triggered by just
filenames to reindex, and gets commands from the global comp DB internally.
This has advantages:

• more of the work can be done asynchronously (fetching compilation commands upfront can be slow for large CDBs)
• we get access to the CDB which can be used to retrieve interpolated commands for headers (useful in some cases where the original TU goes away)
• fits nicely with the filename-only change observation from r347297

The interface to GlobalCompilationDatabase gets extended: when retrieving a
compile command, the GCDB can optionally report the project the file belongs to.
This naturally fits together with getCompileCommand: it's hard to implement one
without the other. But because most callers don't care, I've ended up with an
awkward optional-out-param-in-virtual method pattern - maybe there's a better
one.

This is the main missing integration point between ClangdServer and
BackgroundIndex, after this we should be able to add an auto-index flag.

[kadircet]

LGTM, seems to share a lot of context with what I am preparing for loading shards.
Interfaces looks compatible with what I had before, only difference is it was more tooling::CompileCommand oriented before, now it can only look at filenames, which seems more natural.

Thanks!

clangd/index/Background.cpp

115

s/so //

130

In case of an OverlayCDB with some commands set through LSP configuration, we might end up Project.SourceRoot being an empty string, which will result in creating the index storage directory in cwd. I believe we wouldn't want that, but not sure if it should be handled by BackgroundIndex or DiskBackedIndexStorage. WDYT?

[sammccall]

clangd/index/Background.cpp

130

You're right. To a first approximation, I think that the right behavior is to keep indexing in memory and just not persist the shards.
The storage seems like the right layer because:

• the factory makes it easy to encapsulate this special case
• we may want to change the strategy in future (e.g. write index files into a central location if the builddir is unavailable).

The downside I see is by encapsulating it we can't do things like "downgrade priority of indexing if storage isn't available", but we can find ways around that if we really want to.

I've added a null storage to this patch which is used if the CDB dir is empty.

[sammccall]

If the CDB dir is unknown, don't try to write shards to disk.

clangd/GlobalCompilationDatabase.h

 class CompilationDatabase;
 struct CompileCommand;
 } // namespace tooling
 
 namespace clangd {
 
 class Logger;
 
+struct ProjectInfo {
+  // The directory in which the compilation database was discovered.
+  // Empty if directory-based compilation database discovery was not used.
+  std::string SourceRoot;
+};
+
 /// Provides compilation arguments used for parsing C and C++ files.
 class GlobalCompilationDatabase {
 public:
   virtual ~GlobalCompilationDatabase() = default;
 
   /// If there are any known-good commands for building this file, returns one.
+  /// If the ProjectInfo pointer is set, it will also be populated.
   virtual llvm::Optional<tooling::CompileCommand>
-  getCompileCommand(PathRef File) const = 0;
+  getCompileCommand(PathRef File, ProjectInfo * = nullptr) const = 0;
 
   /// Makes a guess at how to build a file.
   /// The default implementation just runs clang on the file.
   /// Clangd should treat the results as unreliable.
   virtual tooling::CompileCommand getFallbackCommand(PathRef File) const;
 
   using CommandChanged = Event<std::vector<std::string>>;
   /// The callback is notified when files may have new compile commands.

 public:
   DirectoryBasedGlobalCompilationDatabase(
       llvm::Optional<Path> CompileCommandsDir);
   ~DirectoryBasedGlobalCompilationDatabase() override;
 
   /// Scans File's parents looking for compilation databases.
   /// Any extra flags will be added.
   llvm::Optional<tooling::CompileCommand>
-  getCompileCommand(PathRef File) const override;
+  getCompileCommand(PathRef File, ProjectInfo * = nullptr) const override;
 
 private:
-  tooling::CompilationDatabase *getCDBForFile(PathRef File) const;
+  tooling::CompilationDatabase *getCDBForFile(PathRef File,
+                                              ProjectInfo *) const;
   std::pair<tooling::CompilationDatabase *, /*Cached*/ bool>
   getCDBInDirLocked(PathRef File) const;
 
   mutable std::mutex Mutex;
   /// Caches compilation databases loaded from directories(keys are
   /// directories).
   mutable llvm::StringMap<std::unique_ptr<clang::tooling::CompilationDatabase>>
       CompilationDatabases;
 
   /// Used for command argument pointing to folder where compile_commands.json
   /// is located.
   llvm::Optional<Path> CompileCommandsDir;
 };
 
 /// Wraps another compilation database, and supports overriding the commands
 /// using an in-memory mapping.
 class OverlayCDB : public GlobalCompilationDatabase {
 public:
   // Base may be null, in which case no entries are inherited.
   // FallbackFlags are added to the fallback compile command.
   OverlayCDB(const GlobalCompilationDatabase *Base,
              std::vector<std::string> FallbackFlags = {});
 
   llvm::Optional<tooling::CompileCommand>
-  getCompileCommand(PathRef File) const override;
+  getCompileCommand(PathRef File, ProjectInfo * = nullptr) const override;
   tooling::CompileCommand getFallbackCommand(PathRef File) const override;
 
   /// Sets or clears the compilation command for a particular file.
   void
   setCompileCommand(PathRef File,
                     llvm::Optional<tooling::CompileCommand> CompilationCommand);
 
 private:

clangd/GlobalCompilationDatabase.cpp

 DirectoryBasedGlobalCompilationDatabase::
     DirectoryBasedGlobalCompilationDatabase(Optional<Path> CompileCommandsDir)
     : CompileCommandsDir(std::move(CompileCommandsDir)) {}
 
 DirectoryBasedGlobalCompilationDatabase::
     ~DirectoryBasedGlobalCompilationDatabase() = default;
 
 Optional<tooling::CompileCommand>
-DirectoryBasedGlobalCompilationDatabase::getCompileCommand(PathRef File) const {
-  if (auto CDB = getCDBForFile(File)) {
+DirectoryBasedGlobalCompilationDatabase::getCompileCommand(
+    PathRef File, ProjectInfo *Project) const {
+  if (auto CDB = getCDBForFile(File, Project)) {
     auto Candidates = CDB->getCompileCommands(File);
-    if (!Candidates.empty())
+    if (!Candidates.empty()) {
       return std::move(Candidates.front());
+    }
   } else {
     log("Failed to find compilation database for {0}", File);
   }
   return None;
 }
 
 std::pair<tooling::CompilationDatabase *, /*Cached*/ bool>
 DirectoryBasedGlobalCompilationDatabase::getCDBInDirLocked(PathRef Dir) const {
   // FIXME(ibiryukov): Invalidate cached compilation databases on changes
   auto CachedIt = CompilationDatabases.find(Dir);
   if (CachedIt != CompilationDatabases.end())
     return {CachedIt->second.get(), true};
   std::string Error = "";
   auto CDB = tooling::CompilationDatabase::loadFromDirectory(Dir, Error);
   auto Result = CDB.get();
   CompilationDatabases.insert(std::make_pair(Dir, std::move(CDB)));
   return {Result, false};
 }
 
 tooling::CompilationDatabase *
-DirectoryBasedGlobalCompilationDatabase::getCDBForFile(PathRef File) const {
+DirectoryBasedGlobalCompilationDatabase::getCDBForFile(
+    PathRef File, ProjectInfo *Project) const {
   namespace path = sys::path;
   assert((path::is_absolute(File, path::Style::posix) ||
           path::is_absolute(File, path::Style::windows)) &&
          "path must be absolute");
 
   tooling::CompilationDatabase *CDB = nullptr;
   bool Cached = false;
   std::lock_guard<std::mutex> Lock(Mutex);
   if (CompileCommandsDir) {
     std::tie(CDB, Cached) = getCDBInDirLocked(*CompileCommandsDir);
+    if (Project && CDB)
+      Project->SourceRoot = *CompileCommandsDir;
   } else {
     for (auto Path = path::parent_path(File); !CDB && !Path.empty();
          Path = path::parent_path(Path)) {
       std::tie(CDB, Cached) = getCDBInDirLocked(Path);
+      if (Project && CDB)
+        Project->SourceRoot = Path;
     }
   }
   if (CDB && !Cached)
     OnCommandChanged.broadcast(CDB->getAllFiles());
   return CDB;
 }
 
 OverlayCDB::OverlayCDB(const GlobalCompilationDatabase *Base,
                        std::vector<std::string> FallbackFlags)
     : Base(Base), FallbackFlags(std::move(FallbackFlags)) {
   if (Base)
     BaseChanged = Base->watch([this](const std::vector<std::string> Changes) {
       OnCommandChanged.broadcast(Changes);
     });
 }
 
 Optional<tooling::CompileCommand>
-OverlayCDB::getCompileCommand(PathRef File) const {
+OverlayCDB::getCompileCommand(PathRef File, ProjectInfo *Project) const {
   {
     std::lock_guard<std::mutex> Lock(Mutex);
     auto It = Commands.find(File);
-    if (It != Commands.end())
+    if (It != Commands.end()) {
+      if (Project)
+        Project->SourceRoot = "";
       return It->second;
     }
+  }
   return Base ? Base->getCompileCommand(File) : None;
 }
 
 tooling::CompileCommand OverlayCDB::getFallbackCommand(PathRef File) const {
   auto Cmd = Base ? Base->getFallbackCommand(File)
                   : GlobalCompilationDatabase::getFallbackCommand(File);
   std::lock_guard<std::mutex> Lock(Mutex);
   Cmd.CommandLine.insert(Cmd.CommandLine.end(), FallbackFlags.begin(),

clangd/index/Background.h

 //===--- Background.h - Build an index in a background thread ----*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_BACKGROUND_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_BACKGROUND_H
 
 #include "Context.h"
 #include "FSProvider.h"
+#include "GlobalCompilationDatabase.h"
 #include "index/FileIndex.h"
 #include "index/Index.h"
 #include "index/Serialization.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/Support/SHA1.h"
 #include "llvm/Support/Threading.h"
 #include <condition_variable>

 // all commands in a compilation database. Indexing happens in the background.
 // FIXME: it should also persist its state on disk for fast start.
 // FIXME: it should watch for changes to files on disk.
 class BackgroundIndex : public SwapIndex {
 public:
   // FIXME: resource-dir injection should be hoisted somewhere common.
   BackgroundIndex(Context BackgroundContext, llvm::StringRef ResourceDir,
                   const FileSystemProvider &,
+                  const GlobalCompilationDatabase &CDB,
                   BackgroundIndexStorage::Factory IndexStorageFactory,
                   size_t ThreadPoolSize = llvm::hardware_concurrency());
   ~BackgroundIndex(); // Blocks while the current task finishes.
 
-  // Enqueue a translation unit for indexing.
+  // Enqueue translation units for indexing.
   // The indexing happens in a background thread, so the symbols will be
   // available sometime later.
-  void enqueue(llvm::StringRef Directory, tooling::CompileCommand);
-  // Index all TUs described in the compilation database.
-  void enqueueAll(llvm::StringRef Directory,
-                  const tooling::CompilationDatabase &);
+  void enqueue(const std::vector<std::string> &ChangedFiles);
+  void enqueue(const std::string &File);
 
   // Cause background threads to stop after ther current task, any remaining
   // tasks will be discarded.
   void stop();
 
   // Wait until the queue is empty, to allow deterministic testing.
   void blockUntilIdleForTest();
 
   using FileDigest = decltype(llvm::SHA1::hash({}));
 
 private:
   /// Given index results from a TU, only update files in \p FilesToUpdate.
   void update(llvm::StringRef MainFile, SymbolSlab Symbols, RefSlab Refs,
               const llvm::StringMap<FileDigest> &FilesToUpdate,
               BackgroundIndexStorage *IndexStorage);
 
   // configuration
   std::string ResourceDir;
   const FileSystemProvider &FSProvider;
+  const GlobalCompilationDatabase &CDB;
   Context BackgroundContext;
 
   // index state
   llvm::Error index(tooling::CompileCommand,
                     BackgroundIndexStorage *IndexStorage);
 
   FileSymbols IndexedSymbols;
   llvm::StringMap<FileDigest> IndexedFileDigests; // Key is absolute file path.
   std::mutex DigestsMu;
 
   BackgroundIndexStorage::Factory IndexStorageFactory;
 
   // queue management
   using Task = std::function<void()>;
   void run(); // Main loop executed by Thread. Runs tasks from Queue.
+  void enqueueTask(Task T);
   void enqueueLocked(tooling::CompileCommand Cmd,
                      BackgroundIndexStorage *IndexStorage);
   std::mutex QueueMu;
   unsigned NumActiveTasks = 0; // Only idle when queue is empty *and* no tasks.
   std::condition_variable QueueCV;
   bool ShouldStop = false;
   std::deque<Task> Queue;
   std::vector<std::thread> ThreadPool; // FIXME: Abstract this away.
+  GlobalCompilationDatabase::CommandChanged::Subscription CommandsChanged;
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/index/Background.cpp

 #include <string>
 
 using namespace llvm;
 namespace clang {
 namespace clangd {
 
 BackgroundIndex::BackgroundIndex(
     Context BackgroundContext, StringRef ResourceDir,
-    const FileSystemProvider &FSProvider,
+    const FileSystemProvider &FSProvider, const GlobalCompilationDatabase &CDB,
     BackgroundIndexStorage::Factory IndexStorageFactory, size_t ThreadPoolSize)
     : SwapIndex(make_unique<MemIndex>()), ResourceDir(ResourceDir),
-      FSProvider(FSProvider), BackgroundContext(std::move(BackgroundContext)),
-      IndexStorageFactory(std::move(IndexStorageFactory)) {
+      FSProvider(FSProvider), CDB(CDB),
+      BackgroundContext(std::move(BackgroundContext)),
+      IndexStorageFactory(std::move(IndexStorageFactory)),
+      CommandsChanged(
+          CDB.watch([&](const std::vector<std::string> &ChangedFiles) {
+            enqueue(ChangedFiles);
+          })) {
   assert(ThreadPoolSize > 0 && "Thread pool size can't be zero.");
   assert(this->IndexStorageFactory && "Storage factory can not be null!");
   while (ThreadPoolSize--) {
     ThreadPool.emplace_back([this] { run(); });
     // Set priority to low, since background indexing is a long running task we
     // do not want to eat up cpu when there are any other high priority threads.
     // FIXME: In the future we might want a more general way of handling this to
     // support tasks with various priorities.
void BackgroundIndex::run() {
   }
 }
 
 void BackgroundIndex::blockUntilIdleForTest() {
   std::unique_lock<std::mutex> Lock(QueueMu);
   QueueCV.wait(Lock, [&] { return Queue.empty() && NumActiveTasks == 0; });
 }
 
-void BackgroundIndex::enqueue(StringRef Directory,
-                              tooling::CompileCommand Cmd) {
-  BackgroundIndexStorage *IndexStorage = IndexStorageFactory(Directory);
-  {
-    std::lock_guard<std::mutex> Lock(QueueMu);
-    enqueueLocked(std::move(Cmd), IndexStorage);
+void BackgroundIndex::enqueue(const std::vector<std::string> &ChangedFiles) {
+  enqueueTask([this, ChangedFiles] {
+    trace::Span Tracer("BackgroundIndexEnqueue");
+    // We're doing this asynchronously, because we'll read shards here too.
+    // FIXME: read shards here too.
+
+    log("Enqueueing {0} commands for indexing", ChangedFiles.size());
+    SPAN_ATTACH(Tracer, "files", int64_t(ChangedFiles.size()));
+
+    // We shuffle the files because processing them in a random order should
+    // quickly give us good coverage of headers in the project.
+    std::vector<unsigned> Permutation(ChangedFiles.size());
+    std::iota(Permutation.begin(), Permutation.end(), 0);
+    std::mt19937 Generator(std::random_device{}());
+    std::shuffle(Permutation.begin(), Permutation.end(), Generator);
+
+    for (const unsigned I : Permutation)
+      enqueue(ChangedFiles[I]);
+  });
+}
+
+void BackgroundIndex::enqueue(const std::string &File) {
+  ProjectInfo Project;
+  if (auto Cmd = CDB.getCompileCommand(File, &Project)) {
+    auto *Storage = IndexStorageFactory(Project.SourceRoot);
+    enqueueTask(Bind(
+        [this, File, Storage](tooling::CompileCommand Cmd) {
+          Cmd.CommandLine.push_back("-resource-dir=" + ResourceDir);
+          if (auto Error = index(std::move(Cmd), Storage))
+            log("Indexing {0} failed: {1}", File, std::move(Error));
+        },
+        std::move(*Cmd)));
   }
-  QueueCV.notify_all();
 }
 
-void BackgroundIndex::enqueueAll(StringRef Directory,
-                                 const tooling::CompilationDatabase &CDB) {
-  trace::Span Tracer("BackgroundIndexEnqueueCDB");
-  // FIXME: this function may be slow. Perhaps enqueue a task to re-read the CDB
-  // from disk and enqueue the commands asynchronously?
-  auto Cmds = CDB.getAllCompileCommands();
-  BackgroundIndexStorage *IndexStorage = IndexStorageFactory(Directory);
-  SPAN_ATTACH(Tracer, "commands", int64_t(Cmds.size()));
-  std::mt19937 Generator(std::random_device{}());
-  std::shuffle(Cmds.begin(), Cmds.end(), Generator);
-  log("Enqueueing {0} commands for indexing from {1}", Cmds.size(), Directory);
+void BackgroundIndex::enqueueTask(Task T) {
   {
     std::lock_guard<std::mutex> Lock(QueueMu);
-    for (auto &Cmd : Cmds)
-      enqueueLocked(std::move(Cmd), IndexStorage);
+    Queue.push_back(std::move(T));
   }
   QueueCV.notify_all();
 }
 
-void BackgroundIndex::enqueueLocked(tooling::CompileCommand Cmd,
-                                    BackgroundIndexStorage *IndexStorage) {
-  Queue.push_back(Bind(
-      [this, IndexStorage](tooling::CompileCommand Cmd) {
-        std::string Filename = Cmd.Filename;
-        Cmd.CommandLine.push_back("-resource-dir=" + ResourceDir);
-        if (auto Error = index(std::move(Cmd), IndexStorage))
-          log("Indexing {0} failed: {1}", Filename, std::move(Error));
-      },
-      std::move(Cmd)));
-}
-
 static BackgroundIndex::FileDigest digest(StringRef Content) {
   return SHA1::hash({(const uint8_t *)Content.data(), Content.size()});
 }
 
 static Optional<BackgroundIndex::FileDigest> digestFile(const SourceManager &SM,
                                                         FileID FID) {
   bool Invalid = false;
   StringRef Content = SM.getBufferData(FID, &Invalid);

clangd/index/BackgroundIndexStorage.cpp

llvm::Error storeShard(llvm::StringRef ShardIdentifier,
     if (EC)
       return llvm::errorCodeToError(EC);
     OS << Shard;
     OS.close();
     return llvm::errorCodeToError(OS.error());
   }
 };
 
+// Doesn't persist index shards anywhere (used when the CDB dir is unknown).
+// We could consider indexing into ~/.clangd/ or so instead.
+class NullStorage : public BackgroundIndexStorage {
+public:
+  std::unique_ptr<IndexFileIn>
+  loadShard(llvm::StringRef ShardIdentifier) const override {
+    return nullptr;
+  }
+
+  llvm::Error storeShard(llvm::StringRef ShardIdentifier,
+                         IndexFileOut Shard) const override {
+    vlog("Couldn't find project for {0}, indexing in-memory only",
+         ShardIdentifier);
+    return llvm::Error::success();
+  }
+};
+
 // Creates and owns IndexStorages for multiple CDBs.
 class DiskBackedIndexStorageManager {
 public:
   // Creates or fetches to storage from cache for the specified CDB.
   BackgroundIndexStorage *operator()(llvm::StringRef CDBDirectory) {
     std::lock_guard<std::mutex> Lock(*IndexStorageMapMu);
     auto &IndexStorage = IndexStorageMap[CDBDirectory];
     if (!IndexStorage)
-      IndexStorage = llvm::make_unique<DiskBackedIndexStorage>(CDBDirectory);
+      IndexStorage = create(CDBDirectory);
     return IndexStorage.get();
   }
 
   // Creates or fetches to storage from cache for the specified CDB.
   BackgroundIndexStorage *createStorage(llvm::StringRef CDBDirectory);
 
 private:
+  std::unique_ptr<BackgroundIndexStorage> create(llvm::StringRef CDBDirectory) {
+    if (CDBDirectory.empty())
+      return llvm::make_unique<NullStorage>();
+    return llvm::make_unique<DiskBackedIndexStorage>(CDBDirectory);
+  }
+
   llvm::StringMap<std::unique_ptr<BackgroundIndexStorage>> IndexStorageMap;
   std::unique_ptr<std::mutex> IndexStorageMapMu;
 };
 
 } // namespace
 
 BackgroundIndexStorage::Factory
 BackgroundIndexStorage::createDiskBackedStorageFactory() {
   return DiskBackedIndexStorageManager();
 }
 
 } // namespace clangd
 } // namespace clang

unittests/clangd/BackgroundIndexTests.cpp

FS.Files[testPath("root/B.cc")] =
       #define A 0
       #include "A.h"
       void f_b() {
         (void)common;
       })cpp";
   llvm::StringMap<std::string> Storage;
   size_t CacheHits = 0;
   MemoryShardStorage MSS(Storage, CacheHits);
-  BackgroundIndex Idx(Context::empty(), "", FS,
+  OverlayCDB CDB(/*Base=*/nullptr);
+  BackgroundIndex Idx(Context::empty(), "", FS, CDB,
                       [&](llvm::StringRef) { return &MSS; });
 
   tooling::CompileCommand Cmd;
   Cmd.Filename = testPath("root/A.cc");
   Cmd.Directory = testPath("root");
   Cmd.CommandLine = {"clang++", "-DA=1", testPath("root/A.cc")};
-  Idx.enqueue(testPath("root"), Cmd);
+  CDB.setCompileCommand(testPath("root"), Cmd);
 
   Idx.blockUntilIdleForTest();
   EXPECT_THAT(
       runFuzzyFind(Idx, ""),
       UnorderedElementsAre(Named("common"), Named("A_CC"),
                            AllOf(Named("f_b"), Declared(), Not(Defined()))));
 
   Cmd.Filename = testPath("root/B.cc");
   Cmd.CommandLine = {"clang++", Cmd.Filename};
-  Idx.enqueue(testPath("root"), Cmd);
+  CDB.setCompileCommand(testPath("root"), Cmd);
 
   Idx.blockUntilIdleForTest();
   // B_CC is dropped as we don't collect symbols from A.h in this compilation.
   EXPECT_THAT(runFuzzyFind(Idx, ""),
               UnorderedElementsAre(Named("common"), Named("A_CC"),
                                    AllOf(Named("f_b"), Declared(), Defined())));
 
   auto Syms = runFuzzyFind(Idx, "common");
TEST(BackgroundIndexTest, ShardStorageWriteTest) {
   MemoryShardStorage MSS(Storage, CacheHits);
 
   tooling::CompileCommand Cmd;
   Cmd.Filename = testPath("root/A.cc");
   Cmd.Directory = testPath("root");
   Cmd.CommandLine = {"clang++", testPath("root/A.cc")};
   // Check nothing is loaded from Storage, but A.cc and A.h has been stored.
   {
-    BackgroundIndex Idx(Context::empty(), "", FS,
+    OverlayCDB CDB(/*Base=*/nullptr);
+    BackgroundIndex Idx(Context::empty(), "", FS, CDB,
                         [&](llvm::StringRef) { return &MSS; });
-    Idx.enqueue(testPath("root"), Cmd);
+    CDB.setCompileCommand(testPath("root"), Cmd);
     Idx.blockUntilIdleForTest();
   }
   EXPECT_EQ(CacheHits, 0U);
   EXPECT_EQ(Storage.size(), 2U);
 
   auto ShardHeader = MSS.loadShard(testPath("root/A.h"));
   EXPECT_NE(ShardHeader, nullptr);
   EXPECT_THAT(

unittests/clangd/GlobalCompilationDatabaseTests.cpp

 static tooling::CompileCommand cmd(StringRef File, StringRef Arg) {
   return tooling::CompileCommand(testRoot(), File, {"clang", Arg, File}, "");
 }
 
 class OverlayCDBTest : public ::testing::Test {
   class BaseCDB : public GlobalCompilationDatabase {
   public:
     Optional<tooling::CompileCommand>
-    getCompileCommand(StringRef File) const override {
-      if (File == testPath("foo.cc"))
+    getCompileCommand(StringRef File, ProjectInfo *Project) const override {
+      if (File == testPath("foo.cc")) {
+        if (Project)
+          Project->SourceRoot = testRoot();
         return cmd(File, "-DA=1");
+      }
       return None;
     }
 
     tooling::CompileCommand getFallbackCommand(StringRef File) const override {
       return cmd(File, "-DA=2");
     }
   };
 

unittests/clangd/TestFS.h

public:
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap<std::string> Files;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
 class MockCompilationDatabase : public GlobalCompilationDatabase {
 public:
   /// If \p Directory is not empty, use that as the Directory field of the
-  /// CompileCommand.
+  /// CompileCommand, and as project SourceRoot.
   ///
   /// If \p RelPathPrefix is not empty, use that as a prefix in front of the
   /// source file name, instead of using an absolute path.
   MockCompilationDatabase(StringRef Directory = StringRef(),
                           StringRef RelPathPrefix = StringRef());
 
   llvm::Optional<tooling::CompileCommand>
-  getCompileCommand(PathRef File) const override;
+  getCompileCommand(PathRef File, ProjectInfo * = nullptr) const override;
 
   std::vector<std::string> ExtraClangFlags;
 
 private:
   StringRef Directory;
   StringRef RelPathPrefix;
 };
 

unittests/clangd/TestFS.cpp

 MockCompilationDatabase::MockCompilationDatabase(StringRef Directory,
                                                  StringRef RelPathPrefix)
     : ExtraClangFlags({"-ffreestanding"}), Directory(Directory),
       RelPathPrefix(RelPathPrefix) {
   // -ffreestanding avoids implicit stdc-predef.h.
 }
 
 Optional<tooling::CompileCommand>
-MockCompilationDatabase::getCompileCommand(PathRef File) const {
+MockCompilationDatabase::getCompileCommand(PathRef File,
+                                           ProjectInfo *Project) const {
   if (ExtraClangFlags.empty())
     return None;
 
   auto FileName = sys::path::filename(File);
 
   // Build the compile command.
   auto CommandLine = ExtraClangFlags;
   CommandLine.insert(CommandLine.begin(), "clang");
   if (RelPathPrefix.empty()) {
     // Use the absolute path in the compile command.
     CommandLine.push_back(File);
   } else {
     // Build a relative path using RelPathPrefix.
     SmallString<32> RelativeFilePath(RelPathPrefix);
     sys::path::append(RelativeFilePath, FileName);
     CommandLine.push_back(RelativeFilePath.str());
   }
 
+  if (Project)
+    Project->SourceRoot = Directory;
   return {tooling::CompileCommand(
       Directory != StringRef() ? Directory : sys::path::parent_path(File),
       FileName, std::move(CommandLine), "")};
 }
 
 const char *testRoot() {
 #ifdef _WIN32
   return "C:\\clangd-test";