[clangd] Add hot-reload of compile_commands.json and compile_flags.txt

[sammccall]

Summary

When querying the CDB, we stat the underlying file to check it hasn't changed.
We don't do this every time, but only if we didn't check within 5 seconds.

This behavior only exists for compile_commands.json and compile_flags.txt.
The CDB plugin system doesn't expose enough information to handle others.

Slight behavior change: we now only look for build/compile_commands.json
rather than trying every CDB strategy under build subdirectories.

[sammccall]

The compilationdatabase changes won't be landed as part of this patch, they've been sent separately for review.
I can try the git gymnastics to get them out of this patch, but if you don't mind ignoring them that's easier :-)

There might be a sensible way to split this further, but I'm not seeing it for some reason - definitely open to suggestions.

[njames93]

clang-tools-extra/clangd/ClangdLSPServer.cpp

712

Any reason for re-specifying compile_commands.json here?

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

256

[kadircet]

this mostly LGTM. there are some changes to the existing behavior; like discovery order and not looking for other plugins under build/, but they seem like non-harmful changes to me.

my biggest question is, have we considered updating CompilationDatabasePlugin interface to provide a loadFromBuffer and relativeFileName virtual methods. I think loading logic could've been a lot simpler and more uniform that way.
i think we discussed this option but i don't remember the outcome, sorry :(.

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

201

nit: inline MayCache

220

what about symlinks ? i think users usually have cc.json -> obscure_build_dir/cc.json as a symlink.

239

also update cached mtime ?

282

nit: can we have a functor(probably llvm::function_ref) rather than a function pointer?

345

it is not crucial but we used to try to load these from Path + "/build" as well

483

why not log instead of vlog? this shouldn't be too noisy anyway.

clang-tools-extra/clangd/GlobalCompilationDatabase.h

133

why not directly store Options ?

clang-tools-extra/clangd/unittests/GlobalCompilationDatabaseTests.cpp

393

nit: maybe a MATCHER_P instead ?

[sammccall]

Thanks for the review!

my biggest question is, have we considered updating CompilationDatabasePlugin interface to provide a loadFromBuffer and relativeFileName virtual methods. I think loading logic could've been a lot simpler and more uniform that way.

I thought about it, not sure if we discussed it. TL;DR: I feel like we can do a good job of the concrete cases, but don't feel ready to generalize.

It's a larger, complicated design problem and the costs of getting it wrong (either making it too limiting *or* too flexible) are much higher than the benefits we'd get here. Apart from JSON/Fixed CDBs, we know there are downstream CDBs and this is a key extension point. What assumptions should we bake in?

• CDB state is derived from a single file with known path?
• CDBs should be thrown away and reloaded, rather than updated, when data changes?
• CDBs are not invalidated by any other events?

I like the idea of extending the interface at some point. But I'm not sure the model from this patch is the right one to codify.

Meanwhile, most clang tools are batch, so the audience for this is... clangd and ccls? I don't think it's a pressing need.

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

220

RealFileSystem::status() is stat() rather than lstat() (it uses the default follow=true for sys::fs::status) so it will follow symlinks and report "regular file".

345

Yeah, this is on purpose, I think it's better only to do this for compile_commands.json. Now that we're periodically stat'ing we should think a little about how many files are necessary.

The build/ heuristic is basically targetting the "standard cmake setup" which is always compile_commands.json.

I think it's unlikely this is a significant regression for anyone. In a standard build it only affects build/compile_flags.txt which AFAIK is always hand-authored, and I'd expect authors to put it in the project root (we never documented the build/ behavior, and other clang tools don't have it).

FWIW, the old implementation looked in build/ for all plugins only because I wanted to stick to the high-level API rather than piercing the abstraction.

---

As an aside, I am slightly worried this hot-reload feature will make us wary of supporting more build systems that require more IO, because both more periodic stats and having some CDBs only scanned once would be unpalatable options.

Still, given how dominant compile_commands.json is today, we can't let the extensibility argument get in the way of an important feature like this.

483

Basically because "broacdasting" per se isn't meaningful to users.
This log line is going to be sandwiched between "Loaded compilation database from {0}" and "Enqueueing {0} commands for indexing", which tell them what they need to know.

I'm not sure it adds much, and maybe we should just remove it entirely? (But as a developer I do find it a nice reminder of what the flow is)

clang-tools-extra/clangd/unittests/GlobalCompilationDatabaseTests.cpp

393

Well, I find functions easier to reason about than the MATCHER_P factories...

Here it's pretty ugly: if we want to delegate to the two-arg version from the body, we have to pass through the result_listener, so something like

MATCHER_P(HasFlag, Flag, "") {
  return HasFlag(Flag, "dummy.cc").MatchAndExplain(arg, result_listener);
}

however the MATCHER_P macros produce *polymorphic* matchers (which is why we don't need to provide the arg type), so we need something more like

return Matcher<decltype(arg)>(HasFlag(Flag, "dummy.cc")).MatchAndExplain(arg, result_listener);

(Not sure if that's quite right)
I don't think that does a great job of communicating "this is the same HasFlag matcher, but the Path arg is dummy.cc".

[sammccall]

Address review comments

[adamcz]

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

220

Here's a fun thought: what if this is a network or otherwise magical filesystem (like a FUSE) and may be temporarily unavailable (due to expiring credentials, network issues, maybe your machine just woke up from suspend and needs to reconnect).

Do you think it makes sense to check for things like ENOACCES, ENOKEY, ESTALE and treat them as transient?

231

Why throw away the information we have? Better keep the cached metadata around, transient error here could mean the build system is re-creating the file, but it may end up being identical to what we already read, it's just that someone re-run cmake or something.

clang-tools-extra/clangd/unittests/GlobalCompilationDatabaseTests.cpp

393

lint warning here. Either rename or silence with a comment

397

Considering you already use min() in code, I would prefer if the test also included a value that's between min() and max() (i.e. something that's not a special-case)

424

Can you add a couple of calls to test the modtime and size comparison? Like, maybe change DBAZ to DFOO and see that it's picked up despite size being the same if mtime changed?

[sammccall]

address comments

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

220

Maybe in some cases, it depends a bit on the details...

The decision we're making here is really "retry on next access" vs "retry in 30 seconds".
The biggest point of the delay is that when loading background index shards (each corresponding to a source file), we check for each of them whether the stored compile commands are still valid (if not, we still load the shard, but schedule a rebuild). If we stat each possible cdb+config+etc each time, it significantly increases time before the index becomes available.

With this in mind, I don't think forcing ENOACCESS and ENOKEY to try every time is a good idea - the FS might have come back but probably not (user action is probably required). ESTALE sounds more plausible, but I don't know the details there. I'd probably avoid the complexity of this until we have concrete use cases where this is causing problems (e.g. ESTALE isn't portably available through std::error_code AFAIK).

In practice if the CDB is on an unavailable network FS then the sources/shards probably are too so we never really get here, though there can be exceptions

231

Good point, done.

Of course this relies on us getting "lucky" and seeing two different sizes for the file between the calls. If it's being written then we could e.g. see it with size 0 on both stat and read, and then we'll throw away the content.

The way around this would be to e.g. call an unparseable file a transient error (reuse cached value), which requires some structural changes here I think. What do you think about that? (I think this would help with compile_commands.json but not compile_flags.txt)
Another way would be to handle a size of 0 as a special case (presumed invalid), which may catch the practical cases.

clang-tools-extra/clangd/unittests/GlobalCompilationDatabaseTests.cpp

424

Done - I'd forgotten we had mtime support in TestFS. Also added some more comments to the tests as the list gets longer.

(While writing the original test, I had overwritten -DFOO with -DBAR and spent a while debugging why it didn't reload - oops!)

[sammccall]

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

231

Decided not to do anything about this for now, the most common generator is cmake and it does atomic writes. Seems likely other build systems will too.

[thakis]

Seems to break tests on windows: http://45.33.8.238/win/30129/step_9.txt

[sammccall]

In D92663#2462642, @thakis wrote:

Seems to break tests on windows: http://45.33.8.238/win/30129/step_9.txt

Sorry (and for the delay getting back to keyboard). 0336ff0a17e6aec831334aeb50e6685f6b184065 should fix.

[thakis]

Thanks for the fix :)

clang-tools-extra/clangd/ClangdLSPServer.cpp

void ClangdLSPServer::onInitialize(const InitializeParams &Params,
   else if (Params.rootPath && !Params.rootPath->empty())
     Opts.WorkspaceRoot = *Params.rootPath;
   if (Server)
     return Reply(llvm::make_error<LSPError>("server already initialized",
                                             ErrorCode::InvalidRequest));
   if (const auto &Dir = Params.initializationOptions.compilationDatabasePath)
     Opts.CompileCommandsDir = Dir;
   if (Opts.UseDirBasedCDB) {
-    BaseCDB = std::make_unique<DirectoryBasedGlobalCompilationDatabase>(
-        Opts.CompileCommandsDir);
+    DirectoryBasedGlobalCompilationDatabase::Options CDBOpts(TFS);
+    CDBOpts.CompileCommandsDir = Opts.CompileCommandsDir;
+    BaseCDB =
+        std::make_unique<DirectoryBasedGlobalCompilationDatabase>(CDBOpts);
     BaseCDB = getQueryDriverDatabase(llvm::makeArrayRef(Opts.QueryDriverGlobs),
                                      std::move(BaseCDB));
   }
   auto Mangler = CommandMangler::detect();
   if (Opts.ResourceDir)
     Mangler.ResourceDir = *Opts.ResourceDir;
   CDB.emplace(BaseCDB.get(), Params.initializationOptions.fallbackFlags,
               tooling::ArgumentsAdjuster(std::move(Mangler)));

 }
 
 void ClangdLSPServer::onFileEvent(const DidChangeWatchedFilesParams &Params) {
   // We could also reparse all open files here. However:
   //  - this could be frequent, and revalidating all the preambles isn't free
   //  - this is useful e.g. when switching git branches, but we're likely to see
   //    fresh headers but still have the old-branch main-file content
   Server->onFileEvent(Params);
+  // FIXME: observe config files, immediately expire time-based caches, reparse:
+  //  - compile_commands.json and compile_flags.txt
+  //  - .clang_format and .clang-tidy
+  //  - .clangd and clangd/config.yaml
 }
 
 void ClangdLSPServer::onCommand(const ExecuteCommandParams &Params,
                                 Callback<llvm::json::Value> Reply) {
   auto ApplyEdit = [this](WorkspaceEdit WE, std::string SuccessMessage,
                           decltype(Reply) Reply) {
     ApplyWorkspaceEditParams Edit;
     Edit.edit = std::move(WE);

clang-tools-extra/clangd/GlobalCompilationDatabase.h

 //===--- GlobalCompilationDatabase.h -----------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_GLOBALCOMPILATIONDATABASE_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_GLOBALCOMPILATIONDATABASE_H
 
 #include "CompileCommands.h"
 #include "support/Function.h"
 #include "support/Path.h"
+#include "support/ThreadsafeFS.h"
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/StringMap.h"
 #include <memory>
 #include <mutex>
 #include <vector>
 
protected:
   mutable CommandChanged OnCommandChanged;
 };
 
 /// Gets compile args from tooling::CompilationDatabases built for parent
 /// directories.
 class DirectoryBasedGlobalCompilationDatabase
     : public GlobalCompilationDatabase {
 public:
-  DirectoryBasedGlobalCompilationDatabase(
-      llvm::Optional<Path> CompileCommandsDir);
+  struct Options {
+    Options(const ThreadsafeFS &TFS) : TFS(TFS) {}
+
+    const ThreadsafeFS &TFS;
+    // Frequency to check whether e.g. compile_commands.json has changed.
+    std::chrono::steady_clock::duration RevalidateAfter =
+        std::chrono::seconds(5);
+    // Frequency to check whether e.g. compile_commands.json has been created.
+    // (This is more expensive to check frequently, as we check many locations).
+    std::chrono::steady_clock::duration RevalidateMissingAfter =
+        std::chrono::seconds(30);
+    // Only look for a compilation database in this one fixed directory.
+    llvm::Optional<Path> CompileCommandsDir;
+  };
+
+  DirectoryBasedGlobalCompilationDatabase(const Options &Opts);
   ~DirectoryBasedGlobalCompilationDatabase() override;
 
   /// Scans File's parents looking for compilation databases.
   /// Any extra flags will be added.
   /// Might trigger OnCommandChanged, if CDB wasn't broadcasted yet.
   llvm::Optional<tooling::CompileCommand>
   getCompileCommand(PathRef File) const override;
 
   /// Returns the path to first directory containing a compilation database in
   /// \p File's parents.
   llvm::Optional<ProjectInfo> getProjectInfo(PathRef File) const override;
 
 private:
+  Options Opts;
+
   class DirectoryCache;
   // If there's an explicit CompileCommandsDir, cache of the CDB found there.
   mutable std::unique_ptr<DirectoryCache> OnlyDirCache;
 
   // Keyed by possibly-case-folded directory path.
   // We can hand out pointers as they're stable and entries are never removed.
   // Empty if CompileCommandsDir is given (OnlyDirCache is used instead).
   mutable llvm::StringMap<DirectoryCache> DirCaches;
   // DirCaches access must be locked (unlike OnlyDirCache, which is threadsafe).
   mutable std::mutex DirCachesMutex;
 
   std::vector<DirectoryCache *>
   getDirectoryCaches(llvm::ArrayRef<llvm::StringRef> Dirs) const;
 
   struct CDBLookupRequest {
     PathRef FileName;
     // Whether this lookup should trigger discovery of the CDB found.
     bool ShouldBroadcast = false;
+    // Cached results newer than this are considered fresh and not checked
+    // against disk.
+    std::chrono::steady_clock::time_point FreshTime;
+    std::chrono::steady_clock::time_point FreshTimeMissing;
   };
   struct CDBLookupResult {
     std::shared_ptr<const tooling::CompilationDatabase> CDB;
     ProjectInfo PI;
   };
   llvm::Optional<CDBLookupResult> lookupCDB(CDBLookupRequest Request) const;
 
   // Performs broadcast on governed files.
   void broadcastCDB(CDBLookupResult Res) const;
+
+  // cache test calls lookupCDB directly to ensure valid/invalid times.
+  friend class DirectoryBasedGlobalCompilationDatabaseCacheTest;
 };
 
 /// Extracts system include search path from drivers matching QueryDriverGlobs
 /// and adds them to the compile flags. Base may not be nullptr.
 /// Returns Base when \p QueryDriverGlobs is empty.
 std::unique_ptr<GlobalCompilationDatabase>
 getQueryDriverDatabase(llvm::ArrayRef<std::string> QueryDriverGlobs,
                        std::unique_ptr<GlobalCompilationDatabase> Base);

clang-tools-extra/clangd/GlobalCompilationDatabase.cpp

 //===--- GlobalCompilationDatabase.cpp ---------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "GlobalCompilationDatabase.h"
 #include "FS.h"
+#include "SourceCode.h"
 #include "support/Logger.h"
 #include "support/Path.h"
+#include "support/ThreadsafeFS.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/CompilationDatabasePluginRegistry.h"
+#include "clang/Tooling/JSONCompilationDatabase.h"
 #include "llvm/ADT/None.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/ScopeExit.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/FileUtilities.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Program.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include <chrono>
 #include <string>
 #include <tuple>
 #include <vector>
 
 namespace clang {
 namespace clangd {
 namespace {

 // directory we're searching for a CDB.
 // Loading is deferred until first access.
 //
 // The DirectoryBasedCDB keeps a map from path => DirectoryCache.
 // Typical usage is to:
 //  - 1) determine all the paths that might be searched
 //  - 2) acquire the map lock and get-or-create all the DirectoryCache entries
 //  - 3) release the map lock and query the caches as desired
-//
-// FIXME: this should revalidate the cache sometimes
-// FIXME: IO should go through a VFS
 class DirectoryBasedGlobalCompilationDatabase::DirectoryCache {
-  // Absolute canonical path that we're the cache for. (Not case-folded).
-  const std::string Path;
+  using stopwatch = std::chrono::steady_clock;
 
-  // True if we've looked for a CDB here and found none.
-  // (This makes it possible for get() to return without taking a lock)
-  // FIXME: this should have an expiry time instead of lasting forever.
-  std::atomic<bool> FinalizedNoCDB = {false};
+  // CachedFile is used to read a CDB file on disk (e.g. compile_commands.json).
+  // It specializes in being able to quickly bail out if the file is unchanged,
+  // which is the common case.
+  // Internally, it stores file metadata so a stat() can verify it's unchanged.
+  // We don't actually cache the content as it's not needed - if the file is
+  // unchanged then the previous CDB is valid.
+  struct CachedFile {
+    CachedFile(llvm::StringRef Parent, llvm::StringRef Rel) {
+      llvm::SmallString<256> Path = Parent;
+      llvm::sys::path::append(Path, Rel);
+      this->Path = Path.str().str();
+    }
+    std::string Path;
+    size_t Size = NoFileCached;
+    llvm::sys::TimePoint<> ModifiedTime;
+    FileDigest ContentHash;
+
+    static constexpr size_t NoFileCached = -1;
+
+    struct LoadResult {
+      enum {
+        FileNotFound,
+        TransientError,
+        FoundSameData,
+        FoundNewData,
+      } Result;
+      std::unique_ptr<llvm::MemoryBuffer> Buffer; // Set only if FoundNewData
+    };
 
-  // Guards following cache state.
+    LoadResult load(llvm::vfs::FileSystem &FS, bool HasOldData);
+  };
+
+  // If we've looked for a CDB here and found none, the time when that happened.
+  // (Atomics make it possible for get() to return without taking a lock)
+  std::atomic<stopwatch::rep> NoCDBAt = {
+      stopwatch::time_point::min().time_since_epoch().count()};
+
+  // Guards the following cache state.
   std::mutex Mu;
-  // Has cache been filled from disk? FIXME: this should be an expiry time.
-  bool CachePopulated = false;
+  // When was the cache last known to be in sync with disk state?
+  stopwatch::time_point CachePopulatedAt = stopwatch::time_point::min();
   // Whether a new CDB has been loaded but not broadcast yet.
   bool NeedsBroadcast = false;
-  // Last loaded CDB, meaningful if CachePopulated is set.
+  // Last loaded CDB, meaningful if CachePopulatedAt was ever set.
   // shared_ptr so we can overwrite this when callers are still using the CDB.
   std::shared_ptr<tooling::CompilationDatabase> CDB;
+  // File metadata for the CDB files we support tracking directly.
+  CachedFile CompileCommandsJson;
+  CachedFile BuildCompileCommandsJson;
+  CachedFile CompileFlagsTxt;
+  // CachedFile member corresponding to CDB.
+  //   CDB  | ACF  | Scenario
+  //   null | null | no CDB found, or initial empty cache
+  //   set  | null | CDB was loaded via generic plugin interface
+  //   null | set  | found known CDB file, but parsing it failed
+  //   set  | set  | CDB was parsed from a known file
+  CachedFile *ActiveCachedFile = nullptr;
 
 public:
-  DirectoryCache(llvm::StringRef Path) : Path(Path) {
+  DirectoryCache(llvm::StringRef Path)
+      : CompileCommandsJson(Path, "compile_commands.json"),
+        BuildCompileCommandsJson(Path, "build/compile_commands.json"),
+        CompileFlagsTxt(Path, "compile_flags.txt"), Path(Path) {
     assert(llvm::sys::path::is_absolute(Path));
   }
 
+  // Absolute canonical path that we're the cache for. (Not case-folded).
+  const std::string Path;
+
   // Get the CDB associated with this directory.
   // ShouldBroadcast:
   //  - as input, signals whether the caller is willing to broadcast a
   //    newly-discovered CDB. (e.g. to trigger background indexing)
   //  - as output, signals whether the caller should do so.
   // (If a new CDB is discovered and ShouldBroadcast is false, we mark the
   // CDB as needing broadcast, and broadcast it next time we can).
   std::shared_ptr<const tooling::CompilationDatabase>
-  get(bool &ShouldBroadcast) {
+  get(const ThreadsafeFS &TFS, bool &ShouldBroadcast,
+      stopwatch::time_point FreshTime, stopwatch::time_point FreshTimeMissing) {
     // Fast path for common case without taking lock.
-    if (FinalizedNoCDB.load()) {
+    if (stopwatch::time_point(stopwatch::duration(NoCDBAt.load())) >
+        FreshTimeMissing) {
       ShouldBroadcast = false;
       return nullptr;
     }
+
     std::lock_guard<std::mutex> Lock(Mu);
     auto RequestBroadcast = llvm::make_scope_exit([&, OldCDB(CDB.get())] {
       // If we loaded a new CDB, it should be broadcast at some point.
       if (CDB != nullptr && CDB.get() != OldCDB)
         NeedsBroadcast = true;
       else if (CDB == nullptr) // nothing to broadcast anymore!
         NeedsBroadcast = false;
       // If we have something to broadcast, then do so iff allowed.
       if (!ShouldBroadcast)
         return;
       ShouldBroadcast = NeedsBroadcast;
       NeedsBroadcast = false;
     });
 
-    // For now, we never actually attempt to revalidate a populated cache.
-    if (CachePopulated)
+    // If our cache is valid, serve from it.
+    if (CachePopulatedAt > FreshTime)
       return CDB;
-    assert(CDB == nullptr);
 
-    load();
-    CachePopulated = true;
+    if (/*MayCache=*/load(*TFS.view(/*CWD=*/llvm::None))) {
+      // Use new timestamp, as loading may be slow.
+      CachePopulatedAt = stopwatch::now();
+      NoCDBAt.store((CDB ? stopwatch::time_point::min() : CachePopulatedAt)
+                        .time_since_epoch()
+                        .count());
+    }
 
-    if (!CDB)
-      FinalizedNoCDB.store(true);
     return CDB;
   }
 
-  llvm::StringRef path() const { return Path; }
-
 private:
-  // Updates `CDB` from disk state.
-  void load() {
-    std::string Error; // ignored, because it's often "didn't find anything".
-    CDB = tooling::CompilationDatabase::loadFromDirectory(Path, Error);
-    if (!CDB) {
-      // Fallback: check for $src/build, the conventional CMake build root.
-      // Probe existence first to avoid each plugin doing IO if it doesn't
-      // exist.
-      llvm::SmallString<256> BuildDir(Path);
-      llvm::sys::path::append(BuildDir, "build");
-      if (llvm::sys::fs::is_directory(BuildDir)) {
-        vlog("Found candidate build directory {0}", BuildDir);
-        CDB = tooling::CompilationDatabase::loadFromDirectory(BuildDir, Error);
-      }
+  // Updates `CDB` from disk state. Returns false on failure.
+  bool load(llvm::vfs::FileSystem &FS);
+};
+
+DirectoryBasedGlobalCompilationDatabase::DirectoryCache::CachedFile::LoadResult
+DirectoryBasedGlobalCompilationDatabase::DirectoryCache::CachedFile::load(
+    llvm::vfs::FileSystem &FS, bool HasOldData) {
+  auto Stat = FS.status(Path);
+  if (!Stat || !Stat->isRegularFile()) {
+    Size = NoFileCached;
+    ContentHash = {};
+    return {LoadResult::FileNotFound, nullptr};
+  }
+  // If both the size and mtime match, presume unchanged without reading.
+  if (HasOldData && Stat->getLastModificationTime() == ModifiedTime &&
+      Stat->getSize() == Size)
+    return {LoadResult::FoundSameData, nullptr};
+  auto Buf = FS.getBufferForFile(Path);
+  if (!Buf || (*Buf)->getBufferSize() != Stat->getSize()) {
+    // Don't clear the cache - possible we're seeing inconsistent size as the
+    // file is being recreated. If it ends up identical later, great!
+    //
+    // This isn't a complete solution: if we see a partial file but stat/read
+    // agree on its size, we're ultimately going to have spurious CDB reloads.
+    // May be worth fixing if generators don't write atomically (CMake does).
+    elog("Failed to read {0}: {1}", Path,
+         Buf ? "size changed" : Buf.getError().message());
+    return {LoadResult::TransientError, nullptr};
+  }
+
+  FileDigest NewContentHash = digest((*Buf)->getBuffer());
+  if (HasOldData && NewContentHash == ContentHash) {
+    // mtime changed but data is the same: avoid rebuilding the CDB.
+    ModifiedTime = Stat->getLastModificationTime();
+    return {LoadResult::FoundSameData, nullptr};
+  }
+
+  Size = (*Buf)->getBufferSize();
+  ModifiedTime = Stat->getLastModificationTime();
+  ContentHash = NewContentHash;
+  return {LoadResult::FoundNewData, std::move(*Buf)};
+}
+
+// Adapt CDB-loading functions to a common interface for DirectoryCache::load().
+static std::unique_ptr<tooling::CompilationDatabase>
+parseJSON(PathRef Path, llvm::StringRef Data, std::string &Error) {
+  if (auto CDB = tooling::JSONCompilationDatabase::loadFromBuffer(
+          Data, Error, tooling::JSONCommandLineSyntax::AutoDetect)) {
+    // FS used for expanding response files.
+    // FIXME: ExpandResponseFilesDatabase appears not to provide the usual
+    // thread-safety guarantees, as the access to FS is not locked!
+    // For now, use the real FS, which is known to be threadsafe (if we don't
+    // use/change working directory, which ExpandResponseFilesDatabase doesn't).
+    auto FS = llvm::vfs::getRealFileSystem();
+    return tooling::inferTargetAndDriverMode(
+        tooling::inferMissingCompileCommands(
+            expandResponseFiles(std::move(CDB), std::move(FS))));
+  }
+  return nullptr;
+static std::unique_ptr<tooling::CompilationDatabase>
+parseFixed(PathRef Path, llvm::StringRef Data, std::string &Error) {
+  return tooling::FixedCompilationDatabase::loadFromBuffer(Path, Data, Error);
+}
+
+bool DirectoryBasedGlobalCompilationDatabase::DirectoryCache::load(
+    llvm::vfs::FileSystem &FS) {
+  dlog("Probing directory {0}", Path);
+  std::string Error;
+
+  // Load from the specially-supported compilation databases (JSON + Fixed).
+  // For these, we know the files they read and cache their metadata so we can
+  // cheaply validate whether they've changed, and hot-reload if they have.
+  // (As a bonus, these are also VFS-clean)!
+  struct CDBFile {
+    CachedFile *File;
+    // Wrapper for {Fixed,JSON}CompilationDatabase::loadFromBuffer.
+    llvm::function_ref<std::unique_ptr<tooling::CompilationDatabase>(
+        PathRef,
+        /*Data*/ llvm::StringRef,
+        /*ErrorMsg*/ std::string &)>
+        Parser;
+  };
+  for (const auto &Entry : {CDBFile{&CompileCommandsJson, parseJSON},
+                            CDBFile{&BuildCompileCommandsJson, parseJSON},
+                            CDBFile{&CompileFlagsTxt, parseFixed}}) {
+    bool Active = ActiveCachedFile == Entry.File;
+    auto Loaded = Entry.File->load(FS, Active);
+    switch (Loaded.Result) {
+    case CachedFile::LoadResult::FileNotFound:
+      if (Active) {
+        log("Unloaded compilation database from {0}", Entry.File->Path);
+        ActiveCachedFile = nullptr;
+        CDB = nullptr;
       }
-    if (CDB) {
-      log("Loaded compilation database from {0}", Path);
-    } else {
-      vlog("No compilation database at {0}", Path);
+      // Continue looking at other candidates.
+      break;
+    case CachedFile::LoadResult::TransientError:
+      // File existed but we couldn't read it. Reuse the cache, retry later.
+      return false; // Load again next time.
+    case CachedFile::LoadResult::FoundSameData:
+      assert(Active && "CachedFile may not return 'same data' if !HasOldData");
+      // This is the critical file, and it hasn't changed.
+      return true;
+    case CachedFile::LoadResult::FoundNewData:
+      // We have a new CDB!
+      CDB = Entry.Parser(Entry.File->Path, Loaded.Buffer->getBuffer(), Error);
+      if (CDB)
+        log("{0} compilation database from {1}", Active ? "Reloaded" : "Loaded",
+            Entry.File->Path);
+      else
+        elog("Failed to load compilation database from {0}: {1}",
+             Entry.File->Path, Error);
+      ActiveCachedFile = Entry.File;
+      return true;
     }
   }
-};
+
+  // Fall back to generic handling of compilation databases.
+  // We don't know what files they read, so can't efficiently check whether
+  // they need to be reloaded. So we never do that.
+  // FIXME: the interface doesn't provide a way to virtualize FS access.
+
+  // Don't try these more than once. If we've scanned before, we're done.
+  if (CachePopulatedAt > stopwatch::time_point::min())
+    return true;
+  for (const auto &Entry :
+       tooling::CompilationDatabasePluginRegistry::entries()) {
+    // Avoid duplicating the special cases handled above.
+    if (Entry.getName() == "fixed-compilation-database" ||
+        Entry.getName() == "json-compilation-database")
+      continue;
+    auto Plugin = Entry.instantiate();
+    if (auto CDB = Plugin->loadFromDirectory(Path, Error)) {
+      log("Loaded compilation database from {0} with plugin {1}", Path,
+          Entry.getName());
+      this->CDB = std::move(CDB);
+      return true;
+    }
+    // Don't log Error here, it's usually just "couldn't find <file>".
+  }
+  vlog("No compilation database at {0}", Path);
+  return true;
+}
 
 DirectoryBasedGlobalCompilationDatabase::
-    DirectoryBasedGlobalCompilationDatabase(
-        llvm::Optional<Path> CompileCommandsDir) {
-  if (CompileCommandsDir)
-    OnlyDirCache = std::make_unique<DirectoryCache>(*CompileCommandsDir);
+    DirectoryBasedGlobalCompilationDatabase(const Options &Opts)
+    : Opts(Opts) {
+  if (Opts.CompileCommandsDir)
+    OnlyDirCache = std::make_unique<DirectoryCache>(*Opts.CompileCommandsDir);
 }
 
 DirectoryBasedGlobalCompilationDatabase::
     ~DirectoryBasedGlobalCompilationDatabase() = default;
 
 llvm::Optional<tooling::CompileCommand>
 DirectoryBasedGlobalCompilationDatabase::getCompileCommand(PathRef File) const {
   CDBLookupRequest Req;
   Req.FileName = File;
   Req.ShouldBroadcast = true;
+  auto Now = std::chrono::steady_clock::now();
+  Req.FreshTime = Now - Opts.RevalidateAfter;
+  Req.FreshTimeMissing = Now - Opts.RevalidateMissingAfter;
 
   auto Res = lookupCDB(Req);
   if (!Res) {
     log("Failed to find compilation database for {0}", File);
     return llvm::None;
   }
 
   auto Candidates = Res->CDB->getCompileCommands(File);
assert(llvm::sys::path::is_absolute(Request.FileName) &&
          "path must be absolute");
 
   bool ShouldBroadcast = false;
   DirectoryCache *DirCache = nullptr;
   std::shared_ptr<const tooling::CompilationDatabase> CDB = nullptr;
   if (OnlyDirCache) {
     DirCache = OnlyDirCache.get();
     ShouldBroadcast = Request.ShouldBroadcast;
-    CDB = DirCache->get(ShouldBroadcast);
+    CDB = DirCache->get(Opts.TFS, ShouldBroadcast, Request.FreshTime,
+                        Request.FreshTimeMissing);
   } else {
     // Traverse the canonical version to prevent false positives. i.e.:
     // src/build/../a.cc can detect a CDB in /src/build if not canonicalized.
     std::string CanonicalPath = removeDots(Request.FileName);
     std::vector<llvm::StringRef> SearchDirs;
     actOnAllParentDirectories(CanonicalPath, [&](PathRef Path) {
       SearchDirs.push_back(Path);
       return false;
     });
     for (DirectoryCache *Candidate : getDirectoryCaches(SearchDirs)) {
       bool CandidateShouldBroadcast = Request.ShouldBroadcast;
-      if ((CDB = Candidate->get(CandidateShouldBroadcast))) {
+      if ((CDB = Candidate->get(Opts.TFS, CandidateShouldBroadcast,
+                                Request.FreshTime, Request.FreshTimeMissing))) {
         DirCache = Candidate;
         ShouldBroadcast = CandidateShouldBroadcast;
         break;
       }
     }
   }
 
   if (!CDB)
     return llvm::None;
 
   CDBLookupResult Result;
   Result.CDB = std::move(CDB);
-  Result.PI.SourceRoot = DirCache->path().str();
+  Result.PI.SourceRoot = DirCache->Path;
 
   // FIXME: Maybe make the following part async, since this can block
   // retrieval of compile commands.
   if (ShouldBroadcast)
     broadcastCDB(Result);
   return Result;
 }
 
 void DirectoryBasedGlobalCompilationDatabase::broadcastCDB(
     CDBLookupResult Result) const {
+  vlog("Broadcasting compilation database from {0}", Result.PI.SourceRoot);
   assert(Result.CDB && "Trying to broadcast an invalid CDB!");
 
   std::vector<std::string> AllFiles = Result.CDB->getAllFiles();
   // We assume CDB in CompileCommandsDir owns all of its entries, since we don't
   // perform any search in parent paths whenever it is set.
   if (OnlyDirCache) {
-    assert(OnlyDirCache->path() == Result.PI.SourceRoot &&
+    assert(OnlyDirCache->Path == Result.PI.SourceRoot &&
            "Trying to broadcast a CDB outside of CompileCommandsDir!");
     OnCommandChanged.broadcast(std::move(AllFiles));
     return;
   }
 
   // Uniquify all parent directories of all files.
   llvm::StringMap<bool> DirectoryHasCDB;
   std::vector<llvm::StringRef> FileAncestors;
   for (llvm::StringRef File : AllFiles) {
     actOnAllParentDirectories(File, [&](PathRef Path) {
       auto It = DirectoryHasCDB.try_emplace(Path);
       // Already seen this path, and all of its parents.
       if (!It.second)
         return true;
 
       FileAncestors.push_back(It.first->getKey());
       return pathEqual(Path, Result.PI.SourceRoot);
     });
   }
   // Work out which ones have CDBs in them.
+  // Given that we know that CDBs have been moved/generated, don't trust caches.
+  // (This should be rare, so it's OK to add a little latency).
+  constexpr auto IgnoreCache = std::chrono::steady_clock::time_point::max();
   for (DirectoryCache *Dir : getDirectoryCaches(FileAncestors)) {
     bool ShouldBroadcast = false;
-    if (Dir->get(ShouldBroadcast))
-      DirectoryHasCDB.find(Dir->path())->setValue(true);
+    if (Dir->get(Opts.TFS, ShouldBroadcast, /*FreshTime=*/IgnoreCache,
+                 /*FreshTimeMissing=*/IgnoreCache))
+      DirectoryHasCDB.find(Dir->Path)->setValue(true);
   }
 
   std::vector<std::string> GovernedFiles;
   for (llvm::StringRef File : AllFiles) {
     // A file is governed by this CDB if lookup for the file would find it.
     // Independent of whether it has an entry for that file or not.
     actOnAllParentDirectories(File, [&](PathRef Path) {
       if (DirectoryHasCDB.lookup(Path)) {
void DirectoryBasedGlobalCompilationDatabase::broadcastCDB(
   OnCommandChanged.broadcast(std::move(GovernedFiles));
 }
 
 llvm::Optional<ProjectInfo>
 DirectoryBasedGlobalCompilationDatabase::getProjectInfo(PathRef File) const {
   CDBLookupRequest Req;
   Req.FileName = File;
   Req.ShouldBroadcast = false;
+  Req.FreshTime = Req.FreshTimeMissing =
+      std::chrono::steady_clock::time_point::min();
   auto Res = lookupCDB(Req);
   if (!Res)
     return llvm::None;
   return Res->PI;
 }
 
 OverlayCDB::OverlayCDB(const GlobalCompilationDatabase *Base,
                        std::vector<std::string> FallbackFlags,

clang-tools-extra/clangd/tool/Check.cpp

 public:
   // Number of non-fatal errors seen.
   unsigned ErrCount = 0;
 
   Checker(llvm::StringRef File, const ClangdLSPServer::Options &Opts)
       : File(File), Opts(Opts) {}
 
   // Read compilation database and choose a compile command for the file.
-  bool buildCommand() {
+  bool buildCommand(const ThreadsafeFS &TFS) {
     log("Loading compilation database...");
+    DirectoryBasedGlobalCompilationDatabase::Options CDBOpts(TFS);
+    CDBOpts.CompileCommandsDir = Opts.CompileCommandsDir;
     std::unique_ptr<GlobalCompilationDatabase> BaseCDB =
-        std::make_unique<DirectoryBasedGlobalCompilationDatabase>(
-            Opts.CompileCommandsDir);
+        std::make_unique<DirectoryBasedGlobalCompilationDatabase>(CDBOpts);
     BaseCDB = getQueryDriverDatabase(llvm::makeArrayRef(Opts.QueryDriverGlobs),
                                      std::move(BaseCDB));
     auto Mangler = CommandMangler::detect();
     if (Opts.ResourceDir)
       Mangler.ResourceDir = *Opts.ResourceDir;
     auto CDB = std::make_unique<OverlayCDB>(
         BaseCDB.get(), std::vector<std::string>{},
         tooling::ArgumentsAdjuster(std::move(Mangler)));
Contents = R"cpp(
 
       size_t N = 50;
       auto xxx = std::string(N, 'x');
     )cpp";
   }
   log("Testing on source file {0}", File);
 
   Checker C(File, Opts);
-  if (!C.buildCommand() || !C.buildInvocation(TFS, Contents) || !C.buildAST())
+  if (!C.buildCommand(TFS) || !C.buildInvocation(TFS, Contents) ||
+      !C.buildAST())
     return false;
   C.testLocationFeatures();
 
   log("All checks completed, {0} errors", C.ErrCount);
   return C.ErrCount == 0;
 }
 
 } // namespace clangd
 } // namespace clang

clang-tools-extra/clangd/unittests/GlobalCompilationDatabaseTests.cpp

 //===-- GlobalCompilationDatabaseTests.cpp ----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "GlobalCompilationDatabase.h"
 
 #include "Matchers.h"
 #include "TestFS.h"
 #include "support/Path.h"
+#include "support/ThreadsafeFS.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/Optional.h"
+#include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
+#include <chrono>
 #include <fstream>
 #include <string>
 
 namespace clang {
 namespace clangd {
 namespace {
 using ::testing::AllOf;
 using ::testing::Contains;
 using ::testing::ElementsAre;
 using ::testing::EndsWith;
 using ::testing::HasSubstr;
 using ::testing::IsEmpty;
 using ::testing::Not;
 using ::testing::StartsWith;
 using ::testing::UnorderedElementsAre;
 
 TEST(GlobalCompilationDatabaseTest, FallbackCommand) {
-  DirectoryBasedGlobalCompilationDatabase DB(None);
+  MockFS TFS;
+  DirectoryBasedGlobalCompilationDatabase DB(TFS);
   auto Cmd = DB.getFallbackCommand(testPath("foo/bar.cc"));
   EXPECT_EQ(Cmd.Directory, testPath("foo"));
   EXPECT_THAT(Cmd.CommandLine, ElementsAre("clang", testPath("foo/bar.cc")));
   EXPECT_EQ(Cmd.Output, "");
 
   // .h files have unknown language, so they are parsed liberally as obj-c++.
   Cmd = DB.getFallbackCommand(testPath("foo/bar.h"));
   EXPECT_THAT(Cmd.CommandLine, ElementsAre("clang", "-xobjective-c++-header",
TEST_F(OverlayCDBTest, Adjustments) {
 
   // Fallback gets adjusted.
   Cmd = CDB.getFallbackCommand("baz.cc");
   EXPECT_THAT(Cmd.CommandLine, ElementsAre("clang", "-DA=2", "baz.cc",
                                            "-DFallback", "-DAdjust_baz.cc"));
 }
 
 // Allows placement of files for tests and cleans them up after.
+// FIXME: GlobalCompilationDatabase is mostly VFS-clean now, switch to MockFS?
 class ScratchFS {
   llvm::SmallString<128> Root;
 
 public:
   ScratchFS() {
     EXPECT_FALSE(llvm::sys::fs::createUniqueDirectory("clangd-cdb-test", Root))
         << "Failed to create unique directory";
   }
const char *const CDBInner =
         {
           "file": "gen.cc",
           "command": "",
           "directory": "{0}/build",
         }
       ]
       )cdb";
   ScratchFS FS;
+  RealThreadsafeFS TFS;
   FS.write("compile_commands.json", CDBOuter);
   FS.write("build/compile_commands.json", CDBInner);
 
   // Note that gen2.cc goes missing with our following model, not sure this
   // happens in practice though.
   {
-    DirectoryBasedGlobalCompilationDatabase DB(llvm::None);
+    DirectoryBasedGlobalCompilationDatabase DB(TFS);
     std::vector<std::string> DiscoveredFiles;
     auto Sub =
         DB.watch([&DiscoveredFiles](const std::vector<std::string> Changes) {
           DiscoveredFiles = Changes;
         });
 
     DB.getCompileCommand(FS.path("build/../a.cc"));
     EXPECT_THAT(DiscoveredFiles, UnorderedElementsAre(AllOf(
                                      EndsWith("a.cc"), Not(HasSubstr("..")))));
     DiscoveredFiles.clear();
 
     DB.getCompileCommand(FS.path("build/gen.cc"));
     EXPECT_THAT(DiscoveredFiles, UnorderedElementsAre(EndsWith("gen.cc")));
   }
 
   // With a custom compile commands dir.
   {
-    DirectoryBasedGlobalCompilationDatabase DB(FS.root().str());
+    DirectoryBasedGlobalCompilationDatabase::Options Opts(TFS);
+    Opts.CompileCommandsDir = FS.root().str();
+    DirectoryBasedGlobalCompilationDatabase DB(Opts);
     std::vector<std::string> DiscoveredFiles;
     auto Sub =
         DB.watch([&DiscoveredFiles](const std::vector<std::string> Changes) {
           DiscoveredFiles = Changes;
         });
 
     DB.getCompileCommand(FS.path("a.cc"));
     EXPECT_THAT(DiscoveredFiles,
                 UnorderedElementsAre(EndsWith("a.cc"), EndsWith("gen.cc"),
                                      EndsWith("gen2.cc")));
     DiscoveredFiles.clear();
 
     DB.getCompileCommand(FS.path("build/gen.cc"));
     EXPECT_THAT(DiscoveredFiles, IsEmpty());
   }
 }
 
 TEST(GlobalCompilationDatabaseTest, BuildDir) {
   ScratchFS FS;
+  RealThreadsafeFS TFS;
   auto Command = [&](llvm::StringRef Relative) {
-    return DirectoryBasedGlobalCompilationDatabase(llvm::None)
+    DirectoryBasedGlobalCompilationDatabase::Options Opts(TFS);
+    return DirectoryBasedGlobalCompilationDatabase(Opts)
         .getCompileCommand(FS.path(Relative))
         .getValueOr(tooling::CompileCommand())
         .CommandLine;
   };
   EXPECT_THAT(Command("x/foo.cc"), IsEmpty());
-  FS.write("x/build/compile_flags.txt", "-DXYZZY");
+  const char *const CDB =
+      R"cdb(
+      [
+        {
+          "file": "{0}/x/foo.cc",
+          "command": "clang -DXYZZY {0}/x/foo.cc",
+          "directory": "{0}",
+        },
+        {
+          "file": "{0}/bar.cc",
+          "command": "clang -DXYZZY {0}/bar.cc",
+          "directory": "{0}",
+        }
+      ]
+      )cdb";
+  FS.write("x/build/compile_commands.json", CDB);
   EXPECT_THAT(Command("x/foo.cc"), Contains("-DXYZZY"));
   EXPECT_THAT(Command("bar.cc"), IsEmpty())
-      << "x/build/compile_flags.txt only applicable to x/";
+      << "x/build/compile_flags.json only applicable to x/";
 }
 
 TEST(GlobalCompilationDatabaseTest, NonCanonicalFilenames) {
   OverlayCDB DB(nullptr);
   std::vector<std::string> DiscoveredFiles;
   auto Sub =
       DB.watch([&DiscoveredFiles](const std::vector<std::string> Changes) {
         DiscoveredFiles = Changes;
TEST_F(OverlayCDBTest, GetProjectInfo) {
   EXPECT_EQ(DB.getProjectInfo(Header)->SourceRoot, testRoot());
 
   // Shouldn't change after an override.
   DB.setCompileCommand(File, tooling::CompileCommand());
   EXPECT_EQ(DB.getProjectInfo(File)->SourceRoot, testRoot());
   EXPECT_EQ(DB.getProjectInfo(Header)->SourceRoot, testRoot());
 }
 } // namespace
+
+// Friend test has access to internals.
+class DirectoryBasedGlobalCompilationDatabaseCacheTest
+    : public ::testing::Test {
+protected:
+  std::shared_ptr<const tooling::CompilationDatabase>
+  lookupCDB(const DirectoryBasedGlobalCompilationDatabase &GDB,
+            llvm::StringRef Path,
+            std::chrono::steady_clock::time_point FreshTime) {
+    DirectoryBasedGlobalCompilationDatabase::CDBLookupRequest Req;
+    Req.FileName = Path;
+    Req.FreshTime = Req.FreshTimeMissing = FreshTime;
+    if (auto Result = GDB.lookupCDB(Req))
+      return std::move(Result->CDB);
+    return nullptr;
+  }
+};
+
+// Matches non-null CDBs which include the specified flag.
+MATCHER_P2(hasFlag, Flag, Path, "") {
+  if (arg == nullptr)
+    return false;
+  auto Cmds = arg->getCompileCommands(Path);
+  if (Cmds.empty()) {
+    *result_listener << "yields no commands";
+    return false;
+  }
+  if (!llvm::is_contained(Cmds.front().CommandLine, Flag)) {
+    *result_listener << "flags are: "
+                     << llvm::join(Cmds.front().CommandLine, " ");
+    return false;
+  }
+  return true;
+}
+
+auto hasFlag(llvm::StringRef Flag) { return hasFlag(Flag, "dummy.cc"); }
+
+TEST_F(DirectoryBasedGlobalCompilationDatabaseCacheTest, Cacheable) {
+  MockFS FS;
+  auto Stale = std::chrono::steady_clock::now() - std::chrono::minutes(1);
+  auto Fresh = std::chrono::steady_clock::now() + std::chrono::hours(24);
+
+  DirectoryBasedGlobalCompilationDatabase GDB(FS);
+  FS.Files["compile_flags.txt"] = "-DROOT";
+  auto Root = lookupCDB(GDB, testPath("foo/test.cc"), Stale);
+  EXPECT_THAT(Root, hasFlag("-DROOT"));
+
+  // Add a compilation database to a subdirectory - CDB loaded.
+  FS.Files["foo/compile_flags.txt"] = "-DFOO";
+  EXPECT_EQ(Root, lookupCDB(GDB, testPath("foo/test.cc"), Stale))
+      << "cache still valid";
+  auto Foo = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_THAT(Foo, hasFlag("-DFOO")) << "new cdb loaded";
+  EXPECT_EQ(Foo, lookupCDB(GDB, testPath("foo/test.cc"), Stale))
+      << "new cdb in cache";
+
+  // Mtime changed, but no content change - CDB not reloaded.
+  ++FS.Timestamps["foo/compile_flags.txt"];
+  auto FooAgain = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_EQ(Foo, FooAgain) << "Same content, read but not reloaded";
+  // Content changed, but not size or mtime - CDB not reloaded.
+  FS.Files["foo/compile_flags.txt"] = "-DBAR";
+  auto FooAgain2 = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_EQ(Foo, FooAgain2) << "Same filesize, change not detected";
+  // Mtime change forces a re-read, and we notice the different content.
+  ++FS.Timestamps["foo/compile_flags.txt"];
+  auto Bar = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_THAT(Bar, hasFlag("-DBAR")) << "refreshed with mtime change";
+
+  // Size and content both change - CDB reloaded.
+  FS.Files["foo/compile_flags.txt"] = "-DFOOBAR";
+  EXPECT_EQ(Bar, lookupCDB(GDB, testPath("foo/test.cc"), Stale))
+      << "cache still valid";
+  auto FooBar = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_THAT(FooBar, hasFlag("-DFOOBAR")) << "cdb reloaded";
+
+  // compile_commands.json takes precedence over compile_flags.txt.
+  FS.Files["foo/compile_commands.json"] = llvm::formatv(R"json([{
+    "file": "{0}/foo/dummy.cc",
+    "command": "clang -DBAZ dummy.cc",
+    "directory": "{0}/foo",
+  }])json",
+                                                        testRoot());
+  EXPECT_EQ(FooBar, lookupCDB(GDB, testPath("foo/test.cc"), Stale))
+      << "cache still valid";
+  auto Baz = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_THAT(Baz, hasFlag("-DBAZ", testPath("foo/dummy.cc")))
+      << "compile_commands overrides compile_flags";
+
+  // Removing compile_commands.json reveals compile_flags.txt again.
+  // However this *does* cause a CDB reload (we cache only one CDB per dir).
+  FS.Files.erase("foo/compile_commands.json");
+  auto FoobarAgain = lookupCDB(GDB, testPath("foo/test.cc"), Fresh);
+  EXPECT_THAT(FoobarAgain, hasFlag("-DFOOBAR")) << "reloaded compile_flags";
+  EXPECT_NE(FoobarAgain, FooBar) << "CDB discarded (shadowed within directory)";
+
+  // Removing the directory's CDB leaves the parent CDB active.
+  // The parent CDB is *not* reloaded (we cache the CDB per-directory).
+  FS.Files.erase("foo/compile_flags.txt");
+  EXPECT_EQ(Root, lookupCDB(GDB, testPath("foo/test.cc"), Fresh))
+      << "CDB retained (shadowed by another directory)";
+}
+
 } // namespace clangd
 } // namespace clang