[clangd] Cache preambles of closed files

[qchateau]

Summary

When a file is closed, push its preamble to a LRU cache
When a file is opened, try to get the preamble from the LRU cache.

By default store 10 preambles if they are stored on
memory and 1000 if they are stored on disk. That value
can be modified with --keep-preambles=N

[qchateau]

• Fix keep preamble command line option
• Fix tests

[qchateau]

New year's ping ?

[sammccall]

Thanks! This is a really good idea, but not without its risks :-) Sorry for not getting to this for a while!

My only *really* high-level design question is: I wonder what the tradeoffs are in keeping just the preamble vs the whole ASTWorker. I'd *expect* this approach gets almost all the benefit without complicating the basic assumptions around ASTWorker, so it's probably the right way to go. Haven't thought about it much though...

---

I've got a bit of cognitive load because there's a few preamble-related ideas we've had, and I'm trying to work out how they interact. Let's start by writing them down...

1. a *persistent* cache so closing+reopening clangd loses less state. (This is complicated because only the PCH is easily serializable, the rest of the PreambleData struct isn't)
2. building caches of preambles while background-indexing (this would be great for modules but is probably way too big for whole preambles)
3. reusing the "wrong" preamble initially when you open a new file, to give some basic functionality (using existing preamble patching logic, just in a more aggressive scenario)
4. having the disk-based storage unlink the file preemptively, to eliminate any chance of leaking the *.pch

1&2 are both big projects, so I'm not really worried about the overlap or at least not going to try to predict how it'd be resolved.

3 is actually made *easier* by this patch I think: having a central repository of preambles makes it more natural to grab some "other" preamble.

4 I think doesn't really interact at all: the PreambleData object would own a file descriptor instead of a file, but it's all much the same.

So I think in summary there's not really any conflict to resolve with these ideas. cc @kadircet though who's done more thinking about #1 and #3 than I have.

---

I think we need to be fairly careful about policies around this cache. Preambles are large (we frequently see several hundred MB). Some workflows involve opening many files at a time. Some workflows end up running multiple copies of clangd on the same files. Some configurations keep them in memory rather than on disk. So a too-large cache could waste quite a lot of resources.

So, some pointy questions:

• landing so close to the 12 release, should we conservatively default this to 0, and require opt-in?
• is MB or #preambles a better limit to the cache size?
• should we take size into account when deciding what to evict? (my guess is no, cost scales with size, and value scales with size * probability of reuse, so we should purely optimize for probability of reuse)
• can we do better than LRU? The cache is accessed so infrequently and misses are so horrendously expensive that we could certainly affort to e.g. track usage history of every file ever opened, if it would help performance and not add too much complexity.
• not a question, but I can say for sure that 1000 with no size limit isn't a safe default for disk :-(

clang-tools-extra/clangd/TUScheduler.cpp

193

we might instead consider keeping "active" preambles in the cache, and simply considering their cost to be 0/ineligible for eviction if the shared_ptr::usage_count > 1.
This allows this cache to be a "registry" so we can try using a preamble from a different TU as mentioned above.

(but this could also be done later, or could be done with a separate table of weak_ptrs. No change needed for this patch, just thinking out loud)

[qchateau]

Hey, don't worry about the delay, I won't have as much time on my hands anymore anyway.

• we can definitely make this opt-in
• MB instead of # is an idea, it's probably closer to what the user want to configure - if he does - but it would also probably give a worse default value. Your call !
• I mean why not, but this diff was more about bringing the feature in than making it perfect. The heuristic to decide how to evict preambles can be made very complicated if we want to optimize it
• we can do as you suggest, it related to the previous point
• Ah ! What would you recommend ? No forget it, I'll just make it opt-it for now

I've also tried to keep the ASTWorkers alive in a different branch. The result is very different: ASTWorkers always use RAM so we can't keep as many, but keeping them alive makes is even better (15s with no cache, 2s with preamble cache, virtually 0s with ASTWorker cache). I'd say both features are nice but they give different results for a different cost.
Also it would be great to stop the ASTWorker threads when they are just in cache, but the class needs a rework to be able to stop/restart it

[kadircet]

In D93873#2490314, @sammccall wrote:

1. a *persistent* cache so closing+reopening clangd loses less state. (This is complicated because only the PCH is easily serializable, the rest of the PreambleData struct isn't)
2. building caches of preambles while background-indexing (this would be great for modules but is probably way too big for whole preambles)
3. reusing the "wrong" preamble initially when you open a new file, to give some basic functionality (using existing preamble patching logic, just in a more aggressive scenario)
4. having the disk-based storage unlink the file preemptively, to eliminate any chance of leaking the *.pch

It feels like a mixture of 1 and 3 is going to provide the most value for decreasing time until semantic features (but I might be a little biased :D, also we might hit a nice sweet spot with pseudoparsing too).
I don't think having a cache for previously built preambles will ever be enough. As Sam pointed out, scaling is one of the biggest problems, as I don't think it would be feasible to have tens of preambles lying around on the disk, especially when they are costly the built (as it implies increased size).
Surely it optimizes the case of users working on a small set of files but frequently closes and re-opens them. But that's just one use case, it is also quite common to open tens of library headers while investigating an issue, or trying to understand details of some code through chains of go-to-definitions.
Users won't have any preambles for a while on those files and even after building the preamble they'll just be sitting in the cache probably only to be evicted.

So I think having a cache of preambles while optimizing for reusability (by keeping a small set of preambles that cover different set of files, as we can't use a preamble for a source file if it covers the source file in question) and then patching those to be applicable for current file at hand sounds like a better compromise. Surely it won't be as effective for frequent close/re-open use case, but I think the costs of such a cache isn't justified if it is only applicable to a single workflow.

As for mixing idea-1 into the equation, all of these will require clangd to do the work from scratch per instance, if we can have some sort of persistent on-disk cache, we can both share the work (and associated storage costs) across clangd instances and ensure clangd is also responsive even at startup without requiring user to build a bunch of preambles with every new clangd instance first.

clang-tools-extra/clangd/ClangdServer.h

 //===--- ClangdServer.h - Main clangd server code ----------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
struct Options {
     /// If true, force -frecovery-ast flag.
     /// If false, respect the value in clang.
     bool BuildRecoveryAST = false;
 
     /// If true, force -frecovery-ast-type flag.
     /// If false, respect the value in clang.
     bool PreserveRecoveryASTType = false;
 
+    // The number of preambles that will be retained even after the file is
+    // closed
+    size_t KeepPreambles = 0;
+
     /// Clangd's workspace root. Relevant for "workspace" operations not bound
     /// to a particular file.
     /// FIXME: If not set, should use the current working directory.
     llvm::Optional<std::string> WorkspaceRoot;
 
     /// The resource directory is used to find internal headers, overriding
     /// defaults and -resource-dir compiler flag).
     /// If None, ClangdServer calls CompilerInvocation::GetResourcePath() to

clang-tools-extra/clangd/ClangdServer.cpp

 //===--- ClangdServer.cpp - Main clangd server code --------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===-------------------------------------------------------------------===//
 

 
 ClangdServer::Options ClangdServer::optsForTest() {
   ClangdServer::Options Opts;
   Opts.UpdateDebounce = DebouncePolicy::fixed(/*zero*/ {});
   Opts.StorePreamblesInMemory = true;
   Opts.AsyncThreadsCount = 4; // Consistent!
   Opts.TheiaSemanticHighlighting = true;
   Opts.AsyncPreambleBuilds = true;
+  Opts.KeepPreambles = 0;
   return Opts;
 }
 
 ClangdServer::Options::operator TUScheduler::Options() const {
   TUScheduler::Options Opts;
   Opts.AsyncThreadsCount = AsyncThreadsCount;
   Opts.RetentionPolicy = RetentionPolicy;
   Opts.StorePreamblesInMemory = StorePreamblesInMemory;
   Opts.UpdateDebounce = UpdateDebounce;
   Opts.AsyncPreambleBuilds = AsyncPreambleBuilds;
+  Opts.KeepPreambles = KeepPreambles;
   return Opts;
 }
 
 ClangdServer::ClangdServer(const GlobalCompilationDatabase &CDB,
                            const ThreadsafeFS &TFS, const Options &Opts,
                            Callbacks *Callbacks)
     : ConfigProvider(Opts.ConfigProvider), TFS(TFS), ServerCallbacks(Callbacks),
       DynamicIdx(Opts.BuildDynamicSymbolIndex

clang-tools-extra/clangd/TUScheduler.h

 //===--- TUScheduler.h -------------------------------------------*-C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
struct Options {
 
     /// Determines when to keep idle ASTs in memory for future use.
     ASTRetentionPolicy RetentionPolicy;
 
     /// Whether to run PreamblePeer asynchronously.
     /// No-op if AsyncThreadsCount is 0.
     bool AsyncPreambleBuilds = true;
 
+    // The number of preambles that will be retained even after the file is
+    // closed
+    size_t KeepPreambles = 0;
+
     /// Used to create a context that wraps each single operation.
     /// Typically to inject per-file configuration.
     /// If the path is empty, context sholud be "generic".
     std::function<Context(PathRef)> ContextProvider;
   };
 
   TUScheduler(const GlobalCompilationDatabase &CDB, const Options &Opts,
               std::unique_ptr<ParsingCallbacks> ASTCallbacks = nullptr);
private:
   /// This class stores per-file data in the Files map.
   struct FileData;
 
 public:
   /// Responsible for retaining and rebuilding idle ASTs. An implementation is
   /// an LRU cache.
   class ASTCache;
 
+  /// Responsible for retaining preambles.
+  class PreambleCache;
+
   // The file being built/processed in the current thread. This is a hack in
   // order to get the file name into the index implementations. Do not depend on
   // this inside clangd.
   // FIXME: remove this when there is proper index support via build system
   // integration.
   // FIXME: move to ClangdServer via createProcessingContext.
   static llvm::Optional<llvm::StringRef> getFileBeingProcessedInContext();
 
   void profile(MemoryTree &MT) const;
 
 private:
   const GlobalCompilationDatabase &CDB;
   Options Opts;
   std::unique_ptr<ParsingCallbacks> Callbacks; // not nullptr
   Semaphore Barrier;
   llvm::StringMap<std::unique_ptr<FileData>> Files;
+  std::unique_ptr<PreambleCache> CachedPreambles;
   std::unique_ptr<ASTCache> IdleASTs;
   // None when running tasks synchronously and non-None when running tasks
   // asynchronously.
   llvm::Optional<AsyncTaskRunner> PreambleTasks;
   llvm::Optional<AsyncTaskRunner> WorkerThreads;
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clang-tools-extra/clangd/TUScheduler.cpp

 //===--- TUScheduler.cpp -----------------------------------------*-C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 // TUScheduler manages a worker per active file. This ASTWorker processes
private:
 
   std::mutex Mut;
   unsigned MaxRetainedASTs;
   /// Items sorted in LRU order, i.e. first item is the most recently accessed
   /// one.
   std::vector<KVPair> LRU; /* GUARDED_BY(Mut) */
 };
 
+/// LRU cache with amortized O(1) put and take
+/// Preambles can be stored on disk so we may want to store a high
+/// number of entries
+class TUScheduler::PreambleCache {
+public:
+  PreambleCache(size_t MaxSize, bool StorePreamblesInMemory)
+      : MaxSize(MaxSize), StorePreamblesInMemory(StorePreamblesInMemory) {
+    vlog("TUScheduler will cache {0} preambles", MaxSize);
+  }
+
+  /// Get the preamble associated with a \p Key, removing
+  /// it from the cache
+  std::shared_ptr<const PreambleData> take(llvm::StringRef Key) {
+    auto It = Data.find(Key);
+    if (It == Data.end())
+      return nullptr;
+    auto Result = std::move(It->second);
+
+    // Remove the key from all internal data structures
+    auto KeyToLRUIt = KeyToLRU.find(Key);
+    assert(KeyToLRUIt != KeyToLRU.end() && "Key is missing");
+    auto LRUIt = KeyToLRUIt->second;
+    Data.erase(It);
+    KeyToLRU.erase(KeyToLRUIt);
+    LRU.erase(LRUIt);
+
+    return Result;
+  }
+
+  /// Add a \p Preamble associated with a \p Key, the preamble must
+  /// not be in the cache when this function is called
+  void put(llvm::StringRef Key, std::shared_ptr<const PreambleData> Preamble) {
+    assert(KeyToLRU.find(Key) == KeyToLRU.end());
+    if (MaxSize == 0)
+      return;
+
+    LRU.emplace_front(Key.str());
+    // Use the newly created string as the storage
+    Key = LRU.front();
+
+    Data[Key] = std::move(Preamble);
+    KeyToLRU[Key] = LRU.begin();
+    vlog("Added {0} to preamble cache", Key);
+
+    // Trim the LRU to the max size
+    while (LRU.size() > MaxSize) {
+      const auto &OldestKey = LRU.back();
+      KeyToLRU.erase(OldestKey);
+      Data.erase(OldestKey);
+      vlog("Removed {0} from preamble cache", OldestKey);
+      LRU.pop_back();
+    }
+  }
+
+  void profile(MemoryTree &MT) const {
+    auto &ContainersMT = MT.child("containers");
+    ContainersMT.addUsage(LRU.size() * sizeof(void *) * 2);
+    ContainersMT.addUsage(KeyToLRU.getMemorySize());
+    ContainersMT.addUsage(Data.getMemorySize());
+
+    if (!StorePreamblesInMemory)
+      return;
+
+    auto &PreamblesMT = MT.child("preambles");
+    for (const auto &Entry : Data) {
+      auto &EntryMT = PreamblesMT.detail(Entry.first).child("cached_preamble");
+      EntryMT.child("key").addUsage(Entry.first.size());
+      EntryMT.child("preamble").addUsage(Entry.second->Preamble.getSize());
+    }
+  }
+
+private:
+  using LRUType = std::list<std::string>;
+
+  // LRU holds the keys, most recent first
+  // The maps below use references as keys, they reference
+  // string in this LRU list
+  LRUType LRU;
+  llvm::DenseMap<StringRef, LRUType::iterator> KeyToLRU;
+  llvm::DenseMap<StringRef, std::shared_ptr<const PreambleData>> Data;
+
+  size_t MaxSize;
+  bool StorePreamblesInMemory;
+};
+
 namespace {
 /// Threadsafe manager for updating a TUStatus and emitting it after each
 /// update.
 class SynchronizedTUStatus {
 public:
   SynchronizedTUStatus(PathRef FileName, ParsingCallbacks &Callbacks)
       : FileName(FileName), Callbacks(Callbacks) {}
 

 /// Responsible for building preambles. Whenever the thread is idle and the
 /// preamble is outdated, it starts to build a fresh preamble from the latest
 /// inputs. If RunSync is true, preambles are built synchronously in update()
 /// instead.
 class PreambleThread {
 public:
   PreambleThread(llvm::StringRef FileName, ParsingCallbacks &Callbacks,
                  bool StorePreambleInMemory, bool RunSync,
-                 SynchronizedTUStatus &Status, ASTWorker &AW)
-      : FileName(FileName), Callbacks(Callbacks),
-        StoreInMemory(StorePreambleInMemory), RunSync(RunSync), Status(Status),
-        ASTPeer(AW) {}
+                 SynchronizedTUStatus &Status, ASTWorker &AW,
+                 std::shared_ptr<const PreambleData> InitialPreamble)
+      : LatestBuild(std::move(InitialPreamble)), FileName(FileName),
+        Callbacks(Callbacks), StoreInMemory(StorePreambleInMemory),
+        RunSync(RunSync), Status(Status), ASTPeer(AW) {}
 
   /// It isn't guaranteed that each requested version will be built. If there
   /// are multiple update requests while building a preamble, only the last one
   /// will be built.
   void update(std::unique_ptr<CompilerInvocation> CI, ParseInputs PI,
               std::vector<Diag> CIDiags, WantDiagnostics WantDiags) {
     Request Req = {std::move(CI), std::move(PI), std::move(CIDiags), WantDiags,
                    Context::current().clone()};

 /// remove() is called, but its thread may be busy and we don't want to block.
 /// So the workers are accessed via an ASTWorkerHandle. Destroying the handle
 /// signals the worker to exit its run loop and gives up shared ownership of the
 /// worker.
 class ASTWorker {
   friend class ASTWorkerHandle;
   ASTWorker(PathRef FileName, const GlobalCompilationDatabase &CDB,
             TUScheduler::ASTCache &LRUCache, Semaphore &Barrier, bool RunSync,
-            const TUScheduler::Options &Opts, ParsingCallbacks &Callbacks);
+            const TUScheduler::Options &Opts, ParsingCallbacks &Callbacks,
+            std::shared_ptr<const PreambleData> InitialPreamble);
 
 public:
   /// Create a new ASTWorker and return a handle to it.
   /// The processing thread is spawned using \p Tasks. However, when \p Tasks
   /// is null, all requests will be processed on the calling thread
   /// synchronously instead. \p Barrier is acquired when processing each
   /// request, it is used to limit the number of actively running threads.
-  static ASTWorkerHandle create(PathRef FileName,
-                                const GlobalCompilationDatabase &CDB,
-                                TUScheduler::ASTCache &IdleASTs,
-                                AsyncTaskRunner *Tasks, Semaphore &Barrier,
-                                const TUScheduler::Options &Opts,
-                                ParsingCallbacks &Callbacks);
+  static ASTWorkerHandle
+  create(PathRef FileName, const GlobalCompilationDatabase &CDB,
+         TUScheduler::ASTCache &IdleASTs, AsyncTaskRunner *Tasks,
+         Semaphore &Barrier, const TUScheduler::Options &Opts,
+         ParsingCallbacks &Callbacks,
+         std::shared_ptr<const PreambleData> InitialPreamble);
   ~ASTWorker();
 
   void update(ParseInputs Inputs, WantDiagnostics, bool ContentChanged);
   void
   runWithAST(llvm::StringRef Name,
              llvm::unique_function<void(llvm::Expected<InputsAndAST>)> Action,
              TUScheduler::ASTActionInvalidation);
   bool blockUntilIdle(Deadline Timeout) const;
public:
   /// be schedule via the returned reference, i.e. only reads of the preamble
   /// are possible.
   std::shared_ptr<const ASTWorker> lock() { return Worker; }
 
 private:
   std::shared_ptr<ASTWorker> Worker;
 };
 
-ASTWorkerHandle ASTWorker::create(PathRef FileName,
-                                  const GlobalCompilationDatabase &CDB,
-                                  TUScheduler::ASTCache &IdleASTs,
-                                  AsyncTaskRunner *Tasks, Semaphore &Barrier,
-                                  const TUScheduler::Options &Opts,
-                                  ParsingCallbacks &Callbacks) {
-  std::shared_ptr<ASTWorker> Worker(new ASTWorker(
-      FileName, CDB, IdleASTs, Barrier, /*RunSync=*/!Tasks, Opts, Callbacks));
+ASTWorkerHandle
+ASTWorker::create(PathRef FileName, const GlobalCompilationDatabase &CDB,
+                  TUScheduler::ASTCache &IdleASTs, AsyncTaskRunner *Tasks,
+                  Semaphore &Barrier, const TUScheduler::Options &Opts,
+                  ParsingCallbacks &Callbacks,
+                  std::shared_ptr<const PreambleData> InitialPreamble) {
+  std::shared_ptr<ASTWorker> Worker(
+      new ASTWorker(FileName, CDB, IdleASTs, Barrier, /*RunSync=*/!Tasks, Opts,
+                    Callbacks, std::move(InitialPreamble)));
   if (Tasks) {
     Tasks->runAsync("ASTWorker:" + llvm::sys::path::filename(FileName),
                     [Worker]() { Worker->run(); });
     Tasks->runAsync("PreambleWorker:" + llvm::sys::path::filename(FileName),
                     [Worker]() { Worker->PreamblePeer.run(); });
   }
 
   return ASTWorkerHandle(std::move(Worker));
 }
 
 ASTWorker::ASTWorker(PathRef FileName, const GlobalCompilationDatabase &CDB,
                      TUScheduler::ASTCache &LRUCache, Semaphore &Barrier,
                      bool RunSync, const TUScheduler::Options &Opts,
-                     ParsingCallbacks &Callbacks)
+                     ParsingCallbacks &Callbacks,
+                     std::shared_ptr<const PreambleData> InitialPreamble)
     : IdleASTs(LRUCache), RunSync(RunSync), UpdateDebounce(Opts.UpdateDebounce),
       FileName(FileName), ContextProvider(Opts.ContextProvider), CDB(CDB),
       Callbacks(Callbacks), Barrier(Barrier), Done(false),
       Status(FileName, Callbacks),
       PreamblePeer(FileName, Callbacks, Opts.StorePreamblesInMemory,
-                   RunSync || !Opts.AsyncPreambleBuilds, Status, *this) {
+                   RunSync || !Opts.AsyncPreambleBuilds, Status, *this,
+                   InitialPreamble) {
   // Set a fallback command because compile command can be accessed before
   // `Inputs` is initialized. Other fields are only used after initialization
   // from client inputs.
   FileInputs.CompileCommand = CDB.getFallbackCommand(FileName);
+  if (InitialPreamble) {
+    vlog("ASTWorker for {0} using an initial preamble ({1})", FileName,
+         InitialPreamble->Version);
+    LatestPreamble = std::move(InitialPreamble);
+  }
 }
 
 ASTWorker::~ASTWorker() {
   // Make sure we remove the cached AST, if any.
   IdleASTs.take(this);
 #ifndef NDEBUG
   std::lock_guard<std::mutex> Lock(Mutex);
   assert(Done && "handle was not destroyed");

 
 TUScheduler::TUScheduler(const GlobalCompilationDatabase &CDB,
                          const Options &Opts,
                          std::unique_ptr<ParsingCallbacks> Callbacks)
     : CDB(CDB), Opts(Opts),
       Callbacks(Callbacks ? move(Callbacks)
                           : std::make_unique<ParsingCallbacks>()),
       Barrier(Opts.AsyncThreadsCount),
+      CachedPreambles(std::make_unique<PreambleCache>(
+          Opts.KeepPreambles, Opts.StorePreamblesInMemory)),
       IdleASTs(
           std::make_unique<ASTCache>(Opts.RetentionPolicy.MaxRetainedASTs)) {
   // Avoid null checks everywhere.
   if (!Opts.ContextProvider) {
     this->Opts.ContextProvider = [](llvm::StringRef) {
       return Context::current().clone();
     };
   }

 
 bool TUScheduler::update(PathRef File, ParseInputs Inputs,
                          WantDiagnostics WantDiags) {
   std::unique_ptr<FileData> &FD = Files[File];
   bool NewFile = FD == nullptr;
   bool ContentChanged = false;
   if (!FD) {
     // Create a new worker to process the AST-related tasks.
-    ASTWorkerHandle Worker =
-        ASTWorker::create(File, CDB, *IdleASTs,
-                          WorkerThreads ? WorkerThreads.getPointer() : nullptr,
-                          Barrier, Opts, *Callbacks);
+    ASTWorkerHandle Worker = ASTWorker::create(
+        File, CDB, *IdleASTs,
+        WorkerThreads ? WorkerThreads.getPointer() : nullptr, Barrier, Opts,
+        *Callbacks, CachedPreambles->take(File));
     FD = std::unique_ptr<FileData>(
         new FileData{Inputs.Contents, std::move(Worker)});
     ContentChanged = true;
   } else if (FD->Contents != Inputs.Contents) {
     ContentChanged = true;
     FD->Contents = Inputs.Contents;
   }
   FD->Worker->update(std::move(Inputs), WantDiags, ContentChanged);
   return NewFile;
 }
 
 void TUScheduler::remove(PathRef File) {
-  bool Removed = Files.erase(File);
-  if (!Removed)
+  auto It = Files.find(File);
+  if (It == Files.end()) {
     elog("Trying to remove file from TUScheduler that is not tracked: {0}",
          File);
+    return;
+  }
+
+  assert(It->second && "FileData not allocated");
+  if (auto Preamble = It->second->Worker->getPossiblyStalePreamble())
+    CachedPreambles->put(File, std::move(Preamble));
+  Files.erase(It);
 }
 
 llvm::StringMap<std::string> TUScheduler::getAllFileContents() const {
   llvm::StringMap<std::string> Results;
   for (auto &It : Files)
     Results.try_emplace(It.getKey(), It.getValue()->Contents);
   return Results;
 }

 
 DebouncePolicy DebouncePolicy::fixed(clock::duration T) {
   DebouncePolicy P;
   P.Min = P.Max = T;
   return P;
 }
 
 void TUScheduler::profile(MemoryTree &MT) const {
+  auto &FilesMT = MT.child("files");
   for (const auto &Elem : fileStats()) {
-    MT.detail(Elem.first())
+    FilesMT.detail(Elem.first())
         .child("preamble")
         .addUsage(Opts.StorePreamblesInMemory ? Elem.second.UsedBytesPreamble
                                               : 0);
-    MT.detail(Elem.first()).child("ast").addUsage(Elem.second.UsedBytesAST);
+    FilesMT.detail(Elem.first())
+        .child("ast")
+        .addUsage(Elem.second.UsedBytesAST);
   }
+  CachedPreambles->profile(MT.child("cache"));
 }
 } // namespace clangd
 } // namespace clang

clang-tools-extra/clangd/test/memory_tree.test

 # CHECK-NEXT:           },
 # CHECK-NEXT:           "slabs": {
 # CHECK-NEXT:             "_self": {{[0-9]+}},
 # CHECK-NEXT:             "_total": {{[0-9]+}}
 # CHECK-NEXT:           }
 # CHECK-NEXT:         }
 # CHECK-NEXT:       },
 # CHECK-NEXT:       "tuscheduler": {
-# CHECK-NEXT:         "{{.*}}main.cpp": {
-# CHECK-NEXT:           "_self": {{[0-9]+}},
-# CHECK-NEXT:           "_total": {{[0-9]+}},
-# CHECK-NEXT:           "ast": {
-# CHECK-NEXT:             "_self": {{[0-9]+}},
-# CHECK-NEXT:             "_total": {{[0-9]+}}
-# CHECK-NEXT:           },
-# CHECK-NEXT:           "preamble": {
-# CHECK-NEXT:             "_self": {{[0-9]+}},
-# CHECK-NEXT:             "_total": {{[0-9]+}}
-# CHECK-NEXT:           }
-# CHECK-NEXT:         },
-# CHECK-NEXT:         "_self": {{[0-9]+}},
-# CHECK-NEXT:         "_total": {{[0-9]+}}
-# CHECK-NEXT:       }
-# CHECK-NEXT:     }
-# CHECK-NEXT:   }
+# CHECK-NEXT:         "_self": {{[0-9]+}},
+# CHECK-NEXT:         "_total": {{[0-9]+}},
+# CHECK-NEXT:         "cache": {
+# CHECK-NEXT:           "_self": {{[0-9]+}},
+# CHECK-NEXT:           "_total": {{[0-9]+}},
+# CHECK-NEXT:           "containers": {
+# CHECK-NEXT:             "_self": {{[0-9]+}},
+# CHECK-NEXT:             "_total": {{[0-9]+}}
+# CHECK-NEXT:           }
+# CHECK-NEXT:         },
+# CHECK-NEXT:         "files": {
+# CHECK-NEXT:           "{{.*}}main.cpp": {
+# CHECK-NEXT:             "_self": {{[0-9]+}},
+# CHECK-NEXT:             "_total": {{[0-9]+}},
+# CHECK-NEXT:             "ast": {
+# CHECK-NEXT:               "_self": {{[0-9]+}},
+# CHECK-NEXT:               "_total": {{[0-9]+}}
+# CHECK-NEXT:             },
+# CHECK-NEXT:             "preamble": {
+# CHECK-NEXT:               "_self": {{[0-9]+}},
+# CHECK-NEXT:               "_total": {{[0-9]+}}
+# CHECK-NEXT:             }
+# CHECK-NEXT:           },
+# CHECK-NEXT:           "_self": {{[0-9]+}},
+# CHECK-NEXT:           "_total": {{[0-9]+}}
+# CHECK-NEXT:         }
+# CHECK-NEXT:       }
+# CHECK-NEXT:     }
+# CHECK-NEXT:   }
 ---
 {"jsonrpc":"2.0","id":3,"method":"shutdown"}
 ---
 {"jsonrpc":"2.0","method":"exit"}
 

clang-tools-extra/clangd/tool/ClangdMain.cpp

 //===--- ClangdMain.cpp - clangd server loop ------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 

 opt<unsigned> WorkerThreadsCount{
     "j",
     cat(Misc),
     desc("Number of async workers used by clangd. Background index also "
          "uses this many workers."),
     init(getDefaultAsyncThreadsCount()),
 };
 
+// Magic value to know when the user does not specify a value
+constexpr size_t DefaultKeepPreambleValue = std::numeric_limits<size_t>::max();
+constexpr size_t DefaultKeepPreambleMemory = 10;
+constexpr size_t DefaultKeepPreambleDisk = 1000;
+opt<size_t> KeepPreambles{
+    "keep-preambles", cat(Misc),
+    desc("Number of preambles of closed files that clangd will keep in cache.\n"
+         "Note that preambles may be stored in memory or in disk."),
+    init(DefaultKeepPreambleValue)};
+
 opt<Path> IndexFile{
     "index-file",
     cat(Misc),
     desc(
         "Index file to build the static index. The file must have been created "
         "by a compatible clangd-indexer\n"
         "WARNING: This option is experimental only, and will be removed "
         "eventually. Don't rely on it"),
if (StaticIdx) {
     IdxStack.emplace_back(std::move(StaticIdx));
     IdxStack.emplace_back(
         std::make_unique<MergedIndex>(PAI.get(), IdxStack.back().get()));
     Opts.StaticIndex = IdxStack.back().get();
   } else {
     Opts.StaticIndex = PAI.get();
   }
   Opts.AsyncThreadsCount = WorkerThreadsCount;
+
+  if (KeepPreambles == DefaultKeepPreambleValue) // User did not specify a value
+    Opts.KeepPreambles = Opts.StorePreamblesInMemory ? DefaultKeepPreambleMemory
+                                                     : DefaultKeepPreambleDisk;
+  else
+    Opts.KeepPreambles = KeepPreambles;
+
   Opts.BuildRecoveryAST = RecoveryAST;
   Opts.PreserveRecoveryASTType = RecoveryASTType;
   Opts.FoldingRanges = FoldingRanges;
   Opts.MemoryCleanup = getMemoryCleanupFunction();
 
   Opts.CodeComplete.IncludeIneligibleResults = IncludeIneligibleResults;
   Opts.CodeComplete.Limit = LimitResults;
   if (CompletionStyle.getNumOccurrences())

clang-tools-extra/clangd/unittests/ClangdTests.cpp

 //===-- ClangdTests.cpp - Clangd unit tests ---------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
TEST(ClangdServer, MemoryUsageTest) {
   auto FooCpp = testPath("foo.cpp");
   Server.addDocument(FooCpp, "");
   ASSERT_TRUE(Server.blockUntilIdleForTest());
 
   llvm::BumpPtrAllocator Alloc;
   MemoryTree MT(&Alloc);
   Server.profile(MT);
   ASSERT_TRUE(MT.children().count("tuscheduler"));
-  EXPECT_TRUE(MT.child("tuscheduler").children().count(FooCpp));
+  EXPECT_TRUE(MT.child("tuscheduler").child("files").children().count(FooCpp));
 }
 } // namespace
 } // namespace clangd
 } // namespace clang