[clangd] Implemented DraftFileSystem

[LutsenkoDanil]

Summary

Patch changes behavior of preamble building. If headers was changed in editor, preamble will be built with draft versions of them.

[ilya-biryukov]

Thanks for the patch! I believe many people I talked to want this behavior (myself included).
Some people like what we do now more. It feels like it depends on the workflow: for people who auto-save *all* files before build (some editors do it automatically?) the new behavior is the right one, for people who only save current file and rerun build on the console the old behavior is the correct one.
It all boils down to the argument "I want to see same errors that running the compiler would produce".

@klimek, @arphaman, @simark, WDYT?

clangd/FS.h

82

This assumes the Path is absolute and vfs::FileSystem can be called with non-absolute paths too.
One way to make it work with relative paths is to create an InMemoryFileSystem with the headers (it handles the absolute paths conversions) and create an OverlayFileSystem on top of it and the RealFileSystem.

This would also make more complicated things work, e.g. getting the same files when traversing parent directories, etc.

Could we try this approach? WDYT?

[klimek]

In D54077#1287040, @ilya-biryukov wrote:

Thanks for the patch! I believe many people I talked to want this behavior (myself included).
Some people like what we do now more. It feels like it depends on the workflow: for people who auto-save *all* files before build (some editors do it automatically?) the new behavior is the right one, for people who only save current file and rerun build on the console the old behavior is the correct one.
It all boils down to the argument "I want to see same errors that running the compiler would produce".

@klimek, @arphaman, @simark, WDYT?

I'm in yet another camp: I carefully save when I have something that is correct enough syntax, so I only want errors from with changes from the exact file I'm editing and the rest of the files in saved state.

[sammccall]

Disclaimer: I'm on a train with family today, and haven't actually read the
patch...

So I have concerns :-)

1. There's the usual concern that the current behavior is reasonable and

people like it, so adding a second reasonable behavior provides a small
amount of value to the userbase as a whole (because 99%+ will use the
default). If the benefit is small, the comparison to support cost may be
unfavorable.

2. This needs to invalidate preambles more often, which brings both

performance and complexity questions. E.g. we will at some point invalidate
preambles and regenerate diagnostics based on file writes. After this
change, we'll be obligated to do so for edits too. (Remember, LSP has no
concept of "the foreground file"). Invalidating a preamble is expensive,
and edits come rapidly and may invalidate multiple TUs. The current
behavior seems more compatible with being both simple and fast.

So mostly I'd like to be convinced that this is important (or that it's
simple, but that seems unlikely at first glance).

Cheers, Sam

[ilya-biryukov]

There's the usual concern that the current behavior is reasonable and people like it.

I think it would be reasonable to say that a large portion of C++ users are used to the behavior that this patch introduces.
Specifically, all IDEs (Clion, Eclipse, Visual Studio for certain, not sure about XCode) will see the contents of the headers that the user typed and not the other way around.
It does not cause any inconsistencies there, the IDE will save all your open files when you hit "build". That also means reading files from the filesystem gives non-useful errors in that case, because the build will actually see the files from the editors.

[LutsenkoDanil]

@ilya-biryukov, For example, VSCode saves all changed files by default when you press Ctrl+Shift+B (if build task configured for project).

@klimek If behavior will be configurable, is it ok for you?

@sammccall Current behavior may confuse new users, since, other IDEs mostly (all?) shows diagnostics for edited files instead of saved one. And it's unexpected that headers have 2 states - visible and saved (which cannot be viewed in IDE at all). Looks like performance will be same like usage of 'auto save after delay' feature in editor, if we make debounce delay configurable, what do you think?

I suggest inroduce following changes to the patch:

• Changes proposed by @ilya-biryukov
• Configurable behavior
• Debounce timeout option

clangd/FS.h

82

Sounds good. Thank you for review!

[klimek]

In D54077#1287282, @LutsenkoDanil wrote:

@klimek If behavior will be configurable, is it ok for you?

I have the same concerns as Sam for making this an option.

@sammccall Current behavior may confuse new users, since, other IDEs mostly (all?) shows diagnostics for edited files instead of saved one. And it's unexpected that headers have 2 states - visible and saved (which cannot be viewed in IDE at all). Looks like performance will be same like usage of 'auto save after delay' feature in editor, if we make debounce delay configurable, what do you think?

don't most IDEs show whether a file is saved or just modified?

[simark]

In D54077#1287153, @klimek wrote:

I'm in yet another camp: I carefully save when I have something that is correct enough syntax, so I only want errors from with changes from the exact file I'm editing and the rest of the files in saved state.

That sounds like the current behavior, doesn't it?

Personally, I would like the behavior proposed by this patch. I think it perfectly in line with the idea of showing the same diagnostics as the compiler would produce. It's just that it's what the compiler would produce if compiling the files as seen in the editor.

Also, in the LSP, after a didOpen and until the corresponding didClose, the source of truth for that file is supposed to be what is in the editor buffer. So I think it would make sense that if other files include the edited file, they should see the content from the editor.

Of course, that is dependent of having an efficient cancelling mechanism. Even with some debouncing, an update to a header file can trigger the re-processing of many TUs, so if I do another edit to the same header shortly after, all the queued jobs from the first edit should be dropped. But I think we already have that, don't we?

[sammccall]

I don't think either point 1 or 2 above have been addressed, and so I'm not comfortable moving forward with this change.

I think it would be reasonable to say that a large portion of C++ users are used to the behavior that this patch introduces.

I agree, the new behavior is reasonable. The old behavior is also reasonable. This is the basis for point 1 above.

Current behavior may confuse new users, since, other IDEs mostly (all?) shows diagnostics for edited files instead of saved one. And it's unexpected that headers have 2 states - visible and saved (which cannot be viewed in IDE at all).

Maybe part of the issue is culture clash between "IDEs" and "editors" - the distinction is somewhat artificial, but I think the current behavior in editors is somewhat traditional, and certainly the two-states is what editor users expect (the saved state is what you get when you run the compiler).

Looks like performance will be same like usage of 'auto save after delay' feature in editor, if we make debounce delay configurable, what do you think?

I think there will be substantial performance loss *and* additional complexity from this mode once we start updating diagnostics on file change. (Point 2 above).
Additionally, if this needs to be configurable, that adds further complexity.

In D54077#1287301, @simark wrote:

Of course, that is dependent of having an efficient cancelling mechanism. Even with some debouncing, an update to a header file can trigger the re-processing of many TUs, so if I do another edit to the same header shortly after, all the queued jobs from the first edit should be dropped. But I think we already have that, don't we?

Not for preambles (as we don't yet even update diagnostics on file events), and that case is significantly more complicated than main-files:

• there are multiple TUs to consider as you say
• preambles can be 100x more expensive than mainfiles
• this affects background files, so is "usually" unimportant work
• this happens while latency-sensitive operations like code-complete happen in the header file itself

[sammccall]

Someone mentioned to me that the interaction-between-features argument wasn't clear here:

• we don't currently update diagnostics for A.cc when A.h is edited
• we should, this seems more obvious & important than what we do with drafts
• this interacts badly with using draft state, as this patch proposes - there are too many edits

[ioeric]

In D54077#1288387, @sammccall wrote:

Someone mentioned to me that the interaction-between-features argument wasn't clear here:

• we don't currently update diagnostics for A.cc when A.h is edited
• we should, this seems more obvious & important than what we do with drafts
• this interacts badly with using draft state, as this patch proposes - there are too many edits

FWIW, one of my "pain points" when using vim+clangd is:

• Edit A.h in a buffer (and forget to save)
• Switch to A.cc in another buffer
• Realize that I forgot to save A.h
• Go back to save A.h
• Jump back to A.cc file.

I would be very happy if A.cc can see the unsaved A.h when I am editing A.cc. I think the update should be relatively less frequent with this approach, as people don't usually update multiple files at the same time. Not sure if I want all files that depend on A.h to be updated when I edit A.h though; it seems much more complicated and less important (to me).

[klimek]

In D54077#1288404, @ioeric wrote:

In D54077#1288387, @sammccall wrote:

Someone mentioned to me that the interaction-between-features argument wasn't clear here:

• we don't currently update diagnostics for A.cc when A.h is edited
• we should, this seems more obvious & important than what we do with drafts
• this interacts badly with using draft state, as this patch proposes - there are too many edits

FWIW, one of my "pain points" when using vim+clangd is:

• Edit A.h in a buffer (and forget to save)
• Switch to A.cc in another buffer
• Realize that I forgot to save A.h
• Go back to save A.h
• Jump back to A.cc file.

I would be very happy if A.cc can see the unsaved A.h when I am editing A.cc. I think the update should be relatively less frequent with this approach, as people don't usually update multiple files at the same time. Not sure if I want all files that depend on A.h to be updated when I edit A.h though; it seems much more complicated and less important (to me).

I can see this is a problem when you forget to save. On the other hand, one case I encounter frequently enough is:
edit A.h, save
go to B.cc (that includes it), see errors that it causes (here and in other files)
go to A.h, fix error, save
go to B.cc (or a different file that includes A.h) and wanting to see the error gone

[nik]

In D54077#1288404, @ioeric wrote:

I would be very happy if A.cc can see the unsaved A.h when I am editing A.cc.

We have that in Qt Creator (with libclang) and that's quite handy as it can save you some build cycles.

Not sure if I want all files that depend on A.h to be updated when I edit A.h though; it seems much more complicated and less important (to me).

If it helps (and the LSP allows it): In case A.h is edited, we flag it dirty. If the user makes some file depending on it visible (or the current file), we trigger a reparse for that. Not sure whether the LSP has a notion of current or visible files...

[nik]

In D54077#1288413, @nik wrote:

If it helps (and the LSP allows it): In case A.h is edited, we flag it dirty. If the user makes some file depending on it visible (or the current file), we trigger a reparse for that. Not sure whether the LSP has a notion of current or visible files...

Huch..., I meant that we flag the files dirty that depend on A.h.

[LutsenkoDanil]

@sammccall Thank you for detailed explanation!

In D54077#1288387, @sammccall wrote:

Someone mentioned to me that the interaction-between-features argument wasn't clear here:

• we don't currently update diagnostics for A.cc when A.h is edited
• we should, this seems more obvious & important than what we do with drafts
• this interacts badly with using draft state, as this patch proposes - there are too many edits

I already made a patch which introduces such behavior (not uploaded here yet), and looks like it works well with draft fs: diagnostics updates for depended files in 'real-time' on typing for opened files and no seen performance glitches, in multi-threaded mode at least.
I suggest continue discussion when/if dependencies tracking will be implemented and real performance reduce introduced by this patch can be checked with real code.

[ilya-biryukov]

In D54077#1287289, @klimek wrote:

don't most IDEs show whether a file is saved or just modified?

They do, but whenever you run the build from them, they will save all modified files before actually running it.

In D54077#1288455, @LutsenkoDanil wrote:

I already made a patch which introduces such behavior (not uploaded here yet), and looks like it works well with draft fs: diagnostics updates for depended files in 'real-time' on typing for opened files and no seen performance glitches, in multi-threaded mode at least.
I suggest continue discussion when/if dependencies tracking will be implemented and real performance reduce introduced by this patch can be checked with real code.

Exciting! Please send it out! I'm also starting to look into this, but my work is mostly focused on watching files in the filesystem.

[sammccall]

In D54077#1288455, @LutsenkoDanil wrote:

I already made a patch which introduces such behavior (not uploaded here yet), and looks like it works well with draft fs: diagnostics updates for depended files in 'real-time' on typing for opened files and no seen performance glitches, in multi-threaded mode at least.

This does indeed sound exciting! I'm a little skeptical that it "just works", there are some tricky cases:

• TUs where the preamble takes multiple seconds to rebuild
• "thundering herd" problems if several TUs are active that include the edited file
• redundant background work does have costs (e.g. battery)

If it helps (and the LSP allows it): In case A.h is edited, we flag it dirty. If the user makes some file depending on it visible (or the current file), we trigger a reparse for that. Not sure whether the LSP has a notion of current or visible files...

It doesn't. Maybe there's a way to make this idea work anyway (nothing comes to mind).

I suggest continue discussion when/if dependencies tracking will be implemented and real performance reduce introduced by this patch can be checked with real code.

Sounds reasonable. I think maybe the shortest path there is to implement on top of the LSP file change notifications, and have it push a "maybe rebuild" item on every TUScheduler worker.

clangd/ClangdLSPServer.h

private:
   clangd::Transport &Transp;
   class MessageHandler;
   std::unique_ptr<MessageHandler> MsgHandler;
   std::atomic<int> NextCallID = {0};
   std::mutex TranspWriter;
   void call(StringRef Method, llvm::json::Value Params);
   void notify(StringRef Method, llvm::json::Value Params);
 
-  RealFileSystemProvider FSProvider;
+  DraftFileSystemProvider FSProvider;
   /// Options used for code completion
   clangd::CodeCompleteOptions CCOpts;
   /// Options used for diagnostics.
   ClangdDiagnosticOptions DiagOpts;
   /// The supported kinds of the client.
   SymbolKindBitset SupportedSymbolKinds;
   /// The supported completion item kinds of the client.
   CompletionItemKindBitset SupportedCompletionItemKinds;

clangd/ClangdLSPServer.cpp

Server->workspaceSymbols(
             Reply(std::move(*Items));
           },
           std::move(Reply)));
 }
 
 void ClangdLSPServer::onRename(const RenameParams &Params,
                                Callback<WorkspaceEdit> Reply) {
   Path File = Params.textDocument.uri.file();
-  Optional<std::string> Code = DraftMgr.getDraft(File);
-  if (!Code)
+  auto Draft = DraftMgr.getDraft(File);
+  if (!Draft)
     return Reply(make_error<LSPError>("onRename called for non-added file",
                                       ErrorCode::InvalidParams));
 
   Server->rename(
       File, Params.position, Params.newName,
       Bind(
-          [File, Code,
+          [File, Draft,
            Params](decltype(Reply) Reply,
                    Expected<std::vector<tooling::Replacement>> Replacements) {
             if (!Replacements)
               return Reply(Replacements.takeError());
 
             // Turn the replacements into the format specified by the Language
             // Server Protocol. Fuse them into one big JSON array.
             std::vector<TextEdit> Edits;
             for (const auto &R : *Replacements)
-              Edits.push_back(replacementToEdit(*Code, R));
+              Edits.push_back(replacementToEdit(Draft->Content, R));
             WorkspaceEdit WE;
             WE.changes = {{Params.textDocument.uri.uri(), Edits}};
             Reply(WE);
           },
           std::move(Reply)));
 }
 
 void ClangdLSPServer::onDocumentDidClose(
     const DidCloseTextDocumentParams &Params) {
   PathRef File = Params.textDocument.uri.file();
   DraftMgr.removeDraft(File);
   Server->removeDocument(File);
 }
 
 void ClangdLSPServer::onDocumentOnTypeFormatting(
     const DocumentOnTypeFormattingParams &Params,
     Callback<std::vector<TextEdit>> Reply) {
   auto File = Params.textDocument.uri.file();
-  auto Code = DraftMgr.getDraft(File);
-  if (!Code)
+  auto Draft = DraftMgr.getDraft(File);
+  if (!Draft)
     return Reply(make_error<LSPError>(
         "onDocumentOnTypeFormatting called for non-added file",
         ErrorCode::InvalidParams));
 
-  auto ReplacementsOrError = Server->formatOnType(*Code, File, Params.position);
+  auto ReplacementsOrError = Server->formatOnType(Draft->Content, File, Params.position);
   if (ReplacementsOrError)
-    Reply(replacementsToEdits(*Code, ReplacementsOrError.get()));
+    Reply(replacementsToEdits(Draft->Content, ReplacementsOrError.get()));
   else
     Reply(ReplacementsOrError.takeError());
 }
 
 void ClangdLSPServer::onDocumentRangeFormatting(
     const DocumentRangeFormattingParams &Params,
     Callback<std::vector<TextEdit>> Reply) {
   auto File = Params.textDocument.uri.file();
-  auto Code = DraftMgr.getDraft(File);
-  if (!Code)
+  auto Draft = DraftMgr.getDraft(File);
+  if (!Draft)
     return Reply(make_error<LSPError>(
         "onDocumentRangeFormatting called for non-added file",
         ErrorCode::InvalidParams));
 
-  auto ReplacementsOrError = Server->formatRange(*Code, File, Params.range);
+  auto ReplacementsOrError = Server->formatRange(Draft->Content, File, Params.range);
   if (ReplacementsOrError)
-    Reply(replacementsToEdits(*Code, ReplacementsOrError.get()));
+    Reply(replacementsToEdits(Draft->Content, ReplacementsOrError.get()));
   else
     Reply(ReplacementsOrError.takeError());
 }
 
 void ClangdLSPServer::onDocumentFormatting(
     const DocumentFormattingParams &Params,
     Callback<std::vector<TextEdit>> Reply) {
   auto File = Params.textDocument.uri.file();
-  auto Code = DraftMgr.getDraft(File);
-  if (!Code)
+  auto Draft = DraftMgr.getDraft(File);
+  if (!Draft)
     return Reply(
         make_error<LSPError>("onDocumentFormatting called for non-added file",
                              ErrorCode::InvalidParams));
 
-  auto ReplacementsOrError = Server->formatFile(*Code, File);
+  auto ReplacementsOrError = Server->formatFile(Draft->Content, File);
   if (ReplacementsOrError)
-    Reply(replacementsToEdits(*Code, ReplacementsOrError.get()));
+    Reply(replacementsToEdits(Draft->Content, ReplacementsOrError.get()));
   else
     Reply(ReplacementsOrError.takeError());
 }
 
 void ClangdLSPServer::onDocumentSymbol(
     const DocumentSymbolParams &Params,
     Callback<std::vector<SymbolInformation>> Reply) {
   Server->documentSymbols(
Server->findReferences(Params.textDocument.uri.file(), Params.position,
                          std::move(Reply));
 }
 
 ClangdLSPServer::ClangdLSPServer(class Transport &Transp,
                                  const clangd::CodeCompleteOptions &CCOpts,
                                  Optional<Path> CompileCommandsDir,
                                  bool UseDirBasedCDB,
                                  const ClangdServer::Options &Opts)
-    : Transp(Transp), MsgHandler(new MessageHandler(*this)), CCOpts(CCOpts),
+    : Transp(Transp), MsgHandler(new MessageHandler(*this)),
+      FSProvider(DraftMgr), CCOpts(CCOpts),
       SupportedSymbolKinds(defaultSymbolKinds()),
       SupportedCompletionItemKinds(defaultCompletionItemKinds()),
       UseDirBasedCDB(UseDirBasedCDB),
       CompileCommandsDir(std::move(CompileCommandsDir)),
       ClangdServerOpts(Opts) {
   // clang-format off
   MsgHandler->bind("initialize", &ClangdLSPServer::onInitialize);
   MsgHandler->bind("shutdown", &ClangdLSPServer::onShutdown);
notify("textDocument/publishDiagnostics",
          json::Object{
              {"uri", URI},
              {"diagnostics", std::move(LSPDiagnostics)},
          });
 }
 
 void ClangdLSPServer::reparseOpenedFiles() {
   for (const Path &FilePath : DraftMgr.getActiveFiles())
-    Server->addDocument(FilePath, *DraftMgr.getDraft(FilePath),
+    Server->addDocument(FilePath, DraftMgr.getDraft(FilePath)->Content,
                         WantDiagnostics::Auto);
 }
 
 } // namespace clangd
 } // namespace clang

clangd/DraftStore.h

 //===--- DraftStore.h - File contents container -----------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_DRAFTSTORE_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_DRAFTSTORE_H
 
 #include "Path.h"
 #include "Protocol.h"
 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/StringMap.h"
+#include "llvm/Support/Chrono.h"
 #include <mutex>
 #include <string>
 #include <vector>
 
 namespace clang {
 namespace clangd {
 
+struct Draft {
+  std::string Content;
+  llvm::sys::TimePoint<> ModifyTime;
+};
+
 /// A thread-safe container for files opened in a workspace, addressed by
 /// filenames. The contents are owned by the DraftStore. This class supports
 /// both whole and incremental updates of the documents.
 class DraftStore {
 public:
   /// \return Contents of the stored document.
   /// For untracked files, a llvm::None is returned.
-  llvm::Optional<std::string> getDraft(PathRef File) const;
+  llvm::Optional<Draft> getDraft(PathRef File) const;
 
   /// \return List of names of the drafts in this store.
   std::vector<Path> getActiveFiles() const;
 
   /// Replace contents of the draft for \p File with \p Contents.
   void addDraft(PathRef File, StringRef Contents);
 
   /// Update the contents of the draft for \p File based on \p Changes.
   /// If a position in \p Changes is invalid (e.g. out-of-range), the
   /// draft is not modified.
   ///
   /// \return The new version of the draft for \p File, or an error if the
   /// changes couldn't be applied.
   llvm::Expected<std::string>
   updateDraft(PathRef File,
               llvm::ArrayRef<TextDocumentContentChangeEvent> Changes);
 
   /// Remove the draft from the store.
   void removeDraft(PathRef File);
 
 private:
   mutable std::mutex Mutex;
-  llvm::StringMap<std::string> Drafts;
+  llvm::StringMap<Draft> Drafts;
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/DraftStore.cpp

 #include "DraftStore.h"
 #include "SourceCode.h"
 #include "llvm/Support/Errc.h"
 
 using namespace llvm;
 namespace clang {
 namespace clangd {
 
-Optional<std::string> DraftStore::getDraft(PathRef File) const {
+Optional<Draft> DraftStore::getDraft(PathRef File) const {
   std::lock_guard<std::mutex> Lock(Mutex);
 
   auto It = Drafts.find(File);
   if (It == Drafts.end())
     return None;
 
   return It->second;
 }
 
 std::vector<Path> DraftStore::getActiveFiles() const {
   std::lock_guard<std::mutex> Lock(Mutex);
   std::vector<Path> ResultVector;
 
   for (auto DraftIt = Drafts.begin(); DraftIt != Drafts.end(); DraftIt++)
     ResultVector.push_back(DraftIt->getKey());
 
   return ResultVector;
 }
 
 void DraftStore::addDraft(PathRef File, StringRef Contents) {
   std::lock_guard<std::mutex> Lock(Mutex);
 
-  Drafts[File] = Contents;
+  Drafts[File] = { Contents, llvm::sys::TimePoint<>::clock::now() };
 }
 
 Expected<std::string>
 DraftStore::updateDraft(PathRef File,
                         ArrayRef<TextDocumentContentChangeEvent> Changes) {
   std::lock_guard<std::mutex> Lock(Mutex);
 
   auto EntryIt = Drafts.find(File);
   if (EntryIt == Drafts.end()) {
     return make_error<StringError>(
         "Trying to do incremental update on non-added document: " + File,
         llvm::errc::invalid_argument);
   }
 
-  std::string Contents = EntryIt->second;
+  std::string Contents = EntryIt->second.Content;
 
   for (const TextDocumentContentChangeEvent &Change : Changes) {
     if (!Change.range) {
       Contents = Change.text;
       continue;
     }
 
     const Position &Start = Change.range->start;
for (const TextDocumentContentChangeEvent &Change : Changes) {
 
     NewContents = Contents.substr(0, *StartIndex);
     NewContents += Change.text;
     NewContents += Contents.substr(*EndIndex);
 
     Contents = std::move(NewContents);
   }
 
-  EntryIt->second = Contents;
+  Drafts[File] = { Contents, llvm::sys::TimePoint<>::clock::now() };
   return Contents;
 }
 
 void DraftStore::removeDraft(PathRef File) {
   std::lock_guard<std::mutex> Lock(Mutex);
 
   Drafts.erase(File);
 }
 
 } // namespace clangd
 } // namespace clang

clangd/FS.h

 //===--- FS.h - File system related utils ------------------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_FS_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_FS_H
 
+#include "DraftStore.h"
+#include "Logger.h"
 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/Support/VirtualFileSystem.h"
 
 namespace clang {
 namespace clangd {
 
+class DraftFile : public llvm::vfs::File {
+  std::unique_ptr<File> F;
+  Draft DraftInfo;
+
+public:
+  DraftFile(std::unique_ptr<File> F, Draft D)
+      : F(std::move(F)), DraftInfo(std::move(D)) {}
+
+  llvm::ErrorOr<std::string> getName() override { return F->getName(); }
+
+  llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>>
+  getBuffer(const Twine &Name, int64_t FileSize, bool RequiresNullTerminator,
+            bool IsVolatile) override {
+    (void)IsVolatile;
+    (void)RequiresNullTerminator;
+
+    if (FileSize < 0) {
+      FileSize = DraftInfo.Content.size();
+    }
+
+    return llvm::MemoryBuffer::getMemBufferCopy(
+        llvm::StringRef(DraftInfo.Content).substr(0, FileSize), Name);
+  }
+
+  llvm::ErrorOr<llvm::vfs::Status> status() override {
+    auto RealStatus = F->status();
+
+    if (!RealStatus) {
+      return RealStatus;
+    }
+
+    llvm::vfs::Status Result(
+        RealStatus->getName(), RealStatus->getUniqueID(), DraftInfo.ModifyTime,
+        RealStatus->getUser(), RealStatus->getGroup(), DraftInfo.Content.size(),
+        RealStatus->getType(), RealStatus->getPermissions());
+
+    Result.IsVFSMapped = RealStatus->IsVFSMapped;
+    return Result;
+  }
+
+  std::error_code close() override { return F->close(); }
+};
+
+/// Wrapper around real FS. If file was changed in the DraftStrore, returns
+/// changed content, instead of real
+class DraftFileSystem : public llvm::vfs::ProxyFileSystem {
+  DraftStore &DS;
+
+public:
+  DraftFileSystem(IntrusiveRefCntPtr<FileSystem> FS, DraftStore &DS)
+      : llvm::vfs::ProxyFileSystem(FS), DS(DS) {}
+
+  llvm::ErrorOr<std::unique_ptr<llvm::vfs::File>>
+  openFileForRead(const Twine &Path) override {
+    auto Result = getUnderlyingFS().openFileForRead(Path);
+
+    if (!Result) {
+      return Result;
+    }
+
+    if (auto D = DS.getDraft(Path.str())) {
+      return std::unique_ptr<llvm::vfs::File>(
+          new DraftFile(std::move(*Result), std::move(*D)));
+    }
+
+    return Result;
+  }
+};
+
 /// Records status information for files open()ed or stat()ed during preamble
 /// build (except for the main file), so we can avoid stat()s on the underlying
 /// FS when reusing the preamble. For example, code completion can re-stat files
 /// when getting FileID for source locations stored in preamble (e.g. checking
 /// whether a location is in the main file).
 ///
 /// The cache is keyed by absolute path of file name in cached status, as this
 /// is what preamble stores.

clangd/FSProvider.h

 //===--- FSProvider.h - VFS provider for ClangdServer ------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_FSPROVIDER_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_FSPROVIDER_H
 
+#include "FS.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/Support/VirtualFileSystem.h"
 
 namespace clang {
 namespace clangd {
 
 // Wrapper for vfs::FileSystem for use in multithreaded programs like clangd.
 // As FileSystem is not threadsafe, concurrent threads must each obtain one.

 public:
   // FIXME: returns the single real FS instance, which is not threadsafe.
   llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>
   getFileSystem() const override {
     return llvm::vfs::getRealFileSystem();
   }
 };
 
+class DraftFileSystemProvider : public FileSystemProvider {
+  DraftStore &DS;
+
+public:
+  DraftFileSystemProvider(DraftStore &DS) : DS(DS) {}
+
+  llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>
+  getFileSystem() const override {
+    static llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> Result(
+        new DraftFileSystem(llvm::vfs::getRealFileSystem(), DS));
+    return Result;
+  }
+};
+
 } // namespace clangd
 } // namespace clang
 
 #endif

unittests/clangd/DraftStoreTests.cpp

TextDocumentContentChangeEvent Event{
         SrcBefore.range(),
         rangeLength(SrcBefore.code(), SrcBefore.range()),
         Contents.str(),
     };
 
     Expected<std::string> Result = DS.updateDraft(Path, {Event});
     ASSERT_TRUE(!!Result);
     EXPECT_EQ(*Result, SrcAfter.code());
-    EXPECT_EQ(*DS.getDraft(Path), SrcAfter.code());
+    EXPECT_EQ(DS.getDraft(Path)->Content, SrcAfter.code());
   }
 }
 
 /// Send all the changes at once to updateDraft, check only the final result.
 void allAtOnce(ArrayRef<IncrementalTestStep> Steps) {
   DraftStore DS;
   Annotations InitialSrc(Steps.front().Src);
   Annotations FinalSrc(Steps.back().Src);
void allAtOnce(ArrayRef<IncrementalTestStep> Steps) {
 
   // Set the initial content.
   DS.addDraft(Path, InitialSrc.code());
 
   Expected<std::string> Result = DS.updateDraft(Path, Changes);
 
   ASSERT_TRUE(!!Result) << toString(Result.takeError());
   EXPECT_EQ(*Result, FinalSrc.code());
-  EXPECT_EQ(*DS.getDraft(Path), FinalSrc.code());
+  EXPECT_EQ(DS.getDraft(Path)->Content, FinalSrc.code());
 }
 
 TEST(DraftStoreIncrementalUpdateTest, Simple) {
   // clang-format off
   IncrementalTestStep Steps[] =
     {
       // Replace a range
       {
TEST(DraftStoreIncrementalUpdateTest, InvalidRangeInASequence) {
   Change2.text = "something";
 
   Expected<std::string> Result = DS.updateDraft(File, {Change1, Change2});
 
   EXPECT_TRUE(!Result);
   EXPECT_EQ(toString(Result.takeError()),
             "UTF-16 offset 100 is invalid for line 0");
 
-  Optional<std::string> Contents = DS.getDraft(File);
-  EXPECT_TRUE(Contents);
-  EXPECT_EQ(*Contents, OriginalContents);
+  Optional<Draft> NewDraft = DS.getDraft(File);
+  EXPECT_TRUE(NewDraft);
+  EXPECT_EQ(NewDraft->Content, OriginalContents);
 }
 
 } // namespace
 } // namespace clangd
 } // namespace clang