[VFS] Allow multiple RealFileSystem instances with independent CWDs.

[sammccall]

Summary

Previously only one RealFileSystem instance was available, and its working
directory is shared with the process. This doesn't work well for multithreaded
programs that want to work with relative paths - the vfs::FileSystem is assumed
to provide the working directory, but a thread cannot control this exclusively.

The new vfs::createPhysicalFileSystem() factory copies the process's working
directory initially, and then allows it to be independently modified.

This implementation records the working directory path, and glues it to relative
paths to provide the correct absolute path to the sys::fs:: functions.
This will give different results in unusual situations (e.g. the CWD is moved).

The main alternative is the use of openat(), fstatat(), etc to ask the OS to
resolve paths relative to a directory handle which can be kept open. This is
more robust. There are two reasons not to do this initially:

1. these functions are not available on all supported Unixes, and are somewhere between difficult and unavailable on Windows. So we need a path-based fallback anyway.
2. this would mean also adding support at the llvm::sys::fs level, which is a larger project. My clearest idea is an OS-specific BaseDirectory object that can be optionally passed to functions there. Eventually this could be backed by either paths or a fd where openat() is supported. This is a large project, and demonstrating here that a path-based fallback works is a useful prerequisite.

There is some subtlety to the path-manipulation mechanism:

• when setting the working directory, both Specified=makeAbsolute(path) and Resolved=realpath(path) are recorded. These may differ in the presence of symlinks.
• getCurrentWorkingDirectory() and makeAbsolute() use Specified - this is similar to the behavior of $PWD and sys::path::current_path
• IO operations like openFileForRead use Resolved. This is similar to the behavior of an openat() based implementation, that doesn't see changes in symlinks.

There may still be combinations of operations and FS states that yield unhelpful
behavior. This is hard to avoid with symlinks and FS abstractions :(

The caching behavior of the current working directory is removed in this patch.
getRealFileSystem() is now specified to link to the process CWD, so the caching
is incorrect.
The user who needed this so far is clangd, which will immediately switch to
createPhysicalFileSystem().

[labath]

The patch makes sense to me.

I wonder if instead of lambda-wrapping everything we couldn't use a pattern like this:

Twine fixPath(const Twin &Orig, SmallVectorImpl<char> &Storage) {
  if (path_needs_fixing()) {
    fix_path(Orig, Storage);
    return Storage;
  }
  return Orig;
}

T whatever(const Twine &Path) {
  SmallString<N> Storage;
  // This is safe because the returned twine is either a copy of the input twine or a unary twine backed by the provided storage
  Twine Fixed = fixPath(Path, Storage);
  return do_stuff(Fixed);
}

This usage of Twine isn't very nice, but neither is lambda-wrapping everything, so it's kind of a pick-your-poison situation. Anyway, just throwing the idea out there...

lib/Support/VirtualFileSystem.cpp

241

explicit

265

It looks like it should be possible to use the trailing return type syntax to simplify this (auto WithFixedPath(Path, F) -> decltype(F(Path))

[sammccall]

Address comments, avoid weird lambdas.

[sammccall]

In D56545#1352758, @labath wrote:

This usage of Twine isn't very nice, but neither is lambda-wrapping everything, so it's kind of a pick-your-poison situation. Anyway, just throwing the idea out there...

Thanks, I think that's a bit nicer.

I'm almost certain that ripping Twine out of all our filesystem interfaces would improve both performance and readability overall :-/

[labath]

Thanks for jumping on this. Looks good to me, though I would feel better if someone else acked this too.

[JDevlieghere]

Why did you choose to make this a property of the RealFileSystem as opposed to subclassing the RealFileSystem to a PhysicalFileSystem? It seems like the latter would be more in line with the VFS design.

[sammccall]

In D56545#1352933, @JDevlieghere wrote:

Why did you choose to make this a property of the RealFileSystem as opposed to subclassing the RealFileSystem to a PhysicalFileSystem? It seems like the latter would be more in line with the VFS design.

The reason the logic is interleaved like this: the operations aren't a simple "translation" into equivalent operations with absolute paths. Note that the returned files and statuses must report the relative paths provided, not the absolute ones we translate the actual IO into. So building the FS-with-own-WD as an aggregate or derived class of the FS-with-native-WD is going to be awkward and fragile. (And semantically, inheritance seems dubious here).
This could be a hierarchy of three classes where the abstract base tries not to express an opinion (or CRTP, or a template over a policy object), but I didn't find a pattern that seemed less confusing/indirect than just using a single class with if(). I don't expect the performance aspect of having to branch to matter at all.

[JDevlieghere]

In D56545#1352995, @sammccall wrote:

In D56545#1352933, @JDevlieghere wrote:

Why did you choose to make this a property of the RealFileSystem as opposed to subclassing the RealFileSystem to a PhysicalFileSystem? It seems like the latter would be more in line with the VFS design.

The reason the logic is interleaved like this: the operations aren't a simple "translation" into equivalent operations with absolute paths. Note that the returned files and statuses must report the relative paths provided, not the absolute ones we translate the actual IO into. So building the FS-with-own-WD as an aggregate or derived class of the FS-with-native-WD is going to be awkward and fragile. (And semantically, inheritance seems dubious here).
This could be a hierarchy of three classes where the abstract base tries not to express an opinion (or CRTP, or a template over a policy object), but I didn't find a pattern that seemed less confusing/indirect than just using a single class with if(). I don't expect the performance aspect of having to branch to matter at all.

Fair enough, thanks for the explanation!

[jfb]

I did a follow-up to this: D65986
It caused some unexpected breakage.

[sammccall]

Thanks for the heads up!
I think the breakage was rather from D62271 (when this was written, there was no plan to switch the default)

include/llvm/Support/VirtualFileSystem.h

public:
   /// \param Path A path that is modified to be an absolute path.
   /// \returns success if \a path has been made absolute, otherwise a
   ///          platform-specific error_code.
   std::error_code makeAbsolute(SmallVectorImpl<char> &Path) const;
 };
 
 /// Gets an \p vfs::FileSystem for the 'real' file system, as seen by
 /// the operating system.
+/// The working directory is linked to the process's working directory.
+/// (This is usually thread-hostile).
 IntrusiveRefCntPtr<FileSystem> getRealFileSystem();
 
+/// Create an \p vfs::FileSystem for the 'real' file system, as seen by
+/// the operating system.
+/// It has its own working directory, independent of (but initially equal to)
+/// that of the process.
+std::unique_ptr<FileSystem> createPhysicalFileSystem();
+
 /// A file system that allows overlaying one \p AbstractFileSystem on top
 /// of another.
 ///
 /// Consists of a stack of >=1 \p FileSystem objects, which are treated as being
 /// one merged file system. When there is a directory that exists in more than
 /// one file system, the \p OverlayFileSystem contains a directory containing
 /// the union of their contents.  The attributes (permissions, etc.) of the
 /// top-most (most recently added) directory are used.  When there is a file

lib/Support/VirtualFileSystem.cpp

 std::error_code RealFile::close() {
   std::error_code EC = sys::Process::SafelyCloseFileDescriptor(FD);
   FD = -1;
   return EC;
 }
 
 namespace {
 
-/// The file system according to your operating system.
+/// A file system according to your operating system.
+/// This may be linked to the process's working directory, or maintain its own.
+///
+/// Currently, its own working directory is emulated by storing the path and
+/// sending absolute paths to llvm::sys::fs:: functions.
+/// A more principled approach would be to push this down a level, modelling
+/// the working dir as an llvm::sys::fs::WorkingDir or similar.
+/// This would enable the use of openat()-style functions on some platforms.
 class RealFileSystem : public FileSystem {
 public:
+  explicit RealFileSystem(bool LinkCWDToProcess) {
+    if (!LinkCWDToProcess) {
+      SmallString<128> PWD, RealPWD;
+      if (llvm::sys::fs::current_path(PWD))
+        return; // Awful, but nothing to do here.
+      if (auto Err = llvm::sys::fs::real_path(PWD, RealPWD))
+        WD = {PWD, PWD};
+      else
+        WD = {PWD, RealPWD};
+    }
+  }
+
   ErrorOr<Status> status(const Twine &Path) override;
   ErrorOr<std::unique_ptr<File>> openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
 
   llvm::ErrorOr<std::string> getCurrentWorkingDirectory() const override;
   std::error_code setCurrentWorkingDirectory(const Twine &Path) override;
   std::error_code isLocal(const Twine &Path, bool &Result) override;
   std::error_code getRealPath(const Twine &Path,
                               SmallVectorImpl<char> &Output) const override;
 
 private:
-  mutable std::mutex CWDMutex;
-  mutable std::string CWDCache;
+  // If this FS has its own working dir, use it to make Path absolute.
+  // The returned twine is safe to use as long as both Storage and Path live.
+  Twine adjustPath(const Twine &Path, SmallVectorImpl<char> &Storage) const {
+    if (!WD)
+      return Path;
+    Path.toVector(Storage);
+    sys::fs::make_absolute(WD->Resolved, Storage);
+    return Storage;
+  }
+
+  struct WorkingDirectory {
+    // The current working directory, without symlinks resolved. (echo $PWD).
+    SmallString<128> Specified;
+    // The current working directory, with links resolved. (readlink .).
+    SmallString<128> Resolved;
+  };
+  Optional<WorkingDirectory> WD;
 };
 
 } // namespace
 
 ErrorOr<Status> RealFileSystem::status(const Twine &Path) {
+  SmallString<256> Storage;
   sys::fs::file_status RealStatus;
-  if (std::error_code EC = sys::fs::status(Path, RealStatus))
+  if (std::error_code EC =
+          sys::fs::status(adjustPath(Path, Storage), RealStatus))
     return EC;
   return Status::copyWithNewName(RealStatus, Path.str());
 }
 
 ErrorOr<std::unique_ptr<File>>
 RealFileSystem::openFileForRead(const Twine &Name) {
   int FD;
-  SmallString<256> RealName;
-  if (std::error_code EC =
-          sys::fs::openFileForRead(Name, FD, sys::fs::OF_None, &RealName))
+  SmallString<256> RealName, Storage;
+  if (std::error_code EC = sys::fs::openFileForRead(
+          adjustPath(Name, Storage), FD, sys::fs::OF_None, &RealName))
     return EC;
   return std::unique_ptr<File>(new RealFile(FD, Name.str(), RealName.str()));
 }
 
 llvm::ErrorOr<std::string> RealFileSystem::getCurrentWorkingDirectory() const {
-  std::lock_guard<std::mutex> Lock(CWDMutex);
-  if (!CWDCache.empty())
-    return CWDCache;
-  SmallString<256> Dir;
+  if (WD)
+    return WD->Specified.str();
+
+  SmallString<128> Dir;
   if (std::error_code EC = llvm::sys::fs::current_path(Dir))
     return EC;
-  CWDCache = Dir.str();
-  return CWDCache;
+  return Dir.str();
 }
 
 std::error_code RealFileSystem::setCurrentWorkingDirectory(const Twine &Path) {
-  // FIXME: chdir is thread hostile; on the other hand, creating the same
-  // behavior as chdir is complex: chdir resolves the path once, thus
-  // guaranteeing that all subsequent relative path operations work
-  // on the same path the original chdir resulted in. This makes a
-  // difference for example on network filesystems, where symlinks might be
-  // switched during runtime of the tool. Fixing this depends on having a
-  // file system abstraction that allows openat() style interactions.
-  if (auto EC = llvm::sys::fs::set_current_path(Path))
-    return EC;
+  if (!WD)
+    return llvm::sys::fs::set_current_path(Path);
 
-  // Invalidate cache.
-  std::lock_guard<std::mutex> Lock(CWDMutex);
-  CWDCache.clear();
+  SmallString<128> Absolute, Resolved, Storage;
+  adjustPath(Path, Storage).toVector(Absolute);
+  bool IsDir;
+  if (auto Err = llvm::sys::fs::is_directory(Absolute, IsDir))
+    return Err;
+  if (!IsDir)
+    return std::make_error_code(std::errc::not_a_directory);
+  if (auto Err = llvm::sys::fs::real_path(Absolute, Resolved))
+    return Err;
+  WD = {Absolute, Resolved};
   return std::error_code();
 }
 
 std::error_code RealFileSystem::isLocal(const Twine &Path, bool &Result) {
-  return llvm::sys::fs::is_local(Path, Result);
+  SmallString<256> Storage;
+  return llvm::sys::fs::is_local(adjustPath(Path, Storage), Result);
 }
 
 std::error_code
 RealFileSystem::getRealPath(const Twine &Path,
                             SmallVectorImpl<char> &Output) const {
-  return llvm::sys::fs::real_path(Path, Output);
+  SmallString<256> Storage;
+  return llvm::sys::fs::real_path(adjustPath(Path, Storage), Output);
 }
 
 IntrusiveRefCntPtr<FileSystem> vfs::getRealFileSystem() {
-  static IntrusiveRefCntPtr<FileSystem> FS = new RealFileSystem();
+  static IntrusiveRefCntPtr<FileSystem> FS(new RealFileSystem(true));
   return FS;
 }
 
+std::unique_ptr<FileSystem> vfs::createPhysicalFileSystem() {
+  return llvm::make_unique<RealFileSystem>(false);
+}
+
 namespace {
 
 class RealFSDirIter : public llvm::vfs::detail::DirIterImpl {
   llvm::sys::fs::directory_iterator Iter;
 
 public:
   RealFSDirIter(const Twine &Path, std::error_code &EC) : Iter(Path, EC) {
     if (Iter != llvm::sys::fs::directory_iterator())
std::error_code increment() override {
     return EC;
   }
 };
 
 } // namespace
 
 directory_iterator RealFileSystem::dir_begin(const Twine &Dir,
                                              std::error_code &EC) {
-  return directory_iterator(std::make_shared<RealFSDirIter>(Dir, EC));
+  SmallString<128> Storage;
+  return directory_iterator(
+      std::make_shared<RealFSDirIter>(adjustPath(Dir, Storage), EC));
 }
 
 //===-----------------------------------------------------------------------===/
 // OverlayFileSystem implementation
 //===-----------------------------------------------------------------------===/
 
 OverlayFileSystem::OverlayFileSystem(IntrusiveRefCntPtr<FileSystem> BaseFS) {
   FSList.push_back(std::move(BaseFS));

unittests/Support/VirtualFileSystemTest.cpp

struct ScopedLink {
   }
   ~ScopedLink() {
     if (Path != "") {
       EXPECT_FALSE(llvm::sys::fs::remove(Path.str()));
     }
   }
   operator StringRef() { return Path.str(); }
 };
+
+struct ScopedFile {
+  SmallString<128> Path;
+  ScopedFile(const Twine &Path, StringRef Contents) {
+    Path.toVector(this->Path);
+    std::error_code EC;
+    raw_fd_ostream OS(this->Path, EC);
+    EXPECT_FALSE(EC);
+    OS << Contents;
+    OS.flush();
+    EXPECT_FALSE(OS.error());
+    if (EC || OS.error())
+      this->Path = "";
+  }
+  ~ScopedFile() {
+    if (Path != "")
+      EXPECT_FALSE(llvm::sys::fs::remove(Path.str()));
+  }
+};
 } // end anonymous namespace
 
 TEST(VirtualFileSystemTest, BasicRealFSIteration) {
   ScopedDir TestDirectory("virtual-file-system-test", /*Unique*/ true);
   IntrusiveRefCntPtr<vfs::FileSystem> FS = vfs::getRealFileSystem();
 
   std::error_code EC;
   vfs::directory_iterator I = FS->dir_begin(Twine(TestDirectory), EC);
TEST(VirtualFileSystemTest, BasicRealFSIteration) {
   I.increment(EC);
   ASSERT_FALSE(EC);
   ASSERT_NE(vfs::directory_iterator(), I);
   EXPECT_TRUE(I->path().endswith("a") || I->path().endswith("c"));
   I.increment(EC);
   EXPECT_EQ(vfs::directory_iterator(), I);
 }
 
+TEST(VirtualFileSystemTest, MultipleWorkingDirs) {
+  // Our root contains a/aa, b/bb, c, where c is a link to a/.
+  // Run tests both in root/b/ and root/c/ (to test "normal" and symlink dirs).
+  // Interleave operations to show the working directories are independent.
+  ScopedDir Root("r", true), ADir(Root.Path + "/a"), BDir(Root.Path + "/b");
+  ScopedLink C(ADir.Path, Root.Path + "/c");
+  ScopedFile AA(ADir.Path + "/aa", "aaaa"), BB(BDir.Path + "/bb", "bbbb");
+  std::unique_ptr<vfs::FileSystem> BFS = vfs::createPhysicalFileSystem(),
+                                   CFS = vfs::createPhysicalFileSystem();
+
+  ASSERT_FALSE(BFS->setCurrentWorkingDirectory(BDir.Path));
+  ASSERT_FALSE(CFS->setCurrentWorkingDirectory(C.Path));
+  EXPECT_EQ(BDir.Path, *BFS->getCurrentWorkingDirectory());
+  EXPECT_EQ(C.Path, *CFS->getCurrentWorkingDirectory());
+
+  // openFileForRead(), indirectly.
+  auto BBuf = BFS->getBufferForFile("bb");
+  ASSERT_TRUE(BBuf);
+  EXPECT_EQ("bbbb", (*BBuf)->getBuffer());
+
+  auto ABuf = CFS->getBufferForFile("aa");
+  ASSERT_TRUE(ABuf);
+  EXPECT_EQ("aaaa", (*ABuf)->getBuffer());
+
+  // status()
+  auto BStat = BFS->status("bb");
+  ASSERT_TRUE(BStat);
+  EXPECT_EQ("bb", BStat->getName());
+
+  auto AStat = CFS->status("aa");
+  ASSERT_TRUE(AStat);
+  EXPECT_EQ("aa", AStat->getName()); // unresolved name
+
+  // getRealPath()
+  SmallString<128> BPath;
+  ASSERT_FALSE(BFS->getRealPath("bb", BPath));
+  EXPECT_EQ(BB.Path, BPath);
+
+  SmallString<128> APath;
+  ASSERT_FALSE(CFS->getRealPath("aa", APath));
+  EXPECT_EQ(AA.Path, APath); // Reports resolved name.
+
+  // dir_begin
+  std::error_code EC;
+  auto BIt = BFS->dir_begin(".", EC);
+  ASSERT_FALSE(EC);
+  ASSERT_NE(BIt, vfs::directory_iterator());
+  EXPECT_EQ((BDir.Path + "/./bb").str(), BIt->path());
+  BIt.increment(EC);
+  ASSERT_FALSE(EC);
+  ASSERT_EQ(BIt, vfs::directory_iterator());
+
+  auto CIt = CFS->dir_begin(".", EC);
+  ASSERT_FALSE(EC);
+  ASSERT_NE(CIt, vfs::directory_iterator());
+  EXPECT_EQ((ADir.Path + "/./aa").str(), CIt->path()); // Partly resolved name!
+  CIt.increment(EC); // Because likely to read through this path.
+  ASSERT_FALSE(EC);
+  ASSERT_EQ(CIt, vfs::directory_iterator());
+}
+
 #ifdef LLVM_ON_UNIX
 TEST(VirtualFileSystemTest, BrokenSymlinkRealFSIteration) {
   ScopedDir TestDirectory("virtual-file-system-test", /*Unique*/ true);
   IntrusiveRefCntPtr<vfs::FileSystem> FS = vfs::getRealFileSystem();
 
   ScopedLink _a("no_such_file", TestDirectory + "/a");
   ScopedDir _b(TestDirectory + "/b");
   ScopedLink _c("no_such_file", TestDirectory + "/c");