Move RedirectingFileSystem interface into header.

[JDevlieghere]

Summary

This moves the RedirectingFileSystem into the VFS header so we can extend it from LLVM with a custom implementation to get the external path.

[vsapsai]

How do you want it to work with symlinks? Would getRealPath be sufficient for your purpose?

[JDevlieghere]

In D54277#1292180, @vsapsai wrote:

How do you want it to work with symlinks? Would getRealPath be sufficient for your purpose?

I'm not sure how those two are related, other than the confusing name. As far as I can tell this only wraps the realpath(3) call, which is definitely much too expensive to do in the "real" file system case. For symlinks I wouldn't really care, it's up to the user to resolve them if they matter.

[vsapsai]

In D54277#1292267, @JDevlieghere wrote:

In D54277#1292180, @vsapsai wrote:

How do you want it to work with symlinks? Would getRealPath be sufficient for your purpose?

I'm not sure how those two are related, other than the confusing name. As far as I can tell this only wraps the realpath(3) call, which is definitely much too expensive to do in the "real" file system case. For symlinks I wouldn't really care, it's up to the user to resolve them if they matter.

OK. Now I understand that you prefer to avoid realpath as it is too expensive for this purpose.

Do you think Optional is sufficient for handling errors? On one hand it is a simpler interface but on the other hand it loses information why external path might be unavailable.

lib/Support/VirtualFileSystem.cpp

1906–1907 ↗

(On Diff #173229)

Need to double check but at the first glance I have reservations about a case with nested RedirectingFileSystem. Should getExternalPath go only through 1 level of indirection or all the way to real file system?

[JDevlieghere]

In D54277#1293988, @vsapsai wrote:

In D54277#1292267, @JDevlieghere wrote:

In D54277#1292180, @vsapsai wrote:

How do you want it to work with symlinks? Would getRealPath be sufficient for your purpose?

I'm not sure how those two are related, other than the confusing name. As far as I can tell this only wraps the realpath(3) call, which is definitely much too expensive to do in the "real" file system case. For symlinks I wouldn't really care, it's up to the user to resolve them if they matter.

OK. Now I understand that you prefer to avoid realpath as it is too expensive for this purpose.

Do you think Optional is sufficient for handling errors? On one hand it is a simpler interface but on the other hand it loses information why external path might be unavailable.

Yeah, I wasn't sure about this. Usually the client won't care what the error is, but there's no harm in returning an ErrorOr<>. I'll update the patch!

[JDevlieghere]

lib/Support/VirtualFileSystem.cpp

1906–1907 ↗

(On Diff #173229)

Good point. My use case would require all the way to the real FS.

[sammccall]

I don't understand the concept of an "external path". What is it in the general case, when would you want to pass a path through this function vs using it directly?

From the patch description, it sounds like it may be specific to LLDB and/or RedirectingFileSystem. If so, maybe it shouldn't be on the interface?

[JDevlieghere]

In D54277#1296175, @sammccall wrote:

I don't understand the concept of an "external path". What is it in the general case, when would you want to pass a path through this function vs using it directly?

From the patch description, it sounds like it may be specific to LLDB and/or RedirectingFileSystem. If so, maybe it shouldn't be on the interface?

It's true that this is only relevant for VFS implementations that are somehow backed by disk. I'm curious what you have in mind. How would it not be on the interface and still work transparently?

[sammccall]

In D54277#1296400, @JDevlieghere wrote:

In D54277#1296175, @sammccall wrote:

I don't understand the concept of an "external path". What is it in the general case, when would you want to pass a path through this function vs using it directly?

From the patch description, it sounds like it may be specific to LLDB and/or RedirectingFileSystem. If so, maybe it shouldn't be on the interface?

It's true that this is only relevant for VFS implementations that are somehow backed by disk. I'm curious what you have in mind. How would it not be on the interface and still work transparently?

I can't really answer these questions without knowing what an external path is or when you'd want to use it.
Could you describe it here or add some documentation to the patch?

[JDevlieghere]

In D54277#1296661, @sammccall wrote:

In D54277#1296400, @JDevlieghere wrote:

In D54277#1296175, @sammccall wrote:

I don't understand the concept of an "external path". What is it in the general case, when would you want to pass a path through this function vs using it directly?

From the patch description, it sounds like it may be specific to LLDB and/or RedirectingFileSystem. If so, maybe it shouldn't be on the interface?

It's true that this is only relevant for VFS implementations that are somehow backed by disk. I'm curious what you have in mind. How would it not be on the interface and still work transparently?

I can't really answer these questions without knowing what an external path is or when you'd want to use it.
Could you describe it here or add some documentation to the patch?

I'll try here first and we can always improve the patch later.

Let's start from the beginning: I'm adding reproducer functionality to LLDB. Just like in clang, we need files to be available somewhere else form where they actually live on disk. This should also be transparent to the consumer. I want to use the VFS for this rather than hand-rolling my own implementation in LLDB. I did a whole bunch of refactoring to make most of LLDB's file system operations go through the VFS.

One notable difference between llvm/clang and LLDB is how the latter deals with files. In addition to using memory buffers, LLDB manipulates the majority of files through file pointers (FILE*) and file descriptors. It's also a lot less explicit in whether it's opening a file for reading or writing. By this I mean in llvm we have functions like openMemoryBufferForWriting or openMemoryBufferForReading while in LLDB we would just pass a different open flag. While I would love to change how LLDB manipulates files, I simply don't have the bandwidth to do that right now. So that brings me to my current problem: how do I access files from LLDB through the VFS.

I considered extending the VFS API with functions to provide access to file pointers and descriptors, but decided against it. The reason is that (AFAIK) there's no way to access memory through either of those on both windows and unix. I believe you can have a FILE*to memory but that doesn't work on Windows. File descriptors to memory don't make sense on either.

The alternative, proposed in this patch, is to have an API in the VFS to access the "underlying" file through its "external path". This is the path specified in YAML by external-contents. So when the VFS thinks that the file foo lives at /root/foo but it actually lives at /root/whatever/reproducer/foo, I want to be able to get that latter path and open it with (f)open.

Please let me know if this makes things more clear.

[sammccall]

In D54277#1297570, @JDevlieghere wrote:

In D54277#1296661, @sammccall wrote:

In D54277#1296400, @JDevlieghere wrote:

In D54277#1296175, @sammccall wrote:

I don't understand the concept of an "external path". What is it in the general case, when would you want to pass a path through this function vs using it directly?

From the patch description, it sounds like it may be specific to LLDB and/or RedirectingFileSystem. If so, maybe it shouldn't be on the interface?

It's true that this is only relevant for VFS implementations that are somehow backed by disk. I'm curious what you have in mind. How would it not be on the interface and still work transparently?

I can't really answer these questions without knowing what an external path is or when you'd want to use it.
Could you describe it here or add some documentation to the patch?

I'll try here first and we can always improve the patch later.

One notable difference between llvm/clang and LLDB is how the latter deals with files. In addition to using memory buffers, LLDB manipulates the majority of files through file pointers (FILE*) and file descriptors. It's also a lot less explicit in whether it's opening a file for reading or writing. By this I mean in llvm we have functions like openMemoryBufferForWriting or openMemoryBufferForReading while in LLDB we would just pass a different open flag. While I would love to change how LLDB manipulates files, I simply don't have the bandwidth to do that right now. So that brings me to my current problem: how do I access files from LLDB through the VFS.

So if I understand correctly, you want a way to devirtualize the VFS: to take a VFS file and get a concrete FILE* out of it. (In this case, via the filename and fopen()).

The fact that VFS doesn't have any way to do this is an important property, it's what makes the file system virtual.
VFS allows code to work in the same way on OS-native, in-memory, or distributed filesystems, and getExternalPath would break that - any code that called it can only work when the files are 1:1 with native OS files. Code that wants to couple to the FS implementation in this way shouldn't work transparently with VFS, because it breaks the design goal that VFSes are substitutable.

For what you're trying to do, I'd suggest one of the following:

• use a concrete VFS implementation that maps files and exposes the functions you need (e.g. expose RedirectingFileSystem from VFS.h with an appropriate public interface)
• use the existing getRealPath if you know which concrete VFSes you're using and it happens to do what you want
• track the correspondence between virtual and real paths externally (where you set up the FS)
• decouple LLDB's IO from the concrete filesystem (use VFS file instead of FILE*)

[JDevlieghere]

Thanks for bearing with me on this, Sam. I really appreciate you taking the time to find the best solution that works for all of us.

In D54277#1298127, @sammccall wrote:

So if I understand correctly, you want a way to devirtualize the VFS: to take a VFS file and get a concrete FILE* out of it. (In this case, via the filename and fopen()).

The fact that VFS doesn't have any way to do this is an important property, it's what makes the file system virtual.
VFS allows code to work in the same way on OS-native, in-memory, or distributed filesystems, and getExternalPath would break that - any code that called it can only work when the files are 1:1 with native OS files. Code that wants to couple to the FS implementation in this way shouldn't work transparently with VFS, because it breaks the design goal that VFSes are substitutable.

For what you're trying to do, I'd suggest one of the following:

• use a concrete VFS implementation that maps files and exposes the functions you need (e.g. expose RedirectingFileSystem from VFS.h with an appropriate public interface)

I guess this could work if we ignore the concern Volodymyr raised. If we assume that the ExternalFS is the RealFileSystem this could work. The problem is that RedirectingFileSystem takes any FileSystem as its external file system so if that doesn't have the right API we can't "dig deeper".

• use the existing getRealPath if you know which concrete VFSes you're using and it happens to do what you want.

This is not a viable option because it defeats the purpose of the VFS.

• track the correspondence between virtual and real paths externally (where you set up the FS)

This is not a viable option because it defeats the purpose of the VFS.

• decouple LLDB's IO from the concrete filesystem (use VFS file instead of FILE*)

This is also not a viable option because, like I said earlier, this would be too big of a change for LLDB. In addition to the risk such a change brings, I don't have the bandwidth for it and even if I did I'm not sure it be feasible. Pavel said he looked into this and found that some of LLDB's dependencies formed a problem.

I'll update the patch to reflect what (1) would look like.

[JDevlieghere]

I've added an interface ExternalFileSystem (I'm open to a better name) which has a slightly different meaning than before as will become clear later.

There's currently two implementations of this interface:

• The real filesystem.
• The redirecting filesystem.

The implementation for both is trivial. For the real file system it's a no-op. For the redirecting file system it does the translation. It's different from the previous patch in that it follows only one level of indirection. If a RedirectingFileSystem is backed by an in-memory file system it still works, you just get the translated path.

I strongly believe the interface is better than exposing the RedirectingFileSystem because it (1) hides more abstraction and (2) allows me to transparently switch between the two in LLDB.

[sammccall]

Hi Jonas,

I'm not sure about this abstraction: because it's non-recursive, it's hard to give meaningful semantics to getExternalPath() - the thing that would be useful to LLDB (if abstraction-breaking) is "native OS path", but for a general VFS, getExternalPath may give a native path, fail, or give a string that's not a native OS path.
If dispatching between the real VFS and redirecting VFS in this way is the best thing for LLDB, I'd suggest exposing RedirectingFS and adding the indirection on the LLDB side.

I think *if* VFS is to support devirtualization, then putting it on the main VFS interface so it can be recursive as in your original patch is the right way to do it, and it needs to be explicitly optional. (i.e. supporting devirtualization is a dynamic property rather than a static one.)

For example:

// If \p Path corresponds directly to a native operating system path, attempts to return that path.
// (e.g. if this is the native file system, or merely remaps paths for an underlying native file system).
// May return `None` if there is no such native file or its path cannot be determined.
virtual Optional<std::string> getNativeOSPath(StringRef Path) { return None; }

(Note the lack of "error" return value: this makes it safe for VFSes that don't implement this to not distinguish between the "no file" and "can't be determined" case)

But I don't personally think this addition is a good idea at the VFS layer. The upside is it allows LLDB to use VFS in some places, which may eventually turn into true VFS support. The risk is that adding an escape hatch will mean over time we can no longer rely on VFS-aware code being compatible with all VFSes.

I think this could benefit from a discussion in the wider LLVM community - I'll start a discussion on llvm-dev.

[JDevlieghere]

Move RedirectingFileSystem interface into header so we can extend it from LLDB.

[JDevlieghere]

ping

[kristina]

I'm not sure if this is way out of scope for this patch, but if I understand correctly this could be used for extending it to, for example, handle things like Perforce-style paths which could indeed be useful, although I'm not sure if that is possible (ie. would //depot/asdf behave differently on Windows and Linux/Darwin/any other OS that uses Unix-style paths?). If this is with the intention to allow extending RedirectingFileSystem downstream then I think it may be worth splitting it up from the YAML based implementation altogether, allowing custom path handling where needed (as a separate downstream extension), with the concrete reference implementation of RedirectingFileSystem (the YAML based one) being a separate thing. This would make it more extensible from the POV of downstream forks that may desire to implement path redirection that is too complicated to do using YAML mappings, but still fundamentally rely on some sort of path remapping.

Do you think it would be worth decoupling RedirectingFileSystem from YAML for such use cases and exposing both RedirectingFileSystem and, for your use case, something like YAMLRedirectingFileSystem? Just a stray thought, not really sure if it makes sense or is out of scope here (I don't believe it is, since if the interface is being exposed, it may be worth rethinking how it is layered if it's to be exposed through LLVMSupport headers).

[JDevlieghere]

In D54277#1332261, @kristina wrote:

I'm not sure if this is way out of scope for this patch, but if I understand correctly this could be used for extending it to, for example, handle things like Perforce-style paths which could indeed be useful, although I'm not sure if that is possible (ie. would //depot/asdf behave differently on Windows and Linux/Darwin/any other OS that uses Unix-style paths?). If this is with the intention to allow extending RedirectingFileSystem downstream then I think it may be worth splitting it up from the YAML based implementation altogether, allowing custom path handling where needed (as a separate downstream extension), with the concrete reference implementation of RedirectingFileSystem (the YAML based one) being a separate thing. This would make it more extensible from the POV of downstream forks that may desire to implement path redirection that is too complicated to do using YAML mappings, but still fundamentally rely on some sort of path remapping.

Do you think it would be worth decoupling RedirectingFileSystem from YAML for such use cases and exposing both RedirectingFileSystem and, for your use case, something like YAMLRedirectingFileSystem? Just a stray thought, not really sure if it makes sense or is out of scope here (I don't believe it is, since if the interface is being exposed, it may be worth rethinking how it is layered if it's to be exposed through LLVMSupport headers).

It sounds like the scenario you're describing could be handled by just having a custom VFS implementation. What part of the RedirectingFileSystem would you repurpose for this? The reason I want to move it into the header is because I actually need the YAML stuff for LLDB's reproducers.

[JDevlieghere]

post-holiday ping

[sammccall]

This looks reasonable to me, thanks! Just questions about what's exposed where.
(Apologies, was out for all of december and just got back)

include/llvm/Support/VirtualFileSystem.h

496 ↗

(On Diff #176033)

Entry and EntryKind don't seem to apply to anything other than RedirectingVFS, so I think they could be nested within RedirectingVFS or at least given names tying them to that class (rather than to VFS itself)

624 ↗

(On Diff #176033)

does this stuff need to be protected rather than private? I can't find any subclasses

750 ↗

(On Diff #176033)

Unrelated/unused?

[JDevlieghere]

In D54277#1349694, @sammccall wrote:

This looks reasonable to me, thanks! Just questions about what's exposed where.
(Apologies, was out for all of december and just got back)

No worries, thanks for having a look!

I moved the classes and enum into the RedirectingFileSystem.

[JDevlieghere]

include/llvm/Support/VirtualFileSystem.h

750 ↗

(On Diff #176033)

Ah, this is needed for the LLDB reproducers. I'll split it off into a separate patch.

[sammccall]

(Really sorry, I thought I'd accepted this on the previous iteration)

[JDevlieghere]

In D54277#1357735, @sammccall wrote:

(Really sorry, I thought I'd accepted this on the previous iteration)

No worries & thank you!

llvm/trunk/include/llvm/Support/VirtualFileSystem.h

 #include "llvm/ADT/None.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Support/Chrono.h"
 #include "llvm/Support/ErrorOr.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/SourceMgr.h"
 #include <cassert>
 #include <cstdint>
 #include <ctime>
 #include <memory>
 #include <stack>
 #include <string>
 #include <system_error>

 struct YAMLVFSEntry {
   template <typename T1, typename T2>
   YAMLVFSEntry(T1 &&VPath, T2 &&RPath)
       : VPath(std::forward<T1>(VPath)), RPath(std::forward<T2>(RPath)) {}
   std::string VPath;
   std::string RPath;
 };
 
+class VFSFromYamlDirIterImpl;
+class RedirectingFileSystemParser;
+
+/// A virtual file system parsed from a YAML file.
+///
+/// Currently, this class allows creating virtual directories and mapping
+/// virtual file paths to existing external files, available in \c ExternalFS.
+///
+/// The basic structure of the parsed file is:
+/// \verbatim
+/// {
+///   'version': <version number>,
+///   <optional configuration>
+///   'roots': [
+///              <directory entries>
+///            ]
+/// }
+/// \endverbatim
+///
+/// All configuration options are optional.
+///   'case-sensitive': <boolean, default=true>
+///   'use-external-names': <boolean, default=true>
+///   'overlay-relative': <boolean, default=false>
+///   'fallthrough': <boolean, default=true>
+///
+/// Virtual directories are represented as
+/// \verbatim
+/// {
+///   'type': 'directory',
+///   'name': <string>,
+///   'contents': [ <file or directory entries> ]
+/// }
+/// \endverbatim
+///
+/// The default attributes for virtual directories are:
+/// \verbatim
+/// MTime = now() when created
+/// Perms = 0777
+/// User = Group = 0
+/// Size = 0
+/// UniqueID = unspecified unique value
+/// \endverbatim
+///
+/// Re-mapped files are represented as
+/// \verbatim
+/// {
+///   'type': 'file',
+///   'name': <string>,
+///   'use-external-name': <boolean> # Optional
+///   'external-contents': <path to external file>
+/// }
+/// \endverbatim
+///
+/// and inherit their attributes from the external contents.
+///
+/// In both cases, the 'name' field may contain multiple path components (e.g.
+/// /path/to/file). However, any directory that contains more than one child
+/// must be uniquely represented by a directory entry.
+class RedirectingFileSystem : public vfs::FileSystem {
+public:
+  enum EntryKind { EK_Directory, EK_File };
+
+  /// A single file or directory in the VFS.
+  class Entry {
+    EntryKind Kind;
+    std::string Name;
+
+  public:
+    Entry(EntryKind K, StringRef Name) : Kind(K), Name(Name) {}
+    virtual ~Entry() = default;
+
+    StringRef getName() const { return Name; }
+    EntryKind getKind() const { return Kind; }
+  };
+
+  class RedirectingDirectoryEntry : public Entry {
+    std::vector<std::unique_ptr<Entry>> Contents;
+    Status S;
+
+  public:
+    RedirectingDirectoryEntry(StringRef Name,
+                              std::vector<std::unique_ptr<Entry>> Contents,
+                              Status S)
+        : Entry(EK_Directory, Name), Contents(std::move(Contents)),
+          S(std::move(S)) {}
+    RedirectingDirectoryEntry(StringRef Name, Status S)
+        : Entry(EK_Directory, Name), S(std::move(S)) {}
+
+    Status getStatus() { return S; }
+
+    void addContent(std::unique_ptr<Entry> Content) {
+      Contents.push_back(std::move(Content));
+    }
+
+    Entry *getLastContent() const { return Contents.back().get(); }
+
+    using iterator = decltype(Contents)::iterator;
+
+    iterator contents_begin() { return Contents.begin(); }
+    iterator contents_end() { return Contents.end(); }
+
+    static bool classof(const Entry *E) { return E->getKind() == EK_Directory; }
+  };
+
+  class RedirectingFileEntry : public Entry {
+  public:
+    enum NameKind { NK_NotSet, NK_External, NK_Virtual };
+
+  private:
+    std::string ExternalContentsPath;
+    NameKind UseName;
+
+  public:
+    RedirectingFileEntry(StringRef Name, StringRef ExternalContentsPath,
+                         NameKind UseName)
+        : Entry(EK_File, Name), ExternalContentsPath(ExternalContentsPath),
+          UseName(UseName) {}
+
+    StringRef getExternalContentsPath() const { return ExternalContentsPath; }
+
+    /// whether to use the external path as the name for this file.
+    bool useExternalName(bool GlobalUseExternalName) const {
+      return UseName == NK_NotSet ? GlobalUseExternalName
+                                  : (UseName == NK_External);
+    }
+
+    NameKind getUseName() const { return UseName; }
+
+    static bool classof(const Entry *E) { return E->getKind() == EK_File; }
+  };
+
+private:
+  friend class VFSFromYamlDirIterImpl;
+  friend class RedirectingFileSystemParser;
+
+  /// The root(s) of the virtual file system.
+  std::vector<std::unique_ptr<Entry>> Roots;
+
+  /// The file system to use for external references.
+  IntrusiveRefCntPtr<FileSystem> ExternalFS;
+
+  /// If IsRelativeOverlay is set, this represents the directory
+  /// path that should be prefixed to each 'external-contents' entry
+  /// when reading from YAML files.
+  std::string ExternalContentsPrefixDir;
+
+  /// @name Configuration
+  /// @{
+
+  /// Whether to perform case-sensitive comparisons.
+  ///
+  /// Currently, case-insensitive matching only works correctly with ASCII.
+  bool CaseSensitive = true;
+
+  /// IsRelativeOverlay marks whether a ExternalContentsPrefixDir path must
+  /// be prefixed in every 'external-contents' when reading from YAML files.
+  bool IsRelativeOverlay = false;
+
+  /// Whether to use to use the value of 'external-contents' for the
+  /// names of files.  This global value is overridable on a per-file basis.
+  bool UseExternalNames = true;
+
+  /// Whether to attempt a file lookup in external file system after it wasn't
+  /// found in VFS.
+  bool IsFallthrough = true;
+  /// @}
+
+  /// Virtual file paths and external files could be canonicalized without "..",
+  /// "." and "./" in their paths. FIXME: some unittests currently fail on
+  /// win32 when using remove_dots and remove_leading_dotslash on paths.
+  bool UseCanonicalizedPaths =
+#ifdef _WIN32
+      false;
+#else
+      true;
+#endif
+
+  RedirectingFileSystem(IntrusiveRefCntPtr<FileSystem> ExternalFS)
+      : ExternalFS(std::move(ExternalFS)) {}
+
+  /// Looks up the path <tt>[Start, End)</tt> in \p From, possibly
+  /// recursing into the contents of \p From if it is a directory.
+  ErrorOr<Entry *> lookupPath(llvm::sys::path::const_iterator Start,
+                              llvm::sys::path::const_iterator End,
+                              Entry *From) const;
+
+  /// Get the status of a given an \c Entry.
+  ErrorOr<Status> status(const Twine &Path, Entry *E);
+
+public:
+  /// Looks up \p Path in \c Roots.
+  ErrorOr<Entry *> lookupPath(const Twine &Path) const;
+
+  /// Parses \p Buffer, which is expected to be in YAML format and
+  /// returns a virtual file system representing its contents.
+  static RedirectingFileSystem *
+  create(std::unique_ptr<MemoryBuffer> Buffer,
+         SourceMgr::DiagHandlerTy DiagHandler, StringRef YAMLFilePath,
+         void *DiagContext, IntrusiveRefCntPtr<FileSystem> ExternalFS);
+
+  ErrorOr<Status> status(const Twine &Path) override;
+  ErrorOr<std::unique_ptr<File>> openFileForRead(const Twine &Path) override;
+
+  std::error_code getRealPath(const Twine &Path,
+                              SmallVectorImpl<char> &Output) const override;
+
+  llvm::ErrorOr<std::string> getCurrentWorkingDirectory() const override;
+
+  std::error_code setCurrentWorkingDirectory(const Twine &Path) override;
+
+  std::error_code isLocal(const Twine &Path, bool &Result) override;
+
+  directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
+
+  void setExternalContentsPrefixDir(StringRef PrefixDir);
+
+  StringRef getExternalContentsPrefixDir() const;
+
+#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
+  LLVM_DUMP_METHOD void dump() const;
+  LLVM_DUMP_METHOD void dumpEntry(Entry *E, int NumSpaces = 0) const;
+#endif
+};
+
 /// Collect all pairs of <virtual path, real path> entries from the
 /// \p YAMLFilePath. This is used by the module dependency collector to forward
 /// the entries into the reproducer output VFS YAML file.
 void collectVFSFromYAML(
     std::unique_ptr<llvm::MemoryBuffer> Buffer,
     llvm::SourceMgr::DiagHandlerTy DiagHandler, StringRef YAMLFilePath,
     SmallVectorImpl<YAMLVFSEntry> &CollectedEntries,
     void *DiagContext = nullptr,

llvm/trunk/lib/Support/VirtualFileSystem.cpp

 
 } // namespace vfs
 } // namespace llvm
 
 //===-----------------------------------------------------------------------===/
 // RedirectingFileSystem implementation
 //===-----------------------------------------------------------------------===/
 
-namespace {
-
-enum EntryKind { EK_Directory, EK_File };
-
-/// A single file or directory in the VFS.
-class Entry {
-  EntryKind Kind;
-  std::string Name;
-
-public:
-  Entry(EntryKind K, StringRef Name) : Kind(K), Name(Name) {}
-  virtual ~Entry() = default;
-
-  StringRef getName() const { return Name; }
-  EntryKind getKind() const { return Kind; }
-};
-
-class RedirectingDirectoryEntry : public Entry {
-  std::vector<std::unique_ptr<Entry>> Contents;
-  Status S;
-
-public:
-  RedirectingDirectoryEntry(StringRef Name,
-                            std::vector<std::unique_ptr<Entry>> Contents,
-                            Status S)
-      : Entry(EK_Directory, Name), Contents(std::move(Contents)),
-        S(std::move(S)) {}
-  RedirectingDirectoryEntry(StringRef Name, Status S)
-      : Entry(EK_Directory, Name), S(std::move(S)) {}
-
-  Status getStatus() { return S; }
-
-  void addContent(std::unique_ptr<Entry> Content) {
-    Contents.push_back(std::move(Content));
-  }
-
-  Entry *getLastContent() const { return Contents.back().get(); }
-
-  using iterator = decltype(Contents)::iterator;
-
-  iterator contents_begin() { return Contents.begin(); }
-  iterator contents_end() { return Contents.end(); }
-
-  static bool classof(const Entry *E) { return E->getKind() == EK_Directory; }
-};
-
-class RedirectingFileEntry : public Entry {
-public:
-  enum NameKind { NK_NotSet, NK_External, NK_Virtual };
-
-private:
-  std::string ExternalContentsPath;
-  NameKind UseName;
-
-public:
-  RedirectingFileEntry(StringRef Name, StringRef ExternalContentsPath,
-                       NameKind UseName)
-      : Entry(EK_File, Name), ExternalContentsPath(ExternalContentsPath),
-        UseName(UseName) {}
-
-  StringRef getExternalContentsPath() const { return ExternalContentsPath; }
-
-  /// whether to use the external path as the name for this file.
-  bool useExternalName(bool GlobalUseExternalName) const {
-    return UseName == NK_NotSet ? GlobalUseExternalName
-                                : (UseName == NK_External);
-  }
-
-  NameKind getUseName() const { return UseName; }
-
-  static bool classof(const Entry *E) { return E->getKind() == EK_File; }
-};
-
 // FIXME: reuse implementation common with OverlayFSDirIterImpl as these
 // iterators are conceptually similar.
-class VFSFromYamlDirIterImpl : public llvm::vfs::detail::DirIterImpl {
+class llvm::vfs::VFSFromYamlDirIterImpl
+    : public llvm::vfs::detail::DirIterImpl {
   std::string Dir;
-  RedirectingDirectoryEntry::iterator Current, End;
+  RedirectingFileSystem::RedirectingDirectoryEntry::iterator Current, End;
 
   // To handle 'fallthrough' mode we need to iterate at first through
   // RedirectingDirectoryEntry and then through ExternalFS. These operations are
   // done sequentially, we just need to keep a track of what kind of iteration
   // we are currently performing.
 
   /// Flag telling if we should iterate through ExternalFS or stop at the last
   /// RedirectingDirectoryEntry::iterator.
class llvm::vfs::VFSFromYamlDirIterImpl
   std::error_code incrementImpl(bool IsFirstTime);
   /// Responsible for RedirectingDirectoryEntry iteration.
   std::error_code incrementContent(bool IsFirstTime);
   /// Responsible for ExternalFS iteration.
   std::error_code incrementExternal();
   /// @}
 
 public:
-  VFSFromYamlDirIterImpl(const Twine &Path,
-                         RedirectingDirectoryEntry::iterator Begin,
-                         RedirectingDirectoryEntry::iterator End,
-                         bool IterateExternalFS, FileSystem &ExternalFS,
-                         std::error_code &EC);
+  VFSFromYamlDirIterImpl(
+      const Twine &Path,
+      RedirectingFileSystem::RedirectingDirectoryEntry::iterator Begin,
+      RedirectingFileSystem::RedirectingDirectoryEntry::iterator End,
+      bool IterateExternalFS, FileSystem &ExternalFS, std::error_code &EC);
 
   std::error_code increment() override;
 };
 
-/// A virtual file system parsed from a YAML file.
-///
-/// Currently, this class allows creating virtual directories and mapping
-/// virtual file paths to existing external files, available in \c ExternalFS.
-///
-/// The basic structure of the parsed file is:
-/// \verbatim
-/// {
-///   'version': <version number>,
-///   <optional configuration>
-///   'roots': [
-///              <directory entries>
-///            ]
-/// }
-/// \endverbatim
-///
-/// All configuration options are optional.
-///   'case-sensitive': <boolean, default=true>
-///   'use-external-names': <boolean, default=true>
-///   'overlay-relative': <boolean, default=false>
-///   'fallthrough': <boolean, default=true>
-///
-/// Virtual directories are represented as
-/// \verbatim
-/// {
-///   'type': 'directory',
-///   'name': <string>,
-///   'contents': [ <file or directory entries> ]
-/// }
-/// \endverbatim
-///
-/// The default attributes for virtual directories are:
-/// \verbatim
-/// MTime = now() when created
-/// Perms = 0777
-/// User = Group = 0
-/// Size = 0
-/// UniqueID = unspecified unique value
-/// \endverbatim
-///
-/// Re-mapped files are represented as
-/// \verbatim
-/// {
-///   'type': 'file',
-///   'name': <string>,
-///   'use-external-name': <boolean> # Optional
-///   'external-contents': <path to external file>
-/// }
-/// \endverbatim
-///
-/// and inherit their attributes from the external contents.
-///
-/// In both cases, the 'name' field may contain multiple path components (e.g.
-/// /path/to/file). However, any directory that contains more than one child
-/// must be uniquely represented by a directory entry.
-class RedirectingFileSystem : public vfs::FileSystem {
-  friend class RedirectingFileSystemParser;
-
-  /// The root(s) of the virtual file system.
-  std::vector<std::unique_ptr<Entry>> Roots;
-
-  /// The file system to use for external references.
-  IntrusiveRefCntPtr<FileSystem> ExternalFS;
-
-  /// If IsRelativeOverlay is set, this represents the directory
-  /// path that should be prefixed to each 'external-contents' entry
-  /// when reading from YAML files.
-  std::string ExternalContentsPrefixDir;
-
-  /// @name Configuration
-  /// @{
-
-  /// Whether to perform case-sensitive comparisons.
-  ///
-  /// Currently, case-insensitive matching only works correctly with ASCII.
-  bool CaseSensitive = true;
-
-  /// IsRelativeOverlay marks whether a ExternalContentsPrefixDir path must
-  /// be prefixed in every 'external-contents' when reading from YAML files.
-  bool IsRelativeOverlay = false;
-
-  /// Whether to use to use the value of 'external-contents' for the
-  /// names of files.  This global value is overridable on a per-file basis.
-  bool UseExternalNames = true;
-
-  /// Whether to attempt a file lookup in external file system after it wasn't
-  /// found in VFS.
-  bool IsFallthrough = true;
-  /// @}
-
-  /// Virtual file paths and external files could be canonicalized without "..",
-  /// "." and "./" in their paths. FIXME: some unittests currently fail on
-  /// win32 when using remove_dots and remove_leading_dotslash on paths.
-  bool UseCanonicalizedPaths =
-#ifdef _WIN32
-      false;
-#else
-      true;
-#endif
-
-private:
-  RedirectingFileSystem(IntrusiveRefCntPtr<FileSystem> ExternalFS)
-      : ExternalFS(std::move(ExternalFS)) {}
-
-  /// Looks up the path <tt>[Start, End)</tt> in \p From, possibly
-  /// recursing into the contents of \p From if it is a directory.
-  ErrorOr<Entry *> lookupPath(sys::path::const_iterator Start,
-                              sys::path::const_iterator End, Entry *From) const;
-
-  /// Get the status of a given an \c Entry.
-  ErrorOr<Status> status(const Twine &Path, Entry *E);
-
-public:
-  /// Looks up \p Path in \c Roots.
-  ErrorOr<Entry *> lookupPath(const Twine &Path) const;
-
-  /// Parses \p Buffer, which is expected to be in YAML format and
-  /// returns a virtual file system representing its contents.
-  static RedirectingFileSystem *
-  create(std::unique_ptr<MemoryBuffer> Buffer,
-         SourceMgr::DiagHandlerTy DiagHandler, StringRef YAMLFilePath,
-         void *DiagContext, IntrusiveRefCntPtr<FileSystem> ExternalFS);
-
-  ErrorOr<Status> status(const Twine &Path) override;
-  ErrorOr<std::unique_ptr<File>> openFileForRead(const Twine &Path) override;
-
-  std::error_code getRealPath(const Twine &Path,
-                              SmallVectorImpl<char> &Output) const override;
-
-  llvm::ErrorOr<std::string> getCurrentWorkingDirectory() const override {
-    return ExternalFS->getCurrentWorkingDirectory();
-  }
+llvm::ErrorOr<std::string>
+RedirectingFileSystem::getCurrentWorkingDirectory() const {
 
-  std::error_code setCurrentWorkingDirectory(const Twine &Path) override {
-    return ExternalFS->setCurrentWorkingDirectory(Path);
-  }
+std::error_code
+RedirectingFileSystem::setCurrentWorkingDirectory(const Twine &Path) {
 
-  std::error_code isLocal(const Twine &Path, bool &Result) override {
-    return ExternalFS->isLocal(Path, Result);
-  }
+std::error_code RedirectingFileSystem::isLocal(const Twine &Path,
+                                               bool &Result) {
 
-  directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override {
-    ErrorOr<Entry *> E = lookupPath(Dir);
-    if (!E) {
-      EC = E.getError();
-      if (IsFallthrough && EC == errc::no_such_file_or_directory)
-        return ExternalFS->dir_begin(Dir, EC);
-      return {};
-    }
-    ErrorOr<Status> S = status(Dir, *E);
-    if (!S) {
-      EC = S.getError();
-      return {};
-    }
-    if (!S->isDirectory()) {
-      EC = std::error_code(static_cast<int>(errc::not_a_directory),
-                           std::system_category());
-      return {};
-    }
+directory_iterator RedirectingFileSystem::dir_begin(const Twine &Dir,
+                                                    std::error_code &EC) {
+  ErrorOr<RedirectingFileSystem::Entry *> E = lookupPath(Dir);
 
-    auto *D = cast<RedirectingDirectoryEntry>(*E);
-    return directory_iterator(std::make_shared<VFSFromYamlDirIterImpl>(
-        Dir, D->contents_begin(), D->contents_end(),
-        /*IterateExternalFS=*/IsFallthrough, *ExternalFS, EC));
-  }
+  auto *D = cast<RedirectingFileSystem::RedirectingDirectoryEntry>(*E);
 
-  void setExternalContentsPrefixDir(StringRef PrefixDir) {
-    ExternalContentsPrefixDir = PrefixDir.str();
-  }
+void RedirectingFileSystem::setExternalContentsPrefixDir(StringRef PrefixDir) {
 
-  StringRef getExternalContentsPrefixDir() const {
-    return ExternalContentsPrefixDir;
-  }
+StringRef RedirectingFileSystem::getExternalContentsPrefixDir() const {
 
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
-  LLVM_DUMP_METHOD void dump() const {
-    for (const auto &Root : Roots)
-      dumpEntry(Root.get());
-  }
+LLVM_DUMP_METHOD void RedirectingFileSystem::dump() const {
 
-  LLVM_DUMP_METHOD void dumpEntry(Entry *E, int NumSpaces = 0) const {
-    StringRef Name = E->getName();
-    for (int i = 0, e = NumSpaces; i < e; ++i)
-      dbgs() << " ";
-    dbgs() << "'" << Name.str().c_str() << "'"
-           << "\n";
+LLVM_DUMP_METHOD void
+RedirectingFileSystem::dumpEntry(RedirectingFileSystem::Entry *E,
+                                 int NumSpaces) const {
 
-    if (E->getKind() == EK_Directory) {
-      auto *DE = dyn_cast<RedirectingDirectoryEntry>(E);
-      assert(DE && "Should be a directory");
+  if (E->getKind() == RedirectingFileSystem::EK_Directory) {
+    auto *DE = dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(E);
 
-      for (std::unique_ptr<Entry> &SubEntry :
-           llvm::make_range(DE->contents_begin(), DE->contents_end()))
-        dumpEntry(SubEntry.get(), NumSpaces + 2);
-    }
-  }
 #endif
-};
 
 /// A helper class to hold the common YAML parsing state.
-class RedirectingFileSystemParser {
+class llvm::vfs::RedirectingFileSystemParser {
   yaml::Stream &Stream;
 
   void error(yaml::Node *N, const Twine &Msg) { Stream.printError(N, Msg); }
 
   // false on error
   bool parseScalarString(yaml::Node *N, StringRef &Result,
                          SmallVectorImpl<char> &Storage) {
     const auto *S = dyn_cast<yaml::ScalarNode>(N);
for (const auto &I : Keys) {
       if (I.second.Required && !I.second.Seen) {
         error(Obj, Twine("missing key '") + I.first + "'");
         return false;
       }
     }
     return true;
   }
 
-  Entry *lookupOrCreateEntry(RedirectingFileSystem *FS, StringRef Name,
-                             Entry *ParentEntry = nullptr) {
+  RedirectingFileSystem::Entry *
+  lookupOrCreateEntry(RedirectingFileSystem *FS, StringRef Name,
+                      RedirectingFileSystem::Entry *ParentEntry = nullptr) {
     if (!ParentEntry) { // Look for a existent root
       for (const auto &Root : FS->Roots) {
         if (Name.equals(Root->getName())) {
           ParentEntry = Root.get();
           return ParentEntry;
         }
       }
     } else { // Advance to the next component
-      auto *DE = dyn_cast<RedirectingDirectoryEntry>(ParentEntry);
-      for (std::unique_ptr<Entry> &Content :
+      auto *DE = dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(
+          ParentEntry);
+      for (std::unique_ptr<RedirectingFileSystem::Entry> &Content :
            llvm::make_range(DE->contents_begin(), DE->contents_end())) {
-        auto *DirContent = dyn_cast<RedirectingDirectoryEntry>(Content.get());
+        auto *DirContent =
+            dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(
+                Content.get());
         if (DirContent && Name.equals(Content->getName()))
           return DirContent;
       }
     }
 
     // ... or create a new one
-    std::unique_ptr<Entry> E = llvm::make_unique<RedirectingDirectoryEntry>(
-        Name,
-        Status("", getNextVirtualUniqueID(), std::chrono::system_clock::now(),
-               0, 0, 0, file_type::directory_file, sys::fs::all_all));
+    std::unique_ptr<RedirectingFileSystem::Entry> E =
+        llvm::make_unique<RedirectingFileSystem::RedirectingDirectoryEntry>(
+            Name, Status("", getNextVirtualUniqueID(),
+                         std::chrono::system_clock::now(), 0, 0, 0,
+                         file_type::directory_file, sys::fs::all_all));
 
     if (!ParentEntry) { // Add a new root to the overlay
       FS->Roots.push_back(std::move(E));
       ParentEntry = FS->Roots.back().get();
       return ParentEntry;
     }
 
-    auto *DE = dyn_cast<RedirectingDirectoryEntry>(ParentEntry);
+    auto *DE =
+        dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(ParentEntry);
     DE->addContent(std::move(E));
     return DE->getLastContent();
   }
 
-  void uniqueOverlayTree(RedirectingFileSystem *FS, Entry *SrcE,
-                         Entry *NewParentE = nullptr) {
+  void uniqueOverlayTree(RedirectingFileSystem *FS,
+                         RedirectingFileSystem::Entry *SrcE,
+                         RedirectingFileSystem::Entry *NewParentE = nullptr) {
     StringRef Name = SrcE->getName();
     switch (SrcE->getKind()) {
-    case EK_Directory: {
-      auto *DE = dyn_cast<RedirectingDirectoryEntry>(SrcE);
+    case RedirectingFileSystem::EK_Directory: {
+      auto *DE =
+          dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(SrcE);
       assert(DE && "Must be a directory");
       // Empty directories could be present in the YAML as a way to
       // describe a file for a current directory after some of its subdir
       // is parsed. This only leads to redundant walks, ignore it.
       if (!Name.empty())
         NewParentE = lookupOrCreateEntry(FS, Name, NewParentE);
-      for (std::unique_ptr<Entry> &SubEntry :
+      for (std::unique_ptr<RedirectingFileSystem::Entry> &SubEntry :
            llvm::make_range(DE->contents_begin(), DE->contents_end()))
         uniqueOverlayTree(FS, SubEntry.get(), NewParentE);
       break;
     }
-    case EK_File: {
-      auto *FE = dyn_cast<RedirectingFileEntry>(SrcE);
+    case RedirectingFileSystem::EK_File: {
+      auto *FE = dyn_cast<RedirectingFileSystem::RedirectingFileEntry>(SrcE);
       assert(FE && "Must be a file");
       assert(NewParentE && "Parent entry must exist");
-      auto *DE = dyn_cast<RedirectingDirectoryEntry>(NewParentE);
-      DE->addContent(llvm::make_unique<RedirectingFileEntry>(
+      auto *DE = dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(
+          NewParentE);
+      DE->addContent(
+          llvm::make_unique<RedirectingFileSystem::RedirectingFileEntry>(
               Name, FE->getExternalContentsPath(), FE->getUseName()));
       break;
     }
     }
   }
 
-  std::unique_ptr<Entry> parseEntry(yaml::Node *N, RedirectingFileSystem *FS,
-                                    bool IsRootEntry) {
+  std::unique_ptr<RedirectingFileSystem::Entry>
+  parseEntry(yaml::Node *N, RedirectingFileSystem *FS, bool IsRootEntry) {
     auto *M = dyn_cast<yaml::MappingNode>(N);
     if (!M) {
       error(N, "expected mapping node for file or directory entry");
       return nullptr;
     }
 
     KeyStatusPair Fields[] = {
         KeyStatusPair("name", true),
         KeyStatusPair("type", true),
         KeyStatusPair("contents", false),
         KeyStatusPair("external-contents", false),
         KeyStatusPair("use-external-name", false),
     };
 
     DenseMap<StringRef, KeyStatus> Keys(std::begin(Fields), std::end(Fields));
 
     bool HasContents = false; // external or otherwise
-    std::vector<std::unique_ptr<Entry>> EntryArrayContents;
+    std::vector<std::unique_ptr<RedirectingFileSystem::Entry>>
+        EntryArrayContents;
     std::string ExternalContentsPath;
     std::string Name;
     yaml::Node *NameValueNode;
-    auto UseExternalName = RedirectingFileEntry::NK_NotSet;
-    EntryKind Kind;
+    auto UseExternalName =
+        RedirectingFileSystem::RedirectingFileEntry::NK_NotSet;
+    RedirectingFileSystem::EntryKind Kind;
 
     for (auto &I : *M) {
       StringRef Key;
       // Reuse the buffer for key and value, since we don't look at key after
       // parsing value.
       SmallString<256> Buffer;
       if (!parseScalarString(I.getKey(), Key, Buffer))
         return nullptr;
for (auto &I : *M) {
           Name = Path.str();
         } else {
           Name = Value;
         }
       } else if (Key == "type") {
         if (!parseScalarString(I.getValue(), Value, Buffer))
           return nullptr;
         if (Value == "file")
-          Kind = EK_File;
+          Kind = RedirectingFileSystem::EK_File;
         else if (Value == "directory")
-          Kind = EK_Directory;
+          Kind = RedirectingFileSystem::EK_Directory;
         else {
           error(I.getValue(), "unknown value for 'type'");
           return nullptr;
         }
       } else if (Key == "contents") {
         if (HasContents) {
           error(I.getKey(),
                 "entry already has 'contents' or 'external-contents'");
           return nullptr;
         }
         HasContents = true;
         auto *Contents = dyn_cast<yaml::SequenceNode>(I.getValue());
         if (!Contents) {
           // FIXME: this is only for directories, what about files?
           error(I.getValue(), "expected array");
           return nullptr;
         }
 
         for (auto &I : *Contents) {
-          if (std::unique_ptr<Entry> E =
+          if (std::unique_ptr<RedirectingFileSystem::Entry> E =
                   parseEntry(&I, FS, /*IsRootEntry*/ false))
             EntryArrayContents.push_back(std::move(E));
           else
             return nullptr;
         }
       } else if (Key == "external-contents") {
         if (HasContents) {
           error(I.getKey(),
for (auto &I : *M) {
           FullPath = sys::path::remove_leading_dotslash(FullPath);
           sys::path::remove_dots(FullPath, /*remove_dot_dot=*/true);
         }
         ExternalContentsPath = FullPath.str();
       } else if (Key == "use-external-name") {
         bool Val;
         if (!parseScalarBool(I.getValue(), Val))
           return nullptr;
-        UseExternalName = Val ? RedirectingFileEntry::NK_External
-                              : RedirectingFileEntry::NK_Virtual;
+        UseExternalName =
+            Val ? RedirectingFileSystem::RedirectingFileEntry::NK_External
+                : RedirectingFileSystem::RedirectingFileEntry::NK_Virtual;
       } else {
         llvm_unreachable("key missing from Keys");
       }
     }
 
     if (Stream.failed())
       return nullptr;
 
     // check for missing keys
     if (!HasContents) {
       error(N, "missing key 'contents' or 'external-contents'");
       return nullptr;
     }
     if (!checkMissingKeys(N, Keys))
       return nullptr;
 
     // check invalid configuration
-    if (Kind == EK_Directory &&
-        UseExternalName != RedirectingFileEntry::NK_NotSet) {
+    if (Kind == RedirectingFileSystem::EK_Directory &&
+        UseExternalName !=
+            RedirectingFileSystem::RedirectingFileEntry::NK_NotSet) {
       error(N, "'use-external-name' is not supported for directories");
       return nullptr;
     }
 
     if (IsRootEntry && !sys::path::is_absolute(Name)) {
       assert(NameValueNode && "Name presence should be checked earlier");
       error(NameValueNode,
             "entry with relative path at the root level is not discoverable");
       return nullptr;
     }
 
     // Remove trailing slash(es), being careful not to remove the root path
     StringRef Trimmed(Name);
     size_t RootPathLen = sys::path::root_path(Trimmed).size();
     while (Trimmed.size() > RootPathLen &&
            sys::path::is_separator(Trimmed.back()))
       Trimmed = Trimmed.slice(0, Trimmed.size() - 1);
     // Get the last component
     StringRef LastComponent = sys::path::filename(Trimmed);
 
-    std::unique_ptr<Entry> Result;
+    std::unique_ptr<RedirectingFileSystem::Entry> Result;
     switch (Kind) {
-    case EK_File:
-      Result = llvm::make_unique<RedirectingFileEntry>(
+    case RedirectingFileSystem::EK_File:
+      Result = llvm::make_unique<RedirectingFileSystem::RedirectingFileEntry>(
           LastComponent, std::move(ExternalContentsPath), UseExternalName);
       break;
-    case EK_Directory:
-      Result = llvm::make_unique<RedirectingDirectoryEntry>(
+    case RedirectingFileSystem::EK_Directory:
+      Result =
+          llvm::make_unique<RedirectingFileSystem::RedirectingDirectoryEntry>(
               LastComponent, std::move(EntryArrayContents),
-          Status("", getNextVirtualUniqueID(), std::chrono::system_clock::now(),
-                 0, 0, 0, file_type::directory_file, sys::fs::all_all));
+              Status("", getNextVirtualUniqueID(),
+                     std::chrono::system_clock::now(), 0, 0, 0,
+                     file_type::directory_file, sys::fs::all_all));
       break;
     }
 
     StringRef Parent = sys::path::parent_path(Trimmed);
     if (Parent.empty())
       return Result;
 
     // if 'name' contains multiple components, create implicit directory entries
     for (sys::path::reverse_iterator I = sys::path::rbegin(Parent),
                                      E = sys::path::rend(Parent);
          I != E; ++I) {
-      std::vector<std::unique_ptr<Entry>> Entries;
+      std::vector<std::unique_ptr<RedirectingFileSystem::Entry>> Entries;
       Entries.push_back(std::move(Result));
-      Result = llvm::make_unique<RedirectingDirectoryEntry>(
+      Result =
+          llvm::make_unique<RedirectingFileSystem::RedirectingDirectoryEntry>(
               *I, std::move(Entries),
-          Status("", getNextVirtualUniqueID(), std::chrono::system_clock::now(),
-                 0, 0, 0, file_type::directory_file, sys::fs::all_all));
+              Status("", getNextVirtualUniqueID(),
+                     std::chrono::system_clock::now(), 0, 0, 0,
+                     file_type::directory_file, sys::fs::all_all));
     }
     return Result;
   }
 
 public:
   RedirectingFileSystemParser(yaml::Stream &S) : Stream(S) {}
 
   // false on error
KeyStatusPair Fields[] = {
         KeyStatusPair("case-sensitive", false),
         KeyStatusPair("use-external-names", false),
         KeyStatusPair("overlay-relative", false),
         KeyStatusPair("fallthrough", false),
         KeyStatusPair("roots", true),
     };
 
     DenseMap<StringRef, KeyStatus> Keys(std::begin(Fields), std::end(Fields));
-    std::vector<std::unique_ptr<Entry>> RootEntries;
+    std::vector<std::unique_ptr<RedirectingFileSystem::Entry>> RootEntries;
 
     // Parse configuration and 'roots'
     for (auto &I : *Top) {
       SmallString<10> KeyBuffer;
       StringRef Key;
       if (!parseScalarString(I.getKey(), Key, KeyBuffer))
         return false;
 
       if (!checkDuplicateOrUnknownKey(I.getKey(), Key, Keys))
         return false;
 
       if (Key == "roots") {
         auto *Roots = dyn_cast<yaml::SequenceNode>(I.getValue());
         if (!Roots) {
           error(I.getValue(), "expected array");
           return false;
         }
 
         for (auto &I : *Roots) {
-          if (std::unique_ptr<Entry> E =
+          if (std::unique_ptr<RedirectingFileSystem::Entry> E =
                   parseEntry(&I, FS, /*IsRootEntry*/ true))
             RootEntries.push_back(std::move(E));
           else
             return false;
         }
       } else if (Key == "version") {
         StringRef VersionString;
         SmallString<4> Storage;
bool parse(yaml::Node *Root, RedirectingFileSystem *FS) {
     // inside the VFS.
     for (auto &E : RootEntries)
       uniqueOverlayTree(FS, E.get());
 
     return true;
   }
 };
 
-} // namespace
-
 RedirectingFileSystem *
 RedirectingFileSystem::create(std::unique_ptr<MemoryBuffer> Buffer,
                               SourceMgr::DiagHandlerTy DiagHandler,
                               StringRef YAMLFilePath, void *DiagContext,
                               IntrusiveRefCntPtr<FileSystem> ExternalFS) {
   SourceMgr SM;
   yaml::Stream Stream(Buffer->getMemBufferRef(), SM);
 
RedirectingFileSystem::create(std::unique_ptr<MemoryBuffer> Buffer,
   }
 
   if (!P.parse(Root, FS.get()))
     return nullptr;
 
   return FS.release();
 }
 
-ErrorOr<Entry *> RedirectingFileSystem::lookupPath(const Twine &Path_) const {
+ErrorOr<RedirectingFileSystem::Entry *>
+RedirectingFileSystem::lookupPath(const Twine &Path_) const {
   SmallString<256> Path;
   Path_.toVector(Path);
 
   // Handle relative paths
   if (std::error_code EC = makeAbsolute(Path))
     return EC;
 
   // Canonicalize path by removing ".", "..", "./", etc components. This is
   // a VFS request, do bot bother about symlinks in the path components
   // but canonicalize in order to perform the correct entry search.
   if (UseCanonicalizedPaths) {
     Path = sys::path::remove_leading_dotslash(Path);
     sys::path::remove_dots(Path, /*remove_dot_dot=*/true);
   }
 
   if (Path.empty())
     return make_error_code(llvm::errc::invalid_argument);
 
   sys::path::const_iterator Start = sys::path::begin(Path);
   sys::path::const_iterator End = sys::path::end(Path);
   for (const auto &Root : Roots) {
-    ErrorOr<Entry *> Result = lookupPath(Start, End, Root.get());
+    ErrorOr<RedirectingFileSystem::Entry *> Result =
+        lookupPath(Start, End, Root.get());
     if (Result || Result.getError() != llvm::errc::no_such_file_or_directory)
       return Result;
   }
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
-ErrorOr<Entry *>
+ErrorOr<RedirectingFileSystem::Entry *>
 RedirectingFileSystem::lookupPath(sys::path::const_iterator Start,
                                   sys::path::const_iterator End,
-                                  Entry *From) const {
+                                  RedirectingFileSystem::Entry *From) const {
 #ifndef _WIN32
   assert(!isTraversalComponent(*Start) &&
          !isTraversalComponent(From->getName()) &&
          "Paths should not contain traversal components");
 #else
   // FIXME: this is here to support windows, remove it once canonicalized
   // paths become globally default.
   if (Start->equals("."))
if (!FromName.empty()) {
     ++Start;
 
     if (Start == End) {
       // Match!
       return From;
     }
   }
 
-  auto *DE = dyn_cast<RedirectingDirectoryEntry>(From);
+  auto *DE = dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(From);
   if (!DE)
     return make_error_code(llvm::errc::not_a_directory);
 
-  for (const std::unique_ptr<Entry> &DirEntry :
+  for (const std::unique_ptr<RedirectingFileSystem::Entry> &DirEntry :
        llvm::make_range(DE->contents_begin(), DE->contents_end())) {
-    ErrorOr<Entry *> Result = lookupPath(Start, End, DirEntry.get());
+    ErrorOr<RedirectingFileSystem::Entry *> Result =
+        lookupPath(Start, End, DirEntry.get());
     if (Result || Result.getError() != llvm::errc::no_such_file_or_directory)
       return Result;
   }
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
 static Status getRedirectedFileStatus(const Twine &Path, bool UseExternalNames,
                                       Status ExternalStatus) {
   Status S = ExternalStatus;
   if (!UseExternalNames)
     S = Status::copyWithNewName(S, Path.str());
   S.IsVFSMapped = true;
   return S;
 }
 
-ErrorOr<Status> RedirectingFileSystem::status(const Twine &Path, Entry *E) {
+ErrorOr<Status> RedirectingFileSystem::status(const Twine &Path,
+                                              RedirectingFileSystem::Entry *E) {
   assert(E != nullptr);
-  if (auto *F = dyn_cast<RedirectingFileEntry>(E)) {
+  if (auto *F = dyn_cast<RedirectingFileSystem::RedirectingFileEntry>(E)) {
     ErrorOr<Status> S = ExternalFS->status(F->getExternalContentsPath());
     assert(!S || S->getName() == F->getExternalContentsPath());
     if (S)
       return getRedirectedFileStatus(Path, F->useExternalName(UseExternalNames),
                                      *S);
     return S;
   } else { // directory
-    auto *DE = cast<RedirectingDirectoryEntry>(E);
+    auto *DE = cast<RedirectingFileSystem::RedirectingDirectoryEntry>(E);
     return Status::copyWithNewName(DE->getStatus(), Path.str());
   }
 }
 
 ErrorOr<Status> RedirectingFileSystem::status(const Twine &Path) {
-  ErrorOr<Entry *> Result = lookupPath(Path);
+  ErrorOr<RedirectingFileSystem::Entry *> Result = lookupPath(Path);
   if (!Result) {
     if (IsFallthrough &&
         Result.getError() == llvm::errc::no_such_file_or_directory) {
       return ExternalFS->status(Path);
     }
     return Result.getError();
   }
   return status(Path, *Result);
public:
 
   std::error_code close() override { return InnerFile->close(); }
 };
 
 } // namespace
 
 ErrorOr<std::unique_ptr<File>>
 RedirectingFileSystem::openFileForRead(const Twine &Path) {
-  ErrorOr<Entry *> E = lookupPath(Path);
+  ErrorOr<RedirectingFileSystem::Entry *> E = lookupPath(Path);
   if (!E) {
     if (IsFallthrough &&
         E.getError() == llvm::errc::no_such_file_or_directory) {
       return ExternalFS->openFileForRead(Path);
     }
     return E.getError();
   }
 
-  auto *F = dyn_cast<RedirectingFileEntry>(*E);
+  auto *F = dyn_cast<RedirectingFileSystem::RedirectingFileEntry>(*E);
   if (!F) // FIXME: errc::not_a_file?
     return make_error_code(llvm::errc::invalid_argument);
 
   auto Result = ExternalFS->openFileForRead(F->getExternalContentsPath());
   if (!Result)
     return Result;
 
   auto ExternalStatus = (*Result)->status();
   if (!ExternalStatus)
     return ExternalStatus.getError();
 
   // FIXME: Update the status with the name and VFSMapped.
   Status S = getRedirectedFileStatus(Path, F->useExternalName(UseExternalNames),
                                      *ExternalStatus);
   return std::unique_ptr<File>(
       llvm::make_unique<FileWithFixedStatus>(std::move(*Result), S));
 }
 
 std::error_code
 RedirectingFileSystem::getRealPath(const Twine &Path,
                                    SmallVectorImpl<char> &Output) const {
-  ErrorOr<Entry *> Result = lookupPath(Path);
+  ErrorOr<RedirectingFileSystem::Entry *> Result = lookupPath(Path);
   if (!Result) {
     if (IsFallthrough &&
         Result.getError() == llvm::errc::no_such_file_or_directory) {
       return ExternalFS->getRealPath(Path, Output);
     }
     return Result.getError();
   }
 
-  if (auto *F = dyn_cast<RedirectingFileEntry>(*Result)) {
+  if (auto *F =
+          dyn_cast<RedirectingFileSystem::RedirectingFileEntry>(*Result)) {
     return ExternalFS->getRealPath(F->getExternalContentsPath(), Output);
   }
   // Even if there is a directory entry, fall back to ExternalFS if allowed,
   // because directories don't have a single external contents path.
   return IsFallthrough ? ExternalFS->getRealPath(Path, Output)
                        : llvm::errc::invalid_argument;
 }
 
 IntrusiveRefCntPtr<FileSystem>
 vfs::getVFSFromYAML(std::unique_ptr<MemoryBuffer> Buffer,
                     SourceMgr::DiagHandlerTy DiagHandler,
                     StringRef YAMLFilePath, void *DiagContext,
                     IntrusiveRefCntPtr<FileSystem> ExternalFS) {
   return RedirectingFileSystem::create(std::move(Buffer), DiagHandler,
                                        YAMLFilePath, DiagContext,
                                        std::move(ExternalFS));
 }
 
-static void getVFSEntries(Entry *SrcE, SmallVectorImpl<StringRef> &Path,
+static void getVFSEntries(RedirectingFileSystem::Entry *SrcE,
+                          SmallVectorImpl<StringRef> &Path,
                           SmallVectorImpl<YAMLVFSEntry> &Entries) {
   auto Kind = SrcE->getKind();
-  if (Kind == EK_Directory) {
-    auto *DE = dyn_cast<RedirectingDirectoryEntry>(SrcE);
+  if (Kind == RedirectingFileSystem::EK_Directory) {
+    auto *DE = dyn_cast<RedirectingFileSystem::RedirectingDirectoryEntry>(SrcE);
     assert(DE && "Must be a directory");
-    for (std::unique_ptr<Entry> &SubEntry :
+    for (std::unique_ptr<RedirectingFileSystem::Entry> &SubEntry :
          llvm::make_range(DE->contents_begin(), DE->contents_end())) {
       Path.push_back(SubEntry->getName());
       getVFSEntries(SubEntry.get(), Path, Entries);
       Path.pop_back();
     }
     return;
   }
 
-  assert(Kind == EK_File && "Must be a EK_File");
-  auto *FE = dyn_cast<RedirectingFileEntry>(SrcE);
+  assert(Kind == RedirectingFileSystem::EK_File && "Must be a EK_File");
+  auto *FE = dyn_cast<RedirectingFileSystem::RedirectingFileEntry>(SrcE);
   assert(FE && "Must be a file");
   SmallString<128> VPath;
   for (auto &Comp : Path)
     llvm::sys::path::append(VPath, Comp);
   Entries.push_back(YAMLVFSEntry(VPath.c_str(), FE->getExternalContentsPath()));
 }
 
 void vfs::collectVFSFromYAML(std::unique_ptr<MemoryBuffer> Buffer,
                              SourceMgr::DiagHandlerTy DiagHandler,
                              StringRef YAMLFilePath,
                              SmallVectorImpl<YAMLVFSEntry> &CollectedEntries,
                              void *DiagContext,
                              IntrusiveRefCntPtr<FileSystem> ExternalFS) {
   RedirectingFileSystem *VFS = RedirectingFileSystem::create(
       std::move(Buffer), DiagHandler, YAMLFilePath, DiagContext,
       std::move(ExternalFS));
-  ErrorOr<Entry *> RootE = VFS->lookupPath("/");
+  ErrorOr<RedirectingFileSystem::Entry *> RootE = VFS->lookupPath("/");
   if (!RootE)
     return;
   SmallVector<StringRef, 8> Components;
   Components.push_back("/");
   getVFSEntries(*RootE, Components, CollectedEntries);
 }
 
 UniqueID vfs::getNextVirtualUniqueID() {
llvm::sort(Mappings, [](const YAMLVFSEntry &LHS, const YAMLVFSEntry &RHS) {
     return LHS.VPath < RHS.VPath;
   });
 
   JSONWriter(OS).write(Mappings, UseExternalNames, IsCaseSensitive,
                        IsOverlayRelative, OverlayDir);
 }
 
 VFSFromYamlDirIterImpl::VFSFromYamlDirIterImpl(
-    const Twine &_Path, RedirectingDirectoryEntry::iterator Begin,
-    RedirectingDirectoryEntry::iterator End, bool IterateExternalFS,
-    FileSystem &ExternalFS, std::error_code &EC)
+    const Twine &_Path,
+    RedirectingFileSystem::RedirectingDirectoryEntry::iterator Begin,
+    RedirectingFileSystem::RedirectingDirectoryEntry::iterator End,
+    bool IterateExternalFS, FileSystem &ExternalFS, std::error_code &EC)
     : Dir(_Path.str()), Current(Begin), End(End),
       IterateExternalFS(IterateExternalFS), ExternalFS(ExternalFS) {
   EC = incrementImpl(/*IsFirstTime=*/true);
 }
 
 std::error_code VFSFromYamlDirIterImpl::increment() {
   return incrementImpl(/*IsFirstTime=*/false);
 }
std::error_code VFSFromYamlDirIterImpl::incrementContent(bool IsFirstTime) {
   assert((IsFirstTime || Current != End) && "cannot iterate past end");
   if (!IsFirstTime)
     ++Current;
   while (Current != End) {
     SmallString<128> PathStr(Dir);
     llvm::sys::path::append(PathStr, (*Current)->getName());
     sys::fs::file_type Type;
     switch ((*Current)->getKind()) {
-    case EK_Directory:
+    case RedirectingFileSystem::EK_Directory:
       Type = sys::fs::file_type::directory_file;
       break;
-    case EK_File:
+    case RedirectingFileSystem::EK_File:
       Type = sys::fs::file_type::regular_file;
       break;
     }
     CurrentEntry = directory_entry(PathStr.str(), Type);
     return {};
   }
   return incrementExternal();
 }