[This is supporting an RFC (https://lists.llvm.org/pipermail/cfe-dev/2021-January/067576.html / https://lists.llvm.org/pipermail/llvm-dev/2021-January/148124.html); a follow up patch has initial adoption in clang::CompilerInstance; marked WIP for now.]
Add `OutputManager` to the `llvm::vfs` namespaceWIP: Support: Add vfs::OutputBackend to virtualize compiler
The manager is configured with an `OutputBackend`. This patch contains aAdd `OutputBackend` to the `llvm::vfs` namespace to virtualize compiler
number of backends,outputs. The primary interface including:es the following API:
- Null- `getDirectory()` returns an `OutputBackendDirectory`, for ignoring all backends.which is a proxy
backend that uses the requested directory as a working directory.
- `createFile()` creates a file and returns `OutputFile` for writing to
- `createDirectory()` creates a directory and then calls
- OnDiskOutputBackend, `getDirectory()`. for wriThe default implementation assumes creating to disk (the default)a
directory is a no-op, initiallyto support backends that don't have a concept
based on the logic in `clang::CompilerInstance`of a directories without files in them.
- `createUniqueFile()` and `createUniqueDirectory()`, versions of the
above that avoid creating entitity names that already exist.
(`StableUniqueEntityAdaptor<>` can be used by backends that don't have
a way to check what exists.)
This patch contains a number of backends, including:
- NullOutputBackend, for silently ignoring all outputs.
- OnDiskOutputBackend, for writing to disk.
- InMemoryOutputBackend, for writing to an `InMemoryFileSystem`.
- MirroringOutputBackend, for writing to multiple backends.
- FilteringOutputBackend, for filtering which outputs get written to the
An `OutputBackend` needs to implement `createDestination()`, whichThere are also a few helpers for creating other backends:
- `ProxyOutputBackend` forwards all calls to an underlying backend.
returns a `std::u- `StableUnique_ptr<OutputDestination>`. Typically this is anEntityAdaptor<>` adds stable implementations of
associated subclass of `OutputDestination`. `createUniqueFileImpl()` and `createUniqueDirectoryImpl()`.
The primary interface for `OutputFile` includes the following API:
`OutputDestination`s are chained in a linked list (to support- `takeOS()` returns a `std::unique_ptr<raw_pwrite_stream>` for writing
`MirroringOutputBackend`). Each one needs to implement (`getOS()` gives a raw pointer to the stream if it hasn't been taken).
`storeContentImpl- `close()` (at minimum) in order to store contentcommits the file (to disk, in-memory, etc.).
`ContentBuffer` is used to shepherd- `getPath()` gets the content between backends. Somepath to the output.
`OutputDestination`s don't need content to be buffered,- `~Output` erases the output if it hasn't been opened.
`OutputFile` has support for buffering content. One use case is for
concrete subclasses that don't support streams. Another is to support
passing the same content to multiple underlying `OutputFile` instances
(such as for `MirroringOutputBackend`). and can`OutputFile::close()` calls
implement `takeStreamImpl()` / `storeStreamedContentImpl()` to bypass`storeContentBuffer()` with a `ContentBuffer` if this support was used;
it when they're the only destinationotherwise, it calls `storeStreamedContent()`.
An `OutputManager::createOutput()` returns an RAII-managedConfig` is passed when creating a file or directory to
`std::unique_ptr<Ocommunicate semantics about the output>`.. There are a few flags:
- `takeOS()` returns a `std::unique_ptr<raw_pwrite_stream>` fo- Text, for outputs that should be opened in text mode.
- Volatile, for outputs that other processes may modify after writinge.
(`getOS()` gives a raw pointer to the stream if it hasn't been taken).`OnDiskOutputBackend` has support for writing its bytes to an mmap'ed
- `close()` commits the file (to disk, region and passing a file-backed buffer on to other backends. in-memory, etc.).This
optimization is turned off for `Volatile` outputs.)
- `erase()` cancels the output- NoCrashCleanup, cleaning it upfor outputs that should not be removed on crash.
- `~Output` effectively calls `erase()` if the output is still "open".
A `ContentBuffer` is initialized with either- NoAtomicWrite, for outputs that should be written directly, instead
of moved atomically into place.
`std::unique_ptr<llvm::MemoryBuffer>` or `SmallVectorImpl<char> &&`.- NoImplyCreateDirectories, for outputs that require the parent path
If a destination just needs the bytes, it can call `getBytes()`; if it to already exist.
wants a `MemoryBuffer`, it can call `takeBuffer()`- NoOverwrite, which has a variantsfor outputs that are not allowed to clobber any existing
depending on whether the destination needs to own the bytes written to content.
`OnDiskOutputDestination` has support for writing its bytes to anMost of these flags are ignorable by backends that aren't associated
with a filesystem.
- Fix all the header docs. They've mostly bitrotted.
- Document and enforce threading guarantees. Tentatively, all
`OutputBackend` APIs safe to call concurrently. An `OutputFile` cannot
be used concurrently, but two files from the same backend can be.
- Test `OutputDirectory`.
- Test `createDirectory()`.
mmap'ed region and passing a file-backed buffer on to other- Test `createDirectories()`.
destinations- Test `createUnique*()`.