This is an archive of the discontinued LLVM Phabricator instance.

[llvm][vfs] Implement in-memory symlinks
ClosedPublic

Authored by jansvoboda11 on Jan 19 2022, 1:46 AM.

Details

Summary

This patch implements symlinks for the in-memory VFS. Original author: @erik.pilkington.

Depends on D117648 & D117649.

Diff Detail

Event Timeline

jansvoboda11 created this revision.Jan 19 2022, 1:46 AM
jansvoboda11 requested review of this revision.Jan 19 2022, 1:46 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 19 2022, 1:46 AM
jansvoboda11 edited the summary of this revision. (Show Details)Jan 19 2022, 1:52 AM
jansvoboda11 added a subscriber: erik.pilkington.

@sammccall, you've had objections in the past to exposing symbolic links in the vfs::FileSystem API, but this patch doesn't do that. It's just modelling symlink behaviour in an InMemoryFileSystem. Any concerns?

(I have none.)

llvm/lib/Support/VirtualFileSystem.cpp
597

Add a trailing , to avoid a diff if this gets extended again?

@sammccall, you've had objections in the past to exposing symbolic links in the vfs::FileSystem API, but this patch doesn't do that. It's just modelling symlink behaviour in an InMemoryFileSystem. Any concerns?

No, this seems fine to me. And implementation seems solid :-)

(IIRC my previous concerns weren't about VFS->VFS symlinks, but rather ability to "resolve" a VFS file to a non-virtual file in order to open it through native APIs. It's possible I misunderstood that proposal of course)

In principle I don't see a problem with exposing symlinks in the VFS layer either, but I guess it'd be really complicated to get right. (e.g. ensure handling symlinks externally vs within the VFS matches, conceptual problems with overlay filesystems, etc)

llvm/include/llvm/Support/VirtualFileSystem.h
565

FWIW I find From/to confusing, because I had to rewire my brain to think of ln as a variant of cp in order to remember which order the args go in :-)

I think NewLink/Target would be much clearer, but there's value in being consistent with the existing addHardLink too, so up to you.

llvm/lib/Support/VirtualFileSystem.cpp
930

FollowFinalSymlink?
It's wordy, but it's not clear to me what the behavior is if you look up "link/link/file" with FollowSymlinks=false. Also the plural is a bit misleading.

It might be worth adding a comment here with the most important contract: if Follow is true, then it will never return a symlink node.

1013

This fails if ToNode is a symbolic link. Linux allows hardlinks to symlinks, apparently macOS resolves the symlink and hardlinks to the target, posix allows either (implementation-defined) but doesn't allow it to just fail.

This isn't posix so you can do whatever you like, but I think the choice deserves a comment.

1083

This doesn't look correct to me, if the symlink points to a directory we'll claim it's a file and then operations on it will fail.

Since the VFS interface doesn't know about symlinks, it's probably best to resolve the symlink (chain) here and report the target. But returning symlink_file might be ok...

If you resolve and it dangles, you have to choose between reporting regular_file, symlink_file, or hiding the entry entirely... not really sure which is best.

(IIRC my previous concerns weren't about VFS->VFS symlinks, but rather ability to "resolve" a VFS file to a non-virtual file in order to open it through native APIs. It's possible I misunderstood that proposal of course)

Oops, I just remembered about D97288, which is probably what you meant. Yeah, this change is much less expensive because it doesn't affect (other) VFS implementers nor consumers.

(IIRC my previous concerns weren't about VFS->VFS symlinks, but rather ability to "resolve" a VFS file to a non-virtual file in order to open it through native APIs. It's possible I misunderstood that proposal of course)

Oops, I just remembered about D97288, which is probably what you meant. Yeah, this change is much less expensive because it doesn't affect (other) VFS implementers nor consumers.

Yes, I think that's what I was remembering!

ormris removed a subscriber: ormris.Jan 24 2022, 11:08 AM
jansvoboda11 marked 4 inline comments as done.Jun 21 2022, 4:21 AM
jansvoboda11 added inline comments.
llvm/include/llvm/Support/VirtualFileSystem.h
565

You're right, I struggle with that too. I changed that to NewLink/Target for both the new addSymbolicLink() and the existing addHardLink(). Let me know if you'd prefer to extract the change to addHardLink() into a prep-commit. (I assume review wouldn't be necessary.)

llvm/lib/Support/VirtualFileSystem.cpp
930

Fair enough, changed.

1013

Good catch! I chose to follow symlinks to be consistent with macOS for now.

1083

I chose to resolve the symlink to keep the VFS interface symlink-free. In case of dangling symlink, I think it would make sense to report type_unknown, WDYT?

BTW, since I chose to resolve the symlink and directory_entry needs the target path, I was forced to tweak the lookupInMemoryNode to return the resolved target path as well, since with symlinks it will differ from the requested path.

Herald added a project: Restricted Project. · View Herald TranscriptJun 21 2022, 4:21 AM
jansvoboda11 marked 3 inline comments as done.

Follow symlinks when creating hardlinks and iterating over directory entries.
Return the resolved name from InMemoryFileSystem::lookupNode(), update some
documentation.

jansvoboda11 added inline comments.Jun 21 2022, 4:27 AM
llvm/lib/Support/VirtualFileSystem.cpp
1133

We're passing a lambda here since lookupNode is (and should remain) private to InMemoryFileSystem and InMemoryDirIterator is in anonymous namespace so it can't be befriended. An alternative to this solution would be to pull InMemoryDirIterator into the detail namespace and forward-declare it in the friend declaration inside InMemoryFileSystem. I don't have strong feelings either way.

sammccall accepted this revision.Jun 21 2022, 5:50 AM

LG, thanks for the changes!
I wouldn't bother splitting out the commits for something this size.

llvm/lib/Support/VirtualFileSystem.cpp
1133

Yeah :-( I find friending it slightly cleaner, but don't feel strongly, feel free to leave it.

An alternative very similar to friending: forward-declare it as a nested class of InMemoryFileSystem (which makes it implicitly a friend, but also InMemoryFileSystem::DirIterator seems a slightly more obvious name than detail::InMemoryDirIterator.

This revision is now accepted and ready to land.Jun 21 2022, 5:50 AM
This revision was landed with ongoing or failed builds.Jun 21 2022, 7:30 AM
This revision was automatically updated to reflect the committed changes.