This is an archive of the discontinued LLVM Phabricator instance.

[clangd] Loose include-cleaner matching for verbatim headers
ClosedPublic

Authored by sammccall on Jul 20 2023, 12:47 PM.

Details

Summary

This updates clangd to take advantage of the APIs added in D155819.
The main difficulties here are around path normalization.

For layering and performance reasons Includes compares paths lexically, and so
we should have consistent paths that can be compared across addSearchPath() and
add(): symlinks resolved or not, relative or absolute.
This patch spells out that requirement, for most tools consistent use of
FileManager/HeaderSearch is enough.

For clangd this does not work: IncludeStructure doesn't hold FileEntrys due to
the preamble/main-file split. It records paths, but canonicalizes them first.
We choose to use this canonical form as our common representation, so we have
to canonicalize the directory entries too. This is done in preamble-build and
recorded in IncludeStructure, as canonicalization is quite expensive.

Left as future work: while include-cleaner correctly uses FileEntry for Header,
as it cares about inode identity, actual Includes have a single path.
They can be modeled with FileEntryRef, and probably should be.

Diff Detail

Event Timeline

sammccall created this revision.Jul 20 2023, 12:47 PM
Herald added a project: Restricted Project. · View Herald TranscriptJul 20 2023, 12:47 PM
sammccall requested review of this revision.Jul 20 2023, 12:47 PM
sammccall updated this revision to Diff 542653.Jul 20 2023, 1:43 PM

extract isPreferredProvider helper

kadircet added inline comments.Jul 26 2023, 1:58 AM
clang-tools-extra/clangd/Headers.cpp
185

nit: early exit

clang-tools-extra/clangd/Headers.h
179

s/SearchPathCanonical/SearchPathsCanonical/

clang-tools-extra/clangd/Hover.cpp
1250–1253

oops, looks like we were getting away with some dangling references.
the reference here is wrong, convertIncludes returns a value type. can you fix that while here?

clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
483–488

i don't think making this test rely on walkUsed is a good idea. what about just populating the providers directly, like:

EXPECT_TRUE(isPreferredProvider(Decl, Includes, {include_cleaner::Header{"decl.h"}, include_cleaner::Header{"def.h"}}));

and verifying we're recognizing preferred providers purely on that ordering ?

sammccall added inline comments.Jul 26 2023, 5:49 AM
clang-tools-extra/clangd/Headers.cpp
185

I don't think it makes sense here, this isn't "the remaining case" for this function, it's just one of many things this function does

clang-tools-extra/clangd/Hover.cpp
1250–1253

this isn't dangling, it's lifetime extension.

Can change it for style if you like, though?

clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
483–488

That would be a strict unit test, but I think it's too easy to lose track of how this idea of "preferred" relates to the code.

For example, when writing this test, I expected Y to match the includes for both Decl and Def, because they're intuitively both "best matches" and we do allow multiple includes to match.
But of course we force the *providers* to be ranked, so really ~only multiple includes of the *same* header will be accepted as all-preferred.

This wrinkle seems worth recording, and totally disappears if we make the test abstract by describing a provider list directly.

(Can do it though if that's what you do prefer)

kadircet accepted this revision.Jul 26 2023, 6:56 AM

thanks!

clang-tools-extra/clangd/Hover.cpp
1250–1253

you're right. i think it would be better to have it as value though.

clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
483–488

i completely agree with what you said, but I don't see why ranking should matter here. i do feel like all of those concerns you raised can still be inferred from the fact that we're providing a set of ordered providers, but maybe it's just me. i just feel like the downside is we should update this test if ranking for providers changes for whatever reason, and i don't think we should. but that is not a big deal, so feel free to keep it as-is.

This revision is now accepted and ready to land.Jul 26 2023, 6:56 AM
sammccall marked 4 inline comments as done.Jul 27 2023, 10:19 AM
sammccall added inline comments.
clang-tools-extra/clangd/Headers.h
179

Huh, I'd always thought of "search path" singular, like each entry is edges along a path, which makes no sense.
Maybe it's path like a reference to $PATH, which has a singular name for whatever reason.

Anyway, done.

clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
483–488

i completely agree with what you said, but I don't see why ranking should matter here.

I guess I'm basically saying that this function does something that seems reasonable, and the ranking function does something that seems reasonable, but the result when you plug them together is surprising.

That makes me want to write a test that has them plugged together, but I don't feel strongly. Changed it back.

This revision was landed with ongoing or failed builds.Jul 27 2023, 10:21 AM
This revision was automatically updated to reflect the committed changes.
sammccall marked 2 inline comments as done.