This is an archive of the discontinued LLVM Phabricator instance.

[clangd] Add support for the `defaultLibrary` semantic token modifier
ClosedPublic

Authored by dgoldman on Apr 29 2021, 10:45 AM.

Details

Summary

This allows us to differentiate symbols from the system (e.g. system
includes or sysroot) differently than symbols defined in the user's
project, which can be used by editors to display them differently.

This is currently based on FileCharacteristic, but we can
consider alternatives such as Sysroot and file paths in the future.

Diff Detail

Event Timeline

dgoldman created this revision.Apr 29 2021, 10:45 AM
dgoldman requested review of this revision.Apr 29 2021, 10:45 AM
dgoldman updated this revision to Diff 341683.Apr 29 2021, 3:40 PM

Fix casing warnings

kadircet added inline comments.Apr 30 2021, 1:50 AM
clang-tools-extra/clangd/SemanticHighlighting.cpp
413

you can make this a free function by accepting a decl instead. as sourcemanager is accessible via D->getASTContext().getSourceManager(). this way you don't need to modify signatures for scopeModifier overloads (and get rid of the duplicate in CollectExtraHighlightings).

433

why the null check now?

436

I don't use semantic highlighting enough to judge whether it is more useful to say DefaultLibrary than {Class,Function}Scope. (i.e. having the check here vs at the end of the function as a fallback). Can you provide some rationale about the decision (probably as comments in code)?

By looking at the testcase, I suppose you are trying to indicate "overriden"/"extended" attribute of a symbol, which makes sense but would be nice to know if there's more. I am just worried that this is very obj-c specific and won't really generalize to c/c++. As most of the system headers only provide platform specific structs (e.g. posix) and they are almost never extended. So it might be confusing for user to see different colors on some memberexprs.

474

should this be put in a separate patch?

sammccall added inline comments.Apr 30 2021, 2:25 AM
clang-tools-extra/clangd/SemanticHighlighting.cpp
436

I think default-library isn't a scope modifier at all, it should be independent.

e.g. std::string is defaultlLibrary | globalScope, while std::string::push_back() is defaultLibrary | classScope.

465

I'm confused about the interface change - looks like the new parameter never actually used, just passed around

clang-tools-extra/clangd/SemanticHighlighting.h
75

This isn't a scope modifier so shouldn't be grouped with them

dgoldman added inline comments.Apr 30 2021, 7:00 AM
clang-tools-extra/clangd/SemanticHighlighting.cpp
413

TIL decls can reference the ASTContext. I can swap over to that if you'd like, the one advantage the current approach has is that we can easily add a caching layer into the Builder to say cache the results by FileID (but I'm not sure how useful/necessary such caching would be)

436

I can break it up if you'd like - just let me know what you had in mind. I can change the scopeModifier function's name and then have it return a list of modifiers or I can just add a new function which all existing callers of scopeModifier will also need to call.

And what we're really trying to do here is color the system/SDK symbols differently - this is definitely useful for iOS dev since Apple has a very large SDK, but if it's not useful for C++ maybe we could just make this configurable?

465

It's used (currently) as the HighlightingsBuilder is the one who checks for default/systemness

474

If you want I can move it over

sammccall added inline comments.Apr 30 2021, 7:22 AM
clang-tools-extra/clangd/SemanticHighlighting.cpp
436

just let me know what you had in mind

I think this should follow isStatic() etc: just a boolean function that gets called in the minimum number of places needed.

scopeModifier is slightly different pattern because there are a bunch of mutually-exclusive modifiers. defaultLibrary is (mostly) independent of other modifiers.

if it's not useful for C++ maybe we could just make this configurable

No need really - sending more modifiers is cheap (one string at startup and one bit in an integer we're sending anyway).
It's up to clients to style the ones they care about, and at least in VSCode this can be customized per-lang I think. Important thing is that they don't interact with other modifiers.

465

Oops yes, in the other overload.

Please do remove this as Kadir suggests, it's noisy and makes inputs/outputs less clear.

dgoldman updated this revision to Diff 341947.Apr 30 2021, 9:52 AM
dgoldman marked an inline comment as done.

Split defaultLibrary check to separate function

dgoldman marked 5 inline comments as done.Apr 30 2021, 9:53 AM
dgoldman added inline comments.
clang-tools-extra/clangd/SemanticHighlighting.cpp
433

removed and check for it when visit an obj method expr

436

Swapped over, currently don't check on deduced types but we could add our isDefaultLibrary check there as well

dgoldman updated this revision to Diff 341953.Apr 30 2021, 9:57 AM
dgoldman marked 2 inline comments as done.

Remove unnecessary null check

LG from my side.

clang-tools-extra/clangd/SemanticHighlighting.cpp
475

is this guaranteed to be non-null ?

dgoldman updated this revision to Diff 342391.May 3 2021, 8:03 AM

Remove unnecessary scope modifier

dgoldman marked 2 inline comments as done.May 3 2021, 8:03 AM
dgoldman added inline comments.
clang-tools-extra/clangd/SemanticHighlighting.cpp
475

Went ahead and removed it, it's unnecessary now

sammccall accepted this revision.Jun 2 2021, 1:16 AM
sammccall added inline comments.
clang-tools-extra/clangd/SemanticHighlighting.cpp
436

I think it'd be useful to do this for consistency, just VisitDecltypeTypeLoc and VisitDeclaratorDecl need to be changed I think.

We'd want a version of isDefaultLibrary that works on Type. We're only going to see canonical types so I think we need to:

  • unwrap pointers (getPointeeOrArrayElementType)
  • if BuiltinType -> return true
  • for types: find the decl and dispatch to isDefaultLibrary(Decl)

to find the decl it'd be nice to use targetDecl() but also sufficient to handle TagType + ObjCInterfaceType for now I think.

This revision is now accepted and ready to land.Jun 2 2021, 1:16 AM
dgoldman updated this revision to Diff 349259.Jun 2 2021, 7:06 AM
dgoldman marked an inline comment as done.

Add isDefaultLibrary(const Type *) and support auto/decl types

dgoldman marked an inline comment as done.Jun 2 2021, 7:07 AM
dgoldman added inline comments.
clang-tools-extra/clangd/SemanticHighlighting.cpp
436

Done, I think this is fine for a v1 - we don't see much auto/decltype use in objc, can improve it later

This revision was landed with ongoing or failed builds.Jun 2 2021, 7:25 AM
This revision was automatically updated to reflect the committed changes.
dgoldman marked an inline comment as done.