Details
Diff Detail
- Repository
- rG LLVM Github Monorepo
Event Timeline
Some notes on this:
This feature is based on https://github.com/microsoft/language-server-protocol/issues/956, which combines inline parameter hints with other types of annotations that can be shown inline under a single feature called inlay hints. The protocol is designed to accommodate different types of hints, though the only kind currently provided are parameter hints.
The protocol bits are more specifically based on https://github.com/microsoft/vscode-languageserver-node/pull/609/, with some changes to reflect the discussion in the LSP issue (notably, the addition of a position field as discussed in this comment).
I have a corresponding client-side patch here: https://github.com/clangd/vscode-clangd/pull/168
I do have some questions about the details of the protocol, which I've asked inline.
clang-tools-extra/clangd/ClangdLSPServer.cpp | ||
---|---|---|
1478 | I expect the protocol in this patch to be largely in line with the version standardized in LSP, but there may be some minor differences as that's not yet finalized. Does that mean we should rather name this clangd/inlayHints for now? | |
clang-tools-extra/clangd/Protocol.h | ||
1489 | I'm not sure if there's much point in keeping these constants in the protocol. With the addition of InlayHintPosition, the InlayHintKind no longer carries any semantic meaning, beyond allowing the client to enable/disable hints on a per-kind basis. I do think that enable/disable granularity is useful, but it could also be accomplished with a freeform string as the kind. Should I change it to a string? Or keep the enum for clangd's use but serialize it as a string? | |
1524 | In the protocol documentation, I've repeatedly used the term "token" to describe the fragment of code to which a hint applies, but technically it's not necessary a single AST token. For example, for parameter hints, they apply to the entire argument expression. Should I say something like "AST node" instead? |
Thanks for working on this! It looks like it's getting traction in VSCode/LSP now, which makes it much more compelling.
Protocol stuff: obviously in the long run we want to match where LSP goes, and the fewer changes between now and then the better.
But there's enough uncertainty that I don't think we have much chance of guessing right enough to be compatible at this point.
So rather than introducing another variant that's likely to be "wrong", let's use rust-analyzer's protocol as-is?
I don't think leaving out "position" is a big deal as long as we're only providing one type of hint.
(I'd even be tempted to call it "rust-analyzer/inlayHints" but maybe that's too silly)
I dug a bit into the differences based on:
- the pull request you mentioned: https://github.com/microsoft/vscode-languageserver-node/pull/609/files#
- the proposed API inside VSCode: https://github.com/microsoft/vscode/blob/main/src/vs/vscode.proposed.d.ts
- discussion on that PR: https://github.com/microsoft/vscode/pull/113285
Looks like VSCode calls this "inline hints" but has a TODO to change back to "inlay hint". So naming seems likely correct.
Ranges/positions are very different! In particular, I think range has the same name+type but different semantics.
For the parameter hint:
foo(bar: 42); The rust-analyzer protocol wants range = `foo([[42]]);`, label="bar: " position = before The VSCode API wants range = `foo([[]]42);`, label="bar:", whitespaceAfter = true
This is a tough call because I think the rust-analyzer proposal is better (retains more info + flexibility like you say) but the VSCode one seems fairly likely to win :-/ We should provide feedback here.
"text" and "label" seem an arbitrary different choice of name.
clang-tools-extra/clangd/ClangdLSPServer.cpp | ||
---|---|---|
1478 | I think differences are quite likely, as VSCode APIs diverge from what's in the PR. So yeah, I think this should be clangd/inlayHints or rust-analyzer/inlayHints. | |
clang-tools-extra/clangd/Protocol.h | ||
1489 | We should at avoid having string literals scattered all over the impl + tests. I like enum-serialized-as-string a little better than string-with-constants-exposed... (Also it seems reasonably likely they'll pull the same dictionary trick as with semantic-tokens, so we can send enum numbers over the wire) | |
1494 | any point including the unused kinds here, rather than leaving FIXMEs for ideas we have? (I ask because it's not obvious to me that the caret is in the right place in that example, but it seems silly to worry about it now) | |
1494 | implicit conversions/constructor calls is a potentially interesting set of hints too | |
1520 | as mentioned I don't think we need to include position for now. If we do, 0/1 seems like an unlikely representation... | |
1524 | I'd just say something like "the range of source code"... Both "token" and "ast node" seem unnessarily specific. | |
clang-tools-extra/clangd/XRefs.cpp | ||
1977 | It's not obvious to me exactly what cases "implicit constructor call" covers, can you be more precise? | |
1979 | I think you also want to exclude E->isStdListInitialization - in this case you have braces but it's not the arg list | |
1999 | nit: can we take an ArrayRef instead of pointer+length? | |
2004 | do we not want to assume that an incomplete call is a prefix? Actually I guess we'd have to handle RecoveryExpr to get that to work. I did have an idea once that RecoveryExpr would contain a Stmt::Kind indicating what we *tried* to build, which we could use here. Anyway, something for the future. | |
2004 | should we also bail out if numParams > argcount and the function isn't variadic? | |
2012 | Ah, redecls are interesting. Many possible variations on this strategy. For example:
I think it might be worth moving this into getParameterNames (or introducing an intermediate function) to allow some more implementation flexibility (now or in the future). | |
2024 | Very nice. It's extremely annoying that TokenBuffer doesn't capture comments, but I think it would work well enough to simply match the source code up to the insertion point textually. | |
2030 | It's really tempting to include by-ref-ness as foo&: or something, but the distinction between const/non-const is critical and I can't think of a similarly compact way to denote that. | |
2046 | How do you feel about stripping leading underscores from names, in order to prettify standard library calls? e.g. libc++'s std::copy has parameters __first, __last and __result to guard against macros, libstdc++ and MSVCSTL seem to do the same. It's going to look really noisy! (I don't think we currently do this anywhere, it's obviously relevant to signature help & code completion too... but have to start somewhere!) | |
2059 | I don't think param decls can be special names, so instead of printing can you just grab the ASTContext-owned StringRef from the DeclarationName? Then we can return vector<StringRef> with no copies. And while here, it seems like SmallVector<StringRef, 8> or so would be even better. (Can merge this with the anonymous case: either getAsIdentifierInfo gives you an identifier or you insert empty) | |
2066 | Ooh, macros :-) This is fine for now, but because this will be displayed always (for some users) failures are going to be fairly visible. I don't think this check is quite sufficient to be safe: #define PATHOLOGICAL(X) X , X atan2(y: 1 + PATHOLOGICAL(2) + 4); // confusing! prefer no hint and also excludes useful cases. Consider: #define PI 3.14 atan2(PI, PI); // want hints ASSERT(atan2(0, 1) == 0); // want hints We don't really want to place a macro-handling tax on every type of hint we emit, so it would be nice to handle it generically here. Maybe a compromise:
(Doing this with *just* the sourcerange R is probably possible but the heuristics to determine the anchor fileID seem complicated) | |
clang-tools-extra/clangd/XRefs.h | ||
135 | XRefs.cpp has too much stuff in it today. | |
clang-tools-extra/clangd/test/initialize-params.test | ||
75 | I think it's important we namespace this one for now (probably more important than the method itself!) Note that vscode-clangd and clangd aren't version-locked | |
clang-tools-extra/clangd/unittests/InlayHintTests.cpp | ||
188 | I guess/hope you don't want to handle CXXUnresolvedConstructExpr too. If so, there are cases to deal with like when we can't tell whether or not we're calling an init-list-constructor... |
Thanks for the review!
I do plan on working on other hint types (especially TypeHint) as a follow-up.
But we could still leave out "position", and let the client infer before vs. after based on the hint type (this what the Rust client does).
I dug a bit into the differences based on:
- the pull request you mentioned: https://github.com/microsoft/vscode-languageserver-node/pull/609/files#
- the proposed API inside VSCode: https://github.com/microsoft/vscode/blob/main/src/vs/vscode.proposed.d.ts
- discussion on that PR: https://github.com/microsoft/vscode/pull/113285
Ranges/positions are very different! In particular, I think range has the same name+type but different semantics.
For the parameter hint:foo(bar: 42); The rust-analyzer protocol wants range = `foo([[42]]);`, label="bar: " position = before The VSCode API wants range = `foo([[]]42);`, label="bar:", whitespaceAfter = trueThis is a tough call because I think the rust-analyzer proposal is better (retains more info + flexibility like you say) but the VSCode one seems fairly likely to win :-/ We should provide feedback here.
Good catch. Digging into that PR a bit more, it seems like the reason for the range working in this way is that the C# language server has an "overwrite" mode for inline hints that you can enter, where the provided range is overwritten (temporarily) with the hint contents.
For example, if the hint range were [[auto]] x = 42;, in overwrite mode that would be rendered as int x = 42;.
In this model, it makes sense to make the range nonempty for some hint types (like the [[auto]] case), but not others (like parameter hints, where you wouldn't want the parameter name to replace the argument value).
So... if we're going to provide feedback here, we should probably suggest something that accommodates this use case. Perhaps an additional flag in the hint, to indicate if the "overwrite" behaviour is sensible for it? (Or perhaps the hint type is enough to infer that?)
Address some review comments (others remain)
clang-tools-extra/clangd/Protocol.h | ||
---|---|---|
1494 | The motivation for the proposed placement of type hints is that in several other languages (e.g. Rust, TypeScript), you can write both let var : type = init; (where the type is explicitly specified) and let var = init; (where the type is deduced). In those languages, the natural placement of the type hint is the place where the explicit annotation would otherwise go. For consistency / familiarity, I figured it would make sense to use the same placement for C++. Anyways, I've removed the enumerator for type hints now, we can discuss them further in the patch that adds them. | |
clang-tools-extra/clangd/XRefs.cpp | ||
1979 | Good catch, thanks! I tried to write a test for this, but TestTU doesn't seem to like #include <initializer_list> (or any other standard library include). Do you know why that might be? | |
2004 | You make some great points here:
| |
2004 | Ha, I did find a legitimate case where the Callee->getNumParams() < ArgCount condition holds: C-style varargs functions. We currently don't show any hints for those, but presumably it would be better to show hints for the non-vararg parameters. | |
2024 |
In my mind, one of the points of inline parameter hints is that they fulfil the functions of /* foo = */ comments without you actually having to write them. Thus, I would argue for leaving the hints there, as a signal that the comment can now be removed. | |
2046 | Sounds like a good idea to me! | |
2066 |
Surely, people who write this deserve a confusing hint? (I kid, I kid. Your macro suggestions are reasonable and I will give them a try.) | |
clang-tools-extra/clangd/unittests/InlayHintTests.cpp | ||
188 | It hadn't occurred to me, but yeah, that seems like a good reason to avoid handling CXXUnresolvedConstructExpr for now. |
Marking this as "Changes Planned" because there are more review comments I need to address.
clang-tools-extra/clangd/XRefs.cpp | ||
---|---|---|
2024 |
I disagree with that mentality, Not everyone on a project will be using a language server/ide that supports inline hints. A push to remove those comments will harm other contributors who don't see the inline hints. It would also harm readability where the code is hosted as they wont show inline hints. Having said that, inline hints definitely makes code easier to work on so there is still a lot of value in this feature. |
This is looking pretty good! Can't wait to try it, and some of my suggestions are just scope creep, so feel free to draw the line :-)
Sorry, I think phabricator has lost track of line numbers that some of the comments were attached to :-(
Yeah, this would be my preference (mostly because Rust does it and it's fewer fields, not because it's actually better).
I dug a bit into the differences based on:
- the pull request you mentioned: https://github.com/microsoft/vscode-languageserver-node/pull/609/files#
- the proposed API inside VSCode: https://github.com/microsoft/vscode/blob/main/src/vs/vscode.proposed.d.ts
- discussion on that PR: https://github.com/microsoft/vscode/pull/113285
Ranges/positions are very different! In particular, I think range has the same name+type but different semantics.
For the parameter hint:foo(bar: 42); The rust-analyzer protocol wants range = `foo([[42]]);`, label="bar: " position = before The VSCode API wants range = `foo([[]]42);`, label="bar:", whitespaceAfter = trueThis is a tough call because I think the rust-analyzer proposal is better (retains more info + flexibility like you say) but the VSCode one seems fairly likely to win :-/ We should provide feedback here.
Good catch. Digging into that PR a bit more, it seems like the reason for the range working in this way is that the C# language server has an "overwrite" mode for inline hints that you can enter, where the provided range is overwritten (temporarily) with the hint contents.
For example, if the hint range were [[auto]] x = 42;, in overwrite mode that would be rendered as int x = 42;.
In this model, it makes sense to make the range nonempty for some hint types (like the [[auto]] case), but not others (like parameter hints, where you wouldn't want the parameter name to replace the argument value).
So... if we're going to provide feedback here, we should probably suggest something that accommodates this use case. Perhaps an additional flag in the hint, to indicate if the "overwrite" behaviour is sensible for it? (Or perhaps the hint type is enough to infer that?)
Right. There are a few interlocking points here:
- when inserting text, it's nice to be able to say what range it applies to (so it can appear "on demand") - do we have examples of editors where this interaction works well?
- it's also nice to be able to either insert or overwrite (e.g. for expanding deduced types)
- the mode/presentation is related to the hint type, but not in a way that predictably spans languages
One neat way to resolve this is to make position explicit, and have the allowed values be before/after/replace.
My other pieces of feedback would be
- whitespaceBefore/whitespaceAfter doesn't belong in the protocol and can be inferred from the context/content.
- it will be nice to eventually have good support for code actions/commands targeting these hints, as they're often for deduced info that can be explicit
(Happy to leave some version of this myself, but my "experience" is indirect :-))
clang-tools-extra/clangd/InlayHints.cpp | ||
---|---|---|
40 ↗ | (On Diff #335210) | nit: This is *a* reason, but I think the names used in practice wouldn't be much help, either. |
43 ↗ | (On Diff #335210) | nit: you've written the skipped cases (constructor vs call) in different styles, unify? |
67 ↗ | (On Diff #335210) | A few more "obvious" cases to exclude: if there's exactly one parameter and the callee is (a memberexpr/declrefexpr named) "set*"? If the callee is a copy/move constructor? |
70 ↗ | (On Diff #335210) | we may want to rather factor this as shouldHint(Expr *E, StringRef Hint) - I suspect we'll hit other cases where the hints are not useful and may add heuristics |
81 ↗ | (On Diff #335210) | consider E->IgnoreUnlessSpelledInSource() or E->IgnoreParenCasts() or one of the other (countless!) variants... |
clang-tools-extra/clangd/InlayHints.h | ||
9 ↗ | (On Diff #335210) | maybe include a reference to the version being implemented, as there's likely to be more proposals soon |
clang-tools-extra/clangd/XRefs.cpp | ||
1979 | Well, we'd need to include a standard library implementation in the tests (libc++ isn't part of the compiler). This appears to be enough to get init list construction to work: namespace std { template <class> class initializer_list {}; } | |
2004 | Yeah for varargs we should hint the fixed params, and then could choose to apply the vararg name to none/first/all of the varargs.
Agree it's better to just be a bit defensive here. | |
2024 | +1 to njames93 - people are using other tools (we don't have these features in review workflows... yet!) e.g. Google has a style guide that encourages these, and having this feature in clangd won't change that. Given that people *shouldn't* remove the comment, showing the hint as well is annoying. Eventually maybe the "overwrite" mode would be a good fit here (you can make a reasonable case for either the comment or the hint being preferable). But either way I think we need to detect them. | |
2066 |
Yeah, on reflection this case isn't important, feel free to ignore it entirely. Supporting idiomatic macro usage *is* useful but if the filtering/location logic gets too complicated feel free to defer with a FIXME. | |
clang-tools-extra/clangd/unittests/InlayHintTests.cpp | ||
230 | maybe add a FIXME test asserting no-hints-yet for aggregate initialization? |
Hey, I just wanted to say thank you! This is a rust-analyzer feature I really like when writing Rust code but somehow never thought about bringing it to Clangd. This is a nice idea, huge thanks for implementing it!
Address most remaining review comments
I've left a couple of things as FIXMEs, as indicated
clang-tools-extra/clangd/XRefs.cpp | ||
---|---|---|
2024 | Good points, thanks. I'd like to get to a stage where even review tools have these kinds of smarts, but we're not there yet. I've left this as a FIXME for now (along with the setter-function case), but I'm happy to implement it in a follow-up (or in another revision of this patch if you prefer). | |
2059 | I revised the code to avoid printName(), but using StringRef clashes with stripping leading underscores, so I kept it as std::string. | |
2066 | I ended up implementing a simplified version of what you suggest:
The result does support both of your atan2 examples (which I agree are idiomatic and useful to support). |
clang-tools-extra/clangd/ClangdLSPServer.cpp | ||
---|---|---|
1478 | If this is compatible with rust analysers feature. Maybe we should be using their methods and feature flags spec. It will mean make life easier for clients wishing to support this before it gets baked into the official protocol. |
clang-tools-extra/clangd/InlayHints.cpp | ||
---|---|---|
44 ↗ | (On Diff #336774) | apparently user-defined literals are also CallExpr (what?) so I guess you want to exclude UserDefinedLiteral here |
84 ↗ | (On Diff #336774) | nit: SmallVector<8> seems like a clear (minor) win here |
104 ↗ | (On Diff #336774) | A MemberExpr with implicit-this should also not be hinted. We may want to be looser than this, depending how much we value regularity vs brevity when it's obvious.
But we can certainly revisit this later. |
104 ↗ | (On Diff #336774) | can we also avoid printName here? maybe change getDeclRefExpr to StringRef getSimpleExprName or so |
151 ↗ | (On Diff #336774) | nit: or C varargs |
clang-tools-extra/clangd/XRefs.cpp | ||
2059 |
How so? Name.ltrim('_') points to the same data as Name |
Address remaining review comments
clang-tools-extra/clangd/ClangdLSPServer.cpp | ||
---|---|---|
1478 | It's not 100% compatible. For example, for parameter hints, Rust expects the server to provide only the parameter name itself in label, and post-processes the labels on the client side to tack on the ": ". We could make it 100% compatible. Do you think that would be valuable? That is, do you expect there will be cross-language client implementations of the pre-standard protocol, in places that are not a plugin shipped with a language server (like vscode-clangd or the rust-analyzer vscode plugin)? (Would accommodating such client impls limit us from adding things like the suggested "implicit conversion hints"?) | |
clang-tools-extra/clangd/InlayHints.cpp | ||
44 ↗ | (On Diff #336774) | Good call, thanks! |
104 ↗ | (On Diff #336774) | Fixed the "MemberExpr with implicit-this" case. As for the other suggestions: I can see the argument, but I think there's also value in simplicity ("the argument expression matches the hint exactly"), including for the purpose of not leaving the user wondering why we're not showing a hint. |
151 ↗ | (On Diff #336774) | As it happens, for C varargs, the notional "varargs parameter" doesn't count towards getNumParams() in the first place, so this function has (and needs) no logic for them. |
clang-tools-extra/clangd/XRefs.cpp | ||
2059 | Just didn't think of that, thanks :-) |
Still LG
clang-tools-extra/clangd/ClangdLSPServer.cpp | ||
---|---|---|
1478 | FWIW *I* don't think many pre-standard clients are likely or that important in the long run. Rather we should focus on proving the concept, getting a head start on the standard feature, being well-positioned to give feedback on standard proposals. So if it's not totally compatible, I like the clangd prefixes. | |
clang-tools-extra/clangd/InlayHints.cpp | ||
151 ↗ | (On Diff #336774) | Right - but maybe the caller doesn't know that :-) |
I expect the protocol in this patch to be largely in line with the version standardized in LSP, but there may be some minor differences as that's not yet finalized.
Does that mean we should rather name this clangd/inlayHints for now?