This is an archive of the discontinued LLVM Phabricator instance.

[clangd] various fixes
AbandonedPublic

Authored by rwols on Oct 30 2017, 1:49 PM.

Details

Summary
  • Look up in parent directories for a .clang-format file. Use "LLVM" as fallback style.
  • Move cached fixits to ClangdServer, remove fixits once the client closes the document.
  • Fix translations between LSP rows/cols (0-based) and clang row/cols (1-based).

Event Timeline

rwols created this revision.Oct 30 2017, 1:49 PM
rwols added a comment.Oct 30 2017, 1:52 PM

Does anyone have an idea how I would write a test for this?

Also, I should have commit access after approval now. So that's less work for you guys ;)

krasimir edited edge metadata.Oct 30 2017, 3:33 PM

For testing you could do something similar to how clang-format does it.

sammccall edited edge metadata.Nov 2 2017, 10:35 AM

Thanks for this!

clangd/ClangdLSPServer.cpp
130

This function is pretty hard to understand from a high level - it's a bit of an odd split.

Is the duplication really so bad? The straight-line code could be easy to read:

auto File = Params.textDocument.uri.file;
auto Edits = Server.formatRange(File, Params.range);
if (Edits)
  C.reply(replacementsToEdits(Server.getDocument(File), Edits.get()));
else
  C.replyError(Edits.takeError());

(The "[" won't be needed after D39435, and we should add a replyError() overload that takes an Error)

134

It doesn't seem reasonable to assume the error is always .clang-format. Can we just include the message in the error object?

clangd/ClangdServer.cpp
69–70

This needs to use the VFS I think. (I'm not familiar with the details of the VFS usage I'm afraid)

rwols added inline comments.Nov 2 2017, 3:38 PM
clangd/ClangdLSPServer.cpp
130

This is a very nice idea! I'll probably update the diff tomorrow.

rwols updated this revision to Diff 121524.Nov 3 2017, 12:27 PM

Use llvm::Error for beautiful handling of errors.

It's perhaps an idea to write a replyExpected(llvm::Expected<T>), that will
transparently handle llvm::Errors.

I'll write a unit test for these changes next.

It's probably best to wait for https://reviews.llvm.org/D39435 to land, and then rebase.

rwols marked 2 inline comments as done.Nov 3 2017, 12:29 PM
rwols added inline comments.
clangd/ClangdServer.cpp
69–70

Haven't tried implementing this yet, but from some simple experiments it doesn't seem necessary.

rwols updated this revision to Diff 121605.Nov 4 2017, 4:12 PM
  • Rebase to latest SVN revision.
  • Use void reply(llvm:Expected<Twine> Result) for transparent error handling.
  • Use FSProvider from ClangdServer to provide a filesystem for format::getStyle.
  • Organize things around llvm::Expected, I've become a fan of that class :-)

Thanks! Mostly just comments around the IO (which I'm still learning about myself).
Otherwise this LG.

I'm afraid you're going to run into some merge conflicts with rL317486 - I think/hope not too bad, though.

clangd/ClangdServer.cpp
69–70

Thanks for doing this. The reason it's needed is we want to run clangd on systems where the source tree is not a real filesystem. Looking on the real filesystem for .clang-format files would be incorrect.

352–354

As I understand, we tag the return values to ensure we get consistent reads, and describe the consistency of those reads, on systems that support that (e.g. snapshot filesystems).

I don't think we need to do that here, because:

  • formatting only depends on the formatted file, and .clang-format
  • the formatted file's contents are passed directly
  • we should aggressively cache .clang-format reads, I think (see below)
534

getStyle is going to stat several files, in the usual case.
This seems less than ideal, particularly when we're doing format-on-type. Filesystems are not always fast/cheap.

We should really cache this, and I think "when the file is opened" is a reasonable tradeoff between simplicity and producing sensible user-facing behavior.

Unfortunately I'm not sure there's a sensible place to do things at present: addDocument() is fast and doing blocking IO there might be bad. Doing it asynchronously also seems bad (what happens if style's not ready?)

Next best thing would be "first format request when the file is opened". If you feel like doing that here (only computing the style if it's not already known for this file), then feel free, but a FIXME is fine too.

(FWIW, Google has a slow FS that's important to us, but we disable .clang-format scanning on that FS in other ways because it only holds code in one style. Other users might care about this, though)

clangd/ClangdServer.h
320

I don't understand this comment - I think we require and use Code.

If you mean we can just read Code from disk, that won't work: we want to format the dirty buffer in memory, not the last saved version.

rwols updated this revision to Diff 122606.Nov 12 2017, 2:06 PM
  • Merge with upstream, fix merge conflicts
  • Add a FIXME for a caching strategy for .clang-format files
rwols marked 4 inline comments as done.Nov 12 2017, 2:08 PM

Is this ready to merge? I'd like to implement tests in another differential, I'm having trouble referencing a temp dir from lit in a JSON request to clangd.

clangd/ClangdServer.cpp
534

but a FIXME is fine too.

I've added a FIXME note. I think implementing a caching strategy is worthy of its own differential revision.

ilya-biryukov added inline comments.Nov 14 2017, 8:50 AM
clangd/ClangdLSPServer.cpp
146

NIT: remove braces from single-statement branches

156

NIT: remove braces from single-statement branches

clangd/ClangdServer.h
289

Why do we accept Code as a parameter here instead of getting it internally?

Maybe we should consider moving this method out of ClangdServer? Its signature looks pretty self-contained now.

rwols updated this revision to Diff 122904.Nov 14 2017, 1:26 PM
rwols marked an inline comment as done.

Removed braces around single statement if-else.

rwols marked 2 inline comments as done.Nov 14 2017, 1:32 PM
rwols added inline comments.
clangd/ClangdServer.h
289

There are a couple intertwined problems:

  1. replacementsToEdits wants llvm::StringRef Code
  2. ClangdServer::formatCode wants llvm::StringRef Code
  3. ClangdServer::getDocument returns an std::string

So yes, in principle you can call getDocument when you need it for replacementsToEdits, and you can let formatCode itself call getDocument for clang::format::getStyle. But then we create two copies of the document contents for one LSP request.

If getDocument returned an llvm::StringRef, I'd probably vote for removing the Code argument everywhere and call getDocument as needed.

ilya-biryukov added inline comments.Nov 15 2017, 2:15 AM
clangd/ClangdServer.h
289

Oh, I see. Calling getDocument twice does not really make sense.
Maybe we could move a call to replacementsToEdits into formatOnFile and make it return vector<TextEdit>? Seems to be solving both problems.

We could've made getDocument return StringRef, but we'd have to be more careful to ensure it's actually copied when we're scheduling async operations, worth a separate review.

rwols added inline comments.Nov 18 2017, 2:16 AM
clangd/ClangdServer.h
289

Maybe we could move a call to replacementsToEdits into formatOnFile and make it return vector<TextEdit>?

I disagree, this will bring LSP-specific protocols into ClangdServer. The translation from tooling::Replacement to clangd::TextEdit should remain in ClangdLSPServer.

rwols added inline comments.Nov 18 2017, 4:05 AM
clangd/ClangdServer.h
289

I tried this anyway, but ClangdLSPServer::getFixIts also uses replacementsToEdits. And that maintains a map with tooling::Replacements. So, if we want to move replacementsToEdits into ClangdServer, we would have to move that map into ClangdServer too I think.

ilya-biryukov added inline comments.Nov 20 2017, 2:27 AM
clangd/ClangdServer.h
289

I disagree, this will bring LSP-specific protocols into ClangdServer. The translation from tooling::Replacement to clangd::TextEdit should remain in ClangdLSPServer.

+1 to this, I'd also not make ClangdServer LSP-specific. However we already use some LSP structs from Protocol.h there to avoid code duplication (i.e., CompletionItem, though LSP-specific, suits ClangdServer pretty well). In principle, Protocol.h should not be a dependency, though, but we try to manage what we take from there instead. clangd::TextEdit seems to be in line with other things we take from there now (that is, Ranges, Locations, etc)

I tried this anyway, but ClangdLSPServer::getFixIts also uses replacementsToEdits. And that maintains a map with tooling::Replacements. So, if we want to move replacementsToEdits into ClangdServer, we would have to move that map into ClangdServer too I think.

Could we change the map to store clangd::TextEdit instead? It's a bit unfortunate that we'll be making unnecessary tooling::Replacement -> clangd::TextEdit conversions and those aren't free. But honestly, I don't think this should bite us performance-wise.
Given that we already use LSP's Location and Range encoding in other APIs, we'll get a more consistent interface.

rwols updated this revision to Diff 124307.Nov 26 2017, 8:16 AM
  • Rebase to latest upstream revision.
  • Go all-in with TextEdit, even down to ClangdUnit.cpp.
  • Move FixItsMap to ClangdServer. ClangdLSPServer is much cleaner now.
  • Remove the cached FixIts when the client closes the document.
  • Fix (or introduce bug?) wrong conversion between clang line/columns and LSP line/columns. From what I understand, clang line/cols are 1-based, and LSP line/cols are 0-based. The line numbers were already converted correctly (subtracting 1 when going from clang line numbers to LSP line numbers), but the column numbers were not converted correctly. Consequently, a bunch of tests had to be adjusted for the correct (or wrong?) column numbers.
rwols marked 2 inline comments as done.Nov 26 2017, 8:18 AM
rwols added inline comments.Nov 26 2017, 8:22 AM
clangd/ClangdUnit.cpp
168

Why subtract 1 from the line here, but not from the column? When subtracting 1 from both, diagnostics appear correct in my client. I have moved this logic into SourceLocToPosition.

@ilya-biryukov We should get this landed.
I'm happy (my comments were addressed a few rounds ago) - any concerns?

rwols updated this revision to Diff 125267.Dec 2 2017, 6:41 AM
  • Merge with upstream revision 319613
  • Fix FixItHints not getting applied correctly in all cases. The problem is that clang::CharSourceRange can either be a TokenRange or a CharRange. In the case of a TokenRange, we have to determine the end of the token ourselves (this would occur, for example, in things like if (i=1) { ... }; the fixit hint to replace = with == would be a TokenRange for some reason).

    Consequently, a tiny change had to be made to the fixits.test file, where the fixit hint to replace = with == would previously have an empty replacement range, whereas now it has a replacement range of length 1, exactly removing the = and inserting == in its place.

After playing around with these changes more I'm fairly certain I'm on the right
track for converting clang ranges/positions to clangd ranges/positions.

rwols retitled this revision from [clangd] formatting: don't ignore style to [clangd] various fixes.Dec 2 2017, 6:43 AM
rwols edited the summary of this revision. (Show Details)
malaperle edited edge metadata.Dec 3 2017, 9:13 PM
  • Look up in parent directories for a .clang-format file. Use "LLVM" as fallback style.
  • Move cached fixits to ClangdServer, remove fixits once the client closes the document.
  • Fix translations between LSP rows/cols (0-based) and clang row/cols (1-based).

I'd rather keep one commit per fix or feature. I would think the clang-format part should be one commit/review on its own. But I'm not very familiar with the practices here so I'll let others comment.

ilya-biryukov edited edge metadata.Dec 5 2017, 4:58 AM

I'd rather keep one commit per fix or feature. I would think the clang-format part should be one commit/review on its own. But I'm not very familiar with the practices here so I'll let others comment.

+1, please create a separate review per fix or feature.

clangd/ClangdServer.h
288

There are two ways to return fixits from ClangdServer now, one is DiagnosticsConsumer, the other is the new getFixIts method.

I'd also argue the fact that we can't attach fixits to diagnostics is LSP-specific and therefore it should be handled by ClangdLSPServer.
If the problem is that the cache is never cleaned up, we could simply remove the fixits when handling textDocument/didClose.

I think we should not put this method into ClangdServer.

rwols added a comment.Dec 5 2017, 11:17 AM

I will split this diff up into separate parts.

rwols abandoned this revision.Dec 13 2017, 12:28 AM