This is an archive of the discontinued LLVM Phabricator instance.

[clangd] Build index on preamble changes instead of the AST changes
ClosedPublic

Authored by ilya-biryukov on May 23 2018, 11:25 AM.

Details

Summary

This is more efficient and avoids data races when reading files that
come from the preamble. The staleness can occur when reading a file
from disk that changed after the preamble was built. This can lead to
crashes, e.g. when parsing comments.

We do not to rely on symbols from the main file anyway, since any info
that those provide can always be taken from the AST.

Diff Detail

Event Timeline

ilya-biryukov created this revision.May 23 2018, 11:25 AM

We do not to rely on symbols from the main file anyway, since any info hat those provide can always be taken from the AST.

I'll be adding those soon for workspace symbols... And also for document symbols.

We do not to rely on symbols from the main file anyway, since any info hat those provide can always be taken from the AST.

I'll be adding those soon for workspace symbols... And also for document symbols.

I can add extra code to build pieces for the AST later. This is not hard to do, but would require rearranging some code a bit more.
Will try to send the change tomorrow for review tomorrow. Does that SG?

@malaperle, a slight offtopic. I was wondering which completion options do you use for clangd? Defaults?

We do not to rely on symbols from the main file anyway, since any info hat those provide can always be taken from the AST.

I'll be adding those soon for workspace symbols... And also for document symbols.

I can add extra code to build pieces for the AST later. This is not hard to do, but would require rearranging some code a bit more.
Will try to send the change tomorrow for review tomorrow. Does that SG?

Sounds good. Doesn't have to be tomorrow :) Just making sure we're not on incompatible paths.

  • Added forgotten renames

Would it be possible to add some integration tests for file index plus preamble?

clangd/index/FileIndex.cpp
37

Would this give us decls in namespaces? It seems that AST.getTranslationUnitDecl()->decls() are only decls that are immediately inside the TU context.

Would it be possible to add some integration tests for file index plus preamble?

Sure, will do

clangd/index/FileIndex.cpp
37

I'll double check and add a test for that, but I think the indexer visits the namespaces that we provide, I'm not sure if we have tests though.

ioeric added inline comments.May 24 2018, 7:24 AM
clangd/index/FileIndex.cpp
37

Oh, right! In that case, it seems that a single TU decl which contains all interesting decls would be sufficient? Is it possible that preamble doesn't have all available top level decls in TU decl? This seems to be possible for ParsedAST, which is why we used getTopLevelDecls, I think?

  • Added a test for preamble rebuilds.

Added the test.

clangd/index/FileIndex.cpp
37

In that case, it seems that a single TU decl which contains all interesting decls would be sufficient?

I've tried doing the TU decl only and it does not work, because indexTopLevelDecl checks that location of decl that you pass to it is valid xD. Obviously, that's not the case for the TU decl. But it does visit the namespaces recursively, new test also checks for that, so we should be fine with the current code.

This seems to be possible for ParsedAST, which is why we used getTopLevelDecls, I think?

getTopLevelDecls was copied from the ASTUnit without much thought.
IIRC, it allows less deserialization from preamble when traversing the AST for things like "go to def", etc. This is definitely not something we're optimizing for here, since we run on the preamble AST itself.
Now that I think about it, I don't think we need to store top-level decls of preamble at all for any of our features...

ioeric accepted this revision.May 24 2018, 8:46 AM

lg. Thanks for adding the test!

clangd/index/FileIndex.cpp
37

Sounds good. Thanks for the clarification!

This revision is now accepted and ready to land.May 24 2018, 8:46 AM
This revision was automatically updated to reflect the committed changes.