This is an archive of the discontinued LLVM Phabricator instance.

[clangd] Auto-index watches global CDB for changes.
ClosedPublic

Authored by sammccall on Nov 23 2018, 12:38 PM.

Details

Summary

Instead of receiving compilation commands, auto-index is triggered by just
filenames to reindex, and gets commands from the global comp DB internally.
This has advantages:

  • more of the work can be done asynchronously (fetching compilation commands upfront can be slow for large CDBs)
  • we get access to the CDB which can be used to retrieve interpolated commands for headers (useful in some cases where the original TU goes away)
  • fits nicely with the filename-only change observation from r347297

The interface to GlobalCompilationDatabase gets extended: when retrieving a
compile command, the GCDB can optionally report the project the file belongs to.
This naturally fits together with getCompileCommand: it's hard to implement one
without the other. But because most callers don't care, I've ended up with an
awkward optional-out-param-in-virtual method pattern - maybe there's a better
one.

This is the main missing integration point between ClangdServer and
BackgroundIndex, after this we should be able to add an auto-index flag.

Event Timeline

sammccall created this revision.Nov 23 2018, 12:38 PM
kadircet accepted this revision.Nov 26 2018, 1:24 AM

LGTM, seems to share a lot of context with what I am preparing for loading shards.
Interfaces looks compatible with what I had before, only difference is it was more tooling::CompileCommand oriented before, now it can only look at filenames, which seems more natural.

Thanks!

clangd/index/Background.cpp
115

s/so //

130

In case of an OverlayCDB with some commands set through LSP configuration, we might end up Project.SourceRoot being an empty string, which will result in creating the index storage directory in cwd. I believe we wouldn't want that, but not sure if it should be handled by BackgroundIndex or DiskBackedIndexStorage. WDYT?

This revision is now accepted and ready to land.Nov 26 2018, 1:24 AM
sammccall marked 3 inline comments as done.Nov 26 2018, 1:43 AM
sammccall added inline comments.
clangd/index/Background.cpp
130

You're right. To a first approximation, I think that the right behavior is to keep indexing in memory and just not persist the shards.
The storage seems like the right layer because:

  • the factory makes it easy to encapsulate this special case
  • we may want to change the strategy in future (e.g. write index files into a central location if the builddir is unavailable).

The downside I see is by encapsulating it we can't do things like "downgrade priority of indexing if storage isn't available", but we can find ways around that if we really want to.

I've added a null storage to this patch which is used if the CDB dir is empty.

sammccall updated this revision to Diff 175208.Nov 26 2018, 1:43 AM
sammccall marked an inline comment as done.

If the CDB dir is unknown, don't try to write shards to disk.

This revision was automatically updated to reflect the committed changes.