This is an archive of the discontinued LLVM Phabricator instance.

[clangd] Filter out non-governed files from broadcast
ClosedPublic

Authored by kadircet on Jul 5 2019, 7:07 AM.

Event Timeline

kadircet created this revision.Jul 5 2019, 7:07 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 5 2019, 7:07 AM
sammccall added inline comments.Jul 8 2019, 7:52 AM
clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
106

The duplication here is a bit troublesome.

And we have divergences in behavior - probably not important ones, but I can't say I know for sure. (e.g. here we won't create the CDB as a side-effect of getPathInfo() if there's a CompileCommandsDir, and will return the dir even if the CDB doesn't exist)

I think we might want to have some central logic parameterized by a struct, e.g.

struct DirectoryBasedCompilationDatabase::Lookup {
  // inputs
  PathRef File;
  bool ShouldBroadcast;

  // outputs
  tooling::CompilationDatabase *CDB;
  ProjectInfo Info;
  bool EverBroadcast;
};
lookupCDB(Lookup); // replaces getCDBInDirLocked
209

here we're locking/unlocking the mutex maybe thousands of times, to do mostly redundant lookups into the cache

Seems worth doing at least one of:

  • deduplicate directories, so we lock once for each distinct directory and build a string->bool map. Then use that map to filter the results
  • lock around the whole loop and use getCDBInDirLocked()
clang-tools-extra/clangd/index/Background.cpp
653

This looks like a bad assertion: it should be OK to provide compile commands but not project info.

(Otherwise getProjectInfo should be pure virtual, but I'm concerned about the raciness if we're going to insist they're consistent but not provide any way of synchronizing)

I'd suggest we should just not index such files (for now). Later we can start passing "" to the index storage factory to get the fallback storage (I think today we pass "", but the factory doesn't handle it - oops!)

kadircet updated this revision to Diff 208686.Jul 9 2019, 8:08 AM
kadircet marked 3 inline comments as done.
  • Address comments
kadircet added inline comments.Jul 9 2019, 8:12 AM
clang-tools-extra/clangd/index/Background.cpp
653

actually we have a nullstorage for that, which just logs a messsage and doesn't persist the shard.

kadircet updated this revision to Diff 208697.Jul 9 2019, 8:26 AM
  • Rearrange changes to get a better diff
sammccall accepted this revision.Jul 9 2019, 10:19 AM
sammccall added inline comments.
clang-tools-extra/clangd/GlobalCompilationDatabase.cpp
109

I don't think we'll ever fix this, feel free to drop the FIXME if you like.

118

return Path == Result.PI.SourceRoot?

125

(Why not do this in the one pass above? Doesn't seem worth two passes to avoid locking while iterating over AllFiles)

134

assumes isn't strong enough here - we're defining it this way, and should say why.

e.g. "A file is only part of this CDB if lookup for the file would find this CDB."

clang-tools-extra/clangd/GlobalCompilationDatabase.h
115

(random note: we should really rename this something like FixedCDBDir. Probably not in this patch, but what a confusing name)

This revision is now accepted and ready to land.Jul 9 2019, 10:19 AM
kadircet updated this revision to Diff 208890.Jul 10 2019, 1:21 AM
kadircet marked 3 inline comments as done.
  • Address comments
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptJul 10 2019, 7:11 AM