[include-fixer] Refactor include fixer to be usable as a plugin

[bkramer]

Summary

• Refactor the external sema source into a visible class
• Add support for emitting FixIts
• Wrap up include fixer as a plugin as I did with clang-tidy

Test case will follow as soon as I wire this up in libclang.

[hokein]

include-fixer/IncludeFixer.cpp

64 ↗

(On Diff #78210)

also make it const?

124 ↗

(On Diff #78210)

nit: /*AddQualifiers=*/false.

136 ↗

(On Diff #78210)

I have a concern that Placed here might be not the replacement for inserting the new header, becuase the Reps returned from createIncludeFixerReplacements may have some replacements for cleanup.

To make it more correct, maybe we can check whether Placed.getReplacementText() is equal to "#include" + Context.getHeaderInfos().front().Header?

283 ↗

(On Diff #78210)

You can use SM directly here since you have created a SM variable for SourceManager above? The same below.

include-fixer/IncludeFixer.h

119 ↗

(On Diff #78210)

Make it a const member function?

124 ↗

(On Diff #78210)

A blank line below.

include-fixer/plugin/IncludeFixerPlugin.cpp

1 ↗

(On Diff #78210)

nit: IncludeFixerPlugin

91 ↗

(On Diff #78210)

nit: clang-tidy=>include-fixer

[bkramer]

• Address review comments.

include-fixer/IncludeFixer.cpp

136 ↗

(On Diff #78210)

I don't think that will work. We do want to put the replacement into the right position so we have to apply the full cleanup, right? Just comparing with the header path doesn't work because the cleanup is larger than that.

[ioeric]

include-fixer/IncludeFixer.cpp

136 ↗

(On Diff #78210)

Yep, createIncludeFixerReplacements also sorts #includes with formatReplacements, so the returned replacements can be larger than header insertion.

[hokein]

include-fixer/IncludeFixer.cpp

136 ↗

(On Diff #78210)

I see. What I concern is that the behavior won't be correct if there are multiple replacements in *Reps. But looks like if we pass only one Replacement for inserting the new header to createIncludeFixerReplacements, the Reps only has a Replacement inside, which is correct.

Maybe add an assert here?

[ioeric]

include-fixer/IncludeFixer.cpp

136 ↗

(On Diff #78210)

This is implementation detail; the interface does not guarantee that. And I guess there might still be cases where more than one replacement is generated.

[bkramer]

• Turn loop into an assertions, there should never be more than one replacement coming back.

[bkramer]

include-fixer/IncludeFixer.cpp

136 ↗

(On Diff #78210)

We'll see if this comes up. The current implementation wouldn't deal well with multiple replacements as it would format them as alternatives. I'll do the assert for now.

[hokein]

LGTM.

clang-tools-extra/trunk/include-fixer/CMakeLists.txt

add_clang_library(clangIncludeFixer
   clangLex
   clangParse
   clangSema
   clangTooling
   clangToolingCore
   findAllSymbols
   )
 
+add_subdirectory(plugin)
 add_subdirectory(tool)
 add_subdirectory(find-all-symbols)

clang-tools-extra/trunk/include-fixer/IncludeFixer.h

 //===-- IncludeFixer.h - Include inserter -----------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_INCLUDE_FIXER_INCLUDEFIXER_H
 #define LLVM_CLANG_TOOLS_EXTRA_INCLUDE_FIXER_INCLUDEFIXER_H
 
 #include "IncludeFixerContext.h"
 #include "SymbolIndexManager.h"
 #include "clang/Format/Format.h"
+#include "clang/Sema/ExternalSemaSource.h"
 #include "clang/Tooling/Core/Replacement.h"
 #include "clang/Tooling/Tooling.h"
 #include <memory>
 #include <vector>
 
 namespace clang {
 
 class CompilerInvocation;

 /// \return Formatted replacements for inserting, sorting headers and adding
 /// qualifiers on success; otherwise, an llvm::Error carrying llvm::StringError
 /// is returned.
 llvm::Expected<tooling::Replacements> createIncludeFixerReplacements(
     StringRef Code, const IncludeFixerContext &Context,
     const format::FormatStyle &Style = format::getLLVMStyle(),
     bool AddQualifiers = true);
 
+/// Handles callbacks from sema, does the include lookup and turns it into an
+/// IncludeFixerContext.
+class IncludeFixerSemaSource : public clang::ExternalSemaSource {
+public:
+  explicit IncludeFixerSemaSource(SymbolIndexManager &SymbolIndexMgr,
+                                  bool MinimizeIncludePaths,
+                                  bool GenerateDiagnostics)
+      : SymbolIndexMgr(SymbolIndexMgr),
+        MinimizeIncludePaths(MinimizeIncludePaths),
+        GenerateDiagnostics(GenerateDiagnostics) {}
+
+  void setCompilerInstance(CompilerInstance *CI) { this->CI = CI; }
+  void setFilePath(StringRef FilePath) { this->FilePath = FilePath; }
+
+  /// Callback for incomplete types. If we encounter a forward declaration we
+  /// have the fully qualified name ready. Just query that.
+  bool MaybeDiagnoseMissingCompleteType(clang::SourceLocation Loc,
+                                        clang::QualType T) override;
+
+  /// Callback for unknown identifiers. Try to piece together as much
+  /// qualification as we can get and do a query.
+  clang::TypoCorrection CorrectTypo(const DeclarationNameInfo &Typo,
+                                    int LookupKind, Scope *S, CXXScopeSpec *SS,
+                                    CorrectionCandidateCallback &CCC,
+                                    DeclContext *MemberContext,
+                                    bool EnteringContext,
+                                    const ObjCObjectPointerType *OPT) override;
+
+  /// Get the minimal include for a given path.
+  std::string minimizeInclude(StringRef Include,
+                              const clang::SourceManager &SourceManager,
+                              clang::HeaderSearch &HeaderSearch) const;
+
+  /// Get the include fixer context for the queried symbol.
+  IncludeFixerContext
+  getIncludeFixerContext(const clang::SourceManager &SourceManager,
+                         clang::HeaderSearch &HeaderSearch) const;
+
+private:
+  /// Query the database for a given identifier.
+  bool query(StringRef Query, StringRef ScopedQualifiers, tooling::Range Range);
+
+  CompilerInstance *CI;
+
+  /// The client to use to find cross-references.
+  SymbolIndexManager &SymbolIndexMgr;
+
+  /// The information of the symbols being queried.
+  std::vector<IncludeFixerContext::QuerySymbolInfo> QuerySymbolInfos;
+
+  /// All symbol candidates which match QuerySymbol. We only include the first
+  /// discovered identifier to avoid getting caught in results from error
+  /// recovery.
+  std::vector<find_all_symbols::SymbolInfo> MatchedSymbols;
+
+  /// The file path to the file being processed.
+  std::string FilePath;
+
+  /// Whether we should use the smallest possible include path.
+  bool MinimizeIncludePaths = true;
+
+  /// Whether we should generate diagnostics with fixits for missing symbols.
+  bool GenerateDiagnostics = false;
+};
 } // namespace include_fixer
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_INCLUDE_FIXER_INCLUDEFIXER_H

clang-tools-extra/trunk/include-fixer/IncludeFixer.cpp

 //===-- IncludeFixer.cpp - Include inserter based on sema callbacks -------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "IncludeFixer.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Parse/ParseAST.h"
-#include "clang/Sema/ExternalSemaSource.h"
 #include "clang/Sema/Sema.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/raw_ostream.h"
 
 #define DEBUG_TYPE "include-fixer"
 
 using namespace clang;
 
 namespace clang {
 namespace include_fixer {
 namespace {
-
 /// Manages the parse, gathers include suggestions.
-class Action : public clang::ASTFrontendAction,
-               public clang::ExternalSemaSource {
+class Action : public clang::ASTFrontendAction {
 public:
   explicit Action(SymbolIndexManager &SymbolIndexMgr, bool MinimizeIncludePaths)
-      : SymbolIndexMgr(SymbolIndexMgr),
-        MinimizeIncludePaths(MinimizeIncludePaths) {}
+      : SemaSource(SymbolIndexMgr, MinimizeIncludePaths,
+                   /*GenerateDiagnostics=*/false) {}
 
   std::unique_ptr<clang::ASTConsumer>
   CreateASTConsumer(clang::CompilerInstance &Compiler,
                     StringRef InFile) override {
-    FilePath = InFile;
+    SemaSource.setFilePath(InFile);
     return llvm::make_unique<clang::ASTConsumer>();
   }
 
   void ExecuteAction() override {
     clang::CompilerInstance *Compiler = &getCompilerInstance();
     assert(!Compiler->hasSema() && "CI already has Sema");
 
     // Set up our hooks into sema and parse the AST.
     if (hasCodeCompletionSupport() &&
         !Compiler->getFrontendOpts().CodeCompletionAt.FileName.empty())
       Compiler->createCodeCompletionConsumer();
 
     clang::CodeCompleteConsumer *CompletionConsumer = nullptr;
     if (Compiler->hasCodeCompletionConsumer())
       CompletionConsumer = &Compiler->getCodeCompletionConsumer();
 
     Compiler->createSema(getTranslationUnitKind(), CompletionConsumer);
-    Compiler->getSema().addExternalSource(this);
+    SemaSource.setCompilerInstance(Compiler);
+    Compiler->getSema().addExternalSource(&SemaSource);
 
     clang::ParseAST(Compiler->getSema(), Compiler->getFrontendOpts().ShowStats,
                     Compiler->getFrontendOpts().SkipFunctionBodies);
   }
 
-  /// Callback for incomplete types. If we encounter a forward declaration we
-  /// have the fully qualified name ready. Just query that.
-  bool MaybeDiagnoseMissingCompleteType(clang::SourceLocation Loc,
-                                        clang::QualType T) override {
-    // Ignore spurious callbacks from SFINAE contexts.
-    if (getCompilerInstance().getSema().isSFINAEContext())
-      return false;
+  IncludeFixerContext
+  getIncludeFixerContext(const clang::SourceManager &SourceManager,
+                         clang::HeaderSearch &HeaderSearch) const {
+    return SemaSource.getIncludeFixerContext(SourceManager, HeaderSearch);
+  }
+
+private:
+  IncludeFixerSemaSource SemaSource;
+};
+
+} // namespace
+
+IncludeFixerActionFactory::IncludeFixerActionFactory(
+    SymbolIndexManager &SymbolIndexMgr,
+    std::vector<IncludeFixerContext> &Contexts, StringRef StyleName,
+    bool MinimizeIncludePaths)
+    : SymbolIndexMgr(SymbolIndexMgr), Contexts(Contexts),
+      MinimizeIncludePaths(MinimizeIncludePaths) {}
+
+IncludeFixerActionFactory::~IncludeFixerActionFactory() = default;
+
+bool IncludeFixerActionFactory::runInvocation(
+    clang::CompilerInvocation *Invocation, clang::FileManager *Files,
+    std::shared_ptr<clang::PCHContainerOperations> PCHContainerOps,
+    clang::DiagnosticConsumer *Diagnostics) {
+  assert(Invocation->getFrontendOpts().Inputs.size() == 1);
+
+  // Set up Clang.
+  clang::CompilerInstance Compiler(PCHContainerOps);
+  Compiler.setInvocation(Invocation);
+  Compiler.setFileManager(Files);
+
+  // Create the compiler's actual diagnostics engine. We want to drop all
+  // diagnostics here.
+  Compiler.createDiagnostics(new clang::IgnoringDiagConsumer,
+                             /*ShouldOwnClient=*/true);
+  Compiler.createSourceManager(*Files);
+
+  // We abort on fatal errors so don't let a large number of errors become
+  // fatal. A missing #include can cause thousands of errors.
+  Compiler.getDiagnostics().setErrorLimit(0);
+
+  // Run the parser, gather missing includes.
+  auto ScopedToolAction =
+      llvm::make_unique<Action>(SymbolIndexMgr, MinimizeIncludePaths);
+  Compiler.ExecuteAction(*ScopedToolAction);
+
+  Contexts.push_back(ScopedToolAction->getIncludeFixerContext(
+      Compiler.getSourceManager(),
+      Compiler.getPreprocessor().getHeaderSearchInfo()));
+
+  // Technically this should only return true if we're sure that we have a
+  // parseable file. We don't know that though. Only inform users of fatal
+  // errors.
+  return !Compiler.getDiagnostics().hasFatalErrorOccurred();
+}
+
+static void addDiagnosticsForContext(TypoCorrection &Correction,
+                                     const IncludeFixerContext &Context,
+                                     StringRef Code, SourceLocation StartOfFile,
+                                     ASTContext &Ctx) {
+  auto Reps = createIncludeFixerReplacements(
+      Code, Context, format::getLLVMStyle(), /*AddQualifiers=*/false);
+  if (!Reps)
+    return;
+
+  unsigned DiagID = Ctx.getDiagnostics().getCustomDiagID(
+      DiagnosticsEngine::Note, "Add '#include %0' to provide the missing "
+                               "declaration [clang-include-fixer]");
+
+  // FIXME: Currently we only generate a diagnostic for the first header. Give
+  // the user choices.
+  assert(Reps->size() == 1 && "Expected exactly one replacement");
+  const tooling::Replacement &Placed = *Reps->begin();
+
+  auto Begin = StartOfFile.getLocWithOffset(Placed.getOffset());
+  auto End = Begin.getLocWithOffset(Placed.getLength());
+  PartialDiagnostic PD(DiagID, Ctx.getDiagAllocator());
+  PD << Context.getHeaderInfos().front().Header
+     << FixItHint::CreateReplacement(SourceRange(Begin, End),
+                                     Placed.getReplacementText());
+  Correction.addExtraDiagnostic(std::move(PD));
+}
+
+bool IncludeFixerSemaSource::MaybeDiagnoseMissingCompleteType(
+    clang::SourceLocation Loc, clang::QualType T) {
+  if (CI->getSema().isSFINAEContext())
 
-    clang::ASTContext &context = getCompilerInstance().getASTContext();
-    std::string QueryString =
-        T.getUnqualifiedType().getAsString(context.getPrintingPolicy());
-    DEBUG(llvm::dbgs() << "Query missing complete type '" << QueryString
-                       << "'");
-    // Pass an empty range here since we don't add qualifier in this case.
-    query(QueryString, "", tooling::Range());
-    return false;
-  }
+  clang::ASTContext &context = CI->getASTContext();
+  DEBUG(llvm::dbgs() << "Query missing complete type '" << QueryString << "'");
+
+  if (GenerateDiagnostics) {
+    TypoCorrection Correction;
+    FileID FID = CI->getSourceManager().getFileID(Loc);
+    StringRef Code = CI->getSourceManager().getBufferData(FID);
+    SourceLocation StartOfFile =
+        CI->getSourceManager().getLocForStartOfFile(FID);
+    addDiagnosticsForContext(
+        Correction,
+        getIncludeFixerContext(CI->getSourceManager(),
+                               CI->getPreprocessor().getHeaderSearchInfo()),
+        Code, StartOfFile, CI->getASTContext());
+    for (const PartialDiagnostic &PD : Correction.getExtraDiagnostics())
+      CI->getSema().Diag(Loc, PD);
+  }
+  return true;
 
-  /// Callback for unknown identifiers. Try to piece together as much
-  /// qualification as we can get and do a query.
-  clang::TypoCorrection CorrectTypo(const DeclarationNameInfo &Typo,
-                                    int LookupKind, Scope *S, CXXScopeSpec *SS,
-                                    CorrectionCandidateCallback &CCC,
-                                    DeclContext *MemberContext,
-                                    bool EnteringContext,
-                                    const ObjCObjectPointerType *OPT) override {
-    // Ignore spurious callbacks from SFINAE contexts.
-    if (getCompilerInstance().getSema().isSFINAEContext())
-      return clang::TypoCorrection();
+clang::TypoCorrection IncludeFixerSemaSource::CorrectTypo(
+    const DeclarationNameInfo &Typo, int LookupKind, Scope *S, CXXScopeSpec *SS,
+    CorrectionCandidateCallback &CCC, DeclContext *MemberContext,
+    bool EnteringContext, const ObjCObjectPointerType *OPT) {
+  if (CI->getSema().isSFINAEContext())
 
-    // We currently ignore the unidentified symbol which is not from the
-    // main file.
-    //
-    // However, this is not always true due to templates in a non-self contained
-    // header, consider the case:
-    //
-    //   // header.h
-    //   template <typename T>
-    //   class Foo {
-    //     T t;
-    //   };
-    //
-    //   // test.cc
-    //   // We need to add <bar.h> in test.cc instead of header.h.
-    //   class Bar;
-    //   Foo<Bar> foo;
-    //
-    // FIXME: Add the missing header to the header file where the symbol comes
-    // from.
-    if (!getCompilerInstance().getSourceManager().isWrittenInMainFile(
-            Typo.getLoc()))
-      return clang::TypoCorrection();
+  if (!CI->getSourceManager().isWrittenInMainFile(Typo.getLoc()))
 
-    std::string TypoScopeString;
-    if (S) {
-      // FIXME: Currently we only use namespace contexts. Use other context
-      // types for query.
-      for (const auto *Context = S->getEntity(); Context;
-           Context = Context->getParent()) {
-        if (const auto *ND = dyn_cast<NamespaceDecl>(Context)) {
-          if (!ND->getName().empty())
-            TypoScopeString = ND->getNameAsString() + "::" + TypoScopeString;
-        }
-      }
-    }
 
-    auto ExtendNestedNameSpecifier = [this](CharSourceRange Range) {
-      StringRef Source =
-          Lexer::getSourceText(Range, getCompilerInstance().getSourceManager(),
-                               getCompilerInstance().getLangOpts());
+        Lexer::getSourceText(Range, CI->getSourceManager(), CI->getLangOpts());
 
-      // Skip forward until we find a character that's neither identifier nor
-      // colon. This is a bit of a hack around the fact that we will only get a
-      // single callback for a long nested name if a part of the beginning is
-      // unknown. For example:
-      //
-      // llvm::sys::path::parent_path(...)
-      // ^~~~  ^~~
-      //    known
-      //            ^~~~
-      //      unknown, last callback
-      //                  ^~~~~~~~~~~
-      //                  no callback
-      //
-      // With the extension we get the full nested name specifier including
-      // parent_path.
-      // FIXME: Don't rely on source text.
-      const char *End = Source.end();
-      while (isIdentifierBody(*End) || *End == ':')
-        ++End;
 
-      return std::string(Source.begin(), End);
-    };
 
-    /// If we have a scope specification, use that to get more precise results.
-    std::string QueryString;
-    tooling::Range SymbolRange;
-    const auto &SM = getCompilerInstance().getSourceManager();
-    auto CreateToolingRange = [&QueryString, &SM](SourceLocation BeginLoc) {
-      return tooling::Range(SM.getDecomposedLoc(BeginLoc).second,
-                            QueryString.size());
-    };
-    if (SS && SS->getRange().isValid()) {
-      auto Range = CharSourceRange::getTokenRange(SS->getRange().getBegin(),
-                                                  Typo.getLoc());
+  const auto &SM = CI->getSourceManager();
 
-      QueryString = ExtendNestedNameSpecifier(Range);
-      SymbolRange = CreateToolingRange(Range.getBegin());
-    } else if (Typo.getName().isIdentifier() && !Typo.getLoc().isMacroID()) {
-      auto Range =
-          CharSourceRange::getTokenRange(Typo.getBeginLoc(), Typo.getEndLoc());
 
-      QueryString = ExtendNestedNameSpecifier(Range);
-      SymbolRange = CreateToolingRange(Range.getBegin());
-    } else {
-      QueryString = Typo.getAsString();
-      SymbolRange = CreateToolingRange(Typo.getLoc());
-    }
 
-    DEBUG(llvm::dbgs() << "TypoScopeQualifiers: " << TypoScopeString << "\n");
-    query(QueryString, TypoScopeString, SymbolRange);
 
-    // FIXME: We should just return the name we got as input here and prevent
-    // clang from trying to correct the typo by itself. That may change the
-    // identifier to something that's not wanted by the user.
-    return clang::TypoCorrection();
-  }
+  clang::TypoCorrection Correction(Typo.getName());
+  Correction.setCorrectionRange(SS, Typo);
+  if (GenerateDiagnostics) {
+    FileID FID = SM.getFileID(Typo.getLoc());
+    StringRef Code = SM.getBufferData(FID);
+    SourceLocation StartOfFile = SM.getLocForStartOfFile(FID);
+    addDiagnosticsForContext(
+        Correction,
+        getIncludeFixerContext(SM, CI->getPreprocessor().getHeaderSearchInfo()),
+        Code, StartOfFile, CI->getASTContext());
+  }
+  return Correction;
 
-  /// Get the minimal include for a given path.
-  std::string minimizeInclude(StringRef Include,
-                              const clang::SourceManager &SourceManager,
-                              clang::HeaderSearch &HeaderSearch) {
-    if (!MinimizeIncludePaths)
-      return Include;
+std::string IncludeFixerSemaSource::minimizeInclude(
+    StringRef Include, const clang::SourceManager &SourceManager,
+    clang::HeaderSearch &HeaderSearch) const {
 
-    // Get the FileEntry for the include.
-    StringRef StrippedInclude = Include.trim("\"<>");
-    const FileEntry *Entry =
-        SourceManager.getFileManager().getFile(StrippedInclude);
 
-    // If the file doesn't exist return the path from the database.
-    // FIXME: This should never happen.
-    if (!Entry)
-      return Include;
 
-    bool IsSystem;
-    std::string Suggestion =
-        HeaderSearch.suggestPathToFileForDiagnostics(Entry, &IsSystem);
 
-    return IsSystem ? '<' + Suggestion + '>' : '"' + Suggestion + '"';
-  }
 
-  /// Get the include fixer context for the queried symbol.
-  IncludeFixerContext
-  getIncludeFixerContext(const clang::SourceManager &SourceManager,
-                         clang::HeaderSearch &HeaderSearch) {
-    std::vector<find_all_symbols::SymbolInfo> SymbolCandidates;
-    for (const auto &Symbol : MatchedSymbols) {
-      std::string FilePath = Symbol.getFilePath().str();
-      std::string MinimizedFilePath = minimizeInclude(
-          ((FilePath[0] == '"' || FilePath[0] == '<') ? FilePath
-                                                      : "\"" + FilePath + "\""),
-          SourceManager, HeaderSearch);
-      SymbolCandidates.emplace_back(Symbol.getName(), Symbol.getSymbolKind(),
-                                    MinimizedFilePath, Symbol.getLineNumber(),
-                                    Symbol.getContexts(),
-                                    Symbol.getNumOccurrences());
-    }
-    return IncludeFixerContext(FilePath, QuerySymbolInfos, SymbolCandidates);
-  }
+IncludeFixerContext IncludeFixerSemaSource::getIncludeFixerContext(
+    const clang::SourceManager &SourceManager,
+    clang::HeaderSearch &HeaderSearch) const {
 
-private:
-  /// Query the database for a given identifier.
-  bool query(StringRef Query, StringRef ScopedQualifiers,
+bool IncludeFixerSemaSource::query(StringRef Query, StringRef ScopedQualifiers,
                                    tooling::Range Range) {
-    assert(!Query.empty() && "Empty query!");
 
-    // Save all instances of an unidentified symbol.
-    //
-    // We use conservative behavior for detecting the same unidentified symbol
-    // here. The symbols which have the same ScopedQualifier and RawIdentifier
-    // are considered equal. So that include-fixer avoids false positives, and
-    // always adds missing qualifiers to correct symbols.
-    if (!QuerySymbolInfos.empty()) {
-      if (ScopedQualifiers == QuerySymbolInfos.front().ScopedQualifiers &&
-          Query == QuerySymbolInfos.front().RawIdentifier) {
-        QuerySymbolInfos.push_back({Query.str(), ScopedQualifiers, Range});
-      }
-      return false;
-    }
 
-    DEBUG(llvm::dbgs() << "Looking up '" << Query << "' at ");
-    DEBUG(getCompilerInstance()
-              .getSourceManager()
-              .getLocForStartOfFile(
-                  getCompilerInstance().getSourceManager().getMainFileID())
-              .getLocWithOffset(Range.getOffset())
-              .print(llvm::dbgs(), getCompilerInstance().getSourceManager()));
-    DEBUG(llvm::dbgs() << " ...");
+  DEBUG(CI->getSourceManager()
+            .getLocForStartOfFile(CI->getSourceManager().getMainFileID())
+            .print(llvm::dbgs(), CI->getSourceManager()));
 
-    QuerySymbolInfos.push_back({Query.str(), ScopedQualifiers, Range});
 
-    // Query the symbol based on C++ name Lookup rules.
-    // Firstly, lookup the identifier with scoped namespace contexts;
-    // If that fails, falls back to look up the identifier directly.
-    //
-    // For example:
-    //
-    // namespace a {
-    // b::foo f;
-    // }
-    //
-    // 1. lookup a::b::foo.
-    // 2. lookup b::foo.
-    std::string QueryString = ScopedQualifiers.str() + Query.str();
-    // It's unsafe to do nested search for the identifier with scoped namespace
-    // context, it might treat the identifier as a nested class of the scoped
-    // namespace.
-    MatchedSymbols = SymbolIndexMgr.search(QueryString, /*IsNestedSearch=*/false);
-    if (MatchedSymbols.empty())
-      MatchedSymbols = SymbolIndexMgr.search(Query);
-    DEBUG(llvm::dbgs() << "Having found " << MatchedSymbols.size()
-                       << " symbols\n");
-    return !MatchedSymbols.empty();
-  }
 
-  /// The client to use to find cross-references.
-  SymbolIndexManager &SymbolIndexMgr;
-
-  /// The information of the symbols being queried.
-  std::vector<IncludeFixerContext::QuerySymbolInfo> QuerySymbolInfos;
-
-  /// All symbol candidates which match QuerySymbol. We only include the first
-  /// discovered identifier to avoid getting caught in results from error
-  /// recovery.
-  std::vector<find_all_symbols::SymbolInfo> MatchedSymbols;
-
-  /// The file path to the file being processed.
-  std::string FilePath;
-
-  /// Whether we should use the smallest possible include path.
-  bool MinimizeIncludePaths = true;
-};
-
-} // namespace
-
-IncludeFixerActionFactory::IncludeFixerActionFactory(
-    SymbolIndexManager &SymbolIndexMgr,
-    std::vector<IncludeFixerContext> &Contexts, StringRef StyleName,
-    bool MinimizeIncludePaths)
-    : SymbolIndexMgr(SymbolIndexMgr), Contexts(Contexts),
-      MinimizeIncludePaths(MinimizeIncludePaths) {}
-
-IncludeFixerActionFactory::~IncludeFixerActionFactory() = default;
-
-bool IncludeFixerActionFactory::runInvocation(
-    clang::CompilerInvocation *Invocation, clang::FileManager *Files,
-    std::shared_ptr<clang::PCHContainerOperations> PCHContainerOps,
-    clang::DiagnosticConsumer *Diagnostics) {
-  assert(Invocation->getFrontendOpts().Inputs.size() == 1);
-
-  // Set up Clang.
-  clang::CompilerInstance Compiler(PCHContainerOps);
-  Compiler.setInvocation(Invocation);
-  Compiler.setFileManager(Files);
-
-  // Create the compiler's actual diagnostics engine. We want to drop all
-  // diagnostics here.
-  Compiler.createDiagnostics(new clang::IgnoringDiagConsumer,
-                             /*ShouldOwnClient=*/true);
-  Compiler.createSourceManager(*Files);
-
-  // We abort on fatal errors so don't let a large number of errors become
-  // fatal. A missing #include can cause thousands of errors.
-  Compiler.getDiagnostics().setErrorLimit(0);
-
-  // Run the parser, gather missing includes.
-  auto ScopedToolAction =
-      llvm::make_unique<Action>(SymbolIndexMgr, MinimizeIncludePaths);
-  Compiler.ExecuteAction(*ScopedToolAction);
-
-  Contexts.push_back(ScopedToolAction->getIncludeFixerContext(
-      Compiler.getSourceManager(),
-      Compiler.getPreprocessor().getHeaderSearchInfo()));
-
-  // Technically this should only return true if we're sure that we have a
-  // parseable file. We don't know that though. Only inform users of fatal
-  // errors.
-  return !Compiler.getDiagnostics().hasFatalErrorOccurred();
-}
-
 llvm::Expected<tooling::Replacements> createIncludeFixerReplacements(
     StringRef Code, const IncludeFixerContext &Context,
     const clang::format::FormatStyle &Style, bool AddQualifiers) {
   if (Context.getHeaderInfos().empty())
     return tooling::Replacements();
   StringRef FilePath = Context.getFilePath();
   std::string IncludeName =
       "#include " + Context.getHeaderInfos().front().Header + "\n";

clang-tools-extra/trunk/include-fixer/plugin/CMakeLists.txt

+add_clang_library(clangIncludeFixerPlugin
+  IncludeFixerPlugin.cpp
+
+  LINK_LIBS
+  clangAST
+  clangBasic
+  clangFrontend
+  clangIncludeFixer
+  clangParse
+  clangSema
+  clangTooling
+  )

clang-tools-extra/trunk/include-fixer/plugin/IncludeFixerPlugin.cpp

+//===- IncludeFixerPlugin.cpp - clang-include-fixer as a clang plugin -----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "../IncludeFixer.h"
+#include "../YamlSymbolIndex.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Frontend/FrontendPluginRegistry.h"
+#include "clang/Parse/ParseAST.h"
+#include "clang/Sema/Sema.h"
+#include "llvm/Support/Path.h"
+
+namespace clang {
+namespace include_fixer {
+
+/// The core include fixer plugin action. This just provides the AST consumer
+/// and command line flag parsing for using include fixer as a clang plugin.
+class ClangIncludeFixerPluginAction : public PluginASTAction {
+  /// ASTConsumer to keep the symbol index alive. We don't really need an
+  /// ASTConsumer for this plugin (everything is funneled on the side through
+  /// Sema) but we have to keep the symbol index alive until sema is done.
+  struct ASTConsumerManagerWrapper : public ASTConsumer {
+    ASTConsumerManagerWrapper(std::shared_ptr<SymbolIndexManager> SIM)
+        : SymbolIndexMgr(std::move(SIM)) {}
+    std::shared_ptr<SymbolIndexManager> SymbolIndexMgr;
+  };
+
+public:
+  explicit ClangIncludeFixerPluginAction()
+      : SymbolIndexMgr(std::make_shared<SymbolIndexManager>()),
+        SemaSource(new IncludeFixerSemaSource(*SymbolIndexMgr,
+                                              /*MinimizeIncludePaths=*/true,
+                                              /*GenerateDiagnostics=*/true)) {}
+
+  std::unique_ptr<clang::ASTConsumer>
+  CreateASTConsumer(clang::CompilerInstance &CI, StringRef InFile) override {
+    CI.setExternalSemaSource(SemaSource);
+    SemaSource->setFilePath(InFile);
+    SemaSource->setCompilerInstance(&CI);
+    return llvm::make_unique<ASTConsumerManagerWrapper>(SymbolIndexMgr);
+  }
+
+  void ExecuteAction() override {} // Do nothing.
+
+  bool ParseArgs(const CompilerInstance &CI,
+                 const std::vector<std::string> &Args) override {
+    StringRef DB = "yaml";
+    StringRef Input;
+
+    // Parse the extra command line args.
+    // FIXME: This is very limited at the moment.
+    for (StringRef Arg : Args) {
+      if (Arg.startswith("-db="))
+        DB = Arg.substr(strlen("-db="));
+      else if (Arg.startswith("-input="))
+        Input = Arg.substr(strlen("-input="));
+    }
+
+    llvm::ErrorOr<std::unique_ptr<include_fixer::YamlSymbolIndex>> SymbolIdx(
+        nullptr);
+    if (DB == "yaml") {
+      if (!Input.empty()) {
+        SymbolIdx = include_fixer::YamlSymbolIndex::createFromFile(Input);
+      } else {
+        // If we don't have any input file, look in the directory of the first
+        // file and its parents.
+        const FrontendOptions &FO = CI.getFrontendOpts();
+        SmallString<128> AbsolutePath(
+            tooling::getAbsolutePath(FO.Inputs[0].getFile()));
+        StringRef Directory = llvm::sys::path::parent_path(AbsolutePath);
+        SymbolIdx = include_fixer::YamlSymbolIndex::createFromDirectory(
+            Directory, "find_all_symbols_db.yaml");
+      }
+    }
+    SymbolIndexMgr->addSymbolIndex(std::move(*SymbolIdx));
+    return true;
+  }
+
+private:
+  std::shared_ptr<SymbolIndexManager> SymbolIndexMgr;
+  IntrusiveRefCntPtr<IncludeFixerSemaSource> SemaSource;
+};
+} // namespace include_fixer
+} // namespace clang
+
+// This anchor is used to force the linker to link in the generated object file
+// and thus register the include fixer plugin.
+volatile int ClangIncludeFixerPluginAnchorSource = 0;
+
+static clang::FrontendPluginRegistry::Add<
+    clang::include_fixer::ClangIncludeFixerPluginAction>
+    X("clang-include-fixer", "clang-include-fixer");