[clangd] Update symbol collector to use include-cleaner.

[VitaNuo]

Remove redundant variables.

[VitaNuo]

Remove redundancy.

[kadircet]

clang-tools-extra/clangd/index/SymbolCollector.cpp

408–410

sorry i don't understand what this logic is trying to achieve.

we seem to be prefering "first" exporting header, over the headers that directly provide the symbol. Also comment says it's done for objc, but there's nothing limiting this logic to ObjC. any chance this is unintended?

794

s/ASTCtx->getSourceManager()/SM

877

getIncludeHeader might return an empty string

887

this is using legacy mappings for non-objc symbols too

890

you're iterating over multiple headers, and only keeping references for the last one alive.

instead of storing these in the class state, you can have a llvm::DenseMap<Header, std::string> HeaderToSpelling; inside this function and later on just do:

for(const auto& H: It->second) {
  auto [SpellingIt, Inserted] = HeaderToSpelling.try_emplace(H);
  if(Inserted) {
      switch (...) { /* Update *SpellingIt */ }
  }
  NewSym.IncludeHeaders.push_back(*SpellingIt, 1, Directives);

896–898

we actually only want to use HeaderFileURIs->toURI here and not the getIncludeHeader, because:

• We want to make use of mapping logic in include-cleaner solely, and we already have all that information in Providers mapping.
• The extra logic we want to apply for ObjC symbols are already applied before reaching here.

the way we create a FileID here is a little bit iffy, by using toURI, hopefully we can avoid that conversion.

the only remaining issue is, we actually still want to use system header mappings inside CanonicalIncludes until we have better support for them in include-cleaner library itself. So I think we should still call Includes->mapHeader(H.physical()) before using toURI.

clang-tools-extra/clangd/index/SymbolCollector.h

132

unfortunately this alternative is called after we've parsed the file, from clangd/index/FileIndex.cpp, indexSymbols. hence attaching PI to PPCallbacks won't have any effect.

This is working unintentionally because we're still populating CanonicalIncludes with legacy IWYUPragmaHandler in clangd, and using that to perform private -> public mappings.

We should instead propagate the PragmaIncludes we have in ParsedAST and in preamble builds here. You should be able to test the new behaviour clang-tools-extra/clangd/unittests/FileIndexTests.cpp to make sure pragma handling through dynamic indexing code paths keep working as expected using an export pragma.

[kadircet]

clang-tools-extra/clangd/index/SymbolCollector.cpp

844

you can take this by value and std::move into the map

[VitaNuo]

Address review comments.

[VitaNuo]

Thanks for the comments!

clang-tools-extra/clangd/index/SymbolCollector.cpp

408–410

No, this was intended, but possibly I didn't get it right.

comment says it's done for objc, but there's nothing limiting this logic to objc

AFAIU this whole code path will now only be reachable for objc symbols. I have removed pragma includes from the canonical include mapping now, leaving the pragma handling to include cleaner. However, that only triggers for C/C++, so in case we need the export and private pragmas for objc, we need to re-insert the handling for them somewhere.
There is no pre-existing test case for pragmas in objc and there is no usage of these pragmas for objc code in the code base either, so I have no way of knowing if this is actually needed. But I believe you've mentioned in some discussion we should still handle the pragmas for objc.

we seem to be preferring "first" exporting header, over the headers that directly provide the symbol.

Isn't that correct if there's an IWYU export pragma involved? The snippet comes from include_cleaner::findHeaders, with the only difference that it stops at the first exporter (since the canonical include mapping also just stored one mapping for the header).

Let me know how to do it better, or maybe if this is necessary at all. Honestly, I am not sure about this, since, as mentioned, there are no export or private/public pragmas in objc files in the codebase atm.

887

Removed this whole passage.

896–898

Ok SGTM. It seems, though, that we still need the "iffy" FID for the canonical mapping.. Let me know if you know a better way.

clang-tools-extra/clangd/index/SymbolCollector.h

132

Ok thank you. I'm using PragmaIncludes from AST and preamble builds for the dynamic index now.

For the background index, I've finally managed to move pragma recording to the IndexAction now.

I've also removed the redundant (to include cleaner) comment handlers from the AST build and preamble building logic.

Also removed the mapSymbol method from CanonicalIncludes, since it had one usage which should now be covered by the include cleaner, I believe.

[kadircet]

clang-tools-extra/clangd/ParsedAST.cpp

672

can you also add a FIXME here saying that we should attach PragmaInclude callbacks to main file ast build too?

this is not a recent regression as we were not doing it properly in previous version either. IWYU handler we attach only cares about private pragmas, which don't mean much inside the main file. but we actually need it for other pragmas like keep/export/no_include etc. (no need to do it in this patch as it's already getting quite big, let's do that as a follow up)

clang-tools-extra/clangd/Preamble.h

121–123

you need to rebase as we had some big changes around this logic recently. we should pass the PragmaIncludes also as a shared_ptr (and store as one inside the preambledata)

clang-tools-extra/clangd/index/CanonicalIncludes.h

34–35

AFAICT, only users of this endpoint were in IWYUCommentHandler (and some tests). we can also get rid of this one now.

More importantly now this turns into a mapper used only by symbolcollector, that can be cheaply created at use time (we just copy around a pointer every time we want to create it). so you can drop CanonicalIncludes from all of the interfaces, including SymbolCollector::Options and create one on demand in HeaderFileURICache. all we need is langopts, and it's available through preprocessor (not necessarily on construction, but at the time when we want to do the mappings).

As you're already touching all of the interfaces that propagate CanonicalIncludes around, hopefully this change should only make things simpler (and you already need to touch them when you rebase), but if that feels like too much churn in this patch feel free to do that in a follow up as well.

38

let's mention that this will always return a verbatim spelling, e.g. with quotes or angles.

50–51

you can drop this one too, only addMapping would introduce mappings here, but that's going away too.

clang-tools-extra/clangd/index/SymbolCollector.cpp

253

let's change the interface here to take in a FileEntry instead.

254

nit: early exits

if (!SysHeaderMapping)
  return "";
auto Canonical = ...;
if(Canonical.empty())
  return "";
...

261

we should never hit this code path anymore. so feel free to convert the if above to an assert

408–410

existing IWYUCommentHandler only handled private pragmas, so export pragmas were already dropped on the floor, we shouldn't introduce handling for them in here now. the mappings you perform for private/public pragmas and system headers below is enough.

(P.S. using exporters as "alternatives" vs "sole provider" are quite different, policy in include_cleaner does the former whereas logic in here does the latter)

412–413

let's move this logic into mapCanonical too.

415

nit: if (auto Canonical = ...; !Canonical.empty()) return Canonical;

843

what about merging this one and setIncludeLocation ?

e.g:

void SymbolCollector::setIncludeLocation(const Symbol &IdxS, SourceLocation DefLoc, const include_cleaner::Symbol &Sym) {
if (!Opts.CollectIncludePath ||
      shouldCollectIncludePath(S.SymInfo.Kind) == Symbol::Invalid)
  return;
IncludeFiles[SID] = ...;
SymbolProviders[SID] = headersForSymbol(...);
}

846

because you're taking in Headers as const, this move is not actually moving.

867

let's iterate over SymbolProviders instead, as IncludeFiles is a legacy struct now.

867–868

inside of this branch is getting crowded, maybe early exit instead?

const Symbol *S = Symbols.find(SID);
if(!S) continue;
...

881

no need to call insert if we didn't modify IncludeHeaders, so maybe reflow to something like:

if (Directives & Import) {
  if(auto IncHeader = ..; !IncHeader.empty()) {
     auto NewSym = *S;
     NewSym.IncludeHeaders....;
     Symbols.insert(NewSym);
  }
  // FIXME: Use providers from include-cleaner once it's polished for ObjC too.
  continue;
}

887

s/if/assert

903

no need for the getOrCreateFileID after you change parameter for mapCanonical to be a file entry.

905

nit:

if(auto Canonical = mapCanonical(H.physical()); !Canonical.empty())
  SpellingIt->second = Canonical;

907

this should be else if. if we performed a mapping, we no longer care about self-containedness of original header.

908

s/PP->getSourceManager()/SM

911

s/if (SpellingIt->second.empty())/else

clang-tools-extra/clangd/index/SymbolCollector.h

177–179

we seem to be populating this map for all kinds of symbols, as long as shouldCollectIncludePath returns non-invalid. what's up with distinguishing imports from includes?

190

no need to have this in the class state, you can just inline this into ::finish method instead.

193

let's move this closer to the SymbolProviders map

203

we already store this in Opts, can we just use it from there instead of creating an extra name for it?

[kadircet]

clang-tools-extra/clangd/index/SymbolCollector.cpp

843

also sorry for missing this so far, but i think we should limit this to first provider initially. as otherwise we'll have providers that just accidentally declare certain symbols and we don't want them to be inserted (unless they're the only provider and end up at the top) just because our ranking heuristics fail or something.

[VitaNuo]

Address review comments.

[VitaNuo]

Thanks for comments!

clang-tools-extra/clangd/index/CanonicalIncludes.h

34–35

... and create one on demand in HeaderFileURICache. all we need is langopts, and it's available through preprocessor (not necessarily on construction, but at the time when we want to do the mappings

This time sounds a little late, since we don't want to rebuild the system header mapping every time we request a mapping for a particular header.
Cache construction time doesn't work, since it happens before the preprocessor is set in the symbol collector (and we need the preprocessor to retrieve language options).
So far the safe place I've found is right after the preprocessor is set in the symbol collector. Another option is to collect the system header options in the finish() method, but I can't see why we would need to delay until then.

clang-tools-extra/clangd/index/SymbolCollector.cpp

253

I believe it has to be FileEntryRef instead. This is what the CanonicalIncludes::mapHeader expects, and this is also seems right (somewhere in the Clang docs I've read "everything that needs a name should use FileEntryRef", and name is exactly what CanonicalIncludes::mapHeader needs).

For the case of a physical header (i.e., FileEntry instance) I've switched to using getLastRef now. There's also another API FileManager::getFileRef where I could try passing the result of tryGetRealPathName. I am not sure I have enough information to judge if any of these options is distinctly better.

412–413

This would deliver wrong results for C++. We would basically get every IWYU-public header twice: once through the verbatim branch of the include_cleaner results, and then once again when mapping the physical private file to a canonical version (e.g., for system headers).

846

Ok, this is not relevant anymore, but I am not sure I understand why. Am I not allowed to say that I don't need this variable/memory anymore, even though it's const?

867

Not sure I'm following. Iterating over SymbolProviders means retrieving include_cleaner::Headers. How would I handle the entire Obj-C part then, without the FID of the include header?

903

Still need the getLastRef or FileManager::getFileRef, if I understand correctly.

clang-tools-extra/clangd/index/SymbolCollector.h

177–179

Yeah that's right. I've been trying to express that (after populating this map) we are not using this FID to compute the IncludeHeaders anymore (only for the ObjC case), but rather use it to determine if the Directives should contain Symbol::Import (ie., if this FID contains Obj-C constructs/imports).

I'll try to be a bit more verbose and precise.

[kadircet]

clang-tools-extra/clangd/ClangdServer.cpp

85

nit: PI(std::move(PI))

clang-tools-extra/clangd/TUScheduler.cpp

1086

nit: s/PI/std::move(PI)

clang-tools-extra/clangd/index/CanonicalIncludes.cpp

672

we're no longer using any FileEntry pieces of this parameter you can take in just a llvm::StringRef HeaderPath instead.

clang-tools-extra/clangd/index/CanonicalIncludes.h

34–35

This time sounds a little late, since we don't want to rebuild the system header mapping every time we request a mapping for a particular header.

System header mapping is a singleton, we build it statically once on first request throughout the whole process and just use that for the rest of the execution.

clang-tools-extra/clangd/index/FileIndex.cpp

49

again you can have a const include_cleaner::PragmaIncludes & without shared_ptr here.

clang-tools-extra/clangd/index/FileIndex.h

118

you can have a const include_cleaner::PragmaIncludes& PI directly both in here and in indexHeaderSymbols.

after building a preamble we always have a non-null PragmaIncludes and these functions are always executed synchronously, so no need to extend ownership and make the code indirect.

clang-tools-extra/clangd/index/IndexAction.cpp

230

you can have a unique_ptr here instead (once we store a non-owning pointer in Opts.PragmaIncludes)

clang-tools-extra/clangd/index/SymbolCollector.cpp

253

yeah getLastRef is fine, i was mostly trying to avoid dodgy call to getOrCreateFileID.

you can also pass in just a llvm::StringRef HeaderPath now, as we're not using anything but file path. which you can retreive with getName from both FileEntry and FileEntryRef

407

s/SM.getFileEntryForID(FID)/FE

793

you can actually change include_cleaner::Macro to have a const IdentifierInfo* instead of the const_cast here. it's mostly an oversight to store a non-const IdentifierInfo in the first place.

839

once we have the optional<Header> as the value type you can also rewrite this as:

auto [It, Inserted] = SymbolProviders.try_emplace(S.ID);
if (Inserted) {
  auto Providers = include_cleaner::headersForSymbol(Sym, SM, Opts.PragmaIncludes.get());
  if(!Providers.empty()) It->second = Providers.front();
}

that way we can get rid of some redundant calls to headersForSymbol

846

std::move is unfortunately a little bit misleading at times. it is just a cast that returns an rvalue reference. hence on its own it doesn't actually move anything.

but it enables picking rvalue versions of constructors/assignment operators. (e.g. Foo(Foo&&) or operator=(Foo&&)) which can be implemented efficiently with the assumption that RHS object is being "moved from" and will no longer be used. that way certain types implement a move semantics by just consuming the RHS object. unfortunately that is not possbile when RHS is const, and expressions use regular copy semantics instead.

867

we're joining IncludeFiles and SymbolProviders, as they have the same keys.

right now you're iterating over the keys in IncludeFiles and doing a lookup into SymbolProviders using that key to get providers.
we can also do the iteration over SymbolProviders and do the lookup into IncludeFiles instead to find associated FID.

895

you can move the copy into this branch to make sure we're only doing that when necessary, e.g:

if(auto IncludeHeader = ...) {
  Symbol NewSym = *S;
  NewSym.IncludeHeaders...;
  Symbols.insert(NewSym);
}

907

FWIW, i think we should assert(It != SymbolProviders.end()) (or IncludeFiles.end() when you change the outer loop). as we're inserting into these maps simultaneously.

910

we're not really getting any caching benefits as this map is inside the loop. can you put it outside the loop body?

913

we shouldn't continue here, it just means we can directly use SpellingIt to figure out what to put into NewSym.IncludeHeaders.

maybe something like:

auto [SpellingIt, Inserted] = HeaderSpelling.try_emplace(H);
if (Inserted)
  SpellingIt-> second = getSpelling(H, *PP);
if(!SpellingIt->second.empty()) {
  Symbol NewSym = *S;
  NewSym.IncludeHeaders.push_back({SpellingIt->second, 1, Directives});
  Symbols.insert(NewSym);
}

std::string getSpelling(const include_cleaner::Header &H, const Preprocessor &PP) {
  if(H.kind() == Physical) {
   // Check if we have a mapping for the header in system includes.
   // FIXME: get rid of this once stdlib mapping covers these system headers too.
   CanonicalIncludes CI;
   CI.addSystemHeadersMapping(PP.getLangOpts());
    if (auto Canonical = CI.mapHeader(H.physical()->getLastRef()); !Canonical.empty())
        return Canonical;
    if (!tooling::isSelfContainedHeader(...))
        return "";
  }
  // Otherwise use default spelling strategies in include_cleaner.
  return include_cleaner::spellHeader(H);
}

clang-tools-extra/clangd/index/SymbolCollector.h

64

no need to have ownership here, we can have just a const include_cleaner::PragmaIncludes* PI. SymbolCollector shouldn't need to extend lifetimes

130

as mentioned above addSystemHeadersMapping is cheap after the first call in the process, we initialize the mappings only once. so you can just have it as a member in HeaderFileURICache, and drop it completely from the public interface of SymbolCollector

186

can we have a llvm::DenseMap<SymbolID, std::optional<include_cleaner::Header>>, to indicate there are no providers for a symbol, rather than missing the symbol completely from the mapping?

[VitaNuo]

Thank you!

clang-tools-extra/clangd/index/SymbolCollector.cpp

839

Sure, although I'm not sure where the redundant calls would be coming from. I've been under the impression that this function is supposed to be called once for each symbol.

867

Ok, makes sense now, thanks.

913

As agreed offline, it is not easy to split out a spelling function, since HeaderFileURIs is a private member of the symbol collector, and we seem to still need to generate URIs for physical headers going forward.

[VitaNuo]

Address review comments.

[kadircet]

thanks a lot for bearing with me, let's ship it!

clang-tools-extra/clangd/IncludeCleaner.cpp

91

nit: just const include_cleaner::PragmaIncludes* PI

clang-tools-extra/clangd/index/SymbolCollector.cpp

258

SysHeaderMapping doesn't need to be part of class state, just define it here completely. (also gets rid of the need for extra explanation and any possible questions around what would happen when we add system headers multiple times)

423

nit: can you move this above the call to mapCanonical and use it there too?

830–831

nit: early exits might be more readable, as we've more nesting now. (if (!Opts.Collect.. || shouldCollect..() == Symbol::Invalid) return;)

839

yeah unfortunately addDeclaration can be called multiple times (with increasing declaration quality, e.g. once for the forward declaration, then for a complete definition of a class)

872–875

nit: auto FID = IncludeFiles.at(SID);

920

i believe this should be just H.physical()->getLastRef() ? as we're passing in a fileentryref into toURI

921

nit: braces, as we've multiple lines (despite being a single statement)

935–938

if (!SpellingIt->second.empty()) {
  auto NewSym = *S;
  NewSym.IncludeHeaders.push_back({SpellingIt->second, 1, Directives});
  Symbols.insert(NewSym);
}

[VitaNuo]

Address the latest comment batch.

clang-tools-extra/clangd/ClangdServer.cpp

 #include "ParsedAST.h"
 #include "Preamble.h"
 #include "Protocol.h"
 #include "SemanticHighlighting.h"
 #include "SemanticSelection.h"
 #include "SourceCode.h"
 #include "TUScheduler.h"
 #include "XRefs.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/FileIndex.h"
 #include "index/Merge.h"
 #include "index/StdLib.h"
 #include "refactor/Rename.h"
 #include "refactor/Tweak.h"
 #include "support/Cancellation.h"
 #include "support/Logger.h"
 #include "support/MemoryTree.h"

 
 // Update the FileIndex with new ASTs and plumb the diagnostics responses.
 struct UpdateIndexCallbacks : public ParsingCallbacks {
   UpdateIndexCallbacks(FileIndex *FIndex,
                        ClangdServer::Callbacks *ServerCallbacks,
                        const ThreadsafeFS &TFS, AsyncTaskRunner *Tasks,
                        bool CollectInactiveRegions,
                        const ClangdServer::Options &Opts)
-      : FIndex(FIndex), ServerCallbacks(ServerCallbacks),
-        TFS(TFS), Stdlib{std::make_shared<StdLibSet>()}, Tasks(Tasks),
+      : FIndex(FIndex), ServerCallbacks(ServerCallbacks), TFS(TFS),
+        Stdlib{std::make_shared<StdLibSet>()}, Tasks(Tasks),
         CollectInactiveRegions(CollectInactiveRegions), Opts(Opts) {}
 
   void onPreambleAST(
       PathRef Path, llvm::StringRef Version, CapturedASTCtx ASTCtx,
-      const std::shared_ptr<const CanonicalIncludes> CanonIncludes) override {
+      std::shared_ptr<const include_cleaner::PragmaIncludes> PI) override {
 
     if (!FIndex)
       return;
 
     auto &PP = ASTCtx.getPreprocessor();
     auto &CI = ASTCtx.getCompilerInvocation();
     if (auto Loc = Stdlib->add(*CI.getLangOpts(), PP.getHeaderSearchInfo()))
       indexStdlib(CI, std::move(*Loc));
 
     // FIndex outlives the UpdateIndexCallbacks.
     auto Task = [FIndex(FIndex), Path(Path.str()), Version(Version.str()),
-                 ASTCtx(std::move(ASTCtx)),
-                 CanonIncludes(CanonIncludes)]() mutable {
+                 ASTCtx(std::move(ASTCtx)), PI]() mutable {
       trace::Span Tracer("PreambleIndexing");
       FIndex->updatePreamble(Path, Version, ASTCtx.getASTContext(),
-                             ASTCtx.getPreprocessor(), *CanonIncludes);
+                             ASTCtx.getPreprocessor(), PI);
     };
 
     if (Opts.AsyncPreambleIndexing && Tasks) {
       Tasks->runAsync("Preamble indexing for:" + Path + Version,
                       std::move(Task));
     } else
       Task();
   }

clang-tools-extra/clangd/Hover.cpp

 #include "IncludeCleaner.h"
 #include "ParsedAST.h"
 #include "Selection.h"
 #include "SourceCode.h"
 #include "clang-include-cleaner/Analysis.h"
 #include "clang-include-cleaner/IncludeSpeller.h"
 #include "clang-include-cleaner/Types.h"
 #include "index/SymbolCollector.h"
-#include "support/Logger.h"
 #include "support/Markup.h"
 #include "support/Trace.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/ASTDiagnostic.h"
 #include "clang/AST/ASTTypeTraits.h"
 #include "clang/AST/Attr.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclBase.h"

 }
 
 void maybeAddSymbolProviders(ParsedAST &AST, HoverInfo &HI,
                              include_cleaner::Symbol Sym) {
   trace::Span Tracer("Hover::maybeAddSymbolProviders");
 
   const SourceManager &SM = AST.getSourceManager();
   llvm::SmallVector<include_cleaner::Header> RankedProviders =
-      include_cleaner::headersForSymbol(Sym, SM, AST.getPragmaIncludes());
+      include_cleaner::headersForSymbol(Sym, SM, AST.getPragmaIncludes().get());
   if (RankedProviders.empty())
     return;
 
   std::string Result;
   include_cleaner::Includes ConvertedIncludes =
       convertIncludes(SM, AST.getIncludeStructure().MainFileIncludes);
   for (const auto &P : RankedProviders) {
     if (P.kind() == include_cleaner::Header::Physical &&

 void maybeAddUsedSymbols(ParsedAST &AST, HoverInfo &HI, const Inclusion &Inc) {
   const SourceManager &SM = AST.getSourceManager();
   const auto &ConvertedMainFileIncludes =
       convertIncludes(SM, AST.getIncludeStructure().MainFileIncludes);
   const auto &HoveredInclude = convertIncludes(SM, llvm::ArrayRef{Inc});
   llvm::DenseSet<include_cleaner::Symbol> UsedSymbols;
   include_cleaner::walkUsed(
       AST.getLocalTopLevelDecls(), collectMacroReferences(AST),
-      AST.getPragmaIncludes(), SM,
+      AST.getPragmaIncludes().get(), SM,
       [&](const include_cleaner::SymbolReference &Ref,
           llvm::ArrayRef<include_cleaner::Header> Providers) {
         if (Ref.RT != include_cleaner::RefType::Explicit ||
             UsedSymbols.contains(Ref.Target))
           return;
 
         auto Provider =
             firstMatchedProvider(ConvertedMainFileIncludes, Providers);

clang-tools-extra/clangd/IncludeCleaner.cpp

 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Regex.h"
 #include <cassert>
 #include <iterator>
+#include <memory>
 #include <optional>
 #include <string>
 #include <utility>
 #include <vector>
 
 namespace clang::clangd {
 
 static bool AnalyzeStdlib = false;
bool isIgnored(llvm::StringRef HeaderPath, HeaderFilter IgnoreHeaders) {
   llvm::sys::path::native(NormalizedPath, llvm::sys::path::Style::posix);
   for (auto &Filter : IgnoreHeaders) {
     if (Filter(NormalizedPath))
       return true;
   }
   return false;
 }
 
-bool mayConsiderUnused(const Inclusion &Inc, ParsedAST &AST,
-                       const include_cleaner::PragmaIncludes *PI) {
+bool mayConsiderUnused(
+    const Inclusion &Inc, ParsedAST &AST,
+    std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
   // FIXME(kirillbobyrev): We currently do not support the umbrella headers.
   // System headers are likely to be standard library headers.
   // Until we have good support for umbrella headers, don't warn about them.
   if (Inc.Written.front() == '<') {
     if (AnalyzeStdlib && tooling::stdlib::Header::named(Inc.Written))
       return true;
     return false;
   }
IncludeCleanerFindings computeIncludeCleanerFindings(ParsedAST &AST) {
 
   std::vector<include_cleaner::SymbolReference> Macros =
       collectMacroReferences(AST);
   std::vector<MissingIncludeDiagInfo> MissingIncludes;
   llvm::DenseSet<IncludeStructure::HeaderID> Used;
   trace::Span Tracer("include_cleaner::walkUsed");
   include_cleaner::walkUsed(
       AST.getLocalTopLevelDecls(), /*MacroRefs=*/Macros,
-      AST.getPragmaIncludes(), SM,
+      AST.getPragmaIncludes().get(), SM,
       [&](const include_cleaner::SymbolReference &Ref,
           llvm::ArrayRef<include_cleaner::Header> Providers) {
         bool Satisfied = false;
         for (const auto &H : Providers) {
           if (H.kind() == include_cleaner::Header::Physical &&
               (H.physical() == MainFile || H.physical() == PreamblePatch)) {
             Satisfied = true;
             continue;

clang-tools-extra/clangd/ParsedAST.h

 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_PARSEDAST_H
 
 #include "CollectMacros.h"
 #include "Compiler.h"
 #include "Diagnostics.h"
 #include "Headers.h"
 #include "Preamble.h"
 #include "clang-include-cleaner/Record.h"
-#include "index/CanonicalIncludes.h"
 #include "support/Path.h"
 #include "clang/Frontend/FrontendAction.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Tooling/Syntax/Tokens.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 #include <memory>
 #include <optional>
public:
   const std::optional<std::vector<Diag>> &getDiagnostics() const {
     return Diags;
   }
 
   /// Returns the estimated size of the AST and the accessory structures, in
   /// bytes. Does not include the size of the preamble.
   std::size_t getUsedBytes() const;
   const IncludeStructure &getIncludeStructure() const;
-  const CanonicalIncludes &getCanonicalIncludes() const;
 
   /// Gets all macro references (definition, expansions) present in the main
   /// file, including those in the preamble region.
   const MainFileMacros &getMacros() const;
   /// Gets all pragma marks in the main file.
   const std::vector<PragmaMark> &getMarks() const;
   /// Tokens recorded while parsing the main file.
   /// (!) does not have tokens from the preamble.
   const syntax::TokenBuffer &getTokens() const { return Tokens; }
   /// Returns the PramaIncludes from the preamble.
   /// Might be null if AST is built without a preamble.
-  const include_cleaner::PragmaIncludes *getPragmaIncludes() const;
+  std::shared_ptr<const include_cleaner::PragmaIncludes>
+  getPragmaIncludes() const;
 
   /// Returns the version of the ParseInputs this AST was built from.
   llvm::StringRef version() const { return Version; }
 
   /// Returns the path passed by the caller when building this AST.
   PathRef tuPath() const { return TUPath; }
 
   /// Returns the version of the ParseInputs used to build Preamble part of this
   /// AST. Might be std::nullopt if no Preamble is used.
   std::optional<llvm::StringRef> preambleVersion() const;
 
   const HeuristicResolver *getHeuristicResolver() const {
     return Resolver.get();
   }
 
 private:
   ParsedAST(PathRef TUPath, llvm::StringRef Version,
             std::shared_ptr<const PreambleData> Preamble,
             std::unique_ptr<CompilerInstance> Clang,
             std::unique_ptr<FrontendAction> Action, syntax::TokenBuffer Tokens,
             MainFileMacros Macros, std::vector<PragmaMark> Marks,
             std::vector<Decl *> LocalTopLevelDecls,
-            std::optional<std::vector<Diag>> Diags, IncludeStructure Includes,
-            CanonicalIncludes CanonIncludes);
+            std::optional<std::vector<Diag>> Diags, IncludeStructure Includes);
 
   Path TUPath;
   std::string Version;
   // In-memory preambles must outlive the AST, it is important that this member
   // goes before Clang and Action.
   std::shared_ptr<const PreambleData> Preamble;
   // We store an "incomplete" FrontendAction (i.e. no EndSourceFile was called
   // on it) and CompilerInstance used to run it. That way we don't have to do
private:
   std::vector<PragmaMark> Marks;
   // Data, stored after parsing. std::nullopt if AST was built with a stale
   // preamble.
   std::optional<std::vector<Diag>> Diags;
   // Top-level decls inside the current file. Not that this does not include
   // top-level decls from the preamble.
   std::vector<Decl *> LocalTopLevelDecls;
   IncludeStructure Includes;
-  CanonicalIncludes CanonIncludes;
   std::unique_ptr<HeuristicResolver> Resolver;
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_PARSEDAST_H

clang-tools-extra/clangd/ParsedAST.cpp

 #include "Headers.h"
 #include "HeuristicResolver.h"
 #include "IncludeCleaner.h"
 #include "IncludeFixer.h"
 #include "Preamble.h"
 #include "SourceCode.h"
 #include "TidyProvider.h"
 #include "clang-include-cleaner/Record.h"
-#include "index/CanonicalIncludes.h"
 #include "index/Symbol.h"
 #include "support/Logger.h"
 #include "support/Path.h"
 #include "support/Trace.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclGroup.h"
 #include "clang/AST/ExternalASTSource.h"
if (Preamble) {
     Marks = Patch->marks();
   }
   auto &PP = Clang->getPreprocessor();
   PP.addPPCallbacks(std::make_unique<CollectMainFileMacros>(PP, Macros));
 
   PP.addPPCallbacks(
       collectPragmaMarksCallback(Clang->getSourceManager(), Marks));
 
-  // Copy over the includes from the preamble, then combine with the
-  // non-preamble includes below.
-  CanonicalIncludes CanonIncludes;
-  if (Preamble)
-    CanonIncludes = *Preamble->CanonIncludes;
-  else
-    CanonIncludes.addSystemHeadersMapping(Clang->getLangOpts());
-  std::unique_ptr<CommentHandler> IWYUHandler =
-      collectIWYUHeaderMaps(&CanonIncludes);
-  PP.addCommentHandler(IWYUHandler.get());
+  // FIXME: Attach a comment handler to take care of
+  // keep/export/no_include etc. IWYU pragmas.
 
   // Collect tokens of the main file.
   syntax::TokenCollector CollectTokens(PP);
 
   // To remain consistent with preamble builds, these callbacks must be called
   // exactly here, after preprocessor is initialized and BeginSourceFile() was
   // called already.
   for (const auto &L : ASTListeners)
if (PreserveDiags) {
     {
       std::vector<Diag> D = ASTDiags.take(&*CTContext);
       Diags->insert(Diags->end(), D.begin(), D.end());
     }
   }
   ParsedAST Result(Filename, Inputs.Version, std::move(Preamble),
                    std::move(Clang), std::move(Action), std::move(Tokens),
                    std::move(Macros), std::move(Marks), std::move(ParsedDecls),
-                   std::move(Diags), std::move(Includes),
-                   std::move(CanonIncludes));
+                   std::move(Diags), std::move(Includes));
   if (Result.Diags)
     llvm::move(getIncludeCleanerDiags(Result, Inputs.Contents),
                std::back_inserter(*Result.Diags));
   return std::move(Result);
 }
 
 ParsedAST::ParsedAST(ParsedAST &&Other) = default;
 
std::size_t ParsedAST::getUsedBytes() const {
 
   return Total;
 }
 
 const IncludeStructure &ParsedAST::getIncludeStructure() const {
   return Includes;
 }
 
-const CanonicalIncludes &ParsedAST::getCanonicalIncludes() const {
-  return CanonIncludes;
-}
-
 ParsedAST::ParsedAST(PathRef TUPath, llvm::StringRef Version,
                      std::shared_ptr<const PreambleData> Preamble,
                      std::unique_ptr<CompilerInstance> Clang,
                      std::unique_ptr<FrontendAction> Action,
                      syntax::TokenBuffer Tokens, MainFileMacros Macros,
                      std::vector<PragmaMark> Marks,
                      std::vector<Decl *> LocalTopLevelDecls,
                      std::optional<std::vector<Diag>> Diags,
-                     IncludeStructure Includes, CanonicalIncludes CanonIncludes)
+                     IncludeStructure Includes)
     : TUPath(TUPath), Version(Version), Preamble(std::move(Preamble)),
       Clang(std::move(Clang)), Action(std::move(Action)),
       Tokens(std::move(Tokens)), Macros(std::move(Macros)),
       Marks(std::move(Marks)), Diags(std::move(Diags)),
       LocalTopLevelDecls(std::move(LocalTopLevelDecls)),
-      Includes(std::move(Includes)), CanonIncludes(std::move(CanonIncludes)) {
+      Includes(std::move(Includes)) {
   Resolver = std::make_unique<HeuristicResolver>(getASTContext());
   assert(this->Clang);
   assert(this->Action);
 }
 
-const include_cleaner::PragmaIncludes *ParsedAST::getPragmaIncludes() const {
+std::shared_ptr<const include_cleaner::PragmaIncludes>
+ParsedAST::getPragmaIncludes() const {
   if (!Preamble)
     return nullptr;
-  return &Preamble->Pragmas;
+  return Preamble->Pragmas;
 }
 
 std::optional<llvm::StringRef> ParsedAST::preambleVersion() const {
   if (!Preamble)
     return std::nullopt;
   return llvm::StringRef(Preamble->Version);
 }
 
 } // namespace clangd
 } // namespace clang

clang-tools-extra/clangd/Preamble.h

 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_PREAMBLE_H
 
 #include "CollectMacros.h"
 #include "Compiler.h"
 #include "Diagnostics.h"
 #include "FS.h"
 #include "Headers.h"
 #include "clang-include-cleaner/Record.h"
-#include "index/CanonicalIncludes.h"
 #include "support/Path.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
struct PreambleData {
   std::string Version;
   tooling::CompileCommand CompileCommand;
   PrecompiledPreamble Preamble;
   std::vector<Diag> Diags;
   // Processes like code completions and go-to-definitions will need #include
   // information, and their compile action skips preamble range.
   IncludeStructure Includes;
   // Captures #include-mapping information in #included headers.
-  include_cleaner::PragmaIncludes Pragmas;
+  std::shared_ptr<const include_cleaner::PragmaIncludes> Pragmas;
   // Macros defined in the preamble section of the main file.
   // Users care about headers vs main-file, not preamble vs non-preamble.
   // These should be treated as main-file entities e.g. for code completion.
   MainFileMacros Macros;
   // Pragma marks defined in the preamble section of the main file.
   std::vector<PragmaMark> Marks;
   // Cache of FS operations performed when building the preamble.
   // When reusing a preamble, this cache can be consumed to save IO.
   std::shared_ptr<PreambleFileStatusCache> StatCache;
-  std::shared_ptr<const CanonicalIncludes> CanonIncludes;
   // Whether there was a (possibly-incomplete) include-guard on the main file.
   // We need to propagate this information "by hand" to subsequent parses.
   bool MainIsIncludeGuarded = false;
 };
 
 using PreambleParsedCallback =
     std::function<void(CapturedASTCtx ASTCtx,
-                       std::shared_ptr<const CanonicalIncludes> CanonIncludes)>;
+                       std::shared_ptr<const include_cleaner::PragmaIncludes>)>;
 
 /// Timings and statistics from the premble build. Unlike PreambleData, these
 /// do not need to be stored for later, but can be useful for logging, metrics,
 /// etc.
 struct PreambleBuildStats {
   /// Total wall time it took to build preamble, in seconds.
   double TotalBuildTime;
   /// Time spent in filesystem operations during the build, in seconds.

clang-tools-extra/clangd/Preamble.cpp

 #include "CollectMacros.h"
 #include "Compiler.h"
 #include "Config.h"
 #include "Diagnostics.h"
 #include "Headers.h"
 #include "Protocol.h"
 #include "SourceCode.h"
 #include "clang-include-cleaner/Record.h"
-#include "index/CanonicalIncludes.h"
 #include "support/Logger.h"
 #include "support/Path.h"
 #include "support/ThreadsafeFS.h"
 #include "support/Trace.h"
 #include "clang/AST/DeclTemplate.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticLex.h"
 #include "clang/Basic/LangOptions.h"
public:
 
   MainFileMacros takeMacros() { return std::move(Macros); }
 
   std::vector<PragmaMark> takeMarks() { return std::move(Marks); }
 
   include_cleaner::PragmaIncludes takePragmaIncludes() {
     return std::move(Pragmas);
   }
-  CanonicalIncludes takeCanonicalIncludes() { return std::move(CanonIncludes); }
 
   std::optional<CapturedASTCtx> takeLife() { return std::move(CapturedCtx); }
 
   bool isMainFileIncludeGuarded() const { return IsMainFileIncludeGuarded; }
 
   void AfterExecute(CompilerInstance &CI) override {
     // As part of the Preamble compilation, ASTConsumer
     // PrecompilePreambleConsumer/PCHGenerator is setup. This would be called
if (Stats) {
       Stats->BuildSize += PP.getTotalMemory();
       if (PreprocessingRecord *PRec = PP.getPreprocessingRecord())
         Stats->BuildSize += PRec->getTotalMemory();
       Stats->BuildSize += PP.getHeaderSearchInfo().getTotalMemory();
     }
   }
 
   void BeforeExecute(CompilerInstance &CI) override {
-    CanonIncludes.addSystemHeadersMapping(CI.getLangOpts());
     LangOpts = &CI.getLangOpts();
     SourceMgr = &CI.getSourceManager();
     PP = &CI.getPreprocessor();
     Includes.collect(CI);
     Pragmas.record(CI);
     if (BeforeExecuteCallback)
       BeforeExecuteCallback(CI);
   }
 
   std::unique_ptr<PPCallbacks> createPPCallbacks() override {
     assert(SourceMgr && LangOpts && PP &&
            "SourceMgr, LangOpts and PP must be set at this point");
 
     return std::make_unique<PPChainedCallbacks>(
         std::make_unique<CollectMainFileMacros>(*PP, Macros),
         collectPragmaMarksCallback(*SourceMgr, Marks));
   }
 
-  CommentHandler *getCommentHandler() override {
-    IWYUHandler = collectIWYUHeaderMaps(&CanonIncludes);
-    return IWYUHandler.get();
-  }
-
   static bool isLikelyForwardingFunction(FunctionTemplateDecl *FT) {
     const auto *FD = FT->getTemplatedDecl();
     const auto NumParams = FD->getNumParams();
     // Check whether its last parameter is a parameter pack...
     if (NumParams > 0) {
       const auto *LastParam = FD->getParamDecl(NumParams - 1);
       if (const auto *PET = dyn_cast<PackExpansionType>(LastParam->getType())) {
         // ... of the type T&&... or T...
if (auto *FT = llvm::dyn_cast<clang::FunctionTemplateDecl>(D)) {
       }
     }
     return true;
   }
 
 private:
   PathRef File;
   IncludeStructure Includes;
-  CanonicalIncludes CanonIncludes;
   include_cleaner::PragmaIncludes Pragmas;
   MainFileMacros Macros;
   std::vector<PragmaMark> Marks;
   bool IsMainFileIncludeGuarded = false;
-  std::unique_ptr<CommentHandler> IWYUHandler = nullptr;
   const clang::LangOptions *LangOpts = nullptr;
   const SourceManager *SourceMgr = nullptr;
   const Preprocessor *PP = nullptr;
   PreambleBuildStats *Stats;
   bool ParseForwardingFunctions;
   std::function<void(CompilerInstance &)> BeforeExecuteCallback;
   std::optional<CapturedASTCtx> CapturedCtx;
 };
log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
         BuiltPreamble->getSize(), FileName, Inputs.Version,
         PreambleTimer.getTime());
     std::vector<Diag> Diags = PreambleDiagnostics.take();
     auto Result = std::make_shared<PreambleData>(std::move(*BuiltPreamble));
     Result->Version = Inputs.Version;
     Result->CompileCommand = Inputs.CompileCommand;
     Result->Diags = std::move(Diags);
     Result->Includes = CapturedInfo.takeIncludes();
-    Result->Pragmas = CapturedInfo.takePragmaIncludes();
+    Result->Pragmas = std::make_shared<const include_cleaner::PragmaIncludes>(
+        CapturedInfo.takePragmaIncludes());
     Result->Macros = CapturedInfo.takeMacros();
     Result->Marks = CapturedInfo.takeMarks();
-    Result->CanonIncludes = std::make_shared<const CanonicalIncludes>(
-        (CapturedInfo.takeCanonicalIncludes()));
     Result->StatCache = StatCache;
     Result->MainIsIncludeGuarded = CapturedInfo.isMainFileIncludeGuarded();
     if (PreambleCallback) {
       trace::Span Tracer("Running PreambleCallback");
       auto Ctx = CapturedInfo.takeLife();
       // Stat cache is thread safe only when there are no producers. Hence
       // change the VFS underneath to a consuming fs.
       Ctx->getFileManager().setVirtualFileSystem(
           Result->StatCache->getConsumingFS(VFS));
       // While extending the life of FileMgr and VFS, StatCache should also be
       // extended.
       Ctx->setStatCache(Result->StatCache);
-      PreambleCallback(std::move(*Ctx), Result->CanonIncludes);
+      PreambleCallback(std::move(*Ctx), Result->Pragmas);
     }
     return Result;
   }
 
   elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
        Inputs.Version, BuiltPreamble.getError().message());
   for (const Diag &D : PreambleDiagnostics.take()) {
     if (D.Severity < DiagnosticsEngine::Error)

clang-tools-extra/clangd/TUScheduler.h

 //===--- TUScheduler.h -------------------------------------------*-C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_TUSCHEDULER_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_TUSCHEDULER_H
 
 #include "ASTSignals.h"
 #include "Compiler.h"
 #include "Diagnostics.h"
 #include "GlobalCompilationDatabase.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "support/Function.h"
 #include "support/MemoryTree.h"
 #include "support/Path.h"
 #include "support/Threading.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include <chrono>
+#include <memory>
 #include <optional>
 #include <string>
 
 namespace clang {
 namespace clangd {
 class ParsedAST;
 struct PreambleData;
 

 
 class ParsingCallbacks {
 public:
   virtual ~ParsingCallbacks() = default;
 
   /// Called on the AST that was built for emitting the preamble. The built AST
   /// contains only AST nodes from the #include directives at the start of the
   /// file. AST node in the current file should be observed on onMainAST call.
-  virtual void onPreambleAST(PathRef Path, llvm::StringRef Version,
-                             CapturedASTCtx Ctx,
-                             const std::shared_ptr<const CanonicalIncludes>) {}
-
+  virtual void
+  onPreambleAST(PathRef Path, llvm::StringRef Version, CapturedASTCtx Ctx,
+                std::shared_ptr<const include_cleaner::PragmaIncludes>) {}
   /// The argument function is run under the critical section guarding against
   /// races when closing the files.
   using PublishFn = llvm::function_ref<void(llvm::function_ref<void()>)>;
   /// Called on the AST built for the file itself. Note that preamble AST nodes
   /// are not deserialized and should be processed in the onPreambleAST call
   /// instead.
   /// The \p AST always contains all AST nodes for the main file itself, and
   /// only a portion of the AST nodes deserialized from the preamble. Note that

clang-tools-extra/clangd/TUScheduler.cpp

 #include "TUScheduler.h"
 #include "CompileCommands.h"
 #include "Compiler.h"
 #include "Config.h"
 #include "Diagnostics.h"
 #include "GlobalCompilationDatabase.h"
 #include "ParsedAST.h"
 #include "Preamble.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "support/Cancellation.h"
 #include "support/Context.h"
 #include "support/Logger.h"
 #include "support/MemoryTree.h"
 #include "support/Path.h"
 #include "support/ThreadCrashReporter.h"
 #include "support/Threading.h"
 #include "support/Trace.h"
ThreadCrashReporter ScopedReporter([&Inputs]() {
     crashDumpParseInputs(llvm::errs(), Inputs);
   });
 
   PreambleBuildStats Stats;
   bool IsFirstPreamble = !LatestBuild;
   LatestBuild = clang::clangd::buildPreamble(
       FileName, *Req.CI, Inputs, StoreInMemory,
       [&](CapturedASTCtx ASTCtx,
-          std::shared_ptr<const CanonicalIncludes> CanonIncludes) {
+          std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
         Callbacks.onPreambleAST(FileName, Inputs.Version, std::move(ASTCtx),
-                                CanonIncludes);
+                                PI);
       },
       &Stats);
   if (!LatestBuild)
     return;
   reportPreambleBuild(Stats, IsFirstPreamble);
   if (isReliable(LatestBuild->CompileCommand))
     HeaderIncluders.update(FileName, LatestBuild->Includes.allHeaders());
 }

clang-tools-extra/clangd/XRefs.cpp

maybeFindIncludeReferences(ParsedAST &AST, Position Pos,
 
   const auto &Inc = *IncludeOnLine;
   const SourceManager &SM = AST.getSourceManager();
   ReferencesResult Results;
   auto ConvertedMainFileIncludes = convertIncludes(SM, Includes);
   auto ReferencedInclude = convertIncludes(SM, Inc);
   include_cleaner::walkUsed(
       AST.getLocalTopLevelDecls(), collectMacroReferences(AST),
-      AST.getPragmaIncludes(), SM,
+      AST.getPragmaIncludes().get(), SM,
       [&](const include_cleaner::SymbolReference &Ref,
           llvm::ArrayRef<include_cleaner::Header> Providers) {
         if (Ref.RT != include_cleaner::RefType::Explicit)
           return;
 
         auto Provider =
             firstMatchedProvider(ConvertedMainFileIncludes, Providers);
         if (!Provider || ReferencedInclude.match(*Provider).empty())

clang-tools-extra/clangd/index/CanonicalIncludes.h

 //   libstdc++ puts char_traits in bits/char_traits.h, but we #include <string>.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_CANONICALINCLUDES_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_CANONICALINCLUDES_H
 
 #include "clang/Basic/FileEntry.h"
-#include "clang/Lex/Preprocessor.h"
+#include "clang/Basic/LangOptions.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
-#include "llvm/Support/FileSystem/UniqueID.h"
-#include <mutex>
-#include <string>
-#include <vector>
 
 namespace clang {
 namespace clangd {
 
 /// Maps a definition location onto an #include file, based on a set of filename
 /// rules.
 /// Only const methods (i.e. mapHeader) in this class are thread safe.
 class CanonicalIncludes {
 public:
-  /// Adds a file-to-string mapping from \p ID to \p CanonicalPath.
-  void addMapping(FileEntryRef Header, llvm::StringRef CanonicalPath);
-
-  /// Returns the overridden include for a qualified symbol with, or "".
-  /// \p Scope and \p Name concatenation forms the fully qualified name.
-  /// \p Scope is the qualifier with the trailing "::" (e.g. "std::") or empty
-  /// (for global namespace).
-  llvm::StringRef mapSymbol(llvm::StringRef Scope, llvm::StringRef Name,
-                            const LangOptions &L) const;
-
-  /// Returns the overridden include for files in \p Header, or "".
+  /// Returns the overridden verbatim spelling for files in \p Header that can
+  /// be directly included (i.e., contains quotes "" or angled brackets <>), or
+  /// "" if the spelling could not be found.
   llvm::StringRef mapHeader(FileEntryRef Header) const;
 
   /// Adds mapping for system headers and some special symbols (e.g. STL symbols
   /// in <iosfwd> need to be mapped individually). Approximately, the following
   /// system headers are handled:
   ///   - C++ standard library e.g. bits/basic_string.h$ -> <string>
   ///   - Posix library e.g. bits/pthreadtypes.h$ -> <pthread.h>
   ///   - Compiler extensions, e.g. include/avx512bwintrin.h$ -> <immintrin.h>
   /// The mapping is hardcoded and hand-maintained, so it might not cover all
   /// headers.
   void addSystemHeadersMapping(const LangOptions &Language);
 
 private:
-  /// A map from the private header to a canonical include path.
-  llvm::DenseMap<llvm::sys::fs::UniqueID, std::string> FullPathMapping;
   /// A map from a suffix (one or components of a path) to a canonical path.
   /// Used only for mapping standard headers.
   const llvm::StringMap<llvm::StringRef> *StdSuffixHeaderMapping = nullptr;
 };
-
-/// Returns a CommentHandler that parses pragma comment on include files to
-/// determine when we should include a different header from the header that
-/// directly defines a symbol. Mappinps are registered with \p Includes.
-///
-/// Currently it only supports IWYU private pragma:
-/// https://github.com/include-what-you-use/include-what-you-use/blob/master/docs/IWYUPragmas.md#iwyu-pragma-private
-///
-/// We ignore other pragmas:
-/// - keep: this is common but irrelevant: we do not currently remove includes
-/// - export: this is common and potentially interesting, there are three cases:
-///    * Points to a public header (common): we can suppress include2 if you
-///      already have include1. Only marginally useful.
-///    * Points to a private header annotated with `private` (somewhat common):
-///      Not incrementally useful as we support private.
-///    * Points to a private header without pragmas (rare). This is a reversed
-///      private pragma, and is valuable but too rare to be worthwhile.
-/// - no_include: this is about as common as private, but only affects the
-///   current file, so the value is smaller. We could add support.
-/// - friend: this is less common than private, has implementation difficulties,
-///   and affects behavior in a limited scope.
-/// - associated: extremely rare
-std::unique_ptr<CommentHandler>
-collectIWYUHeaderMaps(CanonicalIncludes *Includes);
-
 } // namespace clangd
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_CANONICALINCLUDES_H

clang-tools-extra/clangd/index/CanonicalIncludes.cpp

 //===-- CanonicalIncludes.h - remap #include headers-------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "CanonicalIncludes.h"
-#include "Headers.h"
 #include "clang/Basic/FileEntry.h"
-#include "clang/Tooling/Inclusions/HeaderAnalysis.h"
-#include "clang/Tooling/Inclusions/StandardLibrary.h"
+#include "clang/Basic/LangOptions.h"
 #include "llvm/ADT/StringRef.h"
-#include "llvm/Support/FileSystem/UniqueID.h"
 #include "llvm/Support/Path.h"
-#include <algorithm>
 
 namespace clang {
 namespace clangd {
 namespace {
 const std::pair<llvm::StringRef, llvm::StringRef> IncludeMappings[] = {
     {"include/__stddef_max_align_t.h", "<cstddef>"},
     {"include/__wmmintrin_aes.h", "<wmmintrin.h>"},
     {"include/__wmmintrin_pclmul.h", "<wmmintrin.h>"},
const std::pair<llvm::StringRef, llvm::StringRef> IncludeMappings[] = {
     {"bits/utmpx.h", "<utmpx.h>"},
     {"bits/waitflags.h", "<sys/wait.h>"},
     {"bits/waitstatus.h", "<sys/wait.h>"},
     {"bits/xtitypes.h", "<stropts.h>"},
 };
 
 } // namespace
 
-void CanonicalIncludes::addMapping(FileEntryRef Header,
-                                   llvm::StringRef CanonicalPath) {
-  FullPathMapping[Header.getUniqueID()] = std::string(CanonicalPath);
-}
-
 /// The maximum number of path components in a key from StdSuffixHeaderMapping.
 /// Used to minimize the number of lookups in suffix path mappings.
 constexpr int MaxSuffixComponents = 3;
 
 llvm::StringRef CanonicalIncludes::mapHeader(FileEntryRef Header) const {
-  auto MapIt = FullPathMapping.find(Header.getUniqueID());
-  if (MapIt != FullPathMapping.end())
-    return MapIt->second;
-
   if (!StdSuffixHeaderMapping)
     return "";
 
   int Components = 1;
 
   // FIXME: check that this works on Windows and add tests.
   auto Filename = Header.getName();
   for (auto It = llvm::sys::path::rbegin(Filename),
             End = llvm::sys::path::rend(Filename);
        It != End && Components <= MaxSuffixComponents; ++It, ++Components) {
     auto SubPath = Filename.substr(It->data() - Filename.begin());
     auto MappingIt = StdSuffixHeaderMapping->find(SubPath);
     if (MappingIt != StdSuffixHeaderMapping->end())
       return MappingIt->second;
   }
   return "";
 }
 
-llvm::StringRef CanonicalIncludes::mapSymbol(llvm::StringRef Scope,
-                                             llvm::StringRef Name,
-                                             const LangOptions &L) const {
-  tooling::stdlib::Lang Lang;
-  if (L.CPlusPlus)
-    Lang = tooling::stdlib::Lang::CXX;
-  else if (L.C11)
-    Lang = tooling::stdlib::Lang::C;
-  else
-    return "";
-  // FIXME: remove the following special cases when the tooling stdlib supports
-  // them.
-  // There are two std::move()s, this is by far the most common.
-  if (Scope == "std::" && Name == "move")
-    return "<utility>";
-  if (auto StdSym = tooling::stdlib::Symbol::named(Scope, Name, Lang))
-    if (auto Header = StdSym->header())
-      return Header->name();
-  return "";
-}
-
-std::unique_ptr<CommentHandler>
-collectIWYUHeaderMaps(CanonicalIncludes *Includes) {
-  class PragmaCommentHandler : public clang::CommentHandler {
-  public:
-    PragmaCommentHandler(CanonicalIncludes *Includes) : Includes(Includes) {}
-
-    bool HandleComment(Preprocessor &PP, SourceRange Range) override {
-      auto Pragma = tooling::parseIWYUPragma(
-          PP.getSourceManager().getCharacterData(Range.getBegin()));
-      if (!Pragma || !Pragma->consume_front("private, include "))
-        return false;
-      auto &SM = PP.getSourceManager();
-      // We always insert using the spelling from the pragma.
-      if (auto *FE = SM.getFileEntryForID(SM.getFileID(Range.getBegin())))
-        Includes->addMapping(FE->getLastRef(),
-                             isLiteralInclude(*Pragma)
-                                 ? Pragma->str()
-                                 : ("\"" + *Pragma + "\"").str());
-      return false;
-    }
-
-  private:
-    CanonicalIncludes *const Includes;
-  };
-  return std::make_unique<PragmaCommentHandler>(Includes);
-}
-
 void CanonicalIncludes::addSystemHeadersMapping(const LangOptions &Language) {
   // FIXME: remove the std header mapping once we support ambiguous symbols, now
   // it serves as a fallback to disambiguate:
   //   - symbols with multiple headers (e.g. std::move)
   static const auto *SystemHeaderMap = [] {
     const size_t Size = sizeof(IncludeMappings) / sizeof(IncludeMappings[0]);
     auto *HeaderMap = new llvm::StringMap<llvm::StringRef>(Size);
     for (size_t I = 0; I < Size; I++) {

clang-tools-extra/clangd/index/FileIndex.h

 // updated dynamically.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_FILEINDEX_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_FILEINDEX_H
 
 #include "Headers.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/Index.h"
 #include "index/Merge.h"
 #include "index/Ref.h"
 #include "index/Relation.h"
 #include "index/Serialization.h"
 #include "index/Symbol.h"
 #include "support/MemoryTree.h"
 #include "support/Path.h"

 /// This manages symbols from files and an in-memory index on all symbols.
 /// FIXME: Expose an interface to remove files that are closed.
 class FileIndex : public MergedIndex {
 public:
   FileIndex();
 
   /// Update preamble symbols of file \p Path with all declarations in \p AST
   /// and macros in \p PP.
-  void updatePreamble(PathRef Path, llvm::StringRef Version, ASTContext &AST,
-                      Preprocessor &PP, const CanonicalIncludes &Includes);
+  void
+  updatePreamble(PathRef Path, llvm::StringRef Version, ASTContext &AST,
+                 Preprocessor &PP,
+                 std::shared_ptr<const include_cleaner::PragmaIncludes> PI);
   void updatePreamble(IndexFileIn);
 
   /// Update symbols and references from main file \p Path with
   /// `indexMainDecls`.
   void updateMain(PathRef Path, ParsedAST &AST);
 
   void profile(MemoryTree &MT) const;
 

 
 /// Retrieves symbols and refs of local top level decls in \p AST (i.e.
 /// `AST.getLocalTopLevelDecls()`).
 /// Exposed to assist in unit tests.
 SlabTuple indexMainDecls(ParsedAST &AST);
 
 /// Index declarations from \p AST and macros from \p PP that are declared in
 /// included headers.
-SlabTuple indexHeaderSymbols(llvm::StringRef Version, ASTContext &AST,
-                             Preprocessor &PP,
-                             const CanonicalIncludes &Includes);
+SlabTuple
+indexHeaderSymbols(llvm::StringRef Version, ASTContext &AST, Preprocessor &PP,
+                   std::shared_ptr<const include_cleaner::PragmaIncludes> PI);
 
 /// Takes slabs coming from a TU (multiple files) and shards them per
 /// declaration location.
 struct FileShardedIndex {
   /// \p HintPath is used to convert file URIs stored in symbols into absolute
   /// paths.
   explicit FileShardedIndex(IndexFileIn Input);
 

clang-tools-extra/clangd/index/FileIndex.cpp

 //===--- FileIndex.cpp - Indexes for files. ------------------------ C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "FileIndex.h"
 #include "CollectMacros.h"
 #include "ParsedAST.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/Index.h"
 #include "index/MemIndex.h"
 #include "index/Merge.h"
 #include "index/Ref.h"
 #include "index/Relation.h"
 #include "index/Serialization.h"
 #include "index/Symbol.h"
 #include "index/SymbolCollector.h"

 #include <tuple>
 #include <utility>
 #include <vector>
 
 namespace clang {
 namespace clangd {
 namespace {
 
-SlabTuple indexSymbols(ASTContext &AST, Preprocessor &PP,
-                       llvm::ArrayRef<Decl *> DeclsToIndex,
-                       const MainFileMacros *MacroRefsToIndex,
-                       const CanonicalIncludes &Includes, bool IsIndexMainAST,
-                       llvm::StringRef Version, bool CollectMainFileRefs) {
+SlabTuple indexSymbols(
+    ASTContext &AST, Preprocessor &PP, llvm::ArrayRef<Decl *> DeclsToIndex,
+    std::shared_ptr<const include_cleaner::PragmaIncludes> PI,
+    bool IsIndexMainAST, llvm::StringRef Version, bool CollectMainFileRefs) {
   SymbolCollector::Options CollectorOpts;
   CollectorOpts.CollectIncludePath = true;
-  CollectorOpts.Includes = &Includes;
+  CollectorOpts.PragmaIncludes = PI;
   CollectorOpts.CountReferences = false;
   CollectorOpts.Origin =
       IsIndexMainAST ? SymbolOrigin::Open : SymbolOrigin::Preamble;
   CollectorOpts.CollectMainFileRefs = CollectMainFileRefs;
   // We want stdlib implementation details in the index only if we've opened the
   // file in question. This does means xrefs won't work, though.
   CollectorOpts.CollectReserved = IsIndexMainAST;
 
FileShardedIndex::getShard(llvm::StringRef Uri) const {
   IF.Relations = std::move(RelB).build();
   // Explicit move here is needed by some compilers.
   return std::move(IF);
 }
 
 SlabTuple indexMainDecls(ParsedAST &AST) {
   return indexSymbols(
       AST.getASTContext(), AST.getPreprocessor(), AST.getLocalTopLevelDecls(),
-      &AST.getMacros(), AST.getCanonicalIncludes(),
+      &AST.getMacros(), AST.getPragmaIncludes(),
       /*IsIndexMainAST=*/true, AST.version(), /*CollectMainFileRefs=*/true);
 }
 
-SlabTuple indexHeaderSymbols(llvm::StringRef Version, ASTContext &AST,
-                             Preprocessor &PP,
-                             const CanonicalIncludes &Includes) {
+SlabTuple
+indexHeaderSymbols(llvm::StringRef Version, ASTContext &AST, Preprocessor &PP,
+                   std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
   std::vector<Decl *> DeclsToIndex(
       AST.getTranslationUnitDecl()->decls().begin(),
       AST.getTranslationUnitDecl()->decls().end());
   return indexSymbols(AST, PP, DeclsToIndex,
-                      /*MainFileMacros=*/nullptr, Includes,
+                      /*MainFileMacros=*/nullptr, PI,
                       /*IsIndexMainAST=*/false, Version,
                       /*CollectMainFileRefs=*/false);
 }
 
 FileSymbols::FileSymbols(IndexContents IdxContents)
     : IdxContents(IdxContents) {}
 
 void FileSymbols::update(llvm::StringRef Key,
auto NewIndex = PreambleSymbols.buildIndex(
     PreambleIndex.reset(std::move(NewIndex));
     vlog(
         "Build dynamic index for header symbols with estimated memory usage of "
         "{0} bytes",
         PreambleIndex.estimateMemoryUsage());
   }
 }
 
-void FileIndex::updatePreamble(PathRef Path, llvm::StringRef Version,
-                               ASTContext &AST, Preprocessor &PP,
-                               const CanonicalIncludes &Includes) {
+void FileIndex::updatePreamble(
+    PathRef Path, llvm::StringRef Version, ASTContext &AST, Preprocessor &PP,
+    std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
   IndexFileIn IF;
   std::tie(IF.Symbols, std::ignore, IF.Relations) =
-      indexHeaderSymbols(Version, AST, PP, Includes);
+      indexHeaderSymbols(Version, AST, PP, PI);
   updatePreamble(std::move(IF));
 }
 
 void FileIndex::updateMain(PathRef Path, ParsedAST &AST) {
   auto Contents = indexMainDecls(AST);
   MainFileSymbols.update(
       URI::create(Path).toString(),
       std::make_unique<SymbolSlab>(std::move(std::get<0>(Contents))),

clang-tools-extra/clangd/index/IndexAction.cpp

 //===--- IndexAction.cpp -----------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "IndexAction.h"
 #include "AST.h"
 #include "Headers.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/Relation.h"
+#include "index/SymbolCollector.h"
 #include "index/SymbolOrigin.h"
 #include "clang/AST/ASTConsumer.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendAction.h"
 #include "clang/Index/IndexingAction.h"
private:
   const SourceManager &SM;
   IncludeGraph &IG;
 };
 
 // Wraps the index action and reports index data after each translation unit.
 class IndexAction : public ASTFrontendAction {
 public:
   IndexAction(std::shared_ptr<SymbolCollector> C,
-              std::unique_ptr<CanonicalIncludes> Includes,
+              std::shared_ptr<include_cleaner::PragmaIncludes> PI,
               const index::IndexingOptions &Opts,
               std::function<void(SymbolSlab)> SymbolsCallback,
               std::function<void(RefSlab)> RefsCallback,
               std::function<void(RelationSlab)> RelationsCallback,
               std::function<void(IncludeGraph)> IncludeGraphCallback)
       : SymbolsCallback(SymbolsCallback), RefsCallback(RefsCallback),
         RelationsCallback(RelationsCallback),
-        IncludeGraphCallback(IncludeGraphCallback), Collector(C),
-        Includes(std::move(Includes)), Opts(Opts),
-        PragmaHandler(collectIWYUHeaderMaps(this->Includes.get())) {
+        IncludeGraphCallback(IncludeGraphCallback), Collector(C), PI(PI),
+        Opts(Opts) {
     this->Opts.ShouldTraverseDecl = [this](const Decl *D) {
       // Many operations performed during indexing is linear in terms of depth
       // of the decl (USR generation, name lookups, figuring out role of a
       // reference are some examples). Since we index all the decls nested
       // inside, it becomes quadratic. So we give up on nested symbols.
       if (isDeeplyNested(D))
         return false;
       auto &SM = D->getASTContext().getSourceManager();
       auto FID = SM.getFileID(SM.getExpansionLoc(D->getLocation()));
       if (!FID.isValid())
         return true;
       return Collector->shouldIndexFile(FID);
     };
   }
 
   std::unique_ptr<ASTConsumer>
   CreateASTConsumer(CompilerInstance &CI, llvm::StringRef InFile) override {
-    CI.getPreprocessor().addCommentHandler(PragmaHandler.get());
-    Includes->addSystemHeadersMapping(CI.getLangOpts());
+    PI->record(CI.getPreprocessor());
     if (IncludeGraphCallback != nullptr)
       CI.getPreprocessor().addPPCallbacks(
           std::make_unique<IncludeGraphCollector>(CI.getSourceManager(), IG));
 
     return index::createIndexingASTConsumer(Collector, Opts,
                                             CI.getPreprocessorPtr());
   }
 
#endif
   }
 
 private:
   std::function<void(SymbolSlab)> SymbolsCallback;
   std::function<void(RefSlab)> RefsCallback;
   std::function<void(RelationSlab)> RelationsCallback;
   std::function<void(IncludeGraph)> IncludeGraphCallback;
   std::shared_ptr<SymbolCollector> Collector;
-  std::unique_ptr<CanonicalIncludes> Includes;
+  std::shared_ptr<include_cleaner::PragmaIncludes> PI;
   index::IndexingOptions Opts;
-  std::unique_ptr<CommentHandler> PragmaHandler;
   IncludeGraph IG;
 };
 
 } // namespace
 
 std::unique_ptr<FrontendAction> createStaticIndexingAction(
     SymbolCollector::Options Opts,
     std::function<void(SymbolSlab)> SymbolsCallback,
     std::function<void(RefSlab)> RefsCallback,
     std::function<void(RelationSlab)> RelationsCallback,
     std::function<void(IncludeGraph)> IncludeGraphCallback) {
   index::IndexingOptions IndexOpts;
   IndexOpts.SystemSymbolFilter =
       index::IndexingOptions::SystemSymbolFilterKind::All;
   // We index function-local classes and its member functions only.
   IndexOpts.IndexFunctionLocals = true;
   Opts.CollectIncludePath = true;
   if (Opts.Origin == SymbolOrigin::Unknown)
     Opts.Origin = SymbolOrigin::Static;
   Opts.StoreAllDocumentation = false;
   if (RefsCallback != nullptr) {
     Opts.RefFilter = RefKind::All;
     Opts.RefsInHeaders = true;
   }
-  auto Includes = std::make_unique<CanonicalIncludes>();
-  Opts.Includes = Includes.get();
-  return std::make_unique<IndexAction>(
-      std::make_shared<SymbolCollector>(std::move(Opts)), std::move(Includes),
-      IndexOpts, SymbolsCallback, RefsCallback, RelationsCallback,
-      IncludeGraphCallback);
+  auto PragmaIncludes = std::make_shared<include_cleaner::PragmaIncludes>();
+  Opts.PragmaIncludes = PragmaIncludes;
+  return std::make_unique<IndexAction>(std::make_shared<SymbolCollector>(Opts),
+                                       std::move(PragmaIncludes), IndexOpts,
+                                       SymbolsCallback, RefsCallback,
+                                       RelationsCallback, IncludeGraphCallback);
 }
 
 } // namespace clangd
 } // namespace clang

clang-tools-extra/clangd/index/SymbolCollector.h

 //===--- SymbolCollector.h ---------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_SYMBOLCOLLECTOR_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_INDEX_SYMBOLCOLLECTOR_H
 
 #include "CollectMacros.h"
+#include "clang-include-cleaner/Record.h"
+#include "clang-include-cleaner/Types.h"
 #include "index/CanonicalIncludes.h"
 #include "index/Ref.h"
 #include "index/Relation.h"
 #include "index/Symbol.h"
 #include "index/SymbolID.h"
 #include "index/SymbolOrigin.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexSymbol.h"
 #include "clang/Sema/CodeCompleteConsumer.h"
-#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/SmallVector.h"
 #include <functional>
+#include <memory>
 #include <optional>
 
 namespace clang {
 namespace clangd {
 
 /// Collect declarations (symbols) from an AST.
 /// It collects most declarations except:
 /// - Implicit declarations
public:
   struct Options {
     /// When symbol paths cannot be resolved to absolute paths (e.g. files in
     /// VFS that does not have absolute path), combine the fallback directory
     /// with symbols' paths to get absolute paths. This must be an absolute
     /// path.
     std::string FallbackDir;
     bool CollectIncludePath = false;
     /// If set, this is used to map symbol #include path to a potentially
-    /// different #include path.
-    const CanonicalIncludes *Includes = nullptr;
+    /// different #include path specified by IWYU pragmas.
+    std::shared_ptr<const include_cleaner::PragmaIncludes> PragmaIncludes =
+        nullptr;
     // Populate the Symbol.References field.
     bool CountReferences = false;
     /// The symbol ref kinds that will be collected.
     /// If not set, SymbolCollector will not collect refs.
     /// Note that references of namespace decls are not collected, as they
     /// contribute large part of the index, and they are less useful compared
     /// with other decls.
     RefKind RefFilter = RefKind::Unknown;
public:
   // like call hierarchy.
   static const Decl *getRefContainer(const Decl *Enclosing,
                                      const SymbolCollector::Options &Opts);
 
   void initialize(ASTContext &Ctx) override;
 
   void setPreprocessor(std::shared_ptr<Preprocessor> PP) override {
     this->PP = PP.get();
+    SysHeaderMapping.addSystemHeadersMapping(PP->getLangOpts());
+  }
+  void setPreprocessor(Preprocessor &PP) {
+    this->PP = &PP;
+    SysHeaderMapping.addSystemHeadersMapping(PP.getLangOpts());
   }
-  void setPreprocessor(Preprocessor &PP) { this->PP = &PP; }
 
   bool
   handleDeclOccurrence(const Decl *D, index::SymbolRoleSet Roles,
                        ArrayRef<index::SymbolRelation> Relations,
                        SourceLocation Loc,
                        index::IndexDataConsumer::ASTNodeInfo ASTNode) override;
 
   bool handleMacroOccurrence(const IdentifierInfo *Name, const MacroInfo *MI,
private:
   std::optional<std::string> getIncludeHeader(const Symbol &S, FileID);
 
   SymbolID getSymbolIDCached(const Decl *D);
   SymbolID getSymbolIDCached(const llvm::StringRef MacroName,
                              const MacroInfo *MI, const SourceManager &SM);
 
   // All Symbols collected from the AST.
   SymbolSlab::Builder Symbols;
-  // File IDs for Symbol.IncludeHeaders.
-  // The final spelling is calculated in finish().
+  // File IDs used to determine if the code contains Obj-C constructs.
+  // For Obj-C symbols, these File IDs are used to compute the include
+  // headers.
   llvm::DenseMap<SymbolID, FileID> IncludeFiles;
+  void setIncludeLocation(const Symbol &S, SourceLocation,
+                          const include_cleaner::Symbol &Sym);
+
+  // Providers for Symbol.IncludeHeaders.
+  // The final spelling is calculated in finish().
+  llvm::DenseMap<SymbolID, include_cleaner::Header> SymbolProviders;
+
   // Files which contain ObjC symbols.
   // This is finalized and used in finish().
   llvm::DenseSet<FileID> FilesWithObjCConstructs;
-  void setIncludeLocation(const Symbol &S, SourceLocation);
+
   // Indexed macros, to be erased if they turned out to be include guards.
   llvm::DenseSet<const IdentifierInfo *> IndexedMacros;
   // All refs collected from the AST. It includes:
   //   1) symbols declared in the preamble and referenced from the main file (
   //     which is not a header), or
   //   2) symbols declared and referenced from the main file (which is a header)
   RefSlab::Builder Refs;
   // All relations collected from the AST.
   RelationSlab::Builder Relations;
   ASTContext *ASTCtx;
   Preprocessor *PP = nullptr;
+  // This is used to map symbols to their #include paths in system headers.
+  CanonicalIncludes SysHeaderMapping;
   std::shared_ptr<GlobalCodeCompletionAllocator> CompletionAllocator;
   std::unique_ptr<CodeCompletionTUInfo> CompletionTUInfo;
   Options Opts;
   struct SymbolRef {
     SourceLocation Loc;
     FileID FID;
     index::SymbolRoleSet Roles;
     const Decl *Container;

clang-tools-extra/clangd/index/SymbolCollector.cpp

 //===--- SymbolCollector.cpp -------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "SymbolCollector.h"
 #include "AST.h"
 #include "CodeComplete.h"
 #include "CodeCompletionStrings.h"
 #include "ExpectedTypes.h"
 #include "SourceCode.h"
 #include "URI.h"
+#include "clang-include-cleaner/Analysis.h"
+#include "clang-include-cleaner/Record.h"
+#include "clang-include-cleaner/Types.h"
 #include "index/CanonicalIncludes.h"
 #include "index/Relation.h"
+#include "index/Symbol.h"
 #include "index/SymbolID.h"
 #include "index/SymbolLocation.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclBase.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/AST/DeclTemplate.h"
 #include "clang/AST/DeclarationName.h"
+#include "clang/Basic/FileEntry.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Index/IndexSymbol.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Lex/Token.h"
 #include "clang/Tooling/Inclusions/HeaderAnalysis.h"
 #include "llvm/ADT/ArrayRef.h"
-#include "llvm/Support/Casting.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
+#include <cassert>
+#include <memory>
 #include <optional>
+#include <string>
+#include <utility>
 
 namespace clang {
 namespace clangd {
 namespace {
 
 /// If \p ND is a template specialization, returns the described template.
 /// Otherwise, returns \p ND.
 const NamedDecl &getTemplateOrThis(const NamedDecl &ND) {
struct FrameworkUmbrellaSpelling {
     // <Foundation/Foundation_Private.h>
     std::optional<std::string> PrivateHeader;
   };
   // Weird double-indirect access to PP, which might not be ready yet when
   // HeaderFiles is created but will be by the time it's used.
   // (IndexDataConsumer::setPreprocessor can happen before or after initialize)
   Preprocessor *&PP;
   const SourceManager &SM;
-  const CanonicalIncludes *Includes;
+  const CanonicalIncludes *SysHeaderMapping;
+  std::shared_ptr<const include_cleaner::PragmaIncludes> PI;
   llvm::StringRef FallbackDir;
   llvm::DenseMap<const FileEntry *, const std::string *> CacheFEToURI;
   llvm::StringMap<std::string> CachePathToURI;
   llvm::DenseMap<FileID, llvm::StringRef> CacheFIDToInclude;
   llvm::StringMap<std::string> CachePathToFrameworkSpelling;
   llvm::StringMap<FrameworkUmbrellaSpelling>
       CacheFrameworkToUmbrellaHeaderSpelling;
 
 public:
   HeaderFileURICache(Preprocessor *&PP, const SourceManager &SM,
+                     const CanonicalIncludes *SysHeaderMapping,
                      const SymbolCollector::Options &Opts)
-      : PP(PP), SM(SM), Includes(Opts.Includes), FallbackDir(Opts.FallbackDir) {
-  }
+      : PP(PP), SM(SM), SysHeaderMapping(SysHeaderMapping),
+        PI(Opts.PragmaIncludes), FallbackDir(Opts.FallbackDir) {}
 
   // Returns a canonical URI for the file \p FE.
   // We attempt to make the path absolute first.
   const std::string &toURI(const FileEntryRef FE) {
     auto R = CacheFEToURI.try_emplace(FE);
     if (R.second) {
       auto CanonPath = getCanonicalPath(FE, SM.getFileManager());
       R.first->second = &toURIInternal(CanonPath ? *CanonPath : FE.getName());
public:
   // Returns "" if includes should not be inserted for this file.
   llvm::StringRef getIncludeHeader(FileID FID) {
     auto R = CacheFIDToInclude.try_emplace(FID);
     if (R.second)
       R.first->second = getIncludeHeaderUncached(FID);
     return R.first->second;
   }
 
+  // If a file is mapped by canonical headers, use that mapping, regardless
+  // of whether it's an otherwise-good header (header guards etc).
+  llvm::StringRef mapCanonical(FileEntryRef FE) {
+    if (!SysHeaderMapping)
+      return "";
+    auto Canonical = SysHeaderMapping->mapHeader(FE);
+    if (Canonical.empty())
+      return "";
+    // If we had a mapping, always use it.
+    assert(Canonical.startswith("<") || Canonical.startswith("\""));
+    return Canonical;
+  }
+
 private:
   // This takes care of making paths absolute and path->URI caching, but no
   // FileManager-based canonicalization.
   const std::string &toURIInternal(llvm::StringRef Path) {
     auto R = CachePathToURI.try_emplace(Path);
     if (R.second) {
       llvm::SmallString<256> AbsPath = Path;
       if (!llvm::sys::path::is_absolute(AbsPath) && !FallbackDir.empty())
else
               .str();
     return llvm::StringRef(*CachedHeaderSpelling);
   }
 
   llvm::StringRef getIncludeHeaderUncached(FileID FID) {
     const auto FE = SM.getFileEntryRefForID(FID);
     if (!FE || FE->getName().empty())
       return "";
-    llvm::StringRef Filename = FE->getName();
-    // If a file is mapped by canonical headers, use that mapping, regardless
-    // of whether it's an otherwise-good header (header guards etc).
-    if (Includes) {
-      llvm::StringRef Canonical =
-          Includes->mapHeader(*SM.getFileEntryRefForID(FID));
-      if (!Canonical.empty()) {
-        // If we had a mapping, always use it.
-        if (Canonical.startswith("<") || Canonical.startswith("\""))
-          return Canonical;
-        return toURI(Canonical);
-      }
-    }
+
+    if (auto Verbatim = PI->getPublic(SM.getFileEntryForID(FID));
+        !Verbatim.empty())
+      return Verbatim;
+
+    if (auto Canonical = mapCanonical(*FE); !Canonical.empty())
+
     // Framework headers are spelled as <FrameworkName/Foo.h>, not
     // "path/FrameworkName.framework/Headers/Foo.h".
     auto &HS = PP->getHeaderSearchInfo();
     if (const auto *HFI = HS.getExistingFileInfo(*FE, /*WantExternal*/ false))
       if (!HFI->Framework.empty())
         if (auto Spelling =
                 getFrameworkHeaderIncludeSpelling(*FE, HFI->Framework, HS))
           return *Spelling;
 
+    llvm::StringRef Filename = FE->getName();
     if (!tooling::isSelfContainedHeader(*FE, PP->getSourceManager(),
                                         PP->getHeaderSearchInfo())) {
       // A .inc or .def file is often included into a real header to define
       // symbols (e.g. LLVM tablegen files).
       if (Filename.endswith(".inc") || Filename.endswith(".def"))
         // Don't use cache reentrantly due to iterator invalidation.
         return getIncludeHeaderUncached(SM.getFileID(SM.getIncludeLoc(FID)));
       // Conservatively refuse to insert #includes to files without guards.

 }
 
 SymbolCollector::SymbolCollector(Options Opts) : Opts(std::move(Opts)) {}
 SymbolCollector::~SymbolCollector() = default;
 
 void SymbolCollector::initialize(ASTContext &Ctx) {
   ASTCtx = &Ctx;
   HeaderFileURIs = std::make_unique<HeaderFileURICache>(
-      this->PP, ASTCtx->getSourceManager(), Opts);
+      this->PP, ASTCtx->getSourceManager(), &SysHeaderMapping, Opts);
   CompletionAllocator = std::make_shared<GlobalCodeCompletionAllocator>();
   CompletionTUInfo =
       std::make_unique<CodeCompletionTUInfo>(CompletionAllocator);
 }
 
 bool SymbolCollector::shouldCollectSymbol(const NamedDecl &ND,
                                           const ASTContext &ASTCtx,
                                           const Options &Opts,
bool SymbolCollector::handleMacroOccurrence(const IdentifierInfo *Name,
   std::string Signature;
   std::string SnippetSuffix;
   getSignature(*CCS, &Signature, &SnippetSuffix, SymbolCompletion.Kind,
                SymbolCompletion.CursorKind);
   S.Signature = Signature;
   S.CompletionSnippetSuffix = SnippetSuffix;
 
   IndexedMacros.insert(Name);
-  setIncludeLocation(S, DefLoc);
+
+  setIncludeLocation(
+      S, DefLoc,
+      include_cleaner::Macro{const_cast<IdentifierInfo *>(Name), DefLoc});
   Symbols.insert(S);
   return true;
 }
 
 void SymbolCollector::processRelations(
     const NamedDecl &ND, const SymbolID &ID,
     ArrayRef<index::SymbolRelation> Relations) {
   for (const auto &R : Relations) {
     auto RKind = indexableRelation(R);
for (const auto &R : Relations) {
     // We currently do (B) because it's simpler.
     if (*RKind == RelationKind::BaseOf)
       this->Relations.insert({ID, *RKind, ObjectID});
     else if (*RKind == RelationKind::OverriddenBy)
       this->Relations.insert({ObjectID, *RKind, ID});
   }
 }
 
-void SymbolCollector::setIncludeLocation(const Symbol &S, SourceLocation Loc) {
+void SymbolCollector::setIncludeLocation(const Symbol &S, SourceLocation DefLoc,
+                                         const include_cleaner::Symbol &Sym) {
+  const auto &SM = PP->getSourceManager();
   if (Opts.CollectIncludePath &&
       shouldCollectIncludePath(S.SymInfo.Kind) != Symbol::Invalid)
     // Use the expansion location to get the #include header since this is
     // where the symbol is exposed.
-    IncludeFiles[S.ID] =
-        PP->getSourceManager().getDecomposedExpansionLoc(Loc).first;
+    IncludeFiles[S.ID] = SM.getDecomposedExpansionLoc(DefLoc).first;
+
+  llvm::SmallVector<include_cleaner::Header> Headers =
+      include_cleaner::headersForSymbol(Sym, SM, Opts.PragmaIncludes.get());
+  if (!Headers.empty())
+    SymbolProviders.insert({S.ID, Headers[0]});
 }
 
 void SymbolCollector::finish() {
   // At the end of the TU, add 1 to the refcount of all referenced symbols.
   for (const auto &ID : ReferencedSymbols) {
     if (const auto *S = Symbols.find(ID)) {
       // SymbolSlab::Builder returns const symbols because strings are interned
       // and modifying returned symbols without inserting again wouldn't go
       // well. const_cast is safe here as we're modifying a data owned by the
       // Symbol. This reduces time spent in SymbolCollector by ~1%.
       ++const_cast<Symbol *>(S)->References;
     }
   }
   if (Opts.CollectMacro) {
     assert(PP);
     // First, drop header guards. We can't identify these until EOF.
     for (const IdentifierInfo *II : IndexedMacros) {
       if (const auto *MI = PP->getMacroDefinition(II).getMacroInfo())
         if (auto ID =
                 getSymbolIDCached(II->getName(), MI, PP->getSourceManager()))
           if (MI->isUsedForHeaderGuard())
             Symbols.erase(ID);
     }
   }
   llvm::DenseMap<FileID, bool> FileToContainsImportsOrObjC;
   // Fill in IncludeHeaders.
   // We delay this until end of TU so header guards are all resolved.
   for (const auto &[SID, FID] : IncludeFiles) {
-    if (const Symbol *S = Symbols.find(SID)) {
-      llvm::StringRef IncludeHeader;
-      // Look for an overridden include header for this symbol specifically.
-      if (Opts.Includes) {
-        IncludeHeader =
-            Opts.Includes->mapSymbol(S->Scope, S->Name, ASTCtx->getLangOpts());
-        if (!IncludeHeader.empty()) {
-          if (IncludeHeader.front() != '"' && IncludeHeader.front() != '<')
-            IncludeHeader = HeaderFileURIs->toURI(IncludeHeader);
-          else if (IncludeHeader == "<utility>" && S->Scope == "std::" &&
-                   S->Name == "move" && S->Signature.contains(','))
-            IncludeHeader = "<algorithm>";
-        }
-      }
-      // Otherwise find the approprate include header for the defining file.
-      if (IncludeHeader.empty())
-        IncludeHeader = HeaderFileURIs->getIncludeHeader(FID);
-
-      // Symbols in slabs aren't mutable, insert() has to walk all the strings
-      if (!IncludeHeader.empty()) {
-        Symbol::IncludeDirective Directives = Symbol::Invalid;
-        auto CollectDirectives = shouldCollectIncludePath(S->SymInfo.Kind);
-        if ((CollectDirectives & Symbol::Include) != 0)
-          Directives |= Symbol::Include;
-        // Only allow #import for symbols from ObjC-like files.
-        if ((CollectDirectives & Symbol::Import) != 0) {
-          auto [It, Inserted] = FileToContainsImportsOrObjC.try_emplace(FID);
-          if (Inserted)
-            It->second = FilesWithObjCConstructs.contains(FID) ||
-                         tooling::codeContainsImports(
-                             ASTCtx->getSourceManager().getBufferData(FID));
-          if (It->second)
-            Directives |= Symbol::Import;
-        }
-        if (Directives != Symbol::Invalid) {
-          Symbol NewSym = *S;
-          NewSym.IncludeHeaders.push_back({IncludeHeader, 1, Directives});
-          Symbols.insert(NewSym);
-        }
-      }
+    const Symbol *S = Symbols.find(SID);
+    if (!S)
+      continue;
+    // Determine if the FID is #include'd or #import'ed.
+
+    if (Directives == Symbol::Invalid)
+      continue;
+
+    auto NewSym = *S;
+    // Use the include location-based logic for Objective-C symbols.
+    if (Directives & Symbol::Import) {
+      if (auto IncludeHeader = HeaderFileURIs->getIncludeHeader(FID);
+          !IncludeHeader.empty()) {
+      // FIXME: use providers from include-cleaner library once it's polished
+      // for Objective-C.
+      continue;
+
+    assert(Directives == Symbol::Include);
+    // For #include's, use the providers computed by the include-cleaner
+    // library.
+    auto It = SymbolProviders.find(SID);
+    if (It == SymbolProviders.end())
+      continue;
+    const auto &H = It->second;
+    llvm::DenseMap<include_cleaner::Header, std::string> HeaderSpelling;
+    const auto [SpellingIt, Inserted] = HeaderSpelling.try_emplace(H);
+    if (!Inserted)
+      continue;
+    switch (H.kind()) {
+    case include_cleaner::Header::Kind::Verbatim:
+      SpellingIt->second = H.verbatim().str();
+      break;
+    case include_cleaner::Header::Kind::Standard:
+      SpellingIt->second = H.standard().name().str();
+      break;
+    case include_cleaner::Header::Kind::Physical:
+      auto &SM = ASTCtx->getSourceManager();
+      if (auto Canonical =
+              HeaderFileURIs->mapCanonical(H.physical()->getLastRef());
+          !Canonical.empty())
+        SpellingIt->second = Canonical;
+      else if (!tooling::isSelfContainedHeader(H.physical(), SM,
+                                               PP->getHeaderSearchInfo()))
+        break;
+      else
+        SpellingIt->second =
+            HeaderFileURIs->toURI(H.physical()->tryGetRealPathName());
+      break;
     }
+    if (!SpellingIt->second.empty())
+      NewSym.IncludeHeaders.push_back({SpellingIt->second, 1, Directives});
+    Symbols.insert(NewSym);
   }
 
   ReferencedSymbols.clear();
   IncludeFiles.clear();
+  SymbolProviders.clear();
   FilesWithObjCConstructs.clear();
 }
 
 const Symbol *SymbolCollector::addDeclaration(const NamedDecl &ND, SymbolID ID,
                                               bool IsMainFileOnly) {
   auto &Ctx = ND.getASTContext();
   auto &SM = Ctx.getSourceManager();
 
const Symbol *SymbolCollector::addDeclaration(const NamedDecl &ND, SymbolID ID,
   std::optional<OpaqueType> TypeStorage;
   if (S.Flags & Symbol::IndexedForCodeCompletion) {
     TypeStorage = OpaqueType::fromCompletionResult(*ASTCtx, SymbolCompletion);
     if (TypeStorage)
       S.Type = TypeStorage->raw();
   }
 
   Symbols.insert(S);
-  setIncludeLocation(S, ND.getLocation());
+  setIncludeLocation(S, ND.getLocation(), include_cleaner::Symbol{ND});
   if (S.SymInfo.Lang == index::SymbolLanguage::ObjC)
     FilesWithObjCConstructs.insert(FID);
   return Symbols.find(S.ID);
 }
 
 void SymbolCollector::addDefinition(const NamedDecl &ND,
                                     const Symbol &DeclSym) {
   if (DeclSym.Definition)

clang-tools-extra/clangd/tool/Check.cpp

 #include "ParsedAST.h"
 #include "Preamble.h"
 #include "Protocol.h"
 #include "Selection.h"
 #include "SemanticHighlighting.h"
 #include "SourceCode.h"
 #include "TidyProvider.h"
 #include "XRefs.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/FileIndex.h"
 #include "refactor/Tweak.h"
 #include "support/Context.h"
 #include "support/Logger.h"
 #include "support/ThreadsafeFS.h"
 #include "support/Trace.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/Basic/Diagnostic.h"
public:
   }
 
   // Build preamble and AST, and index them.
   bool buildAST() {
     log("Building preamble...");
     Preamble = buildPreamble(
         File, *Invocation, Inputs, /*StoreInMemory=*/true,
         [&](CapturedASTCtx Ctx,
-            const std::shared_ptr<const CanonicalIncludes> Includes) {
+            std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
           if (!Opts.BuildDynamicSymbolIndex)
             return;
           log("Indexing headers...");
           Index.updatePreamble(File, /*Version=*/"null", Ctx.getASTContext(),
-                               Ctx.getPreprocessor(), *Includes);
+                               Ctx.getPreprocessor(), PI);
         });
     if (!Preamble) {
       elog("Failed to build preamble");
       return false;
     }
     ErrCount += showErrors(Preamble->Diags);
 
     log("Building AST...");

clang-tools-extra/clangd/unittests/CanonicalIncludesTests.cpp

 //===-- CanonicalIncludesTests.cpp - --------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#include "TestFS.h"
 #include "index/CanonicalIncludes.h"
 #include "clang/Basic/FileEntry.h"
 #include "clang/Basic/FileManager.h"
+#include "clang/Basic/FileSystemOptions.h"
 #include "clang/Basic/LangOptions.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Testing/Support/Error.h"
 #include "gtest/gtest.h"
 
 namespace clang {
 namespace clangd {
 namespace {
 
 FileEntryRef addFile(llvm::vfs::InMemoryFileSystem &FS, FileManager &FM,
                      llvm::StringRef Filename) {
   FS.addFile(Filename, 0, llvm::MemoryBuffer::getMemBuffer(""));
   auto File = FM.getFileRef(Filename);
   EXPECT_THAT_EXPECTED(File, llvm::Succeeded());
   return *File;
 }
 
-TEST(CanonicalIncludesTest, CStandardLibrary) {
-  CanonicalIncludes CI;
-  auto Language = LangOptions();
-  Language.C11 = true;
-  CI.addSystemHeadersMapping(Language);
-  // Usual standard library symbols are mapped correctly.
-  EXPECT_EQ("<stdio.h>", CI.mapSymbol("", "printf", Language));
-  EXPECT_EQ("", CI.mapSymbol("", "unknown_symbol", Language));
-}
-
-TEST(CanonicalIncludesTest, CXXStandardLibrary) {
-  CanonicalIncludes CI;
-  auto Language = LangOptions();
-  Language.CPlusPlus = true;
-  CI.addSystemHeadersMapping(Language);
-
-  // Usual standard library symbols are mapped correctly.
-  EXPECT_EQ("<vector>", CI.mapSymbol("std::", "vector", Language));
-  EXPECT_EQ("<cstdio>", CI.mapSymbol("std::", "printf", Language));
-  // std::move is ambiguous, currently always mapped to <utility>
-  EXPECT_EQ("<utility>", CI.mapSymbol("std::", "move", Language));
-  EXPECT_EQ("<cstddef>", CI.mapSymbol("std::", "size_t", Language));
-  // Unknown std symbols aren't mapped.
-  EXPECT_EQ("", CI.mapSymbol("std::", "notathing", Language));
-  // iosfwd declares some symbols it doesn't own.
-  EXPECT_EQ("<ostream>", CI.mapSymbol("std::", "ostream", Language));
-  // And (for now) we assume it owns the others.
-  auto InMemFS = llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>();
-  FileManager Files(FileSystemOptions(), InMemFS);
-  auto File = addFile(*InMemFS, Files, testPath("iosfwd"));
-  EXPECT_EQ("<iosfwd>", CI.mapHeader(File));
-}
-
-TEST(CanonicalIncludesTest, PathMapping) {
-  auto InMemFS = llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>();
-  FileManager Files(FileSystemOptions(), InMemFS);
-  std::string BarPath = testPath("foo/bar");
-  auto Bar = addFile(*InMemFS, Files, BarPath);
-  auto Other = addFile(*InMemFS, Files, testPath("foo/baz"));
-  // As used for IWYU pragmas.
-  CanonicalIncludes CI;
-  CI.addMapping(Bar, "<baz>");
-
-  // We added a mapping for baz.
-  EXPECT_EQ("<baz>", CI.mapHeader(Bar));
-  // Other file doesn't have a mapping.
-  EXPECT_EQ("", CI.mapHeader(Other));
-
-  // Add hard link to "foo/bar" and check that it is also mapped to <baz>, hence
-  // does not depend on the header name.
-  std::string HardLinkPath = testPath("hard/link");
-  InMemFS->addHardLink(HardLinkPath, BarPath);
-  auto HardLinkFile = Files.getFileRef(HardLinkPath);
-  ASSERT_THAT_EXPECTED(HardLinkFile, llvm::Succeeded());
-  EXPECT_EQ("<baz>", CI.mapHeader(*HardLinkFile));
-}
-
-TEST(CanonicalIncludesTest, Precedence) {
+TEST(CanonicalIncludesTest, SystemHeaderMap) {
   auto InMemFS = llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>();
   FileManager Files(FileSystemOptions(), InMemFS);
-  auto File = addFile(*InMemFS, Files, testPath("some/path"));
 
   CanonicalIncludes CI;
-  CI.addMapping(File, "<path>");
   LangOptions Language;
   Language.CPlusPlus = true;
   CI.addSystemHeadersMapping(Language);
 
-  // We added a mapping from some/path to <path>.
-  ASSERT_EQ("<path>", CI.mapHeader(File));
   // We should have a path from 'bits/stl_vector.h' to '<vector>'.
   // FIXME: The Standrad Library map in CanonicalIncludes expects forward
   // slashes and Windows would use backward slashes instead, so the headers are
   // not matched appropriately.
   auto STLVectorFile = addFile(*InMemFS, Files, "bits/stl_vector.h");
   ASSERT_EQ("<vector>", CI.mapHeader(STLVectorFile));
 }
 
 } // namespace
 } // namespace clangd
 } // namespace clang

clang-tools-extra/clangd/unittests/FileIndexTests.cpp

 #include "Compiler.h"
 #include "Headers.h"
 #include "ParsedAST.h"
 #include "SyncAPI.h"
 #include "TestFS.h"
 #include "TestTU.h"
 #include "TestWorkspace.h"
 #include "URI.h"
-#include "index/CanonicalIncludes.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/FileIndex.h"
 #include "index/Index.h"
 #include "index/Ref.h"
 #include "index/Relation.h"
 #include "index/Serialization.h"
 #include "index/Symbol.h"
 #include "index/SymbolID.h"
 #include "support/Threading.h"
 #include "clang/Frontend/CompilerInvocation.h"
-#include "clang/Lex/Preprocessor.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/Support/Allocator.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
+#include <memory>
 #include <utility>
 #include <vector>
 
 using ::testing::_;
 using ::testing::AllOf;
 using ::testing::Contains;
 using ::testing::ElementsAre;
 using ::testing::Gt;

 }
 MATCHER_P(defURI, U, "") {
   return llvm::StringRef(arg.Definition.FileURI) == U;
 }
 MATCHER_P(qName, N, "") { return (arg.Scope + arg.Name).str() == N; }
 MATCHER_P(numReferences, N, "") { return arg.References == N; }
 MATCHER_P(hasOrign, O, "") { return bool(arg.Origin & O); }
 
+MATCHER_P(includeHeader, P, "") {
+  return (arg.IncludeHeaders.size() == 1) &&
+         (arg.IncludeHeaders.begin()->IncludeHeader == P);
+}
+
 namespace clang {
 namespace clangd {
 namespace {
 ::testing::Matcher<const RefSlab &>
 refsAre(std::vector<::testing::Matcher<Ref>> Matchers) {
   return ElementsAre(::testing::Pair(_, UnorderedElementsAreArray(Matchers)));
 }
 

 void update(FileIndex &M, llvm::StringRef Basename, llvm::StringRef Code) {
   TestTU File;
   File.Filename = (Basename + ".cpp").str();
   File.HeaderFilename = (Basename + ".h").str();
   File.HeaderCode = std::string(Code);
   auto AST = File.build();
   M.updatePreamble(testPath(File.Filename), /*Version=*/"null",
                    AST.getASTContext(), AST.getPreprocessor(),
-                   AST.getCanonicalIncludes());
+                   AST.getPragmaIncludes());
 }
 
 TEST(FileIndexTest, CustomizedURIScheme) {
   FileIndex M;
   update(M, "f", "class string {};");
 
   EXPECT_THAT(runFuzzyFind(M, ""), ElementsAre(declURI("unittest:///f.h")));
 }
update(
       "// IWYU pragma: private, include <the/good/header.h>\nclass string {};");
 
   auto Symbols = runFuzzyFind(M, "");
   EXPECT_THAT(Symbols, ElementsAre(_));
   EXPECT_THAT(Symbols.begin()->IncludeHeaders.front().IncludeHeader,
               "<the/good/header.h>");
 }
 
+TEST(FileIndexTest, IWYUPragmaExport) {
+  FileIndex M;
+
+  TestTU File;
+  File.Code = R"cpp(#pragma once
+    #include "exporter.h"
+  )cpp";
+  File.HeaderFilename = "exporter.h";
+  File.HeaderCode = R"cpp(#pragma once
+    #include "private.h" // IWYU pragma: export
+  )cpp";
+  File.AdditionalFiles["private.h"] = "class Foo{};";
+  auto AST = File.build();
+  M.updatePreamble(testPath(File.Filename), /*Version=*/"null",
+                   AST.getASTContext(), AST.getPreprocessor(),
+                   AST.getPragmaIncludes());
+
+  auto Symbols = runFuzzyFind(M, "");
+  EXPECT_THAT(
+      Symbols,
+      UnorderedElementsAre(AllOf(
+          qName("Foo"),
+          includeHeader(URI::create(testPath(File.HeaderFilename)).toString()),
+          declURI(URI::create(testPath("private.h")).toString()))));
+}
+
 TEST(FileIndexTest, HasSystemHeaderMappingsInPreamble) {
   TestTU TU;
   TU.HeaderCode = "class Foo{};";
   TU.HeaderFilename = "algorithm";
 
   auto Symbols = runFuzzyFind(*TU.index(), "");
   EXPECT_THAT(Symbols, ElementsAre(_));
   EXPECT_THAT(Symbols.begin()->IncludeHeaders.front().IncludeHeader,
TEST(FileIndexTest, RebuildWithPreamble) {
   )cpp";
 
   // Rebuild the file.
   IgnoreDiagnostics IgnoreDiags;
   auto CI = buildCompilerInvocation(PI, IgnoreDiags);
 
   FileIndex Index;
   bool IndexUpdated = false;
-  buildPreamble(
-      FooCpp, *CI, PI,
+  buildPreamble(FooCpp, *CI, PI,
                 /*StoreInMemory=*/true,
                 [&](CapturedASTCtx ASTCtx,
-          const std::shared_ptr<const CanonicalIncludes> CanonIncludes) {
+                    std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
                   auto &Ctx = ASTCtx.getASTContext();
                   auto &PP = ASTCtx.getPreprocessor();
-        EXPECT_FALSE(IndexUpdated) << "Expected only a single index update";
+                  EXPECT_FALSE(IndexUpdated)
+                      << "Expected only a single index update";
                   IndexUpdated = true;
-        Index.updatePreamble(FooCpp, /*Version=*/"null", Ctx, PP,
-                             *CanonIncludes);
+                  Index.updatePreamble(FooCpp, /*Version=*/"null", Ctx, PP, PI);
                 });
   ASSERT_TRUE(IndexUpdated);
 
   // Check the index contains symbols from the preamble, but not from the main
   // file.
   FuzzyFindRequest Req;
   Req.Query = "";
   Req.Scopes = {"", "ns_in_header::"};
 
TEST(FileIndexTest, Relations) {
   TestTU TU;
   TU.Filename = "f.cpp";
   TU.HeaderFilename = "f.h";
   TU.HeaderCode = "class A {}; class B : public A {};";
   auto AST = TU.build();
   FileIndex Index;
   Index.updatePreamble(testPath(TU.Filename), /*Version=*/"null",
                        AST.getASTContext(), AST.getPreprocessor(),
-                       AST.getCanonicalIncludes());
+                       AST.getPragmaIncludes());
   SymbolID A = findSymbol(TU.headerSymbols(), "A").ID;
   uint32_t Results = 0;
   RelationsRequest Req;
   Req.Subjects.insert(A);
   Req.Predicate = RelationKind::BaseOf;
   Index.relations(Req, [&](const SymbolID &, const Symbol &) { ++Results; });
   EXPECT_EQ(Results, 1u);
 }
TEST(FileIndexTest, StalePreambleSymbolsDeleted) {
   TestTU File;
   File.HeaderFilename = "a.h";
 
   File.Filename = "f1.cpp";
   File.HeaderCode = "int a;";
   auto AST = File.build();
   M.updatePreamble(testPath(File.Filename), /*Version=*/"null",
                    AST.getASTContext(), AST.getPreprocessor(),
-                   AST.getCanonicalIncludes());
+                   AST.getPragmaIncludes());
   EXPECT_THAT(runFuzzyFind(M, ""), UnorderedElementsAre(qName("a")));
 
   File.Filename = "f2.cpp";
   File.HeaderCode = "int b;";
   AST = File.build();
   M.updatePreamble(testPath(File.Filename), /*Version=*/"null",
                    AST.getASTContext(), AST.getPreprocessor(),
-                   AST.getCanonicalIncludes());
+                   AST.getPragmaIncludes());
   EXPECT_THAT(runFuzzyFind(M, ""), UnorderedElementsAre(qName("b")));
 }
 
 // Verifies that concurrent calls to updateMain don't "lose" any updates.
 TEST(FileIndexTest, Threadsafety) {
   FileIndex M;
   Notification Go;
 

 
 TEST(FileIndexTest, Profile) {
   FileIndex FI;
 
   auto FileName = testPath("foo.cpp");
   auto AST = TestTU::withHeaderCode("int a;").build();
   FI.updateMain(FileName, AST);
   FI.updatePreamble(FileName, "v1", AST.getASTContext(), AST.getPreprocessor(),
-                    AST.getCanonicalIncludes());
+                    AST.getPragmaIncludes());
 
   llvm::BumpPtrAllocator Alloc;
   MemoryTree MT(&Alloc);
   FI.profile(MT);
   ASSERT_THAT(MT.children(),
               UnorderedElementsAre(Pair("preamble", _), Pair("main_file", _)));
 
   ASSERT_THAT(MT.child("preamble").children(),

clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp

 //===-- SymbolCollectorTests.cpp  -------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Annotations.h"
 #include "TestFS.h"
 #include "TestTU.h"
+#include "URI.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/SymbolCollector.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/FileSystemOptions.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Index/IndexingAction.h"
 #include "clang/Index/IndexingOptions.h"
 #include "clang/Tooling/Tooling.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/Error.h"
 #include "gmock/gmock-matchers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 #include <memory>
 #include <optional>
 #include <string>
+#include <utility>
 
 namespace clang {
 namespace clangd {
 namespace {
 
 using ::testing::_;
 using ::testing::AllOf;
 using ::testing::Contains;
build(R"(
     }
   )");
   EXPECT_TRUE(shouldCollect("nx::Top_Level"));
   EXPECT_TRUE(shouldCollect("nx::Kind_Fine"));
 }
 
 class SymbolIndexActionFactory : public tooling::FrontendActionFactory {
 public:
-  SymbolIndexActionFactory(SymbolCollector::Options COpts,
-                           CommentHandler *PragmaHandler)
-      : COpts(std::move(COpts)), PragmaHandler(PragmaHandler) {}
+  SymbolIndexActionFactory(SymbolCollector::Options COpts)
+      : COpts(std::move(COpts)) {}
 
   std::unique_ptr<FrontendAction> create() override {
     class IndexAction : public ASTFrontendAction {
     public:
       IndexAction(std::shared_ptr<index::IndexDataConsumer> DataConsumer,
                   const index::IndexingOptions &Opts,
-                  CommentHandler *PragmaHandler)
+                  std::shared_ptr<include_cleaner::PragmaIncludes> PI)
           : DataConsumer(std::move(DataConsumer)), Opts(Opts),
-            PragmaHandler(PragmaHandler) {}
+            PI(std::move(PI)) {}
 
       std::unique_ptr<ASTConsumer>
       CreateASTConsumer(CompilerInstance &CI, llvm::StringRef InFile) override {
-        if (PragmaHandler)
-          CI.getPreprocessor().addCommentHandler(PragmaHandler);
+        PI->record(CI);
         return createIndexingASTConsumer(DataConsumer, Opts,
                                          CI.getPreprocessorPtr());
       }
 
       bool BeginInvocation(CompilerInstance &CI) override {
         // Make the compiler parse all comments.
         CI.getLangOpts().CommentOpts.ParseAllComments = true;
         return true;
       }
 
     private:
       std::shared_ptr<index::IndexDataConsumer> DataConsumer;
       index::IndexingOptions Opts;
-      CommentHandler *PragmaHandler;
+      std::shared_ptr<include_cleaner::PragmaIncludes> PI;
     };
     index::IndexingOptions IndexOpts;
     IndexOpts.SystemSymbolFilter =
         index::IndexingOptions::SystemSymbolFilterKind::All;
     IndexOpts.IndexFunctionLocals = true;
+    std::shared_ptr<include_cleaner::PragmaIncludes> PI =
+        std::make_shared<include_cleaner::PragmaIncludes>();
+    COpts.PragmaIncludes = PI;
     Collector = std::make_shared<SymbolCollector>(COpts);
     return std::make_unique<IndexAction>(Collector, std::move(IndexOpts),
-                                         PragmaHandler);
+                                         std::move(PI));
   }
 
   std::shared_ptr<SymbolCollector> Collector;
   SymbolCollector::Options COpts;
-  CommentHandler *PragmaHandler;
 };
 
 class SymbolCollectorTest : public ::testing::Test {
 public:
   SymbolCollectorTest()
       : InMemoryFileSystem(new llvm::vfs::InMemoryFileSystem),
         TestHeaderName(testPath("symbol.h")),
         TestFileName(testPath("symbol.cc")) {
     TestHeaderURI = URI::create(TestHeaderName).toString();
     TestFileURI = URI::create(TestFileName).toString();
   }
 
   // Note that unlike TestTU, no automatic header guard is added.
   // HeaderCode should start with #pragma once to be treated as modular.
   bool runSymbolCollector(llvm::StringRef HeaderCode, llvm::StringRef MainCode,
                           const std::vector<std::string> &ExtraArgs = {}) {
     llvm::IntrusiveRefCntPtr<FileManager> Files(
         new FileManager(FileSystemOptions(), InMemoryFileSystem));
 
-    auto Factory = std::make_unique<SymbolIndexActionFactory>(
-        CollectorOpts, PragmaHandler.get());
+    auto Factory = std::make_unique<SymbolIndexActionFactory>(CollectorOpts);
 
     std::vector<std::string> Args = {"symbol_collector", "-fsyntax-only",
                                      "-xc++", "-include", TestHeaderName};
     Args.insert(Args.end(), ExtraArgs.begin(), ExtraArgs.end());
     // This allows to override the "-xc++" with something else, i.e.
     // -xobjective-c++.
     Args.push_back(TestFileName);
 
protected:
   std::string TestHeaderName;
   std::string TestHeaderURI;
   std::string TestFileName;
   std::string TestFileURI;
   SymbolSlab Symbols;
   RefSlab Refs;
   RelationSlab Relations;
   SymbolCollector::Options CollectorOpts;
-  std::unique_ptr<CommentHandler> PragmaHandler;
 };
 
 TEST_F(SymbolCollectorTest, CollectSymbols) {
   const std::string Header = R"(
     class Foo {
       Foo() {}
       Foo(int a) {}
       void f();
TEST_F(SymbolCollectorTest, IncludeHeaderSameAsFileURI) {
   EXPECT_THAT(Symbols, UnorderedElementsAre(
                            AllOf(qName("Foo"), declURI(TestHeaderURI))));
   EXPECT_THAT(Symbols.begin()->IncludeHeaders,
               UnorderedElementsAre(IncludeHeaderWithRef(TestHeaderURI, 1u)));
 }
 
 TEST_F(SymbolCollectorTest, CanonicalSTLHeader) {
   CollectorOpts.CollectIncludePath = true;
-  CanonicalIncludes Includes;
-  auto Language = LangOptions();
-  Language.CPlusPlus = true;
-  Includes.addSystemHeadersMapping(Language);
-  CollectorOpts.Includes = &Includes;
   runSymbolCollector(
       R"cpp(
       namespace std {
         class string {};
         // Move overloads have special handling.
         template <typename _T> T&& move(_T&& __value);
         template <typename _I, typename _O> _O move(_I, _I, _O);
       }
       )cpp",
       /*Main=*/"");
   EXPECT_THAT(
       Symbols,
       UnorderedElementsAre(
           qName("std"),
           AllOf(qName("std::string"), declURI(TestHeaderURI),
                 includeHeader("<string>")),
           // Parameter names are demangled.
           AllOf(labeled("move(T &&value)"), includeHeader("<utility>")),
           AllOf(labeled("move(I, I, O)"), includeHeader("<algorithm>"))));
 }
 
 TEST_F(SymbolCollectorTest, IWYUPragma) {
   CollectorOpts.CollectIncludePath = true;
-  CanonicalIncludes Includes;
-  PragmaHandler = collectIWYUHeaderMaps(&Includes);
-  CollectorOpts.Includes = &Includes;
   const std::string Header = R"(
     // IWYU pragma: private, include the/good/header.h
     class Foo {};
   )";
   runSymbolCollector(Header, /*Main=*/"");
   EXPECT_THAT(Symbols, UnorderedElementsAre(
                            AllOf(qName("Foo"), declURI(TestHeaderURI),
                                  includeHeader("\"the/good/header.h\""))));
 }
 
 TEST_F(SymbolCollectorTest, IWYUPragmaWithDoubleQuotes) {
   CollectorOpts.CollectIncludePath = true;
-  CanonicalIncludes Includes;
-  PragmaHandler = collectIWYUHeaderMaps(&Includes);
-  CollectorOpts.Includes = &Includes;
   const std::string Header = R"(
     // IWYU pragma: private, include "the/good/header.h"
     class Foo {};
   )";
   runSymbolCollector(Header, /*Main=*/"");
   EXPECT_THAT(Symbols, UnorderedElementsAre(
                            AllOf(qName("Foo"), declURI(TestHeaderURI),
                                  includeHeader("\"the/good/header.h\""))));
 }
 
-TEST_F(SymbolCollectorTest, SkipIncFileWhenCanonicalizeHeaders) {
-  auto IncFile = testPath("test.inc");
-  auto IncURI = URI::create(IncFile).toString();
-  InMemoryFileSystem->addFile(IncFile, 0,
-                              llvm::MemoryBuffer::getMemBuffer("class X {};"));
-  llvm::IntrusiveRefCntPtr<FileManager> Files(
-      new FileManager(FileSystemOptions(), InMemoryFileSystem));
-  std::string HeaderCode = "#include \"test.inc\"\nclass Y {};";
-  InMemoryFileSystem->addFile(TestHeaderName, 0,
-                              llvm::MemoryBuffer::getMemBuffer(HeaderCode));
-  auto File = Files->getFileRef(TestHeaderName);
-  ASSERT_THAT_EXPECTED(File, llvm::Succeeded());
-  CanonicalIncludes Includes;
-  Includes.addMapping(*File, "<canonical>");
+TEST_F(SymbolCollectorTest, IWYUPragmaExport) {
   CollectorOpts.CollectIncludePath = true;
-  CollectorOpts.Includes = &Includes;
-  runSymbolCollector(HeaderCode, /*Main=*/"",
+  const std::string Header = R"cpp(#pragma once
+    #include "exporter.h"
+  )cpp";
+  auto ExporterFile = testPath("exporter.h");
+  InMemoryFileSystem->addFile(
+      ExporterFile, 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(#pragma once
+    #include "private.h" // IWYU pragma: export
+  )cpp"));
+  auto PrivateFile = testPath("private.h");
+  InMemoryFileSystem->addFile(
+      PrivateFile, 0, llvm::MemoryBuffer::getMemBuffer("class Foo {};"));
+  runSymbolCollector(Header, /*Main=*/"",
                      /*ExtraArgs=*/{"-I", testRoot()});
-  EXPECT_THAT(Symbols,
-              UnorderedElementsAre(AllOf(qName("X"), declURI(IncURI),
-                                         includeHeader("<canonical>")),
-                                   AllOf(qName("Y"), declURI(TestHeaderURI),
-                                         includeHeader("<canonical>"))));
+  EXPECT_THAT(Symbols, UnorderedElementsAre(AllOf(
+                           qName("Foo"),
+                           includeHeader(URI::create(ExporterFile).toString()),
+                           declURI(URI::create(PrivateFile).toString()))));
 }
 
 TEST_F(SymbolCollectorTest, MainFileIsHeaderWhenSkipIncFile) {
   CollectorOpts.CollectIncludePath = true;
   // To make this case as hard as possible, we won't tell clang main is a
   // header. No extension, no -x c++-header.
   TestFileName = testPath("no_ext_main");
   TestFileURI = URI::create(TestFileName).toString();

clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp

 #include "Diagnostics.h"
 #include "GlobalCompilationDatabase.h"
 #include "Matchers.h"
 #include "ParsedAST.h"
 #include "Preamble.h"
 #include "TUScheduler.h"
 #include "TestFS.h"
 #include "TestIndex.h"
+#include "clang-include-cleaner/Record.h"
 #include "support/Cancellation.h"
 #include "support/Context.h"
 #include "support/Path.h"
 #include "support/TestTracer.h"
 #include "support/Threading.h"
 #include "clang/Basic/DiagnosticDriver.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/FunctionExtras.h"

 
 TEST_F(TUSchedulerTests, AsyncPreambleThread) {
   // Blocks preamble thread while building preamble with \p BlockVersion until
   // \p N is notified.
   class BlockPreambleThread : public ParsingCallbacks {
   public:
     BlockPreambleThread(llvm::StringRef BlockVersion, Notification &N)
         : BlockVersion(BlockVersion), N(N) {}
-    void
-    onPreambleAST(PathRef Path, llvm::StringRef Version, CapturedASTCtx,
-                  const std::shared_ptr<const CanonicalIncludes>) override {
+    void onPreambleAST(
+        PathRef Path, llvm::StringRef Version, CapturedASTCtx,
+        std::shared_ptr<const include_cleaner::PragmaIncludes>) override {
       if (Version == BlockVersion)
         N.wait();
     }
 
   private:
     llvm::StringRef BlockVersion;
     Notification &N;
   };
TEST_F(TUSchedulerTests, PublishWithStalePreamble) {
   // Callbacks that blocks the preamble thread after the first preamble is
   // built and stores preamble/main-file versions for diagnostics released.
   class BlockPreambleThread : public ParsingCallbacks {
   public:
     using DiagsCB = std::function<void(ParsedAST &)>;
     BlockPreambleThread(Notification &UnblockPreamble, DiagsCB CB)
         : UnblockPreamble(UnblockPreamble), CB(std::move(CB)) {}
 
-    void
-    onPreambleAST(PathRef Path, llvm::StringRef Version, CapturedASTCtx,
-                  const std::shared_ptr<const CanonicalIncludes>) override {
+    void onPreambleAST(
+        PathRef Path, llvm::StringRef Version, CapturedASTCtx,
+        std::shared_ptr<const include_cleaner::PragmaIncludes>) override {
       if (BuildBefore)
         ASSERT_TRUE(UnblockPreamble.wait(timeoutSeconds(5)))
             << "Expected notification";
       BuildBefore = true;
     }
 
     void onMainAST(PathRef File, ParsedAST &AST, PublishFn Publish) override {
       CB(AST);
void reset() {
       Callbacks.clear();
     }
   } Throttler;
 
   struct CaptureBuiltFilenames : public ParsingCallbacks {
     std::vector<std::string> &Filenames;
     CaptureBuiltFilenames(std::vector<std::string> &Filenames)
         : Filenames(Filenames) {}
-    void
-    onPreambleAST(PathRef Path, llvm::StringRef Version, CapturedASTCtx,
-                  const std::shared_ptr<const CanonicalIncludes>) override {
+    void onPreambleAST(
+        PathRef Path, llvm::StringRef Version, CapturedASTCtx,
+        std::shared_ptr<const include_cleaner::PragmaIncludes> PI) override {
       // Deliberately no synchronization.
       // The PreambleThrottler should serialize these calls, if not then tsan
       // will find a bug here.
       Filenames.emplace_back(Path);
     }
   };
 
   auto Opts = optsForTest();

clang-tools-extra/clangd/unittests/TestTU.cpp

for (const auto &D : *AST->getDiagnostics())
         std::abort(); // Stop after first error for simplicity.
       }
   }
   return std::move(*AST);
 }
 
 SymbolSlab TestTU::headerSymbols() const {
   auto AST = build();
-  return std::get<0>(indexHeaderSymbols(/*Version=*/"null", AST.getASTContext(),
-                                        AST.getPreprocessor(),
-                                        AST.getCanonicalIncludes()));
+  return std::get<0>(indexHeaderSymbols(
+      /*Version=*/"null", AST.getASTContext(), AST.getPreprocessor(),
+      AST.getPragmaIncludes()));
 }
 
 RefSlab TestTU::headerRefs() const {
   auto AST = build();
   return std::get<1>(indexMainDecls(AST));
 }
 
 std::unique_ptr<SymbolIndex> TestTU::index() const {
   auto AST = build();
   auto Idx = std::make_unique<FileIndex>();
   Idx->updatePreamble(testPath(Filename), /*Version=*/"null",
                       AST.getASTContext(), AST.getPreprocessor(),
-                      AST.getCanonicalIncludes());
+                      AST.getPragmaIncludes());
   Idx->updateMain(testPath(Filename), AST);
   return std::move(Idx);
 }
 
 const Symbol &findSymbol(const SymbolSlab &Slab, llvm::StringRef QName) {
   const Symbol *Result = nullptr;
   for (const Symbol &S : Slab) {
     if (QName != (S.Scope + S.Name).str())

clang-tools-extra/clangd/unittests/TestWorkspace.cpp

 //===--- TestWorkspace.cpp - Utility for writing multi-file tests -*- C++-*===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "TestWorkspace.h"
+#include "clang-include-cleaner/Record.h"
 #include "index/FileIndex.h"
 #include "gtest/gtest.h"
+#include <memory>
 #include <optional>
 
 namespace clang {
 namespace clangd {
 
 std::unique_ptr<SymbolIndex> TestWorkspace::index() {
   auto Index = std::make_unique<FileIndex>();
   for (const auto &Input : Inputs) {
     if (!Input.second.IsMainFile)
       continue;
     TU.Code = Input.second.Code;
     TU.Filename = Input.first().str();
-    TU.preamble(
-        [&](CapturedASTCtx ASTCtx,
-            const std::shared_ptr<const CanonicalIncludes> CanonIncludes) {
-          auto &Ctx = ASTCtx.getASTContext();
-          auto &PP = ASTCtx.getPreprocessor();
-          Index->updatePreamble(testPath(Input.first()), "null", Ctx, PP,
-                                *CanonIncludes);
-        });
+    TU.preamble([&](CapturedASTCtx ASTCtx,
+                    std::shared_ptr<const include_cleaner::PragmaIncludes> PI) {
+      Index->updatePreamble(testPath(Input.first()), "null", Ctx, PP, PI);
     ParsedAST MainAST = TU.build();
     Index->updateMain(testPath(Input.first()), MainAST);
   }
   return Index;
 }
 
 std::optional<ParsedAST> TestWorkspace::openFile(llvm::StringRef Filename) {
   auto It = Inputs.find(Filename);