[clang][ExtractAPI] Refactor ExtractAPIVisitor to make it more extensible

[dang]

Summary

Use CRTP to enable creating statically dispatched subclasses of
ExtractAPIVisitor.
This enables adding extension points and customising the behavior more
easily.

This is used in CXExtractAPI.cpp to create a specialized visitor for
Libclang as well as streamlining the batch implementation in ExtractAPIConsumer.cpp

[clang][ExtractAPI] Improve tests for clang_getSymbolGraphForCursor

Adds a new mode to c-index-test that can fetch a single symbol symbol
graph for a given source location. This way we can be more precise when
writing tests for clang_getSymbolGraphForCursor.
Additionaly this makes it easier to debug the function.

[bnbarham]

clang/lib/ExtractAPI/ExtractAPIConsumer.cpp

170

Accidental?

clang/test/Index/extract-api-cursor.m

36

I personally like to put the run lines next to what's being checked and use [[@LINE+1]]. Up to you though

[zixuw]

Good to see refactoring to make ExtractAPI easier to extend and use. Thanks Daniel!
Some comments on the ExtractAPIVisitor part. I'll leave the libclang part to the experts.

clang/include/clang/ExtractAPI/ExtractAPIVisitor.h

32

Would it be better to call this ExtractAPIVisitorImpl because it provides the visitor implementation, and the ExtractAPIVisitor is actually the base for the second level CRTP for actual visitors?

32

Should this be : public RecursiveASTVisitor<ExtractAPIVisitorBase<Derived>> instead?

34

Should the constructor be made protected for a CRTP base?

57

Why does comment-fetching need to be dispatched?

102

I see that the RecursiveASTVisitor already provides a getDerived() for the top-level CRTP.
My head is still bending trying to get a clear view of the multi-level CRTP here 🙃 , but I'm guessing that this is to avoid the conflict with that method.
In that case could we be more verbose to make clear the purpose of this one? I'm thinking something like getDerivedExtractAPIVisitor(), to communicate that this is for getting the derived/concrete ExtractAPIVisitor subclass.

clang/lib/ExtractAPI/ExtractAPIConsumer.cpp

170

Missed change?

228–234

Couldn't find the original logic in this patch. Could you remind me what are these for? Also good to have more comments here to explain things.

[zixuw]

clang/include/clang/ExtractAPI/ExtractAPIVisitor.h

628

Same as above, should the constructor be protected? I guess it depends if we want people to just instantiate and use a ExtractAPIVisitor.

[dang]

clang/include/clang/ExtractAPI/ExtractAPIVisitor.h

32

I chose to name it "Base" as this seems to be the convention used for ADT, where they use the "Base" suffix for types clients shouldn't be using.

As with the base class, we need to provide RecursiveASTVisitor the most derived class so that when it does the getDerived() dance it can use the most derived methods.

34

agreed

57

This is for the libclang use case. If you request symbol graph data for a symbol in an implementation block for example you should get back the doc comment from the header. The main reason for doing this refactoring is so that we can avoid dynamic dispatch when doing source location lookups and now fetching documentation comments.

102

Not fully sure myself, but I think that using the one from RecursiveASTVisitor would work fine here. I will give it a go and change it if it works.

628

The aim was for clients to be able to instantiate this one, hence why we go through a fair amount of effort to provide base class the most derived type with the std::conditional_t usage above.

clang/lib/ExtractAPI/ExtractAPIConsumer.cpp

170

yup definitely accidental will fix.

228–234

I moved this behavior out of the base type into here, we used to do these checks in the individual VisitXXX methods and bail early, but I needed this behavior to be configurable for the libclang work. I figured it semantically belonged here.

clang/test/Index/extract-api-cursor.m

36

Oh I didn't know about this that sounds like it's definitely more readable, will make the swap.

[dang]

Addressing code review feedback

[dang]

Adding back missing diffs.

[zixuw]

LGTM for the ExtractAPIVisitor part.
Remaining:

• update test with @LINE
• the libclang side

[dang]

In D146656#4220022, @zixuw wrote:

LGTM for the ExtractAPIVisitor part.
Remaining:

• update test with @LINE
• the libclang side

I have decided against doing that, because we can't specify @LINE in the c-index-test command line.
@bnbarham are you happy with the libclang bits?

clang/include/clang/ExtractAPI/ExtractAPIVisitor.h

102

turns out it is needed but not sure why, I will rename to make things clearer.

clang/test/Index/extract-api-cursor.m

36

I tried it out and I actually struggled to follow along so I am leaving it as is.

[bnbarham]

I have decided against doing that, because we can't specify @LINE in the c-index-test command line.

FWIW you can do:

// RUN: .... -pos=%(line+1):7
let bar = Bar()
// CHECK ... [[@LINE-1]]:7

But I don't think this is particularly common, it's just how I write tests. What you have is perfectly reasonable 👍

clang/include/clang/ExtractAPI/ExtractAPIVisitor.h

 //===- ExtractAPI/ExtractAPIVisitor.h ---------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file defines the ExtractAPVisitor AST visitation interface.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_EXTRACTAPI_EXTRACT_API_VISITOR_H
 #define LLVM_CLANG_EXTRACTAPI_EXTRACT_API_VISITOR_H
 
+#include "llvm/ADT/FunctionExtras.h"
+
+#include "clang/AST/ASTContext.h"
+#include "clang/AST/ParentMapContext.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/ExtractAPI/API.h"
-#include "llvm/ADT/FunctionExtras.h"
+#include "clang/ExtractAPI/TypedefUnderlyingTypeResolver.h"
+#include <type_traits>
 
 namespace clang {
 namespace extractapi {
+namespace impl {
 
-/// The RecursiveASTVisitor to traverse symbol declarations and collect API
-/// information.
-class ExtractAPIVisitor : public RecursiveASTVisitor<ExtractAPIVisitor> {
+template <typename Derived>
+class ExtractAPIVisitorBase : public RecursiveASTVisitor<Derived> {
 public:
-  ExtractAPIVisitor(ASTContext &Context,
-                    llvm::unique_function<bool(SourceLocation)> LocationChecker,
-                    APISet &API)
-      : Context(Context), API(API),
-        LocationChecker(std::move(LocationChecker)) {}
+  ExtractAPIVisitorBase(ASTContext &Context, APISet &API)
+      : Context(Context), API(API) {}
 
   const APISet &getAPI() const { return API; }
 
   bool VisitVarDecl(const VarDecl *Decl);
 
   bool VisitFunctionDecl(const FunctionDecl *Decl);
 
   bool VisitEnumDecl(const EnumDecl *Decl);
 
   bool VisitRecordDecl(const RecordDecl *Decl);
 
   bool VisitObjCInterfaceDecl(const ObjCInterfaceDecl *Decl);
 
   bool VisitObjCProtocolDecl(const ObjCProtocolDecl *Decl);
 
   bool VisitTypedefNameDecl(const TypedefNameDecl *Decl);
 
   bool VisitObjCCategoryDecl(const ObjCCategoryDecl *Decl);
 
-private:
+  bool shouldDeclBeIncluded(const Decl *Decl) const;
+
+  const RawComment *fetchRawCommentForDecl(const Decl *Decl) const;
+
+protected:
   /// Collect API information for the enum constants and associate with the
   /// parent enum.
   void recordEnumConstants(EnumRecord *EnumRecord,
                            const EnumDecl::enumerator_range Constants);
 
   /// Collect API information for the struct fields and associate with the
   /// parent struct.
   void recordStructFields(StructRecord *StructRecord,
protected:
   void recordObjCInstanceVariables(
       ObjCContainerRecord *Container,
       const llvm::iterator_range<
           DeclContext::specific_decl_iterator<ObjCIvarDecl>>
           Ivars);
 
   void recordObjCProtocols(ObjCContainerRecord *Container,
                            ObjCInterfaceDecl::protocol_range Protocols);
+
   ASTContext &Context;
   APISet &API;
-  llvm::unique_function<bool(SourceLocation)> LocationChecker;
+
+  StringRef getTypedefName(const TagDecl *Decl) {
+    if (const auto *TypedefDecl = Decl->getTypedefNameForAnonDecl())
+      return TypedefDecl->getName();
+
+    return {};
+  }
+
+  bool isInSystemHeader(const Decl *D) {
+    return Context.getSourceManager().isInSystemHeader(D->getLocation());
+  }
+
+private:
+  Derived &getConcrete() { return *static_cast<Derived *>(this); }
+};
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitVarDecl(const VarDecl *Decl) {
+  // skip function parameters.
+  if (isa<ParmVarDecl>(Decl))
+    return true;
+
+  // Skip non-global variables in records (struct/union/class).
+  if (Decl->getDeclContext()->isRecord())
+    return true;
+
+  // Skip local variables inside function or method.
+  if (!Decl->isDefinedOutsideFunctionOrMethod())
+    return true;
+
+  // If this is a template but not specialization or instantiation, skip.
+  if (Decl->getASTContext().getTemplateOrSpecializationInfo(Decl) &&
+      Decl->getTemplateSpecializationKind() == TSK_Undeclared)
+    return true;
+
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  // Collect symbol information.
+  StringRef Name = Decl->getName();
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  LinkageInfo Linkage = Decl->getLinkageAndVisibility();
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  // Build declaration fragments and sub-heading for the variable.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForVar(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+  // Add the global variable record to the API set.
+  API.addGlobalVar(Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment,
+                   Declaration, SubHeading, isInSystemHeader(Decl));
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitFunctionDecl(
+    const FunctionDecl *Decl) {
+  if (const auto *Method = dyn_cast<CXXMethodDecl>(Decl)) {
+    // Skip member function in class templates.
+    if (Method->getParent()->getDescribedClassTemplate() != nullptr)
+      return true;
+
+    // Skip methods in records.
+    for (auto P : Context.getParents(*Method)) {
+      if (P.template get<CXXRecordDecl>())
+        return true;
+    }
+
+    // Skip ConstructorDecl and DestructorDecl.
+    if (isa<CXXConstructorDecl>(Method) || isa<CXXDestructorDecl>(Method))
+      return true;
+  }
+
+  // Skip templated functions.
+  switch (Decl->getTemplatedKind()) {
+  case FunctionDecl::TK_NonTemplate:
+  case FunctionDecl::TK_DependentNonTemplate:
+    break;
+  case FunctionDecl::TK_MemberSpecialization:
+  case FunctionDecl::TK_FunctionTemplateSpecialization:
+    if (auto *TemplateInfo = Decl->getTemplateSpecializationInfo()) {
+      if (!TemplateInfo->isExplicitInstantiationOrSpecialization())
+        return true;
+    }
+    break;
+  case FunctionDecl::TK_FunctionTemplate:
+  case FunctionDecl::TK_DependentFunctionTemplateSpecialization:
+    return true;
+  }
+
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  // Collect symbol information.
+  StringRef Name = Decl->getName();
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  LinkageInfo Linkage = Decl->getLinkageAndVisibility();
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  // Build declaration fragments, sub-heading, and signature of the function.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForFunction(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+  FunctionSignature Signature =
+      DeclarationFragmentsBuilder::getFunctionSignature(Decl);
+
+  // Add the function record to the API set.
+  API.addGlobalFunction(Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment,
+                        Declaration, SubHeading, Signature,
+                        isInSystemHeader(Decl));
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitEnumDecl(const EnumDecl *Decl) {
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  SmallString<128> QualifiedNameBuffer;
+  // Collect symbol information.
+  StringRef Name = Decl->getName();
+  if (Name.empty())
+    Name = getTypedefName(Decl);
+  if (Name.empty()) {
+    llvm::raw_svector_ostream OS(QualifiedNameBuffer);
+    Decl->printQualifiedName(OS);
+    Name = QualifiedNameBuffer.str();
+  }
+
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  // Build declaration fragments and sub-heading for the enum.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForEnum(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+  EnumRecord *EnumRecord =
+      API.addEnum(API.copyString(Name), USR, Loc, AvailabilitySet(Decl),
+                  Comment, Declaration, SubHeading, isInSystemHeader(Decl));
+
+  // Now collect information about the enumerators in this enum.
+  getConcrete().recordEnumConstants(EnumRecord, Decl->enumerators());
+
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitRecordDecl(const RecordDecl *Decl) {
+  // Skip C++ structs/classes/unions
+  // TODO: support C++ records
+  if (isa<CXXRecordDecl>(Decl))
+    return true;
+
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  // Collect symbol information.
+  StringRef Name = Decl->getName();
+  if (Name.empty())
+    Name = getTypedefName(Decl);
+  if (Name.empty())
+    return true;
+
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  // Build declaration fragments and sub-heading for the struct.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForStruct(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+  StructRecord *StructRecord =
+      API.addStruct(Name, USR, Loc, AvailabilitySet(Decl), Comment, Declaration,
+                    SubHeading, isInSystemHeader(Decl));
+
+  // Now collect information about the fields in this struct.
+  getConcrete().recordStructFields(StructRecord, Decl->fields());
+
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitObjCInterfaceDecl(
+    const ObjCInterfaceDecl *Decl) {
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  // Collect symbol information.
+  StringRef Name = Decl->getName();
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  LinkageInfo Linkage = Decl->getLinkageAndVisibility();
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  // Build declaration fragments and sub-heading for the interface.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForObjCInterface(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+  // Collect super class information.
+  SymbolReference SuperClass;
+  if (const auto *SuperClassDecl = Decl->getSuperClass()) {
+    SuperClass.Name = SuperClassDecl->getObjCRuntimeNameAsString();
+    SuperClass.USR = API.recordUSR(SuperClassDecl);
+  }
+
+  ObjCInterfaceRecord *ObjCInterfaceRecord = API.addObjCInterface(
+      Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment, Declaration,
+      SubHeading, SuperClass, isInSystemHeader(Decl));
+
+  // Record all methods (selectors). This doesn't include automatically
+  // synthesized property methods.
+  getConcrete().recordObjCMethods(ObjCInterfaceRecord, Decl->methods());
+  getConcrete().recordObjCProperties(ObjCInterfaceRecord, Decl->properties());
+  getConcrete().recordObjCInstanceVariables(ObjCInterfaceRecord, Decl->ivars());
+  getConcrete().recordObjCProtocols(ObjCInterfaceRecord, Decl->protocols());
+
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitObjCProtocolDecl(
+    const ObjCProtocolDecl *Decl) {
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  // Collect symbol information.
+  StringRef Name = Decl->getName();
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  // Build declaration fragments and sub-heading for the protocol.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForObjCProtocol(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+  ObjCProtocolRecord *ObjCProtocolRecord =
+      API.addObjCProtocol(Name, USR, Loc, AvailabilitySet(Decl), Comment,
+                          Declaration, SubHeading, isInSystemHeader(Decl));
+
+  getConcrete().recordObjCMethods(ObjCProtocolRecord, Decl->methods());
+  getConcrete().recordObjCProperties(ObjCProtocolRecord, Decl->properties());
+  getConcrete().recordObjCProtocols(ObjCProtocolRecord, Decl->protocols());
+
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitTypedefNameDecl(
+    const TypedefNameDecl *Decl) {
+  // Skip ObjC Type Parameter for now.
+  if (isa<ObjCTypeParamDecl>(Decl))
+    return true;
+
+  if (!Decl->isDefinedOutsideFunctionOrMethod())
+    return true;
+
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  StringRef Name = Decl->getName();
+  StringRef USR = API.recordUSR(Decl);
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+
+  QualType Type = Decl->getUnderlyingType();
+  SymbolReference SymRef =
+      TypedefUnderlyingTypeResolver(Context).getSymbolReferenceForType(Type,
+                                                                       API);
+
+  API.addTypedef(Name, USR, Loc, AvailabilitySet(Decl), Comment,
+                 DeclarationFragmentsBuilder::getFragmentsForTypedef(Decl),
+                 DeclarationFragmentsBuilder::getSubHeading(Decl), SymRef,
+                 isInSystemHeader(Decl));
+
+  return true;
+}
+
+template <typename Derived>
+bool ExtractAPIVisitorBase<Derived>::VisitObjCCategoryDecl(
+    const ObjCCategoryDecl *Decl) {
+  if (!getConcrete().shouldDeclBeIncluded(Decl))
+    return true;
+
+  StringRef Name = Decl->getName();
+  StringRef USR = API.recordUSR(Decl);
+  PresumedLoc Loc =
+      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+  DocComment Comment;
+  if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Decl))
+    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                            Context.getDiagnostics());
+  // Build declaration fragments and sub-heading for the category.
+  DeclarationFragments Declaration =
+      DeclarationFragmentsBuilder::getFragmentsForObjCCategory(Decl);
+  DeclarationFragments SubHeading =
+      DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+  const ObjCInterfaceDecl *InterfaceDecl = Decl->getClassInterface();
+  SymbolReference Interface(InterfaceDecl->getName(),
+                            API.recordUSR(InterfaceDecl));
+
+  ObjCCategoryRecord *ObjCCategoryRecord = API.addObjCCategory(
+      Name, USR, Loc, AvailabilitySet(Decl), Comment, Declaration, SubHeading,
+      Interface, isInSystemHeader(Decl));
+
+  getConcrete().recordObjCMethods(ObjCCategoryRecord, Decl->methods());
+  getConcrete().recordObjCProperties(ObjCCategoryRecord, Decl->properties());
+  getConcrete().recordObjCInstanceVariables(ObjCCategoryRecord, Decl->ivars());
+  getConcrete().recordObjCProtocols(ObjCCategoryRecord, Decl->protocols());
+
+  return true;
+}
+
+/// Collect API information for the enum constants and associate with the
+/// parent enum.
+template <typename Derived>
+void ExtractAPIVisitorBase<Derived>::recordEnumConstants(
+    EnumRecord *EnumRecord, const EnumDecl::enumerator_range Constants) {
+  for (const auto *Constant : Constants) {
+    // Collect symbol information.
+    StringRef Name = Constant->getName();
+    StringRef USR = API.recordUSR(Constant);
+    PresumedLoc Loc =
+        Context.getSourceManager().getPresumedLoc(Constant->getLocation());
+    DocComment Comment;
+    if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Constant))
+      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                              Context.getDiagnostics());
+
+    // Build declaration fragments and sub-heading for the enum constant.
+    DeclarationFragments Declaration =
+        DeclarationFragmentsBuilder::getFragmentsForEnumConstant(Constant);
+    DeclarationFragments SubHeading =
+        DeclarationFragmentsBuilder::getSubHeading(Constant);
+
+    API.addEnumConstant(EnumRecord, Name, USR, Loc, AvailabilitySet(Constant),
+                        Comment, Declaration, SubHeading,
+                        isInSystemHeader(Constant));
+  }
+}
+
+/// Collect API information for the struct fields and associate with the
+/// parent struct.
+template <typename Derived>
+void ExtractAPIVisitorBase<Derived>::recordStructFields(
+    StructRecord *StructRecord, const RecordDecl::field_range Fields) {
+  for (const auto *Field : Fields) {
+    // Collect symbol information.
+    StringRef Name = Field->getName();
+    StringRef USR = API.recordUSR(Field);
+    PresumedLoc Loc =
+        Context.getSourceManager().getPresumedLoc(Field->getLocation());
+    DocComment Comment;
+    if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Field))
+      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                              Context.getDiagnostics());
+
+    // Build declaration fragments and sub-heading for the struct field.
+    DeclarationFragments Declaration =
+        DeclarationFragmentsBuilder::getFragmentsForField(Field);
+    DeclarationFragments SubHeading =
+        DeclarationFragmentsBuilder::getSubHeading(Field);
+
+    API.addStructField(StructRecord, Name, USR, Loc, AvailabilitySet(Field),
+                       Comment, Declaration, SubHeading,
+                       isInSystemHeader(Field));
+  }
+}
+
+/// Collect API information for the Objective-C methods and associate with the
+/// parent container.
+template <typename Derived>
+void ExtractAPIVisitorBase<Derived>::recordObjCMethods(
+    ObjCContainerRecord *Container,
+    const ObjCContainerDecl::method_range Methods) {
+  for (const auto *Method : Methods) {
+    // Don't record selectors for properties.
+    if (Method->isPropertyAccessor())
+      continue;
+
+    StringRef Name = API.copyString(Method->getSelector().getAsString());
+    StringRef USR = API.recordUSR(Method);
+    PresumedLoc Loc =
+        Context.getSourceManager().getPresumedLoc(Method->getLocation());
+    DocComment Comment;
+    if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Method))
+      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                              Context.getDiagnostics());
+
+    // Build declaration fragments, sub-heading, and signature for the method.
+    DeclarationFragments Declaration =
+        DeclarationFragmentsBuilder::getFragmentsForObjCMethod(Method);
+    DeclarationFragments SubHeading =
+        DeclarationFragmentsBuilder::getSubHeading(Method);
+    FunctionSignature Signature =
+        DeclarationFragmentsBuilder::getFunctionSignature(Method);
+
+    API.addObjCMethod(Container, Name, USR, Loc, AvailabilitySet(Method),
+                      Comment, Declaration, SubHeading, Signature,
+                      Method->isInstanceMethod(), isInSystemHeader(Method));
+  }
+}
+
+template <typename Derived>
+void ExtractAPIVisitorBase<Derived>::recordObjCProperties(
+    ObjCContainerRecord *Container,
+    const ObjCContainerDecl::prop_range Properties) {
+  for (const auto *Property : Properties) {
+    StringRef Name = Property->getName();
+    StringRef USR = API.recordUSR(Property);
+    PresumedLoc Loc =
+        Context.getSourceManager().getPresumedLoc(Property->getLocation());
+    DocComment Comment;
+    if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Property))
+      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                              Context.getDiagnostics());
+
+    // Build declaration fragments and sub-heading for the property.
+    DeclarationFragments Declaration =
+        DeclarationFragmentsBuilder::getFragmentsForObjCProperty(Property);
+    DeclarationFragments SubHeading =
+        DeclarationFragmentsBuilder::getSubHeading(Property);
+
+    StringRef GetterName =
+        API.copyString(Property->getGetterName().getAsString());
+    StringRef SetterName =
+        API.copyString(Property->getSetterName().getAsString());
+
+    // Get the attributes for property.
+    unsigned Attributes = ObjCPropertyRecord::NoAttr;
+    if (Property->getPropertyAttributes() &
+        ObjCPropertyAttribute::kind_readonly)
+      Attributes |= ObjCPropertyRecord::ReadOnly;
+
+    API.addObjCProperty(
+        Container, Name, USR, Loc, AvailabilitySet(Property), Comment,
+        Declaration, SubHeading,
+        static_cast<ObjCPropertyRecord::AttributeKind>(Attributes), GetterName,
+        SetterName, Property->isOptional(),
+        !(Property->getPropertyAttributes() &
+          ObjCPropertyAttribute::kind_class),
+        isInSystemHeader(Property));
+  }
+}
+
+template <typename Derived>
+void ExtractAPIVisitorBase<Derived>::recordObjCInstanceVariables(
+    ObjCContainerRecord *Container,
+    const llvm::iterator_range<
+        DeclContext::specific_decl_iterator<ObjCIvarDecl>>
+        Ivars) {
+  for (const auto *Ivar : Ivars) {
+    StringRef Name = Ivar->getName();
+    StringRef USR = API.recordUSR(Ivar);
+    PresumedLoc Loc =
+        Context.getSourceManager().getPresumedLoc(Ivar->getLocation());
+    DocComment Comment;
+    if (auto *RawComment = getConcrete().fetchRawCommentForDecl(Ivar))
+      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                              Context.getDiagnostics());
+
+    // Build declaration fragments and sub-heading for the instance variable.
+    DeclarationFragments Declaration =
+        DeclarationFragmentsBuilder::getFragmentsForField(Ivar);
+    DeclarationFragments SubHeading =
+        DeclarationFragmentsBuilder::getSubHeading(Ivar);
+
+    ObjCInstanceVariableRecord::AccessControl Access =
+        Ivar->getCanonicalAccessControl();
+
+    API.addObjCInstanceVariable(Container, Name, USR, Loc,
+                                AvailabilitySet(Ivar), Comment, Declaration,
+                                SubHeading, Access, isInSystemHeader(Ivar));
+  }
+}
+
+template <typename Derived>
+void ExtractAPIVisitorBase<Derived>::recordObjCProtocols(
+    ObjCContainerRecord *Container,
+    ObjCInterfaceDecl::protocol_range Protocols) {
+  for (const auto *Protocol : Protocols)
+    Container->Protocols.emplace_back(Protocol->getName(),
+                                      API.recordUSR(Protocol));
+}
+
+} // namespace impl
+
+/// The RecursiveASTVisitor to traverse symbol declarations and collect API
+/// information.
+template <typename Derived = void>
+class ExtractAPIVisitor
+    : public impl::ExtractAPIVisitorBase<std::conditional_t<
+          std::is_same_v<Derived, void>, ExtractAPIVisitor<>, Derived>> {
+  using Base = impl::ExtractAPIVisitorBase<std::conditional_t<
+      std::is_same_v<Derived, void>, ExtractAPIVisitor<>, Derived>>;
+
+public:
+  ExtractAPIVisitor(ASTContext &Context, APISet &API) : Base(Context, API) {}
+
+  bool shouldDeclBeIncluded(const Decl *D) const { return true; }
+  const RawComment *fetchRawCommentForDecl(const Decl *D) const {
+    return this->Context.getRawCommentForDeclNoCache(D);
+  }
 };
 
 } // namespace extractapi
 } // namespace clang
 
 #endif // LLVM_CLANG_EXTRACTAPI_EXTRACT_API_VISITOR_H

clang/include/clang/ExtractAPI/TypedefUnderlyingTypeResolver.h

 //===- ExtractAPI/TypedefUnderlyingTypeResolver.h ---------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file defines the UnderlyingTypeResolver which is a helper type for
 /// resolving the undelrying type for a given QualType and exposing that
 /// information in various forms.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_UNDERLYING_TYPE_RESOLVER_H
 #define LLVM_CLANG_UNDERLYING_TYPE_RESOLVER_H
 
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
 #include "clang/ExtractAPI/API.h"
 
 #include <string>
 
 namespace clang {
 namespace extractapi {
 
 struct TypedefUnderlyingTypeResolver {
   /// Gets the underlying type declaration.
   const NamedDecl *getUnderlyingTypeDecl(QualType Type) const;
 
   /// Get a SymbolReference for the given type.
   SymbolReference getSymbolReferenceForType(QualType Type, APISet &API) const;
 
   /// Get a USR for the given type.
   std::string getUSRForType(QualType Type) const;
 
   explicit TypedefUnderlyingTypeResolver(ASTContext &Context)
       : Context(Context) {}
 
 private:
   ASTContext &Context;
 };
 
 } // namespace extractapi
 } // namespace clang
 
 #endif // LLVM_CLANG_UNDERLYING_TYPE_RESOLVER_H

clang/lib/ExtractAPI/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Support
   TargetParser
   )
 
 add_clang_library(clangExtractAPI
   API.cpp
   APIIgnoresList.cpp
   AvailabilityInfo.cpp
   ExtractAPIConsumer.cpp
-  ExtractAPIVisitor.cpp
   DeclarationFragments.cpp
   Serialization/SerializerBase.cpp
   Serialization/SymbolGraphSerializer.cpp
   TypedefUnderlyingTypeResolver.cpp
 
   LINK_LIBS
   clangAST
   clangBasic
   clangFrontend
   clangIndex
   clangLex
   )

clang/lib/ExtractAPI/DeclarationFragments.cpp

 //===- ExtractAPI/DeclarationFragments.cpp ----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file implements Declaration Fragments related classes.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "clang/ExtractAPI/DeclarationFragments.h"
-#include "TypedefUnderlyingTypeResolver.h"
+#include "clang/ExtractAPI/TypedefUnderlyingTypeResolver.h"
 #include "clang/Index/USRGeneration.h"
 #include "llvm/ADT/StringSwitch.h"
 
 using namespace clang::extractapi;
 using namespace llvm;
 
 DeclarationFragments &DeclarationFragments::appendSpace() {
   if (!Fragments.empty()) {

clang/lib/ExtractAPI/ExtractAPIConsumer.cpp

 //===- ExtractAPI/ExtractAPIConsumer.cpp ------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file implements the ExtractAPIAction, and ASTConsumer to collect API
 /// information.
 ///
 //===----------------------------------------------------------------------===//
 
+#include "clang/AST/ASTConcept.h"
 #include "clang/AST/ASTConsumer.h"
 #include "clang/AST/ASTContext.h"
+#include "clang/AST/DeclObjC.h"
 #include "clang/Basic/DiagnosticFrontend.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/ExtractAPI/API.h"
 #include "clang/ExtractAPI/APIIgnoresList.h"
 #include "clang/ExtractAPI/ExtractAPIVisitor.h"
 #include "clang/ExtractAPI/FrontendActions.h"
 #include "clang/ExtractAPI/Serialization/SymbolGraphSerializer.h"
 #include "clang/Frontend/ASTConsumers.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendOptions.h"
 #include "clang/Lex/MacroInfo.h"
 #include "clang/Lex/PPCallbacks.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Regex.h"
 #include "llvm/Support/raw_ostream.h"
 #include <memory>
 #include <optional>
std::optional<std::string> getRelativeIncludeName(const CompilerInstance &CI,
   }
 
   // Couldn't determine a include name, use full path instead.
   return std::nullopt;
 }
 
 struct LocationFileChecker {
   bool operator()(SourceLocation Loc) {
-    // If the loc refers to a macro expansion we need to first get the file
+    // If the loc refersSourceLocationxpansion we need to first get the file
     // location of the expansion.
     auto &SM = CI.getSourceManager();
     auto FileLoc = SM.getFileLoc(Loc);
     FileID FID = SM.getFileID(FileLoc);
     if (FID.isInvalid())
       return false;
 
     const auto *File = SM.getFileEntryForID(FID);

 
 private:
   const CompilerInstance &CI;
   SmallVector<std::pair<SmallString<32>, bool>> &KnownFiles;
   llvm::DenseSet<const FileEntry *> KnownFileEntries;
   llvm::DenseSet<const FileEntry *> ExternalFileEntries;
 };
 
+struct BatchExtractAPIVisitor : ExtractAPIVisitor<BatchExtractAPIVisitor> {
+  bool shouldDeclBeIncluded(const Decl *D) const {
+    bool ShouldBeIncluded = true;
+    // Check that we have the definition for redeclarable types.
+    if (auto *TD = llvm::dyn_cast<TagDecl>(D))
+      ShouldBeIncluded = TD->isThisDeclarationADefinition();
+    else if (auto *Interface = llvm::dyn_cast<ObjCInterfaceDecl>(D))
+      ShouldBeIncluded = Interface->isThisDeclarationADefinition();
+    else if (auto *Protocol = llvm::dyn_cast<ObjCProtocolDecl>(D))
+      ShouldBeIncluded = Protocol->isThisDeclarationADefinition();
+
+    ShouldBeIncluded = ShouldBeIncluded && LCF(D->getLocation());
+    return ShouldBeIncluded;
+  }
+
+  BatchExtractAPIVisitor(LocationFileChecker &LCF, ASTContext &Context,
+                         APISet &API)
+      : ExtractAPIVisitor<BatchExtractAPIVisitor>(Context, API), LCF(LCF) {}
+
+private:
+  LocationFileChecker &LCF;
+};
+
 class ExtractAPIConsumer : public ASTConsumer {
 public:
   ExtractAPIConsumer(ASTContext &Context,
                      std::unique_ptr<LocationFileChecker> LCF, APISet &API)
-      : Visitor(Context, *LCF, API), LCF(std::move(LCF)) {}
+      : Visitor(*LCF, Context, API), LCF(std::move(LCF)) {}
 
   void HandleTranslationUnit(ASTContext &Context) override {
     // Use ExtractAPIVisitor to traverse symbol declarations in the context.
     Visitor.TraverseDecl(Context.getTranslationUnitDecl());
   }
 
 private:
-  ExtractAPIVisitor Visitor;
+  BatchExtractAPIVisitor Visitor;
   std::unique_ptr<LocationFileChecker> LCF;
 };
 
 class MacroCallback : public PPCallbacks {
 public:
   MacroCallback(const SourceManager &SM, LocationFileChecker &LCF, APISet &API,
                 Preprocessor &PP)
       : SM(SM), LCF(LCF), API(API), PP(PP) {}

clang/lib/ExtractAPI/ExtractAPIVisitor.cpp

-//===- ExtractAPI/ExtractAPIVisitor.cpp -------------------------*- C++ -*-===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-///
-/// \file
-/// This file implements the ExtractAPIVisitor an ASTVisitor to collect API
-/// information.
-///
-//===----------------------------------------------------------------------===//
-
-#include "clang/ExtractAPI/ExtractAPIVisitor.h"
-
-#include "TypedefUnderlyingTypeResolver.h"
-#include "clang/AST/ASTConsumer.h"
-#include "clang/AST/ASTContext.h"
-#include "clang/AST/Decl.h"
-#include "clang/AST/DeclCXX.h"
-#include "clang/AST/ParentMapContext.h"
-#include "clang/AST/RawCommentList.h"
-#include "clang/Basic/SourceLocation.h"
-#include "clang/Basic/SourceManager.h"
-#include "clang/Basic/TargetInfo.h"
-#include "clang/ExtractAPI/API.h"
-#include "clang/ExtractAPI/AvailabilityInfo.h"
-#include "clang/ExtractAPI/DeclarationFragments.h"
-#include "clang/Frontend/ASTConsumers.h"
-#include "clang/Frontend/FrontendOptions.h"
-#include "llvm/Support/raw_ostream.h"
-
-using namespace clang;
-using namespace extractapi;
-
-namespace {
-
-StringRef getTypedefName(const TagDecl *Decl) {
-  if (const auto *TypedefDecl = Decl->getTypedefNameForAnonDecl())
-    return TypedefDecl->getName();
-
-  return {};
-}
-
-template <class DeclTy>
-bool isInSystemHeader(const ASTContext &Context, const DeclTy *D) {
-  return Context.getSourceManager().isInSystemHeader(D->getLocation());
-}
-
-} // namespace
-
-bool ExtractAPIVisitor::VisitVarDecl(const VarDecl *Decl) {
-  // skip function parameters.
-  if (isa<ParmVarDecl>(Decl))
-    return true;
-
-  // Skip non-global variables in records (struct/union/class).
-  if (Decl->getDeclContext()->isRecord())
-    return true;
-
-  // Skip local variables inside function or method.
-  if (!Decl->isDefinedOutsideFunctionOrMethod())
-    return true;
-
-  // If this is a template but not specialization or instantiation, skip.
-  if (Decl->getASTContext().getTemplateOrSpecializationInfo(Decl) &&
-      Decl->getTemplateSpecializationKind() == TSK_Undeclared)
-    return true;
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  LinkageInfo Linkage = Decl->getLinkageAndVisibility();
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  // Build declaration fragments and sub-heading for the variable.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForVar(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-
-  // Add the global variable record to the API set.
-  API.addGlobalVar(Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment,
-                   Declaration, SubHeading, isInSystemHeader(Context, Decl));
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitFunctionDecl(const FunctionDecl *Decl) {
-  if (const auto *Method = dyn_cast<CXXMethodDecl>(Decl)) {
-    // Skip member function in class templates.
-    if (Method->getParent()->getDescribedClassTemplate() != nullptr)
-      return true;
-
-    // Skip methods in records.
-    for (auto P : Context.getParents(*Method)) {
-      if (P.get<CXXRecordDecl>())
-        return true;
-    }
-
-    // Skip ConstructorDecl and DestructorDecl.
-    if (isa<CXXConstructorDecl>(Method) || isa<CXXDestructorDecl>(Method))
-      return true;
-  }
-
-  // Skip templated functions.
-  switch (Decl->getTemplatedKind()) {
-  case FunctionDecl::TK_NonTemplate:
-  case FunctionDecl::TK_DependentNonTemplate:
-    break;
-  case FunctionDecl::TK_MemberSpecialization:
-  case FunctionDecl::TK_FunctionTemplateSpecialization:
-    if (auto *TemplateInfo = Decl->getTemplateSpecializationInfo()) {
-      if (!TemplateInfo->isExplicitInstantiationOrSpecialization())
-        return true;
-    }
-    break;
-  case FunctionDecl::TK_FunctionTemplate:
-  case FunctionDecl::TK_DependentFunctionTemplateSpecialization:
-    return true;
-  }
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  LinkageInfo Linkage = Decl->getLinkageAndVisibility();
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  // Build declaration fragments, sub-heading, and signature of the function.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForFunction(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-  FunctionSignature Signature =
-      DeclarationFragmentsBuilder::getFunctionSignature(Decl);
-
-  // Add the function record to the API set.
-  API.addGlobalFunction(Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment,
-                        Declaration, SubHeading, Signature,
-                        isInSystemHeader(Context, Decl));
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitEnumDecl(const EnumDecl *Decl) {
-  if (!Decl->isComplete())
-    return true;
-
-  // Skip forward declaration.
-  if (!Decl->isThisDeclarationADefinition())
-    return true;
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  SmallString<128> QualifiedNameBuffer;
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  if (Name.empty())
-    Name = getTypedefName(Decl);
-  if (Name.empty()) {
-    llvm::raw_svector_ostream OS(QualifiedNameBuffer);
-    Decl->printQualifiedName(OS);
-    Name = QualifiedNameBuffer.str();
-  }
-
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  // Build declaration fragments and sub-heading for the enum.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForEnum(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-
-  EnumRecord *EnumRecord = API.addEnum(
-      API.copyString(Name), USR, Loc, AvailabilitySet(Decl), Comment,
-      Declaration, SubHeading, isInSystemHeader(Context, Decl));
-
-  // Now collect information about the enumerators in this enum.
-  recordEnumConstants(EnumRecord, Decl->enumerators());
-
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitRecordDecl(const RecordDecl *Decl) {
-  if (!Decl->isCompleteDefinition())
-    return true;
-
-  // Skip C++ structs/classes/unions
-  // TODO: support C++ records
-  if (isa<CXXRecordDecl>(Decl))
-    return true;
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  if (Name.empty())
-    Name = getTypedefName(Decl);
-  if (Name.empty())
-    return true;
-
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  // Build declaration fragments and sub-heading for the struct.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForStruct(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-
-  StructRecord *StructRecord =
-      API.addStruct(Name, USR, Loc, AvailabilitySet(Decl), Comment, Declaration,
-                    SubHeading, isInSystemHeader(Context, Decl));
-
-  // Now collect information about the fields in this struct.
-  recordStructFields(StructRecord, Decl->fields());
-
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitObjCInterfaceDecl(const ObjCInterfaceDecl *Decl) {
-  // Skip forward declaration for classes (@class)
-  if (!Decl->isThisDeclarationADefinition())
-    return true;
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  LinkageInfo Linkage = Decl->getLinkageAndVisibility();
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  // Build declaration fragments and sub-heading for the interface.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForObjCInterface(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-
-  // Collect super class information.
-  SymbolReference SuperClass;
-  if (const auto *SuperClassDecl = Decl->getSuperClass()) {
-    SuperClass.Name = SuperClassDecl->getObjCRuntimeNameAsString();
-    SuperClass.USR = API.recordUSR(SuperClassDecl);
-  }
-
-  ObjCInterfaceRecord *ObjCInterfaceRecord = API.addObjCInterface(
-      Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment, Declaration,
-      SubHeading, SuperClass, isInSystemHeader(Context, Decl));
-
-  // Record all methods (selectors). This doesn't include automatically
-  // synthesized property methods.
-  recordObjCMethods(ObjCInterfaceRecord, Decl->methods());
-  recordObjCProperties(ObjCInterfaceRecord, Decl->properties());
-  recordObjCInstanceVariables(ObjCInterfaceRecord, Decl->ivars());
-  recordObjCProtocols(ObjCInterfaceRecord, Decl->protocols());
-
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitObjCProtocolDecl(const ObjCProtocolDecl *Decl) {
-  // Skip forward declaration for protocols (@protocol).
-  if (!Decl->isThisDeclarationADefinition())
-    return true;
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  // Build declaration fragments and sub-heading for the protocol.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForObjCProtocol(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-
-  ObjCProtocolRecord *ObjCProtocolRecord = API.addObjCProtocol(
-      Name, USR, Loc, AvailabilitySet(Decl), Comment, Declaration, SubHeading,
-      isInSystemHeader(Context, Decl));
-
-  recordObjCMethods(ObjCProtocolRecord, Decl->methods());
-  recordObjCProperties(ObjCProtocolRecord, Decl->properties());
-  recordObjCProtocols(ObjCProtocolRecord, Decl->protocols());
-
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitTypedefNameDecl(const TypedefNameDecl *Decl) {
-  // Skip ObjC Type Parameter for now.
-  if (isa<ObjCTypeParamDecl>(Decl))
-    return true;
-
-  if (!Decl->isDefinedOutsideFunctionOrMethod())
-    return true;
-
-  if (!LocationChecker(Decl->getLocation()))
-    return true;
-
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  StringRef Name = Decl->getName();
-  StringRef USR = API.recordUSR(Decl);
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-
-  QualType Type = Decl->getUnderlyingType();
-  SymbolReference SymRef =
-      TypedefUnderlyingTypeResolver(Context).getSymbolReferenceForType(Type,
-                                                                       API);
-
-  API.addTypedef(Name, USR, Loc, AvailabilitySet(Decl), Comment,
-                 DeclarationFragmentsBuilder::getFragmentsForTypedef(Decl),
-                 DeclarationFragmentsBuilder::getSubHeading(Decl), SymRef,
-                 isInSystemHeader(Context, Decl));
-
-  return true;
-}
-
-bool ExtractAPIVisitor::VisitObjCCategoryDecl(const ObjCCategoryDecl *Decl) {
-  // Collect symbol information.
-  StringRef Name = Decl->getName();
-  StringRef USR = API.recordUSR(Decl);
-  PresumedLoc Loc =
-      Context.getSourceManager().getPresumedLoc(Decl->getLocation());
-  DocComment Comment;
-  if (auto *RawComment = Context.getRawCommentForDeclNoCache(Decl))
-    Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                            Context.getDiagnostics());
-  // Build declaration fragments and sub-heading for the category.
-  DeclarationFragments Declaration =
-      DeclarationFragmentsBuilder::getFragmentsForObjCCategory(Decl);
-  DeclarationFragments SubHeading =
-      DeclarationFragmentsBuilder::getSubHeading(Decl);
-
-  const ObjCInterfaceDecl *InterfaceDecl = Decl->getClassInterface();
-  SymbolReference Interface(InterfaceDecl->getName(),
-                            API.recordUSR(InterfaceDecl));
-
-  ObjCCategoryRecord *ObjCCategoryRecord = API.addObjCCategory(
-      Name, USR, Loc, AvailabilitySet(Decl), Comment, Declaration, SubHeading,
-      Interface, isInSystemHeader(Context, Decl));
-
-  recordObjCMethods(ObjCCategoryRecord, Decl->methods());
-  recordObjCProperties(ObjCCategoryRecord, Decl->properties());
-  recordObjCInstanceVariables(ObjCCategoryRecord, Decl->ivars());
-  recordObjCProtocols(ObjCCategoryRecord, Decl->protocols());
-
-  return true;
-}
-
-/// Collect API information for the enum constants and associate with the
-/// parent enum.
-void ExtractAPIVisitor::recordEnumConstants(
-    EnumRecord *EnumRecord, const EnumDecl::enumerator_range Constants) {
-  for (const auto *Constant : Constants) {
-    // Collect symbol information.
-    StringRef Name = Constant->getName();
-    StringRef USR = API.recordUSR(Constant);
-    PresumedLoc Loc =
-        Context.getSourceManager().getPresumedLoc(Constant->getLocation());
-    DocComment Comment;
-    if (auto *RawComment = Context.getRawCommentForDeclNoCache(Constant))
-      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                              Context.getDiagnostics());
-
-    // Build declaration fragments and sub-heading for the enum constant.
-    DeclarationFragments Declaration =
-        DeclarationFragmentsBuilder::getFragmentsForEnumConstant(Constant);
-    DeclarationFragments SubHeading =
-        DeclarationFragmentsBuilder::getSubHeading(Constant);
-
-    API.addEnumConstant(EnumRecord, Name, USR, Loc, AvailabilitySet(Constant),
-                        Comment, Declaration, SubHeading,
-                        isInSystemHeader(Context, Constant));
-  }
-}
-
-/// Collect API information for the struct fields and associate with the
-/// parent struct.
-void ExtractAPIVisitor::recordStructFields(
-    StructRecord *StructRecord, const RecordDecl::field_range Fields) {
-  for (const auto *Field : Fields) {
-    // Collect symbol information.
-    StringRef Name = Field->getName();
-    StringRef USR = API.recordUSR(Field);
-    PresumedLoc Loc =
-        Context.getSourceManager().getPresumedLoc(Field->getLocation());
-    DocComment Comment;
-    if (auto *RawComment = Context.getRawCommentForDeclNoCache(Field))
-      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                              Context.getDiagnostics());
-
-    // Build declaration fragments and sub-heading for the struct field.
-    DeclarationFragments Declaration =
-        DeclarationFragmentsBuilder::getFragmentsForField(Field);
-    DeclarationFragments SubHeading =
-        DeclarationFragmentsBuilder::getSubHeading(Field);
-
-    API.addStructField(StructRecord, Name, USR, Loc, AvailabilitySet(Field),
-                       Comment, Declaration, SubHeading,
-                       isInSystemHeader(Context, Field));
-  }
-}
-
-/// Collect API information for the Objective-C methods and associate with the
-/// parent container.
-void ExtractAPIVisitor::recordObjCMethods(
-    ObjCContainerRecord *Container,
-    const ObjCContainerDecl::method_range Methods) {
-  for (const auto *Method : Methods) {
-    // Don't record selectors for properties.
-    if (Method->isPropertyAccessor())
-      continue;
-
-    StringRef Name = API.copyString(Method->getSelector().getAsString());
-    StringRef USR = API.recordUSR(Method);
-    PresumedLoc Loc =
-        Context.getSourceManager().getPresumedLoc(Method->getLocation());
-    DocComment Comment;
-    if (auto *RawComment = Context.getRawCommentForDeclNoCache(Method))
-      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                              Context.getDiagnostics());
-
-    // Build declaration fragments, sub-heading, and signature for the method.
-    DeclarationFragments Declaration =
-        DeclarationFragmentsBuilder::getFragmentsForObjCMethod(Method);
-    DeclarationFragments SubHeading =
-        DeclarationFragmentsBuilder::getSubHeading(Method);
-    FunctionSignature Signature =
-        DeclarationFragmentsBuilder::getFunctionSignature(Method);
-
-    API.addObjCMethod(Container, Name, USR, Loc, AvailabilitySet(Method),
-                      Comment, Declaration, SubHeading, Signature,
-                      Method->isInstanceMethod(),
-                      isInSystemHeader(Context, Method));
-  }
-}
-
-void ExtractAPIVisitor::recordObjCProperties(
-    ObjCContainerRecord *Container,
-    const ObjCContainerDecl::prop_range Properties) {
-  for (const auto *Property : Properties) {
-    StringRef Name = Property->getName();
-    StringRef USR = API.recordUSR(Property);
-    PresumedLoc Loc =
-        Context.getSourceManager().getPresumedLoc(Property->getLocation());
-    DocComment Comment;
-    if (auto *RawComment = Context.getRawCommentForDeclNoCache(Property))
-      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                              Context.getDiagnostics());
-
-    // Build declaration fragments and sub-heading for the property.
-    DeclarationFragments Declaration =
-        DeclarationFragmentsBuilder::getFragmentsForObjCProperty(Property);
-    DeclarationFragments SubHeading =
-        DeclarationFragmentsBuilder::getSubHeading(Property);
-
-    StringRef GetterName =
-        API.copyString(Property->getGetterName().getAsString());
-    StringRef SetterName =
-        API.copyString(Property->getSetterName().getAsString());
-
-    // Get the attributes for property.
-    unsigned Attributes = ObjCPropertyRecord::NoAttr;
-    if (Property->getPropertyAttributes() &
-        ObjCPropertyAttribute::kind_readonly)
-      Attributes |= ObjCPropertyRecord::ReadOnly;
-
-    API.addObjCProperty(
-        Container, Name, USR, Loc, AvailabilitySet(Property), Comment,
-        Declaration, SubHeading,
-        static_cast<ObjCPropertyRecord::AttributeKind>(Attributes), GetterName,
-        SetterName, Property->isOptional(),
-        !(Property->getPropertyAttributes() &
-          ObjCPropertyAttribute::kind_class),
-        isInSystemHeader(Context, Property));
-  }
-}
-
-void ExtractAPIVisitor::recordObjCInstanceVariables(
-    ObjCContainerRecord *Container,
-    const llvm::iterator_range<
-        DeclContext::specific_decl_iterator<ObjCIvarDecl>>
-        Ivars) {
-  for (const auto *Ivar : Ivars) {
-    StringRef Name = Ivar->getName();
-    StringRef USR = API.recordUSR(Ivar);
-    PresumedLoc Loc =
-        Context.getSourceManager().getPresumedLoc(Ivar->getLocation());
-    DocComment Comment;
-    if (auto *RawComment = Context.getRawCommentForDeclNoCache(Ivar))
-      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
-                                              Context.getDiagnostics());
-
-    // Build declaration fragments and sub-heading for the instance variable.
-    DeclarationFragments Declaration =
-        DeclarationFragmentsBuilder::getFragmentsForField(Ivar);
-    DeclarationFragments SubHeading =
-        DeclarationFragmentsBuilder::getSubHeading(Ivar);
-
-    ObjCInstanceVariableRecord::AccessControl Access =
-        Ivar->getCanonicalAccessControl();
-
-    API.addObjCInstanceVariable(
-        Container, Name, USR, Loc, AvailabilitySet(Ivar), Comment, Declaration,
-        SubHeading, Access, isInSystemHeader(Context, Ivar));
-  }
-}
-
-void ExtractAPIVisitor::recordObjCProtocols(
-    ObjCContainerRecord *Container,
-    ObjCInterfaceDecl::protocol_range Protocols) {
-  for (const auto *Protocol : Protocols)
-    Container->Protocols.emplace_back(Protocol->getName(),
-                                      API.recordUSR(Protocol));
-}

clang/lib/ExtractAPI/TypedefUnderlyingTypeResolver.cpp

 //===- ExtractAPI/TypedefUnderlyingTypeResolver.cpp -------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file implements UnderlyingTypeResolver.
 ///
 //===----------------------------------------------------------------------===//
 
-#include "TypedefUnderlyingTypeResolver.h"
+#include "clang/ExtractAPI/TypedefUnderlyingTypeResolver.h"
 #include "clang/Index/USRGeneration.h"
 
 using namespace clang;
 using namespace extractapi;
 
 const NamedDecl *
 TypedefUnderlyingTypeResolver::getUnderlyingTypeDecl(QualType Type) const {
   const NamedDecl *TypeDecl = nullptr;

clang/test/Index/extract-api-cursor.m

+// Test is line- and column-sensitive. Run lines are below
+
 /// Foo docs
 struct Foo {
     /// Bar docs
     int bar;
 };
 
 /// Base docs
 @interface Base

 @end
 
 /// Derived docs
 @interface Derived: Base
 /// Derived method docs
 - (void)derivedMethodWithValue:(id<Protocol>)value;
 @end
 
-/// This won't show up in docs because we can't serialize it
-@interface Derived ()
-/// Derived method in category docs, won't show up either.
-- (void)derivedMethodInCategory;
+@implementation Derived
+- (void)derivedMethodWithValue:(id<Protocol>)value {
+    int a = 5;
+}
 @end
 
-// RUN: c-index-test -single-symbol-sgfs local %s | FileCheck %s
-
-// Checking for Foo
-// CHECK: "parentContexts":[]
-// CHECK-SAME: "relatedSymbols":[]
-// CHECK-SAME: "relationships":[]
-// CHECK-SAME: "text":"Foo docs"
-// CHECK-SAME: "kind":{"displayName":"Structure","identifier":"objective-c.struct"}
-// CHECK-SAME: "title":"Foo"
-
-// Checking for bar
-// CHECK-NEXT: "parentContexts":[{"kind":"objective-c.struct","name":"Foo","usr":"c:@S@Foo"}]
-// CHECK-SAME: "relatedSymbols":[]
-// CHECK-SAME: "relationships":[{"kind":"memberOf","source":"c:@S@Foo@FI@bar","target":"c:@S@Foo"
-// CHECK-SAME: "text":"Bar docs"
-// CHECK-SAME: "kind":{"displayName":"Instance Property","identifier":"objective-c.property"}
-// CHECK-SAME: "title":"bar"
-
-// Checking for Base
-// CHECK-NEXT: "parentContexts":[]
-// CHECK-SAME: "relatedSymbols":[]
-// CHECK-SAME: "relationships":[]
-// CHECK-SAME: "text":"Base docs"
-// CHECK-SAME: "kind":{"displayName":"Class","identifier":"objective-c.class"}
-// CHECK-SAME: "title":"Base"
-
-// Checking for baseProperty
-// CHECK-NEXT: "parentContexts":[{"kind":"objective-c.class","name":"Base","usr":"c:objc(cs)Base"}]
-// CHECK-SAME: "relatedSymbols":[{"accessLevel":"public","declarationLanguage":"objective-c"
-// CHECK-SAME: "isSystem":false
-// CHECK-SAME: "usr":"c:@S@Foo"}]
-// CHECK-SAME: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Base(py)baseProperty","target":"c:objc(cs)Base"
-// CHECK-SAME: "text":"Base property docs"
-// CHECK-SAME: "kind":{"displayName":"Instance Property","identifier":"objective-c.property"}
-// CHECK-SAME: "title":"baseProperty"
-
-// Checking for baseMethodWithArg
-// CHECK-NEXT: "parentContexts":[{"kind":"objective-c.class","name":"Base","usr":"c:objc(cs)Base"}]
-// CHECK-SAME: "relatedSymbols":[]
-// CHECK-SAME: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Base(im)baseMethodWithArg:","target":"c:objc(cs)Base"
-// CHECK-SAME: "text":"Base method docs"
-// CHECK-SAME: "kind":{"displayName":"Instance Method","identifier":"objective-c.method"}
-// CHECK-SAME: "title":"baseMethodWithArg:"
-
-// Checking for Protocol
-// CHECK-NEXT: "parentContexts":[]
-// CHECK-SAME: "relatedSymbols":[]
-// CHECK-SAME: "relationships":[]
-// CHECK-SAME: "text":"Protocol docs"
-// CHECK-SAME: "kind":{"displayName":"Protocol","identifier":"objective-c.protocol"}
-// CHECK-SAME: "title":"Protocol"
-
-// Checking for protocolProperty
-// CHECK-NEXT: "parentContexts":[{"kind":"objective-c.protocol","name":"Protocol","usr":"c:objc(pl)Protocol"}]
-// CHECK-SAME: "relatedSymbols":[{"accessLevel":"public","declarationLanguage":"objective-c"
-// CHECK-SAME: "isSystem":false
-// CHECK-SAME: "usr":"c:@S@Foo"}]
-// CHECK-SAME: "relationships":[{"kind":"memberOf","source":"c:objc(pl)Protocol(py)protocolProperty","target":"c:objc(pl)Protocol"
-// CHECK-SAME: "text":"Protocol property docs"
-// CHECK-SAME: "kind":{"displayName":"Instance Property","identifier":"objective-c.property"}
-// CHECK-SAME: "title":"protocolProperty"
-
-// Checking for Derived
-// CHECK-NEXT: "parentContexts":[]
-// CHECK-SAME: "relatedSymbols":[{"accessLevel":"public","declarationLanguage":"objective-c"
-// CHECK-SAME: "isSystem":false
-// CHECK-SAME: "usr":"c:objc(cs)Base"}]
-// CHECK-SAME: "relationships":[{"kind":"inheritsFrom","source":"c:objc(cs)Derived","target":"c:objc(cs)Base"
-// CHECK-SAME: "text":"Derived docs"
-// CHECK-SAME: "kind":{"displayName":"Class","identifier":"objective-c.class"}
-// CHECK-SAME: "title":"Derived"
-
-// Checking for derivedMethodWithValue
-// CHECK-NEXT: "parentContexts":[{"kind":"objective-c.class","name":"Derived","usr":"c:objc(cs)Derived"}]
-// CHECK-SAME: "relatedSymbols":[]
-// CHECK-SAME: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Derived(im)derivedMethodWithValue:","target":"c:objc(cs)Derived"
-// CHECK-SAME: "text":"Derived method docs"
-// CHECK-SAME: "kind":{"displayName":"Instance Method","identifier":"objective-c.method"}
-// CHECK-SAME: "title":"derivedMethodWithValue:"
-
-// CHECK-NOT: This won't show up in docs because we can't serialize it
-// CHECK-NOT: Derived method in category docs, won't show up either.
+// RUN: c-index-test -single-symbol-sgf-at=%s:4:9 local %s | FileCheck -check-prefix=CHECK-FOO %s
+// CHECK-FOO: "parentContexts":[]
+// CHECK-FOO: "relatedSymbols":[]
+// CHECK-FOO: "relationships":[]
+// CHECK-FOO: "text":"Foo docs"
+// CHECK-FOO: "kind":{"displayName":"Structure","identifier":"objective-c.struct"}
+// CHECK-FOO: "title":"Foo"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:6:9 local %s | FileCheck -check-prefix=CHECK-BAR %s
+// CHECK-BAR: "parentContexts":[{"kind":"objective-c.struct","name":"Foo","usr":"c:@S@Foo"}]
+// CHECK-BAR: "relatedSymbols":[]
+// CHECK-BAR: "relationships":[{"kind":"memberOf","source":"c:@S@Foo@FI@bar","target":"c:@S@Foo"
+// CHECK-BAR: "text":"Bar docs"
+// CHECK-BAR: "kind":{"displayName":"Instance Property","identifier":"objective-c.property"}
+// CHECK-BAR: "title":"bar"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:10:11 local %s | FileCheck -check-prefix=CHECK-BASE %s
+// CHECK-BASE: "parentContexts":[]
+// CHECK-BASE: "relatedSymbols":[]
+// CHECK-BASE: "relationships":[]
+// CHECK-BASE: "text":"Base docs"
+// CHECK-BASE: "kind":{"displayName":"Class","identifier":"objective-c.class"}
+// CHECK-BASE: "title":"Base"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:12:25 local %s | FileCheck -check-prefix=CHECK-BASE-PROP %s
+// CHECK-BASE-PROP: "parentContexts":[{"kind":"objective-c.class","name":"Base","usr":"c:objc(cs)Base"}]
+// CHECK-BASE-PROP: "relatedSymbols":[{"accessLevel":"public","declarationLanguage":"objective-c"
+// CHECK-BASE-PROP: "isSystem":false
+// CHECK-BASE-PROP: "usr":"c:@S@Foo"}]
+// CHECK-BASE-PROP: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Base(py)baseProperty","target":"c:objc(cs)Base"
+// CHECK-BASE-PROP: "text":"Base property docs"
+// CHECK-BASE-PROP: "kind":{"displayName":"Instance Property","identifier":"objective-c.property"}
+// CHECK-BASE-PROP: "title":"baseProperty"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:15:9 local %s | FileCheck -check-prefix=CHECK-BASE-METHOD %s
+// CHECK-BASE-METHOD: "parentContexts":[{"kind":"objective-c.class","name":"Base","usr":"c:objc(cs)Base"}]
+// CHECK-BASE-METHOD: "relatedSymbols":[]
+// CHECK-BASE-METHOD: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Base(im)baseMethodWithArg:","target":"c:objc(cs)Base"
+// CHECK-BASE-METHOD: "text":"Base method docs"
+// CHECK-BASE-METHOD: "kind":{"displayName":"Instance Method","identifier":"objective-c.method"}
+// CHECK-BASE-METHOD: "title":"baseMethodWithArg:"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:19:11 local %s | FileCheck -check-prefix=CHECK-PROTOCOL %s
+// CHECK-PROTOCOL: "parentContexts":[]
+// CHECK-PROTOCOL: "relatedSymbols":[]
+// CHECK-PROTOCOL: "relationships":[]
+// CHECK-PROTOCOL: "text":"Protocol docs"
+// CHECK-PROTOCOL: "kind":{"displayName":"Protocol","identifier":"objective-c.protocol"}
+// CHECK-PROTOCOL: "title":"Protocol"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:21:27 local %s | FileCheck -check-prefix=CHECK-PROTOCOL-PROP %s
+// CHECK-PROTOCOL-PROP: "parentContexts":[{"kind":"objective-c.protocol","name":"Protocol","usr":"c:objc(pl)Protocol"}]
+// CHECK-PROTOCOL-PROP: "relatedSymbols":[{"accessLevel":"public","declarationLanguage":"objective-c"
+// CHECK-PROTOCOL-PROP: "isSystem":false
+// CHECK-PROTOCOL-PROP: "usr":"c:@S@Foo"}]
+// CHECK-PROTOCOL-PROP: "relationships":[{"kind":"memberOf","source":"c:objc(pl)Protocol(py)protocolProperty","target":"c:objc(pl)Protocol"
+// CHECK-PROTOCOL-PROP: "text":"Protocol property docs"
+// CHECK-PROTOCOL-PROP: "kind":{"displayName":"Instance Property","identifier":"objective-c.property"}
+// CHECK-PROTOCOL-PROP: "title":"protocolProperty"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:25:15 local %s | FileCheck -check-prefix=CHECK-DERIVED %s
+// CHECK-DERIVED: "parentContexts":[]
+// CHECK-DERIVED: "relatedSymbols":[{"accessLevel":"public","declarationLanguage":"objective-c"
+// CHECK-DERIVED: "isSystem":false
+// CHECK-DERIVED: "usr":"c:objc(cs)Base"}]
+// CHECK-DERIVED: "relationships":[{"kind":"inheritsFrom","source":"c:objc(cs)Derived","target":"c:objc(cs)Base"
+// CHECK-DERIVED: "text":"Derived docs"
+// CHECK-DERIVED: "kind":{"displayName":"Class","identifier":"objective-c.class"}
+// CHECK-DERIVED: "title":"Derived"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:27:11 local %s | FileCheck -check-prefix=CHECK-DERIVED-METHOD %s
+// CHECK-DERIVED-METHOD: "parentContexts":[{"kind":"objective-c.class","name":"Derived","usr":"c:objc(cs)Derived"}]
+// CHECK-DERIVED-METHOD: "relatedSymbols":[]
+// CHECK-DERIVED-METHOD: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Derived(im)derivedMethodWithValue:","target":"c:objc(cs)Derived"
+// CHECK-DERIVED-METHOD: "text":"Derived method docs"
+// CHECK-DERIVED-METHOD: "kind":{"displayName":"Instance Method","identifier":"objective-c.method"}
+// CHECK-DERIVED-METHOD: "title":"derivedMethodWithValue:"
+
+// RUN: c-index-test -single-symbol-sgf-at=%s:31:11 local %s | FileCheck -check-prefix=CHECK-DERIVED-METHOD-IMPL %s
+// CHECK-DERIVED-METHOD-IMPL: "parentContexts":[{"kind":"objective-c.class","name":"Derived","usr":"c:objc(cs)Derived"}]
+// CHECK-DERIVED-METHOD-IMPL: "relatedSymbols":[]
+// CHECK-DERIVED-METHOD-IMPL: "relationships":[{"kind":"memberOf","source":"c:objc(cs)Derived(im)derivedMethodWithValue:","target":"c:objc(cs)Derived"
+// CHECK-DERIVED-METHOD-IMPL: "text":"Derived method docs"
+// CHECK-DERIVED-METHOD-IMPL: "kind":{"displayName":"Instance Method","identifier":"objective-c.method"}
+// CHECK-DERIVED-METHOD-IMPL: "title":"derivedMethodWithValue:"

clang/tools/c-index-test/c-index-test.c

 /* c-index-test.c */
 
 #include "clang-c/BuildSystem.h"
 #include "clang-c/CXCompilationDatabase.h"
 #include "clang-c/CXErrorCode.h"
+#include "clang-c/CXSourceLocation.h"
 #include "clang-c/CXString.h"
 #include "clang-c/Documentation.h"
 #include "clang-c/Index.h"
 #include "clang/Config/config.h"
 #include <assert.h>
 #include <ctype.h>
 #include <stdio.h>
 #include <stdlib.h>
dispose_tu:
   clang_disposeTranslationUnit(TU);
 free_remapped_files:
   free_remapped_files(unsaved_files, num_unsaved_files);
 dispose_index:
   clang_disposeIndex(Idx);
   return result;
 }
 
+static void inspect_single_symbol_sgf_cursor(CXCursor Cursor) {
+  CXSourceLocation CursorLoc;
+  CXString SGFData;
+  const char *SGF;
+  unsigned line, column;
+  CursorLoc = clang_getCursorLocation(Cursor);
+  clang_getSpellingLocation(CursorLoc, 0, &line, &column, 0);
+  printf("%d:%d: ", line, column);
+
+  SGFData = clang_getSymbolGraphForCursor(Cursor);
+  SGF = clang_getCString(SGFData);
+  if (SGF)
+    printf("%s\n", SGF);
+}
+
 /******************************************************************************/
 /* Command line processing.                                                   */
 /******************************************************************************/
 
 static CXCursorVisitor GetVisitor(const char *s) {
   if (s[0] == '\0')
     return FilteredPrintingVisitor;
   if (strcmp(s, "-usrs") == 0)
fprintf(stderr,
     "       c-index-test -test-print-type-size {<args>}*\n"
     "       c-index-test -test-print-bitwidth {<args>}*\n"
     "       c-index-test -test-print-target-info {<args>}*\n"
     "       c-index-test -test-print-type-declaration {<args>}*\n"
     "       c-index-test -print-usr [<CursorKind> {<args>}]*\n"
     "       c-index-test -print-usr-file <file>\n");
   fprintf(stderr,
           "       c-index-test -single-symbol-sgfs <symbol filter> {<args>*}\n"
+          "       c-index-test -single-symbol-sgf-at=<site> {<args>*}\n"
           "       c-index-test -single-symbol-sgf-for=<usr> {<args>}*\n");
   fprintf(stderr,
     "       c-index-test -write-pch <file> <compiler arguments>\n"
     "       c-index-test -compilation-db [lookup <filename>] database\n");
   fprintf(stderr,
     "       c-index-test -print-build-session-timestamp\n");
   fprintf(stderr,
     "       c-index-test -read-diagnostics <file>\n\n");
else if (argc > 2 && strcmp(argv[1], "-write-pch") == 0)
     return write_pch_file(argv[2], argc - 3, argv + 3);
   else if (argc > 2 && strcmp(argv[1], "-compilation-db") == 0)
     return perform_test_compilation_db(argv[argc-1], argc - 3, argv + 2);
   else if (argc == 2 && strcmp(argv[1], "-print-build-session-timestamp") == 0)
     return perform_print_build_session_timestamp();
   else if (argc > 3 && strcmp(argv[1], "-single-symbol-sgfs") == 0)
     return perform_test_load_source(argc - 3, argv + 3, argv[2],
                                     PrintSingleSymbolSGFs, NULL);
+  else if (argc > 2 && strstr(argv[1], "-single-symbol-sgf-at=") == argv[1])
+    return inspect_cursor_at(
+        argc, argv, "-single-symbol-sgf-at=", inspect_single_symbol_sgf_cursor);
   else if (argc > 2 && strstr(argv[1], "-single-symbol-sgf-for=") == argv[1])
     return perform_test_single_symbol_sgf(argv[1], argc - 2, argv + 2);
 
   print_usage();
   return 1;
 }
 
 /***/

clang/tools/libclang/CXExtractAPI.cpp

 #include "CXCursor.h"
 #include "CXString.h"
 #include "CXTranslationUnit.h"
 #include "clang-c/CXErrorCode.h"
 #include "clang-c/Documentation.h"
 #include "clang-c/Index.h"
 #include "clang-c/Platform.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/DeclObjC.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/ExtractAPI/API.h"
 #include "clang/ExtractAPI/ExtractAPIVisitor.h"
 #include "clang/ExtractAPI/Serialization/SymbolGraphSerializer.h"
 #include "clang/Frontend/ASTUnit.h"
 #include "clang/Frontend/FrontendOptions.h"
 #include "clang/Index/USRGeneration.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Support/CBindingWrapping.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/JSON.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace clang;
 using namespace clang::extractapi;
 
+namespace {
+struct LibClangExtractAPIVisitor
+    : ExtractAPIVisitor<LibClangExtractAPIVisitor> {
+  using Base = ExtractAPIVisitor<LibClangExtractAPIVisitor>;
+
+  LibClangExtractAPIVisitor(ASTContext &Context, APISet &API)
+      : ExtractAPIVisitor<LibClangExtractAPIVisitor>(Context, API) {}
+
+  const RawComment *fetchRawCommentForDecl(const Decl *D) const {
+    return Context.getRawCommentForAnyRedecl(D);
+  }
+
+  // We need to visit implementations as well to ensure that when a user clicks
+  // on a method defined only within the implementation that we can still
+  // provide a symbol graph for it.
+  bool VisitObjCImplementationDecl(const ObjCImplementationDecl *Decl) {
+    if (!shouldDeclBeIncluded(Decl))
+      return true;
+
+    const ObjCInterfaceDecl *Interface = Decl->getClassInterface();
+    StringRef Name = Interface->getName();
+    StringRef USR = API.recordUSR(Decl);
+    PresumedLoc Loc =
+        Context.getSourceManager().getPresumedLoc(Decl->getLocation());
+    LinkageInfo Linkage = Decl->getLinkageAndVisibility();
+    DocComment Comment;
+    if (auto *RawComment = fetchRawCommentForDecl(Interface))
+      Comment = RawComment->getFormattedLines(Context.getSourceManager(),
+                                              Context.getDiagnostics());
+
+    // Build declaration fragments and sub-heading by generating them for the
+    // interface.
+    DeclarationFragments Declaration =
+        DeclarationFragmentsBuilder::getFragmentsForObjCInterface(Interface);
+    DeclarationFragments SubHeading =
+        DeclarationFragmentsBuilder::getSubHeading(Decl);
+
+    // Collect super class information.
+    SymbolReference SuperClass;
+    if (const auto *SuperClassDecl = Decl->getSuperClass()) {
+      SuperClass.Name = SuperClassDecl->getObjCRuntimeNameAsString();
+      SuperClass.USR = API.recordUSR(SuperClassDecl);
+    }
+
+    ObjCInterfaceRecord *ObjCInterfaceRecord = API.addObjCInterface(
+        Name, USR, Loc, AvailabilitySet(Decl), Linkage, Comment, Declaration,
+        SubHeading, SuperClass, isInSystemHeader(Decl));
+
+    // Record all methods (selectors). This doesn't include automatically
+    // synthesized property methods.
+    recordObjCMethods(ObjCInterfaceRecord, Decl->methods());
+    recordObjCProperties(ObjCInterfaceRecord, Decl->properties());
+    recordObjCInstanceVariables(ObjCInterfaceRecord, Decl->ivars());
+
+    return true;
+  }
+};
+} // namespace
+
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(APISet, CXAPISet)
 
-static void WalkupFromMostDerivedType(ExtractAPIVisitor &Visitor, Decl *D);
+static void WalkupFromMostDerivedType(LibClangExtractAPIVisitor &Visitor,
+                                      Decl *D);
 
 template <typename DeclTy>
 static bool WalkupParentContext(DeclContext *Parent,
-                                ExtractAPIVisitor &Visitor) {
+                                LibClangExtractAPIVisitor &Visitor) {
   if (auto *D = dyn_cast<DeclTy>(Parent)) {
     WalkupFromMostDerivedType(Visitor, D);
     return true;
   }
   return false;
 }
 
-static void WalkupFromMostDerivedType(ExtractAPIVisitor &Visitor, Decl *D) {
+static void WalkupFromMostDerivedType(LibClangExtractAPIVisitor &Visitor,
+                                      Decl *D) {
   switch (D->getKind()) {
 #define ABSTRACT_DECL(DECL)
 #define DECL(CLASS, BASE)                                                      \
   case Decl::CLASS:                                                            \
     Visitor.WalkUpFrom##CLASS##Decl(static_cast<CLASS##Decl *>(D));            \
     break;
 #include "clang/AST/DeclNodes.inc"
   }
if (cxtu::isNotUsableTU(tu) || !out_api)
     return CXError_InvalidArguments;
 
   ASTUnit *Unit = cxtu::getASTUnit(tu);
 
   auto &Ctx = Unit->getASTContext();
   auto Lang = Unit->getInputKind().getLanguage();
   APISet *API = new APISet(Ctx.getTargetInfo().getTriple(), Lang,
                            Unit->getMainFileName().str());
-  ExtractAPIVisitor Visitor(
-      Ctx, [](SourceLocation Loc) { return true; }, *API);
+  LibClangExtractAPIVisitor Visitor(Ctx, *API);
 
   for (auto It = Unit->top_level_begin(); It != Unit->top_level_end(); ++It) {
     Visitor.TraverseDecl(*It);
   }
 
   *out_api = wrap(API);
   return CXError_Success;
 }
 
 void clang_disposeAPISet(CXAPISet api) { delete unwrap(api); }
 
 CXString clang_getSymbolGraphForUSR(const char *usr, CXAPISet api) {
   auto *API = unwrap(api);
 
   if (auto SGF = SymbolGraphSerializer::serializeSingleSymbolSGF(usr, *API))
     return GenerateCXStringFromSymbolGraphData(std::move(*SGF));
 
   return cxstring::createNull();
 }
 
 CXString clang_getSymbolGraphForCursor(CXCursor cursor) {
+  cursor = clang_getCursorReferenced(cursor);
   CXCursorKind Kind = clang_getCursorKind(cursor);
-  if (clang_isDeclaration(Kind)) {
-    const Decl *D = cxcursor::getCursorDecl(cursor);
+  if (!clang_isDeclaration(Kind))
+    return cxstring::createNull();
+
 
-    if (!D)
-      return cxstring::createNull();
 
-    CXTranslationUnit TU = cxcursor::getCursorTU(cursor);
-    if (!TU)
-      return cxstring::createNull();
 
-    ASTUnit *Unit = cxtu::getASTUnit(TU);
 
-    auto &Ctx = Unit->getASTContext();
-    auto Lang = Unit->getInputKind().getLanguage();
-    APISet API(Ctx.getTargetInfo().getTriple(), Lang,
-               Unit->getMainFileName().str());
-    ExtractAPIVisitor Visitor(
-        Ctx, [](SourceLocation Loc) { return true; }, API);
+
+  LibClangExtractAPIVisitor Visitor(Ctx, API);
+
+  const Decl *CanonicalDecl = D->getCanonicalDecl();
+  CanonicalDecl = CanonicalDecl ? CanonicalDecl : D;
 
-    SmallString<128> USR;
-    if (index::generateUSRForDecl(D, USR))
-      return cxstring::createNull();
+  if (index::generateUSRForDecl(CanonicalDecl, USR))
 
-    WalkupFromMostDerivedType(Visitor, const_cast<Decl *>(D));
-    auto *Record = API.findRecordForUSR(USR);
+  WalkupFromMostDerivedType(Visitor, const_cast<Decl *>(CanonicalDecl));
 
-    if (!Record)
-      return cxstring::createNull();
 
-    for (const auto &Fragment : Record->Declaration.getFragments()) {
-      if (Fragment.Declaration)
-        WalkupFromMostDerivedType(Visitor,
-                                  const_cast<Decl *>(Fragment.Declaration));
-    }
 
-    if (auto SGF = SymbolGraphSerializer::serializeSingleSymbolSGF(USR, API))
-      return GenerateCXStringFromSymbolGraphData(std::move(*SGF));
-  }
 
   return cxstring::createNull();
 }