[ASTMatchers] Add matchers available through casting to derived

[steveire]

These commits are available on github if it's convenient to see it all together there:

https://github.com/steveire/clang/commits/matcher-output

Here is context for the changes I'm making in case it is useful:

https://steveire.wordpress.com/2018/11/11/future-developments-in-clang-query/

Note that the commits pushed for review so far don't implement everything I wrote about there.

[steveire]

Update

[aaron.ballman]

lib/ASTMatchers/Dynamic/Registry.cpp

645

Don't use auto here (and if you did. the const would go on the other side).

647–651

I think this can be replaced with llvm::find_if().

659

I'd prefer this to not be a default argument (the callers can pass it in). It might also be nice to use an enum here rather than a bool, but alternative to that, you should stick in comments in the callers that describe the parameter name. e.g. getMatchingMatchersImpl(yada, /*ExactOnly*/true);

711

Don't use auto.

716–718

Why do these need their own inner block scopes?

719

auto.

720–721

if (TypeForMatcherOpt && StaticType.isBaseOf(...)) {

722–723

auto, here and everywhere in this file.

749

Can remove whitespace around here.

761–766

It seems like getMatchingMatchersImpl() should just be renamed to getMatchingMatchers().

[steveire]

Some updates

[lebedev.ri]

Differential lacks description.

[aaron.ballman]

lib/ASTMatchers/Dynamic/Registry.cpp

642

targetCtor -> TargetCtor

644

it -> It

645

ctor -> Ctor

648–650

Elide braces

709

Can combine with the previous if statement, I believe.

715

Don't use auto here please.

718

Don't use auto here please.

731

Don't use auto here please.

737

item -> Item

[steveire]

Case

[Abpostelnicu]

@aaron.ballman
I think the auto usage improves and simplifies the code, however, as I would really like to see this code landed, I would be ok to help Stephen to finish this work and replace auto by the "old" way. Do you think we can have this approved if we make the changes mentioned earlier?

[aaron.ballman]

In D54408#1659301, @Abpostelnicu wrote:

@aaron.ballman
I think the auto usage improves and simplifies the code, however, as I would really like to see this code landed, I would be ok to help Stephen to finish this work and replace auto by the "old" way. Do you think we can have this approved if we make the changes mentioned earlier?

Nothing jumps out at me as blocking this, aside from completing the usual review activities. Have at it!

[steveire]

@aaron.ballman I think we agreed in Belfast in November (after the most recent comment) to get this in as it is and not be as draconian about auto. Is anything blocking this?

[aaron.ballman]

In D54408#2056251, @steveire wrote:

@aaron.ballman I think we agreed in Belfast in November (after the most recent comment) to get this in as it is and not be as draconian about auto. Is anything blocking this?

Nothing is blocking this, but I think we may have slightly different understandings of what we agreed to. Reviewers asking for code to follow the coding guidelines is not draconian; it's expected for code to follow the coding guidelines and for patch authors to field requests for changes like these. I re-reviewed the things I was asking for changes on and have added a comment where I definitely insist on changes. LGTM with that nit fixed though.

lib/ASTMatchers/Dynamic/Registry.cpp

731

Please change this use of auto -- I have no idea what the type is for NestedResult without hunting for it and so I have no idea what the type is for Item and whether using a const reference is reasonable or not (after doing the hunting, it looks fine though).

lib/ASTMatchers/Dynamic/Registry.cpp

for (const auto &M : RegistryData->constructors()) {
     }
 
     if (!RetKinds.empty() && MaxSpecificity > 0) {
       std::forward<Callable>(Func)(Name, Matcher, RetKinds, ArgsKinds, MaxSpecificity);
     }
   }
 }
 
+using SR = ast_matchers::dynamic::SourceRange;
+
+llvm::Optional<std::pair<ASTNodeKind, std::string>>
+getNodeConstructorType(MatcherCtor targetCtor) {
+  auto const &ctors = RegistryData->nodeConstructors();
+
+  for (auto ctor : ctors) {
+    if (ctor.second.second == targetCtor)
+      return std::make_pair(ctor.first, ctor.second.first);
+  }
+  return llvm::None;
+}
+
 std::vector<MatchingMatcher>
-Registry::getMatchingMatchers(ast_type_traits::ASTNodeKind StaticType) {
+getDerivedResults(ast_type_traits::ASTNodeKind StaticType, StringRef Name);
+
+std::vector<MatchingMatcher>
+getMatchingMatchersImpl(ast_type_traits::ASTNodeKind StaticType,
+                        bool ExactOnly = false) {
+
   std::vector<MatchingMatcher> Result;
 
   // Exclude matchers which can't unambiguously
   // be used with particular arguments.
   static std::vector<StringRef> ExcludedMatchers{
       "allOf",
       "anyOf",
       "anything",
static std::vector<StringRef> ExcludedMatchers{
       "matchesName",
       "matchesSelector",
       "unless"};
   assert(std::is_sorted(ExcludedMatchers.begin(), ExcludedMatchers.end()));
 
   std::vector<ArgKind> AcceptedTypes{StaticType};
 
   processAcceptableMatchers(
-      AcceptedTypes, [&Result](StringRef Name, const MatcherDescriptor &,
-                               std::set<ASTNodeKind> &,
-                               std::vector<std::vector<ArgKind>>, unsigned) {
+      AcceptedTypes, [&](StringRef Name, const MatcherDescriptor &Matcher,
+        {
+          unsigned Specificity;
+          ASTNodeKind LeastDerivedKind;
+          if (ExactOnly) {
+            auto IsConvertible = Matcher.isConvertibleTo(
+                StaticType, &Specificity, &LeastDerivedKind);
+            if (IsConvertible && !LeastDerivedKind.isSame(StaticType))
+              return;
+          }
+        }
+
+        {
+          auto TypeForMatcherOpt = getNodeConstructorType(&Matcher);
+          if (TypeForMatcherOpt) {
+            if (StaticType.isBaseOf(TypeForMatcherOpt->first)) {
+              auto DerivedResults =
+                  getDerivedResults(TypeForMatcherOpt->first, Name);
+              Result.insert(Result.end(), DerivedResults.begin(),
+                            DerivedResults.end());
+              return;
+            }
+          }
+        }
+
         if (!std::binary_search(ExcludedMatchers.begin(),
                                 ExcludedMatchers.end(), Name)) {
           Result.emplace_back((Name + "()").str());
         }
       });
 
   return Result;
 }
 
+std::vector<MatchingMatcher>
+getDerivedResults(ast_type_traits::ASTNodeKind StaticType, StringRef Name) {
+  auto NestedResult = getMatchingMatchersImpl(StaticType, true);
+
+  std::vector<MatchingMatcher> Result;
+
+  Diagnostics DiagnosticIgnorer;
+
+  for (auto item : NestedResult) {
+
+    auto nestedMatcherName = item.MatcherString;
+
+    nestedMatcherName = Name.str() + "(" + nestedMatcherName + ")";
+
+    Result.emplace_back(nestedMatcherName);
+  }
+
+  return Result;
+}
+
+std::vector<MatchingMatcher>
+Registry::getMatchingMatchers(ast_type_traits::ASTNodeKind StaticType) {
+
+  std::vector<MatchingMatcher> Result = getMatchingMatchersImpl(StaticType);
+
+  return Result;
+}
+
 std::vector<MatcherCompletion>
 Registry::getMatcherCompletions(ArrayRef<ArgKind> AcceptedTypes) {
   std::vector<MatcherCompletion> Completions;
 
   processAcceptableMatchers(
       AcceptedTypes,
       [&Completions](StringRef Name, const MatcherDescriptor &Matcher,
                      std::set<ASTNodeKind> &RetKinds,

unittests/ASTMatchers/Dynamic/RegistryTest.cpp

TEST_F(RegistryTest, Completion) {
   EXPECT_TRUE(hasCompletion(Comps, "whileStmt(",
                             "Matcher<Stmt> whileStmt(Matcher<WhileStmt>...)"));
   // Polymorphic.
   EXPECT_TRUE(hasCompletion(
       Comps, "hasDescendant(",
       "Matcher<NestedNameSpecifierLoc|QualType|TypeLoc|...> "
       "hasDescendant(Matcher<NestedNameSpecifierLoc|QualType|TypeLoc|...>)"));
 
+  CompVector FuncDeclComps = getCompletions("functionDecl", 0);
+
+  EXPECT_TRUE(!FuncDeclComps.empty());
+
+  // Exclude bases and self
+  EXPECT_TRUE(!hasCompletion(FuncDeclComps, "decl("));
+  EXPECT_TRUE(!hasCompletion(FuncDeclComps, "namedDecl("));
+  EXPECT_TRUE(!hasCompletion(FuncDeclComps, "valueDecl("));
+  EXPECT_TRUE(!hasCompletion(FuncDeclComps, "declaratorDecl("));
+  EXPECT_TRUE(!hasCompletion(FuncDeclComps, "functionDecl("));
+
   CompVector WhileComps = getCompletions("whileStmt", 0);
 
   EXPECT_TRUE(hasCompletion(WhileComps, "hasBody(",
                             "Matcher<WhileStmt> hasBody(Matcher<Stmt>)"));
   EXPECT_TRUE(hasCompletion(
       WhileComps, "hasParent(",
       "Matcher<Stmt> "
       "hasParent(Matcher<NestedNameSpecifierLoc|TypeLoc|Decl|...>)"));
return llvm::find_if(C, [Name](const MatchingMatcher &M) {
            }) != C.end();
   };
 
   EXPECT_TRUE(Contains(Matchers, "isPublic()"));
   EXPECT_TRUE(Contains(Matchers, "hasName()"));
   EXPECT_TRUE(Contains(Matchers, "hasType()"));
   EXPECT_TRUE(Contains(Matchers, "hasTypeLoc()"));
   EXPECT_TRUE(Contains(Matchers, "parameterCountIs()"));
+  EXPECT_TRUE(Contains(Matchers, "cxxMethodDecl(isOverride())"));
 
   EXPECT_TRUE(!Contains(Matchers, "decl()"));
   EXPECT_TRUE(!Contains(Matchers, "namedDecl()"));
   EXPECT_TRUE(!Contains(Matchers, "valueDecl()"));
   EXPECT_TRUE(!Contains(Matchers, "declaratorDecl()"));
   EXPECT_TRUE(!Contains(Matchers, "functionDecl()"));
-
-  EXPECT_TRUE(Contains(Matchers, "cxxMethodDecl()"));
+  EXPECT_TRUE(!Contains(Matchers, "cxxMethodDecl()"));
 
   EXPECT_TRUE(!Contains(Matchers, "has()"));
 }
 
 } // end anonymous namespace
 } // end namespace dynamic
 } // end namespace ast_matchers
 } // end namespace clang