[clang] Adapt ASTMatcher to explicit(bool) specifier

[Tyker]

Summary

Changes:

• add an ast matcher for deductiong guide.
• allow isExplicit matcher for deductiong guide.
• add hasExplicitSpecifier matcher which give access to the expression of the explicit specifier if present.

[rsmith]

This seems reasonable to me, but I'll leave it to @klimek or someone else to judge whether the isExplicit / hasExplicitSpecifier approach is the right way to expose this functionality to matcher users.

[aaron.ballman]

You should also regenerate the documentation by running clang/docs/tools/dump_ast_matchers.py.

clang/include/clang/ASTMatchers/ASTMatchers.h

1141

Including implicit deduction guides? Might be worth clarifying that.

6168

conversion function and deduction -> conversion function, and deduction

6197

Oxford comma here as well, please.

6219

I think this is a somewhat useful AST matcher, but I think we are going to need a way to match on the expression contained within the explicit specifier too, won't we? Perhaps a different approach is to make a matcher named explicitSpecifier() that allows traversing to the inner expression?

clang/lib/ASTMatchers/Dynamic/Registry.cpp

163

This is missing the new hasExplicitSpecifier() matcher.

[Tyker]

i didn't manage to get "clang/docs/tools/dump_ast_matchers.py" to work. it keep failing to parse and generating empty files as a result.

hasExplicitSpecifier now gives access to the expression if present.

[aaron.ballman]

In D61552#1530730, @Tyker wrote:

i didn't manage to get "clang/docs/tools/dump_ast_matchers.py" to work. it keep failing to parse and generating empty files as a result.

Are you getting errors from running it, or just incorrect output?

clang/include/clang/ASTMatchers/ASTMatchers.h

6190

Why won't it match #2?

6220

Not keen on the name hasExplicitExpr -- I think hasExplicitSpecifier() would be a more intuitive name.

clang/lib/ASTMatchers/Dynamic/Registry.cpp

171

Please sort alphabetically.

263

Please sort alphabetically.

[Tyker]

fixed requested changes.

Are you getting errors from running it, or just incorrect output?

the issue happens to me even on master so i suppose the input is correct.
here is the error report:

[tyker@tyker tools]$ ./dump_ast_matchers.py 
*** Unparsable: " #ifndef LLVM_CLANG_ASTMATCHERS_ASTMATCHERS_H #define LLVM_CLANG_ASTMATCHERS_ASTMATCHERS_H " ***
*** Unparsable: " #include "clang/AST/ASTContext.h" #include "clang/AST/ASTTypeTraits.h" #include "clang/AST/Attr.h" #include "clang/AST/Decl.h" #include "clang/AST/DeclCXX.h" #include "clang/AST/DeclFriend.h" #include "clang/AST/DeclObjC.h" #include "clang/AST/DeclTemplate.h" #include "clang/AST/Expr.h" #include "clang/AST/ExprCXX.h" #include "clang/AST/ExprObjC.h" #include "clang/AST/NestedNameSpecifier.h" #include "clang/AST/OpenMPClause.h" #include "clang/AST/OperationKinds.h" #include "clang/AST/Stmt.h" #include "clang/AST/StmtCXX.h" #include "clang/AST/StmtObjC.h" #include "clang/AST/StmtOpenMP.h" #include "clang/AST/TemplateBase.h" #include "clang/AST/TemplateName.h" #include "clang/AST/Type.h" #include "clang/AST/TypeLoc.h" #include "clang/ASTMatchers/ASTMatchersInternal.h" #include "clang/ASTMatchers/ASTMatchersMacros.h" #include "clang/Basic/AttrKinds.h" #include "clang/Basic/ExceptionSpecificationType.h" #include "clang/Basic/IdentifierTable.h" #include "clang/Basic/LLVM.h" #include "clang/Basic/SourceManager.h" #include "clang/Basic/Specifiers.h" #include "clang/Basic/TypeTraits.h" #include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/SmallVector.h" #include "llvm/ADT/StringRef.h" #include "llvm/Support/Casting.h" #include "llvm/Support/Compiler.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/Regex.h" #include <cassert> #include <cstddef> #include <iterator> #include <limits> #include <string> #include <utility> #include <vector> " ***
*** Unparsable: " namespace clang {" ***
*** Unparsable: " using DeclarationMatcher = internal::Matcher<Decl>;" ***
*** Unparsable: " using StatementMatcher = internal::Matcher<Stmt>;" ***
*** Unparsable: " using TypeMatcher = internal::Matcher<QualType>;" ***
*** Unparsable: " using TypeLocMatcher = internal::Matcher<TypeLoc>;" ***
*** Unparsable: " using NestedNameSpecifierMatcher = internal::Matcher<NestedNameSpecifier>;" ***
*** Unparsable: " using NestedNameSpecifierLocMatcher = internal::Matcher<NestedNameSpecifierLoc>;" ***
*** Unparsable: " using CXXCtorInitializerMatcher = internal::Matcher<CXXCtorInitializer>;" ***
Probing https://clang.llvm.org/doxygen/classclang_1_1Decl.html...

after this the script keep probing with no issues.
i tested using python 2.7 and python 3.7 the same error occurs

[aaron.ballman]

In D61552#1550080, @Tyker wrote:

fixed requested changes.

Are you getting errors from running it, or just incorrect output?

the issue happens to me even on master so i suppose the input is correct.
here is the error report:

[tyker@tyker tools]$ ./dump_ast_matchers.py 
*** Unparsable: " #ifndef LLVM_CLANG_ASTMATCHERS_ASTMATCHERS_H #define LLVM_CLANG_ASTMATCHERS_ASTMATCHERS_H " ***
*** Unparsable: " #include "clang/AST/ASTContext.h" #include "clang/AST/ASTTypeTraits.h" #include "clang/AST/Attr.h" #include "clang/AST/Decl.h" #include "clang/AST/DeclCXX.h" #include "clang/AST/DeclFriend.h" #include "clang/AST/DeclObjC.h" #include "clang/AST/DeclTemplate.h" #include "clang/AST/Expr.h" #include "clang/AST/ExprCXX.h" #include "clang/AST/ExprObjC.h" #include "clang/AST/NestedNameSpecifier.h" #include "clang/AST/OpenMPClause.h" #include "clang/AST/OperationKinds.h" #include "clang/AST/Stmt.h" #include "clang/AST/StmtCXX.h" #include "clang/AST/StmtObjC.h" #include "clang/AST/StmtOpenMP.h" #include "clang/AST/TemplateBase.h" #include "clang/AST/TemplateName.h" #include "clang/AST/Type.h" #include "clang/AST/TypeLoc.h" #include "clang/ASTMatchers/ASTMatchersInternal.h" #include "clang/ASTMatchers/ASTMatchersMacros.h" #include "clang/Basic/AttrKinds.h" #include "clang/Basic/ExceptionSpecificationType.h" #include "clang/Basic/IdentifierTable.h" #include "clang/Basic/LLVM.h" #include "clang/Basic/SourceManager.h" #include "clang/Basic/Specifiers.h" #include "clang/Basic/TypeTraits.h" #include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/SmallVector.h" #include "llvm/ADT/StringRef.h" #include "llvm/Support/Casting.h" #include "llvm/Support/Compiler.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/Regex.h" #include <cassert> #include <cstddef> #include <iterator> #include <limits> #include <string> #include <utility> #include <vector> " ***
*** Unparsable: " namespace clang {" ***
*** Unparsable: " using DeclarationMatcher = internal::Matcher<Decl>;" ***
*** Unparsable: " using StatementMatcher = internal::Matcher<Stmt>;" ***
*** Unparsable: " using TypeMatcher = internal::Matcher<QualType>;" ***
*** Unparsable: " using TypeLocMatcher = internal::Matcher<TypeLoc>;" ***
*** Unparsable: " using NestedNameSpecifierMatcher = internal::Matcher<NestedNameSpecifier>;" ***
*** Unparsable: " using NestedNameSpecifierLocMatcher = internal::Matcher<NestedNameSpecifierLoc>;" ***
*** Unparsable: " using CXXCtorInitializerMatcher = internal::Matcher<CXXCtorInitializer>;" ***
Probing https://clang.llvm.org/doxygen/classclang_1_1Decl.html...

after this the script keep probing with no issues.
i tested using python 2.7 and python 3.7 the same error occurs

That output matches what I get, and my docs generate for me. Are you sure the .html file is not being generated properly for you?

[Tyker]

i was confuse because the log seemed like an error and it was happening in python2.7 and python3.7
but the actual error that was preventing generating the documentation only occurs in 3.7.

it works in python2.7.
sorry for my confusion.

clang/include/clang/ASTMatchers/ASTMatchers.h

6190

#2 matches.

[aaron.ballman]

LGTM aside from a documentation nit.

clang/include/clang/ASTMatchers/ASTMatchers.h

6190

I figured it would -- the comment should be updated to correct that (and then generate the docs again).

[Tyker]

clang/include/clang/ASTMatchers/ASTMatchers.h

6190

but the comment says it matches

cxxConstructorDecl(isExplicit()) will match #2, but not #1.

[aaron.ballman]

clang/include/clang/ASTMatchers/ASTMatchers.h

6190

Because this comment says it doesn't:

/// cxxConstructorDecl(isExplicit()) will match #8, but not #7 or #9.

I see now that there are two different comments that say different and incorrect things. I'd say they should be combined into one correct comment.

clang/include/clang/ASTMatchers/ASTMatchers.h

 ///
 /// Example matches the operator.
 /// \code
 ///   class X { operator int() const; };
 /// \endcode
 extern const internal::VariadicDynCastAllOfMatcher<Decl, CXXConversionDecl>
     cxxConversionDecl;
 
+/// Matches deduction guide declarations.
+///
+/// Example matches the deduction guide.
+/// \code
+///   template<typename T>
+///   class X { X(int) };
+///   X(int) -> X<int>;
+/// \endcode
+extern const internal::VariadicDynCastAllOfMatcher<Decl, CXXDeductionGuideDecl>
+    cxxDeductionGuideDecl;
+
 /// Matches variable declarations.
 ///
 /// Note: this does not match declarations of member variables, which are
 /// "field" declarations in Clang parlance.
 ///
 /// Example matches a
 /// \code
 ///   int a;

 ///   S::S() : S(0) {} // #4
 /// \endcode
 /// cxxConstructorDecl(isDelegatingConstructor()) will match #3 and #4, but not
 /// #1 or #2.
 AST_MATCHER(CXXConstructorDecl, isDelegatingConstructor) {
   return Node.isDelegatingConstructor();
 }
 
-/// Matches constructor and conversion declarations that are marked with
-/// the explicit keyword.
+/// Matches constructor, conversion function and deduction guide declarations
+/// that have an explicit specifier if this explicit specifier is resolved to
+/// true.
 ///
 /// Given
 /// \code
+///   template<bool b>
 ///   struct S {
 ///     S(int); // #1
 ///     explicit S(double); // #2
 ///     operator int(); // #3
 ///     explicit operator bool(); // #4
+///     explicit(false) S(bool) // # 7
+///     explicit(true) S(char) // # 8
+///     explicit(b) S(S) // # 9
 ///   };
+///   S(int) -> S<true> // #5
+///   explicit S(double) -> S<false> // #6
 /// \endcode
 /// cxxConstructorDecl(isExplicit()) will match #2, but not #1.
 /// cxxConversionDecl(isExplicit()) will match #4, but not #3.
-AST_POLYMORPHIC_MATCHER(isExplicit,
-                        AST_POLYMORPHIC_SUPPORTED_TYPES(CXXConstructorDecl,
-                                                        CXXConversionDecl)) {
-  // FIXME : it's not clear whether this should match a dependent
-  //         explicit(....). this matcher should also be able to match
-  //         CXXDeductionGuideDecl with explicit specifier.
+/// cxxDeductionGuideDecl(isExplicit()) will match #6, but not #5.
+/// cxxConstructorDecl(isExplicit()) will match #8, but not #7 or #9.
+AST_POLYMORPHIC_MATCHER(isExplicit, AST_POLYMORPHIC_SUPPORTED_TYPES(
+                                        CXXConstructorDecl, CXXConversionDecl,
+                                        CXXDeductionGuideDecl)) {
   return Node.isExplicit();
 }
 
+/// Matches constructor, conversion function and deduction guide declarations
+/// that have an explicit specifier.
+///
+/// Given
+/// \code
+///   template<bool b>
+///   struct S {
+///     S(int); // #1
+///     explicit S(double); // #2
+///     operator int(); // #3
+///     explicit operator bool(); // #4
+///     explicit(false) S(bool) // # 7
+///     explicit(true) S(char) // # 8
+///     explicit(b) S(S) // # 9
+///   };
+///   S(int) -> S<true> // #5
+///   explicit S(double) -> S<false> // #6
+/// \endcode
+/// cxxConstructorDecl(isExplicit()) will match #2, #7, #8 and #9, but not #1.
+/// cxxConversionDecl(isExplicit()) will match #4, but not #3.
+/// cxxDeductionGuideDecl(isExplicit()) will match #6, but not #5.
+AST_POLYMORPHIC_MATCHER(
+    hasExplicitSpecifier,
+    AST_POLYMORPHIC_SUPPORTED_TYPES(CXXConstructorDecl, CXXConversionDecl,
+                                    CXXDeductionGuideDecl)) {
+  return Node.getExplicitSpecifier().isSpecified();
+}
+
 /// Matches function and namespace declarations that are marked with
 /// the inline keyword.
 ///
 /// Given
 /// \code
 ///   inline void f();
 ///   void g();
 ///   namespace n {

clang/lib/ASTMatchers/Dynamic/Registry.cpp

RegistryMaps::RegistryMaps() {
   REGISTER_MATCHER(complexType);
   REGISTER_MATCHER(compoundLiteralExpr);
   REGISTER_MATCHER(compoundStmt);
   REGISTER_MATCHER(conditionalOperator);
   REGISTER_MATCHER(constantArrayType);
   REGISTER_MATCHER(constantExpr);
   REGISTER_MATCHER(containsDeclaration);
   REGISTER_MATCHER(continueStmt);
   REGISTER_MATCHER(cudaKernelCallExpr);
   REGISTER_MATCHER(cxxBindTemporaryExpr);
   REGISTER_MATCHER(cxxBoolLiteral);
   REGISTER_MATCHER(cxxCatchStmt);
   REGISTER_MATCHER(cxxConstCastExpr);
   REGISTER_MATCHER(cxxConstructExpr);
   REGISTER_MATCHER(cxxConstructorDecl);
   REGISTER_MATCHER(cxxConversionDecl);
+  REGISTER_MATCHER(cxxDeductionGuideDecl);
   REGISTER_MATCHER(cxxCtorInitializer);
   REGISTER_MATCHER(cxxDefaultArgExpr);
   REGISTER_MATCHER(cxxDeleteExpr);
   REGISTER_MATCHER(cxxDependentScopeMemberExpr);
   REGISTER_MATCHER(cxxDestructorDecl);
   REGISTER_MATCHER(cxxDynamicCastExpr);
   REGISTER_MATCHER(cxxForRangeStmt);
   REGISTER_MATCHER(cxxFunctionalCastExpr);
RegistryMaps::RegistryMaps() {
   REGISTER_MATCHER(hasCaseConstant);
   REGISTER_MATCHER(hasCastKind);
   REGISTER_MATCHER(hasCondition);
   REGISTER_MATCHER(hasConditionVariableStatement);
   REGISTER_MATCHER(hasDecayedType);
   REGISTER_MATCHER(hasDeclContext);
   REGISTER_MATCHER(hasDeclaration);
   REGISTER_MATCHER(hasDeducedType);
   REGISTER_MATCHER(hasDefaultArgument);
   REGISTER_MATCHER(hasDefinition);
   REGISTER_MATCHER(hasDescendant);
   REGISTER_MATCHER(hasDestinationType);
   REGISTER_MATCHER(hasDynamicExceptionSpec);
   REGISTER_MATCHER(hasEitherOperand);
   REGISTER_MATCHER(hasElementType);
   REGISTER_MATCHER(hasElse);
   REGISTER_MATCHER(hasExternalFormalLinkage);

clang/unittests/ASTMatchers/ASTMatchersNarrowingTest.cpp

EXPECT_TRUE(
                  unresolvedMemberExpr(isArrow())));
 }
 
 TEST(ConversionDeclaration, IsExplicit) {
   EXPECT_TRUE(matches("struct S { explicit operator int(); };",
                       cxxConversionDecl(isExplicit())));
   EXPECT_TRUE(notMatches("struct S { operator int(); };",
                          cxxConversionDecl(isExplicit())));
+  EXPECT_TRUE(matchesConditionally(
+      "template<bool b> struct S { explicit(b) operator int(); };",
+      cxxConversionDecl(isExplicit()), false, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally(
+      "struct S { explicit(true) operator int(); };",
+      cxxConversionDecl(isExplicit()), true, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally(
+      "struct S { explicit(false) operator int(); };",
+      cxxConversionDecl(isExplicit()), false, "-std=c++2a"));
+}
+
+TEST(ConversionDeclaration, HasExplicitSpecifier) {
+  EXPECT_TRUE(matches("struct S { explicit operator int(); };",
+                      cxxConversionDecl(hasExplicitSpecifier())));
+  EXPECT_TRUE(notMatches("struct S { operator int(); };",
+                         cxxConversionDecl(hasExplicitSpecifier())));
+  EXPECT_TRUE(matchesConditionally(
+      "template<bool b> struct S { explicit(b) operator int(); };",
+      cxxConversionDecl(hasExplicitSpecifier()), true, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally(
+      "struct S { explicit(true) operator int(); };",
+      cxxConversionDecl(hasExplicitSpecifier()), true, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally(
+      "struct S { explicit(false) operator int(); };",
+      cxxConversionDecl(hasExplicitSpecifier()), true, "-std=c++2a"));
 }
 
 TEST(Matcher, ArgumentCount) {
   StatementMatcher Call1Arg = callExpr(argumentCountIs(1));
 
   EXPECT_TRUE(matches("void x(int) { x(0); }", Call1Arg));
   EXPECT_TRUE(matches("class X { void x(int) { x(0); } };", Call1Arg));
   EXPECT_TRUE(notMatches("void x(int, int) { x(0, 0); }", Call1Arg));
EXPECT_TRUE(matches("struct A { int x; } a = {0}, b = a; void f() { a = b; }",
                       cxxMethodDecl(isImplicit(), hasName("operator="))));
 }
 
 TEST(ConstructorDeclaration, IsExplicit) {
   EXPECT_TRUE(matches("struct S { explicit S(int); };",
                       cxxConstructorDecl(isExplicit())));
   EXPECT_TRUE(notMatches("struct S { S(int); };",
                          cxxConstructorDecl(isExplicit())));
+  EXPECT_TRUE(matchesConditionally(
+      "template<bool b> struct S { explicit(b) S(int);};",
+      cxxConstructorDecl(isExplicit()), false, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("struct S { explicit(true) S(int);};",
+                                   cxxConstructorDecl(isExplicit()), true,
+                                   "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("struct S { explicit(false) S(int);};",
+                                   cxxConstructorDecl(isExplicit()), false,
+                                   "-std=c++2a"));
+}
+
+TEST(ConstructorDeclaration, HasExplicitSpecifier) {
+  EXPECT_TRUE(matches("struct S { explicit S(int); };",
+                      cxxConstructorDecl(hasExplicitSpecifier())));
+  EXPECT_TRUE(notMatches("struct S { S(int); };",
+                         cxxConstructorDecl(hasExplicitSpecifier())));
+  EXPECT_TRUE(matchesConditionally(
+      "template<bool b> struct S { explicit(b) S(int);};",
+      cxxConstructorDecl(hasExplicitSpecifier()), true, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("struct S { explicit(true) S(int);};",
+                                   cxxConstructorDecl(hasExplicitSpecifier()),
+                                   true, "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("struct S { explicit(false) S(int);};",
+                                   cxxConstructorDecl(hasExplicitSpecifier()),
+                                   true, "-std=c++2a"));
+}
+
+TEST(DeductionGuideDeclaration, IsExplicit) {
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(isExplicit()), false,
+                                   "-std=c++17"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "explicit S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(isExplicit()), true,
+                                   "-std=c++17"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "explicit(true) S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(isExplicit()), true,
+                                   "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "explicit(false) S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(isExplicit()), false,
+                                   "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "template<bool b = true> explicit(b) S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(isExplicit()), false,
+                                   "-std=c++2a"));
+}
+
+TEST(DeductionGuideDeclaration, HasExplicitSpecifier) {
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(hasExplicitSpecifier()), false,
+                                   "-std=c++17"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "explicit S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(hasExplicitSpecifier()), true,
+                                   "-std=c++17"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "explicit(true) S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(hasExplicitSpecifier()), true,
+                                   "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "explicit(false) S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(hasExplicitSpecifier()), true,
+                                   "-std=c++2a"));
+  EXPECT_TRUE(matchesConditionally("template<typename T> struct S { S(int);};"
+                                   "template<bool b = true> explicit(b) S(int) -> S<int>;",
+                                   cxxDeductionGuideDecl(hasExplicitSpecifier()), true,
+                                   "-std=c++2a"));
 }
 
 TEST(ConstructorDeclaration, Kinds) {
   EXPECT_TRUE(matches(
       "struct S { S(); };",
       cxxConstructorDecl(isDefaultConstructor(), unless(isImplicit()))));
   EXPECT_TRUE(notMatches(
       "struct S { S(); };",