[ASTMatchers] Add binaryOperation matcher

[steveire]

Summary

This is a simple utility which allows matching on binaryOperator and
cxxOperatorCallExpr. It can also be extended to support
cxxRewrittenBinaryOperator.

[aaron.ballman]

clang/include/clang/ASTMatchers/ASTMatchers.h

2735

2781

Missing changes to Registry.cpp to expose this to clang-query?

[steveire]

Update

[aaron.ballman]

LGTM aside from some minor NFC improvements.

clang/lib/ASTMatchers/Dynamic/Marshallers.h

957

964–968

clang/docs/LibASTMatchersReference.html

 </pre></td></tr>
 
 
 <tr><td>Matcher&lt;<a href="https://clang.llvm.org/doxygen/classclang_1_1Stmt.html">Stmt</a>&gt;</td><td class="name" onclick="toggle('binaryOperator0')"><a name="binaryOperator0Anchor">binaryOperator</a></td><td>Matcher&lt;<a href="https://clang.llvm.org/doxygen/classclang_1_1BinaryOperator.html">BinaryOperator</a>&gt;...</td></tr>
 <tr><td colspan="4" class="doc" id="binaryOperator0"><pre>Matches binary operator expressions.
 
 Example matches a || b
   !(a || b)
+See also the binaryOperation() matcher for more-general matching.
 </pre></td></tr>
 
 
 <tr><td>Matcher&lt;<a href="https://clang.llvm.org/doxygen/classclang_1_1Stmt.html">Stmt</a>&gt;</td><td class="name" onclick="toggle('blockExpr0')"><a name="blockExpr0Anchor">blockExpr</a></td><td>Matcher&lt;<a href="https://clang.llvm.org/doxygen/classclang_1_1BlockExpr.html">BlockExpr</a>&gt;...</td></tr>
 <tr><td colspan="4" class="doc" id="blockExpr0"><pre>Matches a reference to a block.
 
 Example: matches "^{}":
   void f() { ^{}(); }

 Currently it does not match operators such as new delete.
 FIXME: figure out why these do not match?
 
 Example matches both operator&lt;&lt;((o &lt;&lt; b), c) and operator&lt;&lt;(o, b)
     (matcher = cxxOperatorCallExpr())
   ostream &amp;operator&lt;&lt; (ostream &amp;out, int i) { };
   ostream &amp;o; int b = 1, c = 1;
   o &lt;&lt; b &lt;&lt; c;
+See also the binaryOperation() matcher for more-general matching of binary
+uses of this AST node.
 </pre></td></tr>
 
 
 <tr><td>Matcher&lt;<a href="https://clang.llvm.org/doxygen/classclang_1_1Stmt.html">Stmt</a>&gt;</td><td class="name" onclick="toggle('cxxReinterpretCastExpr0')"><a name="cxxReinterpretCastExpr0Anchor">cxxReinterpretCastExpr</a></td><td>Matcher&lt;<a href="https://clang.llvm.org/doxygen/classclang_1_1CXXReinterpretCastExpr.html">CXXReinterpretCastExpr</a>&gt;...</td></tr>
 <tr><td colspan="4" class="doc" id="cxxReinterpretCastExpr0"><pre>Matches a reinterpret_cast expression.
 
 Either the source expression or the destination type can be matched
 using has(), but hasDestinationType() is more specific and can be

 <p>Note that there are special traversal matchers (has, hasDescendant, forEach and
 forEachDescendant) which work on all nodes and allow users to write more generic
 match expressions.</p>
 
 <table>
 <tr style="text-align:left"><th>Return type</th><th>Name</th><th>Parameters</th></tr>
 <!-- START_TRAVERSAL_MATCHERS -->
 
+<tr><td>Matcher&lt;*&gt;</td><td class="name" onclick="toggle('binaryOperation0')"><a name="binaryOperation0Anchor">binaryOperation</a></td><td>Matcher&lt;*&gt;...Matcher&lt;*&gt;</td></tr>
+<tr><td colspan="4" class="doc" id="binaryOperation0"><pre>Matches nodes which can be used with binary operators.
+
+The code
+  var1 != var2;
+might be represented in the clang AST as a binaryOperator or a
+cxxOperatorCallExpr, depending on
+
+* whether the types of var1 and var2 are fundamental (binaryOperator) or at
+  least one is a class type (cxxOperatorCallExpr)
+* whether the code appears in a template declaration, if at least one of the
+  vars is a dependent-type (binaryOperator)
+
+This matcher elides details in places where the matchers for the nodes are
+compatible.
+
+Given
+  binaryOperation(
+    hasOperatorName("!="),
+    hasLHS(expr().bind("lhs")),
+    hasRHS(expr().bind("rhs"))
+  )
+matches each use of "!=" in:
+  struct S{
+      bool operator!=(const S&amp;) const;
+  };
+
+  void foo()
+  {
+     1 != 2;
+     S() != S();
+  }
+
+  template&lt;typename T&gt;
+  void templ()
+  {
+     1 != 2;
+     T() != S();
+  }
+</pre></td></tr>
+
+
 <tr><td>Matcher&lt;*&gt;</td><td class="name" onclick="toggle('eachOf0')"><a name="eachOf0Anchor">eachOf</a></td><td>Matcher&lt;*&gt;, ..., Matcher&lt;*&gt;</td></tr>
 <tr><td colspan="4" class="doc" id="eachOf0"><pre>Matches if any of the given matchers matches.
 
 Unlike anyOf, eachOf will generate a match result for each
 matching submatcher.
 
 For example, in:
   class A { int a; int b; };

clang/docs/tools/dump_ast_matchers.py

if m:
       min_args, max_args, name = m.groups()[:3]
       if max_args == '1':
         add_matcher('*', name, 'Matcher<*>', comment)
         return
       elif max_args == 'std::numeric_limits<unsigned>::max()':
         add_matcher('*', name, 'Matcher<*>, ..., Matcher<*>', comment)
         return
 
+    m = re.match(
+        r"""^.*MapAnyOfMatcher<.*>\s*
+              ([a-zA-Z]*);$""",
+        declaration, flags=re.X)
+    if m:
+      name = m.groups()[0]
+      add_matcher('*', name, 'Matcher<*>...Matcher<*>', comment)
+      return
 
     # Parse free standing matcher functions, like:
     #   Matcher<ResultType> Name(Matcher<ArgumentType> InnerMatcher) {
     m = re.match(r"""^\s*(?:template\s+<\s*(?:class|typename)\s+(.+)\s*>\s+)?   
                      (.*)\s+
                      ([^\s\(]+)\s*\(
                      (.*)
                      \)\s*{""", declaration, re.X)

clang/include/clang/ASTMatchers/ASTMatchers.h

 ///
 /// Example matches both operator<<((o << b), c) and operator<<(o, b)
 ///     (matcher = cxxOperatorCallExpr())
 /// \code
 ///   ostream &operator<< (ostream &out, int i) { };
 ///   ostream &o; int b = 1, c = 1;
 ///   o << b << c;
 /// \endcode
+/// See also the binaryOperation() matcher for more-general matching of binary
+/// uses of this AST node.
 extern const internal::VariadicDynCastAllOfMatcher<Stmt, CXXOperatorCallExpr>
     cxxOperatorCallExpr;
 
 /// Matches expressions.
 ///
 /// Example matches x()
 /// \code
 ///   void f() { x(); }

 extern const internal::VariadicDynCastAllOfMatcher<Stmt, StmtExpr> stmtExpr;
 
 /// Matches binary operator expressions.
 ///
 /// Example matches a || b
 /// \code
 ///   !(a || b)
 /// \endcode
+/// See also the binaryOperation() matcher for more-general matching.
 extern const internal::VariadicDynCastAllOfMatcher<Stmt, BinaryOperator>
     binaryOperator;
 
 /// Matches unary operator expressions.
 ///
 /// Example matches !a
 /// \code
 ///   !a || b

 /// The with() chain-call accepts zero or more matchers which are combined
 /// as-if with allOf() in each of the node matchers.
 /// Usable as: Any Matcher
 template <typename T, typename... U>
 auto mapAnyOf(internal::VariadicDynCastAllOfMatcher<T, U> const &...) {
   return internal::MapAnyOfHelper<U...>();
 }
 
+/// Matches nodes which can be used with binary operators.
+///
+/// The code
+/// \code
+///   var1 != var2;
+/// \endcode
+/// might be represented in the clang AST as a binaryOperator or a
+/// cxxOperatorCallExpr, depending on
+///
+/// * whether the types of var1 and var2 are fundamental (binaryOperator) or at
+///   least one is a class type (cxxOperatorCallExpr)
+/// * whether the code appears in a template declaration, if at least one of the
+///   vars is a dependent-type (binaryOperator)
+///
+/// This matcher elides details in places where the matchers for the nodes are
+/// compatible.
+///
+/// Given
+/// \code
+///   binaryOperation(
+///     hasOperatorName("!="),
+///     hasLHS(expr().bind("lhs")),
+///     hasRHS(expr().bind("rhs"))
+///   )
+/// \endcode
+/// matches each use of "!=" in:
+/// \code
+///   struct S{
+///       bool operator!=(const S&) const;
+///   };
+///
+///   void foo()
+///   {
+///      1 != 2;
+///      S() != S();
+///   }
+///
+///   template<typename T>
+///   void templ()
+///   {
+///      1 != 2;
+///      T() != S();
+///   }
+/// \endcode
+extern const internal::MapAnyOfMatcher<BinaryOperator, CXXOperatorCallExpr>
+    binaryOperation;
+
 /// Matches unary expressions that have a specific type of argument.
 ///
 /// Given
 /// \code
 ///   int a, c; float b; int s = sizeof(a) + sizeof(b) + alignof(c);
 /// \endcode
 /// unaryExprOrTypeTraitExpr(hasArgumentOfType(asString("int"))
 ///   matches \c sizeof(a) and \c alignof(c)

clang/lib/ASTMatchers/ASTMatchersInternal.cpp

 const internal::VariadicDynCastAllOfMatcher<Stmt, ChooseExpr> chooseExpr;
 const internal::VariadicDynCastAllOfMatcher<Stmt, GNUNullExpr> gnuNullExpr;
 const internal::VariadicDynCastAllOfMatcher<Stmt, GenericSelectionExpr>
     genericSelectionExpr;
 const internal::VariadicDynCastAllOfMatcher<Stmt, AtomicExpr> atomicExpr;
 const internal::VariadicDynCastAllOfMatcher<Stmt, StmtExpr> stmtExpr;
 const internal::VariadicDynCastAllOfMatcher<Stmt, BinaryOperator>
     binaryOperator;
+const internal::MapAnyOfMatcher<BinaryOperator, CXXOperatorCallExpr>
+    binaryOperation;
 const internal::VariadicDynCastAllOfMatcher<Stmt, UnaryOperator> unaryOperator;
 const internal::VariadicDynCastAllOfMatcher<Stmt, ConditionalOperator>
     conditionalOperator;
 const internal::VariadicDynCastAllOfMatcher<Stmt, BinaryConditionalOperator>
     binaryConditionalOperator;
 const internal::VariadicDynCastAllOfMatcher<Stmt, OpaqueValueExpr>
     opaqueValueExpr;
 const internal::VariadicDynCastAllOfMatcher<Decl, StaticAssertDecl>

clang/lib/ASTMatchers/Dynamic/Marshallers.h

 
 private:
   const unsigned MinCount;
   const unsigned MaxCount;
   const VarOp Op;
   const StringRef MatcherName;
 };
 
+template <typename CladeType, typename... MatcherT>
+class MapAnyOfMatcherDescriptor : public MatcherDescriptor {
+  std::vector<DynCastAllOfMatcherDescriptor> Funcs;
+
+public:
+  MapAnyOfMatcherDescriptor(StringRef MatcherName)
+      : Funcs{DynCastAllOfMatcherDescriptor(
+            ast_matchers::internal::VariadicDynCastAllOfMatcher<CladeType,
+                                                                MatcherT>{},
+            MatcherName)...} {}
+
+  VariantMatcher create(SourceRange NameRange, ArrayRef<ParserValue> Args,
+                        Diagnostics *Error) const override {
+    std::vector<VariantMatcher> InnerArgs;
+
+    for (auto const &F : Funcs) {
+      InnerArgs.push_back(F.create(NameRange, Args, Error));
+      if (!Error->errors().empty())
+        return {};
+    }
+    return VariantMatcher::SingleMatcher(
+        ast_matchers::internal::BindableMatcher<CladeType>(
+            VariantMatcher::VariadicOperatorMatcher(
+                ast_matchers::internal::DynTypedMatcher::VO_AnyOf,
+                std::move(InnerArgs))
+                .getTypedMatcher<CladeType>()));
+  }
+
+  bool isVariadic() const override { return true; }
+  unsigned getNumArgs() const override { return 0; }
+
+  void getArgKinds(ASTNodeKind ThisKind, unsigned,
+                   std::vector<ArgKind> &Kinds) const override {
+    Kinds.push_back(ThisKind);
+  }
+
+  bool isConvertibleTo(ASTNodeKind Kind, unsigned *Specificity,
+                       ASTNodeKind *LeastDerivedKind) const override {
+    return llvm::all_of(Funcs, [=](const auto &F) {
+      return F.isConvertibleTo(Kind, Specificity, LeastDerivedKind);
+    });
+  }
+};
+
 /// Helper functions to select the appropriate marshaller functions.
 /// They detect the number of arguments, arguments types and return type.
 
 /// 0-arg overload
 template <typename ReturnType>
 std::unique_ptr<MatcherDescriptor>
 makeMatcherAutoMarshall(ReturnType (*Func)(), StringRef MatcherName) {
   std::vector<ASTNodeKind> RetTypes;

 std::unique_ptr<MatcherDescriptor> makeMatcherAutoMarshall(
     ast_matchers::internal::VariadicOperatorMatcherFunc<MinCount, MaxCount>
         Func,
     StringRef MatcherName) {
   return std::make_unique<VariadicOperatorMatcherDescriptor>(
       MinCount, MaxCount, Func.Op, MatcherName);
 }
 
+template <typename CladeType, typename... MatcherT>
+std::unique_ptr<MatcherDescriptor> makeMatcherAutoMarshall(
+    ast_matchers::internal::MapAnyOfMatcherImpl<CladeType, MatcherT...>,
+    StringRef MatcherName) {
+  return std::make_unique<MapAnyOfMatcherDescriptor<CladeType, MatcherT...>>(
+      MatcherName);
+}
+
 } // namespace internal
 } // namespace dynamic
 } // namespace ast_matchers
 } // namespace clang
 
 #endif // LLVM_CLANG_AST_MATCHERS_DYNAMIC_MARSHALLERS_H

clang/lib/ASTMatchers/Dynamic/Registry.cpp

RegistryMaps::RegistryMaps() {
   REGISTER_MATCHER(asString);
   REGISTER_MATCHER(asmStmt);
   REGISTER_MATCHER(atomicExpr);
   REGISTER_MATCHER(atomicType);
   REGISTER_MATCHER(autoType);
   REGISTER_MATCHER(autoreleasePoolStmt)
   REGISTER_MATCHER(binaryConditionalOperator);
   REGISTER_MATCHER(binaryOperator);
+  REGISTER_MATCHER(binaryOperation);
   REGISTER_MATCHER(blockDecl);
   REGISTER_MATCHER(blockExpr);
   REGISTER_MATCHER(blockPointerType);
   REGISTER_MATCHER(booleanType);
   REGISTER_MATCHER(breakStmt);
   REGISTER_MATCHER(builtinType);
   REGISTER_MATCHER(cStyleCastExpr);
   REGISTER_MATCHER(callExpr);

clang/unittests/ASTMatchers/ASTMatchersNarrowingTest.cpp

EXPECT_TRUE(
               cxxCatchStmt(anyOf(hasDescendant(varDecl()), isCatchAll()))));
 }
 
 TEST_P(ASTMatchersTest, MapAnyOf) {
   if (!GetParam().isCXX()) {
     return;
   }
 
+  if (GetParam().hasDelayedTemplateParsing()) {
+    return;
+  }
+
   StringRef Code = R"cpp(
 void F() {
   if (true) {}
   for ( ; false; ) {}
 }
 )cpp";
 
   auto trueExpr = cxxBoolLiteral(equals(true));

 
 void opFree()
 {
     HasOpFree s1;
     HasOpFree s2;
     if (s1 != s2)
         return;
 }
+
+template<typename T>
+void templ()
+{
+    T s1;
+    T s2;
+    if (s1 != s2)
+        return;
+}
 )cpp";
 
   EXPECT_TRUE(matches(
       Code,
       traverse(TK_IgnoreUnlessSpelledInSource,
                mapAnyOf(binaryOperator, cxxOperatorCallExpr)
                    .with(hasOperatorName("!="),
                          forFunction(functionDecl(hasName("binop"))),
EXPECT_TRUE(matches(
                                forFunction(functionDecl(hasName("opMem")))))));
 
   EXPECT_TRUE(matches(
       Code, traverse(TK_IgnoreUnlessSpelledInSource,
                      mapAnyOf(binaryOperator, cxxOperatorCallExpr)
                          .with(hasAnyOperatorName("==", "!="),
                                forFunction(functionDecl(hasName("opFree")))))));
 
+  EXPECT_TRUE(matches(
+      Code, traverse(TK_IgnoreUnlessSpelledInSource,
+                     binaryOperation(
+                         hasOperatorName("!="),
+                         forFunction(functionDecl(hasName("binop"))),
+                         hasLHS(declRefExpr(to(varDecl(hasName("s1"))))),
+                         hasRHS(declRefExpr(to(varDecl(hasName("s2")))))))));
+
+  EXPECT_TRUE(matches(
+      Code, traverse(TK_IgnoreUnlessSpelledInSource,
+                     binaryOperation(
+                         hasOperatorName("!="),
+                         forFunction(functionDecl(hasName("opMem"))),
+                         hasLHS(declRefExpr(to(varDecl(hasName("s1"))))),
+                         hasRHS(declRefExpr(to(varDecl(hasName("s2")))))))));
+
+  EXPECT_TRUE(matches(
+      Code, traverse(TK_IgnoreUnlessSpelledInSource,
+                     binaryOperation(
+                         hasOperatorName("!="),
+                         forFunction(functionDecl(hasName("opFree"))),
+                         hasLHS(declRefExpr(to(varDecl(hasName("s1"))))),
+                         hasRHS(declRefExpr(to(varDecl(hasName("s2")))))))));
+
+  EXPECT_TRUE(matches(
+      Code, traverse(TK_IgnoreUnlessSpelledInSource,
+                     binaryOperation(
+                         hasOperatorName("!="),
+                         forFunction(functionDecl(hasName("templ"))),
+                         hasLHS(declRefExpr(to(varDecl(hasName("s1"))))),
+                         hasRHS(declRefExpr(to(varDecl(hasName("s2")))))))));
+
   Code = R"cpp(
 struct HasOpBangMem
 {
     bool operator!() const
     {
         return false;
     }
 };