Support named values in the autocomplete feature.

[sbenza]

Summary

This includes:

• Passing a Sema to completeExpression to allow for named values in the expression.
• Update the Sema interface to include completion for matchers and values.
• Change the parser to use the Sema for completion, instead of going directly to Registry.

[pcc]

Apologies for not reviewing this earlier, it slipped through the cracks. In future feel free to ping if I haven't looked at a change within a week.

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

This approach requires a bunch of common logic to be duplicated between individual Sema implementations. Instead, you could have a virtual function in this class that returns (a const reference to) a map of named values, which would be called by this function and getNamedValue.

You might also consider changing the parser API to pass the map separately (along with Sema) to the parser. That way, you could also avoid complicating Sema too much as a result of having to allow completeExpression to take custom Semas.

lib/ASTMatchers/Dynamic/Parser.cpp

578 ↗

(On Diff #8839)

Nit: specificity.

unittests/ASTMatchers/Dynamic/RegistryTest.cpp

456 ↗

(On Diff #8839)

Instead of removing these checks, you could probably change them to instead check the relative specificity of the results.

[sbenza]

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

Having this in the Sema object is following the same logic as having the matchers.
It allows an implementation of it to do whatever it wants. It could be backed by an in-memory std::map, be generated on demand, or by some other source.
Earlier, I added 'RegistrySema' because it was common to use that one.
I was thinking in providing a MappedValueSema, and make both of these mix-ins that the user can add to their implementations.

wrt duplicated logic, I added most of the required logic in the support classes (VariantValue, etc). The implementation of this method of a backing string->value map is just a few lines.

[pcc]

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

I realize why you added this flexibility, and I sort of see the value in it, but I'm not convinced that it is necessary.

If I were designing this, I would make the std::map part of the interface to the parser, and defer the design of any additional flexibility for when it is needed, so that I'd have something concrete to base my design on. For example, it could be done incrementally on top of the std::map design by adding something to Sema that lets a client augment the contents of the map. I won't insist on you changing your current design though.

[sbenza]

Reworked the changelist to pass a map of values instead of using the Sema for it.

[sbenza]

Sorry for the very long delay.
I applied your suggestions.

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

Followed your suggestion.
To make it consistent I had to remove the getNamedValue() method also. This will break clang-query temporarily.

lib/ASTMatchers/Dynamic/Parser.cpp

578 ↗

(On Diff #8839)

Fixed

[sbenza]

Print the name in the "declaration" to show it on the autocomplete suggestion.

[pcc]

Thank you, this looks close.

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

It looks like it should be easy to fix clang-query. Can you please also upload a diff for clang-tools-extra?

lib/ASTMatchers/Dynamic/Parser.cpp

457 ↗

(On Diff #12248)

Shouldn't this require updates to the unit tests? If not, why not?

[sbenza]

Fix the tests to pass with the new behavior of completion logic.

[sbenza]

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

It is easy to fix. I have another change for that, but I am not sure if I can send them atomically.
Aren't they in different repos?

lib/ASTMatchers/Dynamic/Parser.cpp

457 ↗

(On Diff #12248)

Yes. It does. Apparently I ran the wrong tests before.

[pcc]

include/clang/ASTMatchers/Dynamic/Parser.h

86 ↗

(On Diff #8839)

You can just create another differential revision.

[sbenza]

The change in clang-query in diff D4851

[pcc]

LGTM

Please remember to commit D4851 at the same time.

[sbenza]

Closed by commit rL215472 (authored by @sbenza).

cfe/trunk/include/clang/ASTMatchers/Dynamic/Parser.h

public:
   /// However, a more complex processor might decide to intercept the matcher
   /// creation and do some extra work. For example, it could apply some
   /// transformation to the matcher by adding some id() nodes, or could detect
   /// specific matcher nodes for more efficient lookup.
   class Sema {
   public:
     virtual ~Sema();
 
-    /// \brief Lookup a value by name.
-    ///
-    /// This can be used in the Sema layer to declare known constants or to
-    /// allow to split an expression in pieces.
-    ///
-    /// \param Name The name of the value to lookup.
-    ///
-    /// \return The named value. It could be any type that VariantValue
-    ///   supports. An empty value means that the name is not recognized.
-    virtual VariantValue getNamedValue(StringRef Name);
-
     /// \brief Process a matcher expression.
     ///
     /// All the arguments passed here have already been processed.
     ///
     /// \param Ctor A matcher constructor looked up by lookupMatcherCtor.
     ///
     /// \param NameRange The location of the name in the matcher source.
     ///   Useful for error reporting.
public:
     /// \brief Look up a matcher by name.
     ///
     /// \param MatcherName The matcher name found by the parser.
     ///
     /// \return The matcher constructor, or Optional<MatcherCtor>() if not
     /// found.
     virtual llvm::Optional<MatcherCtor>
     lookupMatcherCtor(StringRef MatcherName) = 0;
+
+    /// \brief Compute the list of completion types for \p Context.
+    ///
+    /// Each element of \p Context represents a matcher invocation, going from
+    /// outermost to innermost. Elements are pairs consisting of a reference to
+    /// the matcher constructor and the index of the next element in the
+    /// argument list of that matcher (or for the last element, the index of
+    /// the completion point in the argument list). An empty list requests
+    /// completion for the root matcher.
+    virtual std::vector<ArgKind> getAcceptedCompletionTypes(
+        llvm::ArrayRef<std::pair<MatcherCtor, unsigned>> Context);
+
+    /// \brief Compute the list of completions that match any of
+    /// \p AcceptedTypes.
+    ///
+    /// \param All types accepted for this completion.
+    ///
+    /// \return All completions for the specified types.
+    /// Completions should be valid when used in \c lookupMatcherCtor().
+    /// The matcher constructed from the return of \c lookupMatcherCtor()
+    /// should be convertible to some type in \p AcceptedTypes.
+    virtual std::vector<MatcherCompletion>
+    getMatcherCompletions(llvm::ArrayRef<ArgKind> AcceptedTypes);
   };
 
   /// \brief Sema implementation that uses the matcher registry to process the
   ///   tokens.
   class RegistrySema : public Parser::Sema {
    public:
     virtual ~RegistrySema();
 
     llvm::Optional<MatcherCtor>
     lookupMatcherCtor(StringRef MatcherName) override;
 
     VariantMatcher actOnMatcherExpression(MatcherCtor Ctor,
                                           const SourceRange &NameRange,
                                           StringRef BindID,
                                           ArrayRef<ParserValue> Args,
                                           Diagnostics *Error) override;
+
+    std::vector<ArgKind> getAcceptedCompletionTypes(
+        llvm::ArrayRef<std::pair<MatcherCtor, unsigned>> Context) override;
+
+    std::vector<MatcherCompletion>
+    getMatcherCompletions(llvm::ArrayRef<ArgKind> AcceptedTypes) override;
   };
 
-  /// \brief Parse a matcher expression, creating matchers from the registry.
-  ///
-  /// This overload creates matchers calling directly into the registry. If the
-  /// caller needs more control over how the matchers are created, then it can
-  /// use the overload below that takes a Sema.
-  ///
-  /// \param MatcherCode The matcher expression to parse.
-  ///
-  /// \return The matcher object constructed, or an empty Optional if an error
-  ///   occurred.
-  ///   In that case, \c Error will contain a description of the error.
-  ///   The caller takes ownership of the DynTypedMatcher object returned.
-  static llvm::Optional<DynTypedMatcher>
-  parseMatcherExpression(StringRef MatcherCode, Diagnostics *Error);
+  typedef llvm::StringMap<VariantValue> NamedValueMap;
 
   /// \brief Parse a matcher expression.
   ///
   /// \param MatcherCode The matcher expression to parse.
   ///
   /// \param S The Sema instance that will help the parser
-  ///   construct the matchers.
+  ///   construct the matchers. If null, it uses the default registry.
+  ///
+  /// \param NamedValues A map of precomputed named values.  This provides
+  ///   the dictionary for the <NamedValue> rule of the grammar.
+  ///   If null, it is ignored.
+  ///
   /// \return The matcher object constructed by the processor, or an empty
   ///   Optional if an error occurred. In that case, \c Error will contain a
   ///   description of the error.
   ///   The caller takes ownership of the DynTypedMatcher object returned.
   static llvm::Optional<DynTypedMatcher>
-  parseMatcherExpression(StringRef MatcherCode, Sema *S, Diagnostics *Error);
-
-  /// \brief Parse an expression, creating matchers from the registry.
-  ///
-  /// Parses any expression supported by this parser. In general, the
-  /// \c parseMatcherExpression function is a better approach to get a matcher
-  /// object.
-  static bool parseExpression(StringRef Code, VariantValue *Value,
-                              Diagnostics *Error);
+  parseMatcherExpression(StringRef MatcherCode, Sema *S,
+                         const NamedValueMap *NamedValues,
+  static llvm::Optional<DynTypedMatcher>
+  parseMatcherExpression(StringRef MatcherCode, Sema *S,
+                         Diagnostics *Error) {
+    return parseMatcherExpression(MatcherCode, S, nullptr, Error);
+  }
+  static llvm::Optional<DynTypedMatcher>
+  parseMatcherExpression(StringRef MatcherCode, Diagnostics *Error) {
+    return parseMatcherExpression(MatcherCode, nullptr, Error);
+  }
 
   /// \brief Parse an expression.
   ///
   /// Parses any expression supported by this parser. In general, the
   /// \c parseMatcherExpression function is a better approach to get a matcher
   /// object.
+  ///
+  /// \param S The Sema instance that will help the parser
+  ///   construct the matchers. If null, it uses the default registry.
+  ///
+  /// \param NamedValues A map of precomputed named values.  This provides
+  ///   the dictionary for the <NamedValue> rule of the grammar.
+  ///   If null, it is ignored.
   static bool parseExpression(StringRef Code, Sema *S,
+                              const NamedValueMap *NamedValues,
                               VariantValue *Value, Diagnostics *Error);
+  static bool parseExpression(StringRef Code, Sema *S,
+                              VariantValue *Value, Diagnostics *Error) {
+    return parseExpression(Code, S, nullptr, Value, Error);
+  }
+  static bool parseExpression(StringRef Code, VariantValue *Value,
+                              Diagnostics *Error) {
+    return parseExpression(Code, nullptr, Value, Error);
+  }
 
   /// \brief Complete an expression at the given offset.
   ///
+  /// \param S The Sema instance that will help the parser
+  ///   construct the matchers. If null, it uses the default registry.
+  ///
+  /// \param NamedValues A map of precomputed named values.  This provides
+  ///   the dictionary for the <NamedValue> rule of the grammar.
+  ///   If null, it is ignored.
+  ///
   /// \return The list of completions, which may be empty if there are no
   /// available completions or if an error occurred.
   static std::vector<MatcherCompletion>
-  completeExpression(StringRef Code, unsigned CompletionOffset);
+  completeExpression(StringRef Code, unsigned CompletionOffset, Sema *S,
+                     const NamedValueMap *NamedValues);
+  static std::vector<MatcherCompletion>
+  completeExpression(StringRef Code, unsigned CompletionOffset, Sema *S) {
+    return completeExpression(Code, CompletionOffset, S, nullptr);
+  }
+  static std::vector<MatcherCompletion>
+  completeExpression(StringRef Code, unsigned CompletionOffset) {
+    return completeExpression(Code, CompletionOffset, nullptr);
+  }
 
 private:
   class CodeTokenizer;
   struct ScopedContextEntry;
   struct TokenInfo;
 
   Parser(CodeTokenizer *Tokenizer, Sema *S,
+         const NamedValueMap *NamedValues,
          Diagnostics *Error);
 
   bool parseExpressionImpl(VariantValue *Value);
   bool parseMatcherExpressionImpl(const TokenInfo &NameToken,
                                   VariantValue *Value);
   bool parseIdentifierPrefixImpl(VariantValue *Value);
 
-  void addCompletion(const TokenInfo &CompToken, StringRef TypedText,
-                     StringRef Decl);
+  void addCompletion(const TokenInfo &CompToken,
+                     const MatcherCompletion &Completion);
   void addExpressionCompletions();
 
+  std::vector<MatcherCompletion>
+  getNamedValueCompletions(ArrayRef<ArgKind> AcceptedTypes);
+
   CodeTokenizer *const Tokenizer;
   Sema *const S;
+  const NamedValueMap *const NamedValues;
   Diagnostics *const Error;
 
   typedef std::vector<std::pair<MatcherCtor, unsigned> > ContextStackTy;
   ContextStackTy ContextStack;
   std::vector<MatcherCompletion> Completions;
 };
 
 }  // namespace dynamic
 }  // namespace ast_matchers
 }  // namespace clang
 
 #endif  // LLVM_CLANG_AST_MATCHERS_DYNAMIC_PARSER_H

cfe/trunk/include/clang/ASTMatchers/Dynamic/Registry.h

 namespace internal {
 class MatcherDescriptor;
 }
 
 typedef const internal::MatcherDescriptor *MatcherCtor;
 
 struct MatcherCompletion {
   MatcherCompletion() {}
-  MatcherCompletion(StringRef TypedText, StringRef MatcherDecl)
-      : TypedText(TypedText), MatcherDecl(MatcherDecl) {}
+  MatcherCompletion(StringRef TypedText, StringRef MatcherDecl,
+                    unsigned Specificity)
+      : TypedText(TypedText), MatcherDecl(MatcherDecl),
+        Specificity(Specificity) {}
 
   /// \brief The text to type to select this matcher.
   std::string TypedText;
 
   /// \brief The "declaration" of the matcher, with type information.
   std::string MatcherDecl;
 
+  /// \brief Value corresponding to the "specificity" of the converted matcher.
+  ///
+  /// Zero specificity indicates that this conversion would produce a trivial
+  /// matcher that will either always or never match.
+  /// Such matchers are excluded from code completion results.
+  unsigned Specificity;
+
   bool operator==(const MatcherCompletion &Other) const {
     return TypedText == Other.TypedText && MatcherDecl == Other.MatcherDecl;
   }
 };
 
 class Registry {
 public:
   /// \brief Look up a matcher in the registry by name,
   ///
   /// \return An opaque value which may be used to refer to the matcher
   /// constructor, or Optional<MatcherCtor>() if not found.
   static llvm::Optional<MatcherCtor> lookupMatcherCtor(StringRef MatcherName);
 
-  /// \brief Compute the list of completions for \p Context.
+  /// \brief Compute the list of completion types for \p Context.
   ///
   /// Each element of \p Context represents a matcher invocation, going from
-  /// outermost to innermost. Elements are pairs consisting of a reference to the
-  /// matcher constructor and the index of the next element in the argument list
-  /// of that matcher (or for the last element, the index of the completion
-  /// point in the argument list). An empty list requests completion for the
-  /// root matcher.
-  ///
-  /// The completions are ordered first by decreasing relevance, then
-  /// alphabetically.  Relevance is determined by how closely the matcher's
-  /// type matches that of the context. For example, if the innermost matcher
-  /// takes a FunctionDecl matcher, the FunctionDecl matchers are returned
-  /// first, followed by the ValueDecl matchers, then NamedDecl, then Decl, then
-  /// polymorphic matchers.
-  ///
-  /// Matchers which are technically convertible to the innermost context but
-  /// which would match either all or no nodes are excluded. For example,
-  /// namedDecl and varDecl are excluded in a FunctionDecl context, because
-  /// those matchers would match respectively all or no nodes in such a context.
+  /// outermost to innermost. Elements are pairs consisting of a reference to
+  /// the matcher constructor and the index of the next element in the
+  /// argument list of that matcher (or for the last element, the index of
+  /// the completion point in the argument list). An empty list requests
+  /// completion for the root matcher.
+  static std::vector<ArgKind> getAcceptedCompletionTypes(
+      llvm::ArrayRef<std::pair<MatcherCtor, unsigned>> Context);
+
+  /// \brief Compute the list of completions that match any of
+  /// \p AcceptedTypes.
+  ///
+  /// \param All types accepted for this completion.
+  ///
+  /// \return All completions for the specified types.
+  /// Completions should be valid when used in \c lookupMatcherCtor().
+  /// The matcher constructed from the return of \c lookupMatcherCtor()
+  /// should be convertible to some type in \p AcceptedTypes.
   static std::vector<MatcherCompletion>
-  getCompletions(ArrayRef<std::pair<MatcherCtor, unsigned> > Context);
+  getMatcherCompletions(ArrayRef<ArgKind> AcceptedTypes);
 
   /// \brief Construct a matcher from the registry.
   ///
   /// \param Ctor The matcher constructor to instantiate.
   ///
   /// \param NameRange The location of the name in the matcher source.
   ///   Useful for error reporting.
   ///

cfe/trunk/include/clang/ASTMatchers/Dynamic/VariantValue.h

 #include "llvm/ADT/Twine.h"
 #include <memory>
 #include <vector>
 
 namespace clang {
 namespace ast_matchers {
 namespace dynamic {
 
+/// \brief Kind identifier.
+///
+/// It supports all types that VariantValue can contain.
+class ArgKind {
+ public:
+  enum Kind {
+    AK_Matcher,
+    AK_Unsigned,
+    AK_String
+  };
+  /// \brief Constructor for non-matcher types.
+  ArgKind(Kind K) : K(K) { assert(K != AK_Matcher); }
+
+  /// \brief Constructor for matcher types.
+  ArgKind(ast_type_traits::ASTNodeKind MatcherKind)
+      : K(AK_Matcher), MatcherKind(MatcherKind) {}
+
+  Kind getArgKind() const { return K; }
+  ast_type_traits::ASTNodeKind getMatcherKind() const {
+    assert(K == AK_Matcher);
+    return MatcherKind;
+  }
+
+  /// \brief Determines if this type can be converted to \p To.
+  ///
+  /// \param To the requested destination type.
+  ///
+  /// \param Value corresponding to the "specificity" of the convertion.
+  bool isConvertibleTo(ArgKind To, unsigned *Specificity) const;
+
+  bool operator<(const ArgKind &Other) const {
+    if (K == AK_Matcher && Other.K == AK_Matcher)
+      return MatcherKind < Other.MatcherKind;
+    return K < Other.K;
+  }
+
+  /// \brief String representation of the type.
+  std::string asString() const;
+
+private:
+  Kind K;
+  ast_type_traits::ASTNodeKind MatcherKind;
+};
+
 using ast_matchers::internal::DynTypedMatcher;
 
 /// \brief A variant matcher object.
 ///
 /// The purpose of this object is to abstract simple and polymorphic matchers
 /// into a single object type.
 /// Polymorphic matchers might be implemented as a list of all the possible
 /// overloads of the matcher. \c VariantMatcher knows how to select the
class VariantMatcher {
   ///
   /// It follows a similar interface as VariantMatcher itself.
   class Payload : public RefCountedBaseVPTR {
   public:
     virtual ~Payload();
     virtual llvm::Optional<DynTypedMatcher> getSingleMatcher() const = 0;
     virtual std::string getTypeAsString() const = 0;
     virtual void makeTypedMatcher(MatcherOps &Ops) const = 0;
+    virtual bool isConvertibleTo(ast_type_traits::ASTNodeKind Kind,
+                                 unsigned *Specificity) const = 0;
   };
 
 public:
   /// \brief A null matcher.
   VariantMatcher();
 
   /// \brief Clones the provided matcher.
   static VariantMatcher SingleMatcher(const DynTypedMatcher &Matcher);
public:
   /// that can, the result would be ambiguous and false is returned.
   template <class T>
   bool hasTypedMatcher() const {
     TypedMatcherOps<T> Ops;
     if (Value) Value->makeTypedMatcher(Ops);
     return Ops.hasMatcher();
   }
 
+  /// \brief Determines if the contained matcher can be converted to \p Kind.
+  ///
+  /// \param Kind the requested destination type.
+  ///
+  /// \param Value corresponding to the "specificity" of the convertion.
+  bool isConvertibleTo(ast_type_traits::ASTNodeKind Kind,
+                       unsigned *Specificity) const {
+    if (Value)
+      return Value->isConvertibleTo(Kind, Specificity);
+    return false;
+  }
+
   /// \brief Return this matcher as a \c Matcher<T>.
   ///
   /// Handles the different types (Single, Polymorphic) accordingly.
   /// Asserts that \c hasTypedMatcher<T>() is true.
   template <class T>
   ast_matchers::internal::Matcher<T> getTypedMatcher() const {
     TypedMatcherOps<T> Ops;
     Value->makeTypedMatcher(Ops);
public:
   const std::string &getString() const;
   void setString(const std::string &String);
 
   /// \brief Matcher value functions.
   bool isMatcher() const;
   const VariantMatcher &getMatcher() const;
   void setMatcher(const VariantMatcher &Matcher);
 
+  /// \brief Determines if the contained value can be converted to \p Kind.
+  ///
+  /// \param Kind the requested destination type.
+  ///
+  /// \param Value corresponding to the "specificity" of the convertion.
+  bool isConvertibleTo(ArgKind Kind, unsigned* Specificity) const;
+
+  /// \brief Determines if the contained value can be converted to any kind
+  /// in \p Kinds.
+  ///
+  /// \param Kinds the requested destination types.
+  ///
+  /// \param Value corresponding to the "specificity" of the convertion. It is
+  /// the maximum specificity of all the possible conversions.
+  bool isConvertibleTo(ArrayRef<ArgKind> Kinds, unsigned *Specificity) const;
+
   /// \brief String representation of the type of the value.
   std::string getTypeAsString() const;
 
 private:
   void reset();
 
   /// \brief All supported value types.
   enum ValueType {

cfe/trunk/lib/ASTMatchers/Dynamic/Marshallers.h

 #include "clang/ASTMatchers/Dynamic/VariantValue.h"
 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/STLExtras.h"
 #include <string>
 
 namespace clang {
 namespace ast_matchers {
 namespace dynamic {
-
 namespace internal {
 
-struct ArgKind {
-  enum Kind {
-    AK_Matcher,
-    AK_Unsigned,
-    AK_String
-  };
-  ArgKind(Kind K)
-      : K(K) {}
-  ArgKind(ast_type_traits::ASTNodeKind MatcherKind)
-      : K(AK_Matcher), MatcherKind(MatcherKind) {}
-
-  std::string asString() const {
-    switch (getArgKind()) {
-    case AK_Matcher:
-      return (Twine("Matcher<") + MatcherKind.asStringRef() + ">").str();
-    case AK_Unsigned:
-      return "unsigned";
-    case AK_String:
-      return "string";
-    }
-    llvm_unreachable("unhandled ArgKind");
-  }
-
-  Kind getArgKind() const { return K; }
-  ast_type_traits::ASTNodeKind getMatcherKind() const {
-    assert(K == AK_Matcher);
-    return MatcherKind;
-  }
-
-  bool operator<(const ArgKind &Other) const {
-    if (K == AK_Matcher && Other.K == AK_Matcher)
-      return MatcherKind < Other.MatcherKind;
-    return K < Other.K;
-  }
-
-private:
-  Kind K;
-  ast_type_traits::ASTNodeKind MatcherKind;
-};
 
 /// \brief Helper template class to just from argument type to the right is/get
 ///   functions in VariantValue.
 /// Used to verify and extract the matcher arguments below.
 template <class T> struct ArgTypeTraits;
 template <class T> struct ArgTypeTraits<const T &> : public ArgTypeTraits<T> {
 };
 
public:
   /// matcher of type T.
   virtual bool isPolymorphic() const { return false; }
 };
 
 inline bool isRetKindConvertibleTo(
     ArrayRef<ast_type_traits::ASTNodeKind> RetKinds,
     ast_type_traits::ASTNodeKind Kind, unsigned *Specificity,
     ast_type_traits::ASTNodeKind *LeastDerivedKind) {
-  for (ArrayRef<ast_type_traits::ASTNodeKind>::const_iterator
-           i = RetKinds.begin(),
-           e = RetKinds.end();
-       i != e; ++i) {
-    unsigned Distance;
-    if (i->isBaseOf(Kind, &Distance)) {
-      if (Specificity)
-        *Specificity = 100 - Distance;
+  for (const ast_type_traits::ASTNodeKind &NodeKind : RetKinds) {
+    if (ArgKind(NodeKind).isConvertibleTo(Kind, Specificity)) {
       if (LeastDerivedKind)
-        *LeastDerivedKind = *i;
+        *LeastDerivedKind = NodeKind;
       return true;
     }
   }
   return false;
 }
 
 /// \brief Simple callback implementation. Marshaller and function are provided.
 ///

cfe/trunk/lib/ASTMatchers/Dynamic/Parser.cpp

 ///
 //===----------------------------------------------------------------------===//
 
 #include "clang/ASTMatchers/Dynamic/Parser.h"
 #include "clang/ASTMatchers/Dynamic/Registry.h"
 #include "clang/Basic/CharInfo.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/Twine.h"
+#include "llvm/Support/ManagedStatic.h"
 #include <string>
 #include <vector>
 
 namespace clang {
 namespace ast_matchers {
 namespace dynamic {
 
 /// \brief Simple structure to hold information for one token from the parser.
private:
   unsigned Line;
   Diagnostics *Error;
   TokenInfo NextToken;
   const char *CodeCompletionLocation;
 };
 
 Parser::Sema::~Sema() {}
 
-VariantValue Parser::Sema::getNamedValue(StringRef Name) {
-  return VariantValue();
+std::vector<ArgKind> Parser::Sema::getAcceptedCompletionTypes(
+    llvm::ArrayRef<std::pair<MatcherCtor, unsigned>> Context) {
+  return std::vector<ArgKind>();
+}
+
+std::vector<MatcherCompletion>
+Parser::Sema::getMatcherCompletions(llvm::ArrayRef<ArgKind> AcceptedTypes) {
+  return std::vector<MatcherCompletion>();
 }
 
 struct Parser::ScopedContextEntry {
   Parser *P;
 
   ScopedContextEntry(Parser *P, MatcherCtor C) : P(P) {
     P->ContextStack.push_back(std::make_pair(C, 0u));
   }

 /// This function can parse named values and matchers.
 /// In case of failure it will try to determine the user's intent to give
 /// an appropriate error message.
 bool Parser::parseIdentifierPrefixImpl(VariantValue *Value) {
   const TokenInfo NameToken = Tokenizer->consumeNextToken();
 
   if (Tokenizer->nextTokenKind() != TokenInfo::TK_OpenParen) {
     // Parse as a named value.
-    if (const VariantValue NamedValue = S->getNamedValue(NameToken.Text)) {
+    if (const VariantValue NamedValue =
+            NamedValues ? NamedValues->lookup(NameToken.Text)
+                        : VariantValue()) {
       *Value = NamedValue;
       return true;
     }
     // If the syntax is correct and the name is not a matcher either, report
     // unknown named value.
     if ((Tokenizer->nextTokenKind() == TokenInfo::TK_Comma ||
          Tokenizer->nextTokenKind() == TokenInfo::TK_CloseParen ||
          Tokenizer->nextTokenKind() == TokenInfo::TK_Eof) &&
bool Parser::parseMatcherExpressionImpl(const TokenInfo &NameToken,
   }
 
   std::string BindID;
   if (Tokenizer->peekNextToken().Kind == TokenInfo::TK_Period) {
     // Parse .bind("foo")
     Tokenizer->consumeNextToken();  // consume the period.
     const TokenInfo BindToken = Tokenizer->consumeNextToken();
     if (BindToken.Kind == TokenInfo::TK_CodeCompletion) {
-      addCompletion(BindToken, "bind(\"", "bind");
+      addCompletion(BindToken, MatcherCompletion("bind(\"", "bind", 1));
       return false;
     }
 
     const TokenInfo OpenToken = Tokenizer->consumeNextToken();
     const TokenInfo IDToken = Tokenizer->consumeNextToken();
     const TokenInfo CloseToken = Tokenizer->consumeNextToken();
 
     // TODO: We could use different error codes for each/some to be more
bool Parser::parseMatcherExpressionImpl(const TokenInfo &NameToken,
   if (Result.isNull()) return false;
 
   *Value = Result;
   return true;
 }
 
 // If the prefix of this completion matches the completion token, add it to
 // Completions minus the prefix.
-void Parser::addCompletion(const TokenInfo &CompToken, StringRef TypedText,
-                           StringRef Decl) {
-  if (TypedText.size() >= CompToken.Text.size() &&
-      TypedText.substr(0, CompToken.Text.size()) == CompToken.Text) {
-    Completions.push_back(
-        MatcherCompletion(TypedText.substr(CompToken.Text.size()), Decl));
+void Parser::addCompletion(const TokenInfo &CompToken,
+                           const MatcherCompletion& Completion) {
+  if (StringRef(Completion.TypedText).startswith(CompToken.Text) &&
+      Completion.Specificity > 0) {
+    Completions.emplace_back(Completion.TypedText.substr(CompToken.Text.size()),
+                             Completion.MatcherDecl, Completion.Specificity);
+  }
+}
+
+std::vector<MatcherCompletion> Parser::getNamedValueCompletions(
+    ArrayRef<ArgKind> AcceptedTypes) {
+  if (!NamedValues) return std::vector<MatcherCompletion>();
+  std::vector<MatcherCompletion> Result;
+  for (const auto &Entry : *NamedValues) {
+    unsigned Specificity;
+    if (Entry.getValue().isConvertibleTo(AcceptedTypes, &Specificity)) {
+      std::string Decl =
+          (Entry.getValue().getTypeAsString() + " " + Entry.getKey()).str();
+      Result.emplace_back(Entry.getKey(), Decl, Specificity);
+    }
   }
+  return Result;
 }
 
 void Parser::addExpressionCompletions() {
   const TokenInfo CompToken = Tokenizer->consumeNextToken();
   assert(CompToken.Kind == TokenInfo::TK_CodeCompletion);
 
   // We cannot complete code if there is an invalid element on the context
   // stack.
   for (ContextStackTy::iterator I = ContextStack.begin(),
                                 E = ContextStack.end();
        I != E; ++I) {
     if (!I->first)
       return;
   }
 
-  std::vector<MatcherCompletion> RegCompletions =
-      Registry::getCompletions(ContextStack);
-  for (std::vector<MatcherCompletion>::iterator I = RegCompletions.begin(),
-                                                E = RegCompletions.end();
-       I != E; ++I) {
-    addCompletion(CompToken, I->TypedText, I->MatcherDecl);
+  auto AcceptedTypes = S->getAcceptedCompletionTypes(ContextStack);
+  for (const auto &Completion : S->getMatcherCompletions(AcceptedTypes)) {
+    addCompletion(CompToken, Completion);
+  }
+
+  for (const auto &Completion : getNamedValueCompletions(AcceptedTypes)) {
+    addCompletion(CompToken, Completion);
   }
 }
 
 /// \brief Parse an <Expresssion>
 bool Parser::parseExpressionImpl(VariantValue *Value) {
   switch (Tokenizer->nextTokenKind()) {
   case TokenInfo::TK_Literal:
     *Value = Tokenizer->consumeNextToken().Value;
case TokenInfo::TK_InvalidChar:
     const TokenInfo Token = Tokenizer->consumeNextToken();
     Error->addError(Token.Range, Error->ET_ParserInvalidToken) << Token.Text;
     return false;
   }
 
   llvm_unreachable("Unknown token kind.");
 }
 
+static llvm::ManagedStatic<Parser::RegistrySema> DefaultRegistrySema;
+
 Parser::Parser(CodeTokenizer *Tokenizer, Sema *S,
-               Diagnostics *Error)
-    : Tokenizer(Tokenizer), S(S), Error(Error) {}
+               const NamedValueMap *NamedValues, Diagnostics *Error)
+    : Tokenizer(Tokenizer), S(S ? S : &*DefaultRegistrySema),
+      NamedValues(NamedValues), Error(Error) {}
 
 Parser::RegistrySema::~RegistrySema() {}
 
 llvm::Optional<MatcherCtor>
 Parser::RegistrySema::lookupMatcherCtor(StringRef MatcherName) {
   return Registry::lookupMatcherCtor(MatcherName);
 }
 
 VariantMatcher Parser::RegistrySema::actOnMatcherExpression(
     MatcherCtor Ctor, const SourceRange &NameRange, StringRef BindID,
     ArrayRef<ParserValue> Args, Diagnostics *Error) {
   if (BindID.empty()) {
     return Registry::constructMatcher(Ctor, NameRange, Args, Error);
   } else {
     return Registry::constructBoundMatcher(Ctor, NameRange, BindID, Args,
                                            Error);
   }
 }
 
-bool Parser::parseExpression(StringRef Code, VariantValue *Value,
-                             Diagnostics *Error) {
-  RegistrySema S;
-  return parseExpression(Code, &S, Value, Error);
+std::vector<ArgKind> Parser::RegistrySema::getAcceptedCompletionTypes(
+    ArrayRef<std::pair<MatcherCtor, unsigned>> Context) {
+  return Registry::getAcceptedCompletionTypes(Context);
+}
+
+std::vector<MatcherCompletion> Parser::RegistrySema::getMatcherCompletions(
+    ArrayRef<ArgKind> AcceptedTypes) {
+  return Registry::getMatcherCompletions(AcceptedTypes);
 }
 
 bool Parser::parseExpression(StringRef Code, Sema *S,
+                             const NamedValueMap *NamedValues,
                              VariantValue *Value, Diagnostics *Error) {
   CodeTokenizer Tokenizer(Code, Error);
-  if (!Parser(&Tokenizer, S, Error).parseExpressionImpl(Value)) return false;
+  if (!Parser(&Tokenizer, S, NamedValues, Error).parseExpressionImpl(Value))
+    return false;
   if (Tokenizer.peekNextToken().Kind != TokenInfo::TK_Eof) {
     Error->addError(Tokenizer.peekNextToken().Range,
                     Error->ET_ParserTrailingCode);
     return false;
   }
   return true;
 }
 
 std::vector<MatcherCompletion>
-Parser::completeExpression(StringRef Code, unsigned CompletionOffset) {
+Parser::completeExpression(StringRef Code, unsigned CompletionOffset, Sema *S,
+                           const NamedValueMap *NamedValues) {
   Diagnostics Error;
   CodeTokenizer Tokenizer(Code, &Error, CompletionOffset);
-  RegistrySema S;
-  Parser P(&Tokenizer, &S, &Error);
+  Parser P(&Tokenizer, S, NamedValues, &Error);
   VariantValue Dummy;
   P.parseExpressionImpl(&Dummy);
 
-  return P.Completions;
-}
+  // Sort by specificity, then by name.
+  std::sort(P.Completions.begin(), P.Completions.end(),
+            [](const MatcherCompletion &A, const MatcherCompletion &B) {
+    if (A.Specificity != B.Specificity)
+      return A.Specificity > B.Specificity;
+    return A.TypedText < B.TypedText;
+  });
 
-llvm::Optional<DynTypedMatcher>
-Parser::parseMatcherExpression(StringRef Code, Diagnostics *Error) {
-  RegistrySema S;
-  return parseMatcherExpression(Code, &S, Error);
+  return P.Completions;
 }
 
 llvm::Optional<DynTypedMatcher>
-Parser::parseMatcherExpression(StringRef Code, Parser::Sema *S,
+Parser::parseMatcherExpression(StringRef Code, Sema *S,
+                               const NamedValueMap *NamedValues,
                                Diagnostics *Error) {
   VariantValue Value;
-  if (!parseExpression(Code, S, &Value, Error))
+  if (!parseExpression(Code, S, NamedValues, &Value, Error))
     return llvm::Optional<DynTypedMatcher>();
   if (!Value.isMatcher()) {
     Error->addError(SourceRange(), Error->ET_ParserNotAMatcher);
     return llvm::Optional<DynTypedMatcher>();
   }
   llvm::Optional<DynTypedMatcher> Result =
       Value.getMatcher().getSingleMatcher();
   if (!Result.hasValue()) {

cfe/trunk/lib/ASTMatchers/Dynamic/Registry.cpp

if (Count++ == 3) {
       OS << "...";
       break;
     }
     OS << *I;
   }
   return OS;
 }
 
-struct ReverseSpecificityThenName {
-  bool operator()(const std::pair<unsigned, std::string> &A,
-                  const std::pair<unsigned, std::string> &B) const {
-    return A.first > B.first || (A.first == B.first && A.second < B.second);
-  }
-};
-
-}
+}  // namespace
 
-std::vector<MatcherCompletion> Registry::getCompletions(
-    ArrayRef<std::pair<MatcherCtor, unsigned> > Context) {
+std::vector<ArgKind> Registry::getAcceptedCompletionTypes(
+    ArrayRef<std::pair<MatcherCtor, unsigned>> Context) {
   ASTNodeKind InitialTypes[] = {
       ASTNodeKind::getFromNodeKind<Decl>(),
       ASTNodeKind::getFromNodeKind<QualType>(),
       ASTNodeKind::getFromNodeKind<Type>(),
       ASTNodeKind::getFromNodeKind<Stmt>(),
       ASTNodeKind::getFromNodeKind<NestedNameSpecifier>(),
       ASTNodeKind::getFromNodeKind<NestedNameSpecifierLoc>(),
-    ASTNodeKind::getFromNodeKind<TypeLoc>()
-  };
-  ArrayRef<ASTNodeKind> InitialTypesRef(InitialTypes);
+      ASTNodeKind::getFromNodeKind<TypeLoc>()};
 
   // Starting with the above seed of acceptable top-level matcher types, compute
   // the acceptable type set for the argument indicated by each context element.
-  std::set<ASTNodeKind> TypeSet(InitialTypesRef.begin(), InitialTypesRef.end());
-  for (ArrayRef<std::pair<MatcherCtor, unsigned> >::iterator
-           CtxI = Context.begin(),
-           CtxE = Context.end();
-       CtxI != CtxE; ++CtxI) {
-    std::vector<internal::ArgKind> NextTypeSet;
-    for (std::set<ASTNodeKind>::iterator I = TypeSet.begin(), E = TypeSet.end();
-         I != E; ++I) {
-      if (CtxI->first->isConvertibleTo(*I) &&
-          (CtxI->first->isVariadic() ||
-           CtxI->second < CtxI->first->getNumArgs()))
-        CtxI->first->getArgKinds(*I, CtxI->second, NextTypeSet);
+  std::set<ArgKind> TypeSet(std::begin(InitialTypes), std::end(InitialTypes));
+  for (const auto &CtxEntry : Context) {
+    MatcherCtor Ctor = CtxEntry.first;
+    unsigned ArgNumber = CtxEntry.second;
+    std::vector<ArgKind> NextTypeSet;
+    for (const ArgKind &Kind : TypeSet) {
+      if (Kind.getArgKind() == Kind.AK_Matcher &&
+          Ctor->isConvertibleTo(Kind.getMatcherKind()) &&
+          (Ctor->isVariadic() || ArgNumber < Ctor->getNumArgs()))
+        Ctor->getArgKinds(Kind.getMatcherKind(), ArgNumber, NextTypeSet);
     }
     TypeSet.clear();
-    for (std::vector<internal::ArgKind>::iterator I = NextTypeSet.begin(),
-                                                  E = NextTypeSet.end();
-         I != E; ++I) {
-      if (I->getArgKind() == internal::ArgKind::AK_Matcher)
-        TypeSet.insert(I->getMatcherKind());
-    }
-  }
+    TypeSet.insert(NextTypeSet.begin(), NextTypeSet.end());
+  return std::vector<ArgKind>(TypeSet.begin(), TypeSet.end());
 
-  typedef std::map<std::pair<unsigned, std::string>, MatcherCompletion,
-                   ReverseSpecificityThenName> CompletionsTy;
-  CompletionsTy Completions;
+std::vector<MatcherCompletion>
+Registry::getMatcherCompletions(ArrayRef<ArgKind> AcceptedTypes) {
+  std::vector<MatcherCompletion> Completions;
 
-  // TypeSet now contains the list of acceptable types for the argument we are
-  // completing.  Search the registry for acceptable matchers.
+  // Search the registry for acceptable matchers.
   for (ConstructorMap::const_iterator I = RegistryData->constructors().begin(),
                                       E = RegistryData->constructors().end();
        I != E; ++I) {
     std::set<ASTNodeKind> RetKinds;
     unsigned NumArgs = I->second->isVariadic() ? 1 : I->second->getNumArgs();
     bool IsPolymorphic = I->second->isPolymorphic();
-    std::vector<std::vector<internal::ArgKind> > ArgsKinds(NumArgs);
+    std::vector<std::vector<ArgKind>> ArgsKinds(NumArgs);
     unsigned MaxSpecificity = 0;
-    for (std::set<ASTNodeKind>::iterator TI = TypeSet.begin(),
-                                         TE = TypeSet.end();
-         TI != TE; ++TI) {
+    for (const ArgKind& Kind : AcceptedTypes) {
+      if (Kind.getArgKind() != Kind.AK_Matcher)
+        continue;
       unsigned Specificity;
       ASTNodeKind LeastDerivedKind;
-      if (I->second->isConvertibleTo(*TI, &Specificity, &LeastDerivedKind)) {
+      if (I->second->isConvertibleTo(Kind.getMatcherKind(), &Specificity,
+                                     &LeastDerivedKind)) {
         if (MaxSpecificity < Specificity)
           MaxSpecificity = Specificity;
         RetKinds.insert(LeastDerivedKind);
         for (unsigned Arg = 0; Arg != NumArgs; ++Arg)
-          I->second->getArgKinds(*TI, Arg, ArgsKinds[Arg]);
+          I->second->getArgKinds(Kind.getMatcherKind(), Arg, ArgsKinds[Arg]);
         if (IsPolymorphic)
           break;
       }
     }
 
     if (!RetKinds.empty() && MaxSpecificity > 0) {
       std::string Decl;
       llvm::raw_string_ostream OS(Decl);
 
       if (IsPolymorphic) {
         OS << "Matcher<T> " << I->first() << "(Matcher<T>";
       } else {
         OS << "Matcher<" << RetKinds << "> " << I->first() << "(";
-        for (std::vector<std::vector<internal::ArgKind> >::iterator
-                 KI = ArgsKinds.begin(),
-                 KE = ArgsKinds.end();
-             KI != KE; ++KI) {
-          if (KI != ArgsKinds.begin())
+        for (const std::vector<ArgKind> &Arg : ArgsKinds) {
+          if (&Arg != &ArgsKinds[0])
             OS << ", ";
           // This currently assumes that a matcher may not overload a
           // non-matcher, and all non-matcher overloads have identical
           // arguments.
-          if ((*KI)[0].getArgKind() == internal::ArgKind::AK_Matcher) {
+          if (Arg[0].getArgKind() == ArgKind::AK_Matcher) {
             std::set<ASTNodeKind> MatcherKinds;
-            std::transform(
-                KI->begin(), KI->end(),
+            std::transform(Arg.begin(), Arg.end(),
                            std::inserter(MatcherKinds, MatcherKinds.end()),
-                std::mem_fun_ref(&internal::ArgKind::getMatcherKind));
+                           std::mem_fun_ref(&ArgKind::getMatcherKind));
             OS << "Matcher<" << MatcherKinds << ">";
           } else {
-            OS << (*KI)[0].asString();
+            OS << Arg[0].asString();
           }
         }
       }
       if (I->second->isVariadic())
         OS << "...";
       OS << ")";
 
       std::string TypedText = I->first();
       TypedText += "(";
       if (ArgsKinds.empty())
         TypedText += ")";
-      else if (ArgsKinds[0][0].getArgKind() == internal::ArgKind::AK_String)
+      else if (ArgsKinds[0][0].getArgKind() == ArgKind::AK_String)
         TypedText += "\"";
 
-      Completions[std::make_pair(MaxSpecificity, I->first())] =
-          MatcherCompletion(TypedText, OS.str());
+      Completions.emplace_back(TypedText, OS.str(), MaxSpecificity);
     }
   }
 
-  std::vector<MatcherCompletion> RetVal;
-  for (CompletionsTy::iterator I = Completions.begin(), E = Completions.end();
-       I != E; ++I)
-    RetVal.push_back(I->second);
-  return RetVal;
+  return Completions;
 }
 
 // static
 VariantMatcher Registry::constructMatcher(MatcherCtor Ctor,
                                           const SourceRange &NameRange,
                                           ArrayRef<ParserValue> Args,
                                           Diagnostics *Error) {
   return Ctor->create(NameRange, Args, Error);

cfe/trunk/lib/ASTMatchers/Dynamic/VariantValue.cpp

 #include "clang/ASTMatchers/Dynamic/VariantValue.h"
 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/STLExtras.h"
 
 namespace clang {
 namespace ast_matchers {
 namespace dynamic {
 
+std::string ArgKind::asString() const {
+  switch (getArgKind()) {
+  case AK_Matcher:
+    return (Twine("Matcher<") + MatcherKind.asStringRef() + ">").str();
+  case AK_Unsigned:
+    return "unsigned";
+  case AK_String:
+    return "string";
+  }
+  llvm_unreachable("unhandled ArgKind");
+}
+
+bool ArgKind::isConvertibleTo(ArgKind To, unsigned *Specificity) const {
+  if (K != To.K)
+    return false;
+  if (K != AK_Matcher) {
+    if (Specificity)
+      *Specificity = 1;
+    return true;
+  }
+  unsigned Distance;
+  if (!MatcherKind.isBaseOf(To.MatcherKind, &Distance))
+    return false;
+
+  if (Specificity)
+    *Specificity = 100 - Distance;
+  return true;
+}
+
 VariantMatcher::MatcherOps::~MatcherOps() {}
 VariantMatcher::Payload::~Payload() {}
 
 class VariantMatcher::SinglePayload : public VariantMatcher::Payload {
 public:
   SinglePayload(const DynTypedMatcher &Matcher) : Matcher(Matcher) {}
 
-  virtual llvm::Optional<DynTypedMatcher> getSingleMatcher() const {
+  llvm::Optional<DynTypedMatcher> getSingleMatcher() const override {
     return Matcher;
   }
 
-  virtual std::string getTypeAsString() const {
+  std::string getTypeAsString() const override {
     return (Twine("Matcher<") + Matcher.getSupportedKind().asStringRef() + ">")
         .str();
   }
 
-  virtual void makeTypedMatcher(MatcherOps &Ops) const {
+  void makeTypedMatcher(MatcherOps &Ops) const override {
     bool Ignore;
     if (Ops.canConstructFrom(Matcher, Ignore))
       Ops.constructFrom(Matcher);
   }
 
+  bool isConvertibleTo(ast_type_traits::ASTNodeKind Kind,
+                       unsigned *Specificity) const override {
+    return ArgKind(Matcher.getSupportedKind())
+        .isConvertibleTo(Kind, Specificity);
+  }
+
 private:
   const DynTypedMatcher Matcher;
 };
 
 class VariantMatcher::PolymorphicPayload : public VariantMatcher::Payload {
 public:
   PolymorphicPayload(std::vector<DynTypedMatcher> MatchersIn)
       : Matchers(std::move(MatchersIn)) {}
 
-  virtual ~PolymorphicPayload() {}
+  ~PolymorphicPayload() override {}
 
-  virtual llvm::Optional<DynTypedMatcher> getSingleMatcher() const {
+  llvm::Optional<DynTypedMatcher> getSingleMatcher() const override {
     if (Matchers.size() != 1)
       return llvm::Optional<DynTypedMatcher>();
     return Matchers[0];
   }
 
-  virtual std::string getTypeAsString() const {
+  std::string getTypeAsString() const override {
     std::string Inner;
     for (size_t i = 0, e = Matchers.size(); i != e; ++i) {
       if (i != 0)
         Inner += "|";
       Inner += Matchers[i].getSupportedKind().asStringRef();
     }
     return (Twine("Matcher<") + Inner + ">").str();
   }
 
-  virtual void makeTypedMatcher(MatcherOps &Ops) const {
+  void makeTypedMatcher(MatcherOps &Ops) const override {
     bool FoundIsExact = false;
     const DynTypedMatcher *Found = nullptr;
     int NumFound = 0;
     for (size_t i = 0, e = Matchers.size(); i != e; ++i) {
       bool IsExactMatch;
       if (Ops.canConstructFrom(Matchers[i], IsExactMatch)) {
         if (Found) {
           if (FoundIsExact) {
             assert(!IsExactMatch && "We should not have two exact matches.");
             continue;
           }
         }
         Found = &Matchers[i];
         FoundIsExact = IsExactMatch;
         ++NumFound;
       }
     }
     // We only succeed if we found exactly one, or if we found an exact match.
     if (Found && (FoundIsExact || NumFound == 1))
       Ops.constructFrom(*Found);
   }
 
+  bool isConvertibleTo(ast_type_traits::ASTNodeKind Kind,
+                       unsigned *Specificity) const override {
+    unsigned MaxSpecificity = 0;
+    for (const DynTypedMatcher &Matcher : Matchers) {
+      unsigned ThisSpecificity;
+      if (ArgKind(Matcher.getSupportedKind())
+              .isConvertibleTo(Kind, &ThisSpecificity)) {
+        MaxSpecificity = std::max(MaxSpecificity, ThisSpecificity);
+      }
+    }
+    if (Specificity)
+      *Specificity = MaxSpecificity;
+    return MaxSpecificity > 0;
+  }
+
   const std::vector<DynTypedMatcher> Matchers;
 };
 
 class VariantMatcher::VariadicOpPayload : public VariantMatcher::Payload {
 public:
   VariadicOpPayload(ast_matchers::internal::VariadicOperatorFunction Func,
                     std::vector<VariantMatcher> Args)
       : Func(Func), Args(std::move(Args)) {}
 
-  virtual llvm::Optional<DynTypedMatcher> getSingleMatcher() const {
+  llvm::Optional<DynTypedMatcher> getSingleMatcher() const override {
     return llvm::Optional<DynTypedMatcher>();
   }
 
-  virtual std::string getTypeAsString() const {
+  std::string getTypeAsString() const override {
     std::string Inner;
     for (size_t i = 0, e = Args.size(); i != e; ++i) {
       if (i != 0)
         Inner += "&";
       Inner += Args[i].getTypeAsString();
     }
     return Inner;
   }
 
-  virtual void makeTypedMatcher(MatcherOps &Ops) const {
+  void makeTypedMatcher(MatcherOps &Ops) const override {
     Ops.constructVariadicOperator(Func, Args);
   }
 
+  bool isConvertibleTo(ast_type_traits::ASTNodeKind Kind,
+                       unsigned *Specificity) const override {
+    for (const VariantMatcher &Matcher : Args) {
+      if (!Matcher.isConvertibleTo(Kind, Specificity))
+        return false;
+    }
+    return true;
+  }
+
 private:
   const ast_matchers::internal::VariadicOperatorFunction Func;
   const std::vector<VariantMatcher> Args;
 };
 
 VariantMatcher::VariantMatcher() {}
 
 VariantMatcher VariantMatcher::SingleMatcher(const DynTypedMatcher &Matcher) {

 }
 
 void VariantValue::setMatcher(const VariantMatcher &NewValue) {
   reset();
   Type = VT_Matcher;
   Value.Matcher = new VariantMatcher(NewValue);
 }
 
+bool VariantValue::isConvertibleTo(ArgKind Kind, unsigned *Specificity) const {
+  switch (Kind.getArgKind()) {
+  case ArgKind::AK_Unsigned:
+    if (!isUnsigned())
+      return false;
+    *Specificity = 1;
+    return true;
+
+  case ArgKind::AK_String:
+    if (!isString())
+      return false;
+    *Specificity = 1;
+    return true;
+
+  case ArgKind::AK_Matcher:
+    if (!isMatcher())
+      return false;
+    return getMatcher().isConvertibleTo(Kind.getMatcherKind(), Specificity);
+  }
+  llvm_unreachable("Invalid Type");
+}
+
+bool VariantValue::isConvertibleTo(ArrayRef<ArgKind> Kinds,
+                                   unsigned *Specificity) const {
+  unsigned MaxSpecificity = 0;
+  for (const ArgKind& Kind : Kinds) {
+    unsigned ThisSpecificity;
+    if (!isConvertibleTo(Kind, &ThisSpecificity))
+      continue;
+    MaxSpecificity = std::max(MaxSpecificity, ThisSpecificity);
+  }
+  if (Specificity && MaxSpecificity > 0) {
+    *Specificity = MaxSpecificity;
+  }
+  return MaxSpecificity > 0;
+}
+
 std::string VariantValue::getTypeAsString() const {
   switch (Type) {
   case VT_String: return "String";
   case VT_Matcher: return getMatcher().getTypeAsString();
   case VT_Unsigned: return "Unsigned";
   case VT_Nothing: return "Nothing";
   }
   llvm_unreachable("Invalid Type");
 }
 
 } // end namespace dynamic
 } // end namespace ast_matchers
 } // end namespace clang

cfe/trunk/unittests/ASTMatchers/Dynamic/ParserTest.cpp

TEST(ParserTest, ParseMatcher) {
   EXPECT_EQ(2ULL, Foo.Args.size());
   EXPECT_EQ(ExpectedBar, getSingleMatcher(Foo.Args[0].Value)->getID());
   EXPECT_EQ(ExpectedBaz, getSingleMatcher(Foo.Args[1].Value)->getID());
   EXPECT_EQ("Yo!", Foo.BoundID);
 }
 
 using ast_matchers::internal::Matcher;
 
+Parser::NamedValueMap getTestNamedValues() {
+  Parser::NamedValueMap Values;
+  Values["nameX"] = std::string("x");
+  Values["hasParamA"] =
+      VariantMatcher::SingleMatcher(hasParameter(0, hasName("a")));
+  return Values;
+}
+
 TEST(ParserTest, FullParserTest) {
   Diagnostics Error;
   llvm::Optional<DynTypedMatcher> VarDecl(Parser::parseMatcherExpression(
       "varDecl(hasInitializer(binaryOperator(hasLHS(integerLiteral()),"
       "                                      hasOperatorName(\"+\"))))",
       &Error));
   EXPECT_EQ("", Error.toStringFull());
   Matcher<Decl> M = VarDecl->unconditionalConvertTo<Decl>();
   EXPECT_TRUE(matches("int x = 1 + false;", M));
   EXPECT_FALSE(matches("int x = true + 1;", M));
   EXPECT_FALSE(matches("int x = 1 - false;", M));
   EXPECT_FALSE(matches("int x = true - 1;", M));
 
   llvm::Optional<DynTypedMatcher> HasParameter(Parser::parseMatcherExpression(
       "functionDecl(hasParameter(1, hasName(\"x\")))", &Error));
   EXPECT_EQ("", Error.toStringFull());
   M = HasParameter->unconditionalConvertTo<Decl>();
 
   EXPECT_TRUE(matches("void f(int a, int x);", M));
   EXPECT_FALSE(matches("void f(int x, int a);", M));
 
   // Test named values.
-  struct NamedSema : public Parser::RegistrySema {
-   public:
-    virtual VariantValue getNamedValue(StringRef Name) {
-      if (Name == "nameX")
-        return std::string("x");
-      if (Name == "param0")
-        return VariantMatcher::SingleMatcher(hasParameter(0, hasName("a")));
-      return VariantValue();
-    }
-  };
-  NamedSema Sema;
+  auto NamedValues = getTestNamedValues();
   llvm::Optional<DynTypedMatcher> HasParameterWithNamedValues(
       Parser::parseMatcherExpression(
-          "functionDecl(param0, hasParameter(1, hasName(nameX)))", &Sema,
-          &Error));
+          "functionDecl(hasParamA, hasParameter(1, hasName(nameX)))",
+          nullptr, &NamedValues, &Error));
   EXPECT_EQ("", Error.toStringFull());
   M = HasParameterWithNamedValues->unconditionalConvertTo<Decl>();
 
   EXPECT_TRUE(matches("void f(int a, int x);", M));
   EXPECT_FALSE(matches("void f(int x, int a);", M));
 
 
   EXPECT_TRUE(!Parser::parseMatcherExpression(
TEST(ParserTest, OverloadErrors) {
   EXPECT_EQ("1:1: Error building matcher callee.\n"
             "1:8: Candidate 1: Incorrect type for arg 1. "
             "(Expected = Matcher<Stmt>) != (Actual = String)\n"
             "1:8: Candidate 2: Incorrect type for arg 1. "
             "(Expected = Matcher<Decl>) != (Actual = String)",
             ParseWithError("callee(\"A\")"));
 }
 
-TEST(ParserTest, Completion) {
+TEST(ParserTest, CompletionRegistry) {
   std::vector<MatcherCompletion> Comps =
       Parser::completeExpression("while", 5);
   ASSERT_EQ(1u, Comps.size());
   EXPECT_EQ("Stmt(", Comps[0].TypedText);
   EXPECT_EQ("Matcher<Stmt> whileStmt(Matcher<WhileStmt>...)",
             Comps[0].MatcherDecl);
 
   Comps = Parser::completeExpression("whileStmt().", 12);
   ASSERT_EQ(1u, Comps.size());
   EXPECT_EQ("bind(\"", Comps[0].TypedText);
   EXPECT_EQ("bind", Comps[0].MatcherDecl);
 }
 
+TEST(ParserTest, CompletionNamedValues) {
+  // Can complete non-matcher types.
+  auto NamedValues = getTestNamedValues();
+  StringRef Code = "functionDecl(hasName(";
+  std::vector<MatcherCompletion> Comps =
+      Parser::completeExpression(Code, Code.size(), nullptr, &NamedValues);
+  ASSERT_EQ(1u, Comps.size());
+  EXPECT_EQ("nameX", Comps[0].TypedText);
+  EXPECT_EQ("String nameX", Comps[0].MatcherDecl);
+
+  // Can complete if there are names in the expression.
+  Code = "methodDecl(hasName(nameX), ";
+  Comps = Parser::completeExpression(Code, Code.size(), nullptr, &NamedValues);
+  EXPECT_LT(0u, Comps.size());
+
+  // Can complete names and registry together.
+  Code = "methodDecl(hasP";
+  Comps = Parser::completeExpression(Code, Code.size(), nullptr, &NamedValues);
+  ASSERT_EQ(3u, Comps.size());
+  EXPECT_EQ("aramA", Comps[0].TypedText);
+  EXPECT_EQ("Matcher<FunctionDecl> hasParamA", Comps[0].MatcherDecl);
+
+  EXPECT_EQ("arameter(", Comps[1].TypedText);
+  EXPECT_EQ(
+      "Matcher<FunctionDecl> hasParameter(unsigned, Matcher<ParmVarDecl>)",
+      Comps[1].MatcherDecl);
+
+  EXPECT_EQ("arent(", Comps[2].TypedText);
+  EXPECT_EQ("Matcher<Decl> hasParent(Matcher<Decl|Stmt>)",
+            Comps[2].MatcherDecl);
+}
+
 }  // end anonymous namespace
 }  // end namespace dynamic
 }  // end namespace ast_matchers
 }  // end namespace clang

cfe/trunk/unittests/ASTMatchers/Dynamic/RegistryTest.cpp

if (Ctor)
                                        Error);
     EXPECT_EQ("", DummyError.toStringFull());
     return Out;
   }
 
   typedef std::vector<MatcherCompletion> CompVector;
 
   CompVector getCompletions() {
-    return Registry::getCompletions(
-        ArrayRef<std::pair<MatcherCtor, unsigned> >());
+    std::vector<std::pair<MatcherCtor, unsigned> > Context;
+    return Registry::getMatcherCompletions(
+        Registry::getAcceptedCompletionTypes(Context));
   }
 
   CompVector getCompletions(StringRef MatcherName1, unsigned ArgNo1) {
     std::vector<std::pair<MatcherCtor, unsigned> > Context;
     llvm::Optional<MatcherCtor> Ctor = lookupMatcherCtor(MatcherName1);
     if (!Ctor)
       return CompVector();
     Context.push_back(std::make_pair(*Ctor, ArgNo1));
-    return Registry::getCompletions(Context);
+    return Registry::getMatcherCompletions(
+        Registry::getAcceptedCompletionTypes(Context));
   }
 
   CompVector getCompletions(StringRef MatcherName1, unsigned ArgNo1,
                             StringRef MatcherName2, unsigned ArgNo2) {
     std::vector<std::pair<MatcherCtor, unsigned> > Context;
     llvm::Optional<MatcherCtor> Ctor = lookupMatcherCtor(MatcherName1);
     if (!Ctor)
       return CompVector();
     Context.push_back(std::make_pair(*Ctor, ArgNo1));
     Ctor = lookupMatcherCtor(MatcherName2);
     if (!Ctor)
       return CompVector();
     Context.push_back(std::make_pair(*Ctor, ArgNo2));
-    return Registry::getCompletions(Context);
+    return Registry::getMatcherCompletions(
+        Registry::getAcceptedCompletionTypes(Context));
   }
 
   bool hasCompletion(const CompVector &Comps, StringRef TypedText,
-                     StringRef MatcherDecl = StringRef(),
-                     unsigned *Index = nullptr) {
+                     StringRef MatcherDecl = StringRef()) {
     for (CompVector::const_iterator I = Comps.begin(), E = Comps.end(); I != E;
          ++I) {
       if (I->TypedText == TypedText &&
           (MatcherDecl.empty() || I->MatcherDecl == MatcherDecl)) {
-        if (Index)
-          *Index = I - Comps.begin();
         return true;
       }
     }
     return false;
   }
 };
 
 TEST_F(RegistryTest, CanConstructNoArgs) {
TEST_F(RegistryTest, Completion) {
   CompVector Comps = getCompletions();
   EXPECT_TRUE(hasCompletion(
       Comps, "hasParent(", "Matcher<Decl|Stmt> hasParent(Matcher<Decl|Stmt>)"));
   EXPECT_TRUE(hasCompletion(Comps, "whileStmt(",
                             "Matcher<Stmt> whileStmt(Matcher<WhileStmt>...)"));
 
   CompVector WhileComps = getCompletions("whileStmt", 0);
 
-  unsigned HasBodyIndex, HasParentIndex, AllOfIndex;
   EXPECT_TRUE(hasCompletion(WhileComps, "hasBody(",
-                            "Matcher<WhileStmt> hasBody(Matcher<Stmt>)",
-                            &HasBodyIndex));
+                            "Matcher<WhileStmt> hasBody(Matcher<Stmt>)"));
   EXPECT_TRUE(hasCompletion(WhileComps, "hasParent(",
-                            "Matcher<Stmt> hasParent(Matcher<Decl|Stmt>)",
-                            &HasParentIndex));
-  EXPECT_TRUE(hasCompletion(WhileComps, "allOf(",
-                            "Matcher<T> allOf(Matcher<T>...)", &AllOfIndex));
-  EXPECT_GT(HasParentIndex, HasBodyIndex);
-  EXPECT_GT(AllOfIndex, HasParentIndex);
+                            "Matcher<Stmt> hasParent(Matcher<Decl|Stmt>)"));
+  EXPECT_TRUE(
+      hasCompletion(WhileComps, "allOf(", "Matcher<T> allOf(Matcher<T>...)"));
 
   EXPECT_FALSE(hasCompletion(WhileComps, "whileStmt("));
   EXPECT_FALSE(hasCompletion(WhileComps, "ifStmt("));
 
   CompVector AllOfWhileComps =
       getCompletions("allOf", 0, "whileStmt", 0);
   ASSERT_EQ(AllOfWhileComps.size(), WhileComps.size());
   EXPECT_TRUE(std::equal(WhileComps.begin(), WhileComps.end(),