Clang Code Completion Filtering

[CrisCristescu]

Summary

The patch is enabling filtering for code completion.

The stem upon which the filtering is to be done is known at the moment of parsing.
A get/set function for the completion stem has been added to the Token class in order to save the stem as the identifier of the code_complete token type. Once the token has been set, it has to be passed through to the SemaCodeCompleteX functions, which subsequently passes it to the HandleCodeCompletionResults and which passes it to the PrintingCodeCompleteConsumer::ProcessCodeCompleteResults where the actual filtering is happening.

[milianw]

I'm not yet acquainted enough with the code at hand, but I wonder whether I'm understanding the code correctly:

Could it be that you only filter before printing to the output stream? The other consumers, e.g. the one used by the clang-c API function clang_codeCompleteAt is thus not affected? That would be a shame, imo. This functionality should also be applied there, don't you think?

[akyrtzi]

We should not bake-in filtering into the clang CodeComplete API. What kind of filtering to use should be on a higher level, different clients may want different filtering, e.g. prefix filtering, fuzzy filtering, etc.

[milianw]

@akyrtzi raises a very valid point - I did not think about that. KDevelop uses the clang-c API and does fuzzy matching on top of the results, e.g. for camel-case matching. But, we currently always request code completion at a word start boundary so nothing would change for us. That said, I see how this patch could be seen as a breaking behavior change, and thus should probably only get enabled by an explicit option - if at all.

[akyrtzi]

In D17820#366638, @milianw wrote:

But, we currently always request code completion at a word start boundary so nothing would change for us. That said, I see how this patch could be seen as a breaking behavior change, and thus should probably only get enabled by an explicit option - if at all.

I actually think it is fine changing it so clang code-completion automatically adjusts to word boundary, I cannot think of a case where it is useful to not do that, or being aware of a client that does not already adjusts anyway.

[CrisCristescu]

This is a new way of filtering which is could be less intrusive and has less changes.

The code_completion token now stores the information, if there is one,
about the identifier that it is lexing.
During the parsing there is a common entry point which is ConsumerToken
where we can set the information necessary for filtering in the Sema
in the case of a code_completion token.
The information is retrieved when doing the actual filtering.

This introduces a change in functionality which means that some tests
had to be modified(6 tests for ObjC) due to the fact that if you press
tab at the end of a word which could be a language specific keyword,
you will get the possible completions of this identifier,
not what could be written after the space. This is in accordance with the
behaviour of IDEs, for instance XCode.
As an example:
const<TAB> will only give completions for variables like constVar,
and if you have const<SPACE><TAB> the you get the possible completions:
const double, const int etc.

[CrisCristescu]

For the filtering itself the motivation for not doing it on the client side is the following in our use case:

• we do not re-filter on every key stroke, we only filter when the <TAB> key is pressed i.e there is one completion point;
• we have a PCH which will help with the generation time;
• performance is acceptable with the patch included: less than 1s even when the file contains several "heavier" includes such as <string> or <vector>;
• without the patch if the <TAB> key is pressed in the middle of the identifier name (i.e setDouble) an error is thrown:

fiter-member-access.cpp:19:17: error: no member named 'set' in 'MyClass'
        objectMyClass->set
        ~~~~~~~~~~~~~  ^

• and most importantly, we plan to propose another set of patches which would do lookup's only on subsets of the DeclContext of the identifier i.e incomplete lookup-s in order to reduce the amount of PCH deserializations.

[benlangmuir]

Overall approach seems good to me, but this needs tests. Other feedback inline.

include/clang/Sema/CodeCompleteConsumer.h

920

I would expect the default to be false, since if you don't implement filtering presumably you want every result.

Also: Please fix alignment of the results parameter.

lib/Sema/CodeCompleteConsumer.cpp

435

Can Declaration ever be legitimately null? We're going to unconditionally filter it out if so, which seems odd. Same applies to the other cases below.

450

Is this reachable? If not, consider using llvm_unreachable() instead of return.

[CrisCristescu]

Addresses the previous comments and adds testing for filtering.
All the tests already in CodeCompletion are testing both the backward compatibility and the new functionality itself.

[benlangmuir]

LGTM with one comment about the doxygen comments, but you should probably wait to hear from @akyrtzi too.

include/clang/Sema/CodeCompleteConsumer.h

916

Do we really need a group here for just one method? If we do need the group, there should be a matching @}. And it's missing one of the slashes (should be /// @{).

[karies]

Regarding the concerns raised by @akyrtzi : maybe it's a question of making the completer interface useful at low cost (with filtering for generic use cases and less ASTReading) versus not breaking existing use cases (fuzzy match). Would you still object to this change if the filtering could be disabled in the CodeCompleteConsumer?

[v.g.vassilev]

include/clang/Lex/Preprocessor.h

270 ↗

(On Diff #56115)

Can you initialize this variable in the PP's ctor?

[akyrtzi]

Thanks for making the changes; I'd recommend to go ahead and commit and we can iterate post-commit if necessary.

[CrisCristescu]

Addressing some sugesstions.

[CrisCristescu]

PP CodeCompletionII initialisation.

[CrisCristescu]

Some more cosmetics.

[v.g.vassilev]

lib/Lex/Preprocessor.cpp

748 ↗

(On Diff #65508)

It seems the indentation is too much here.

[CrisCristescu]

Wrong indentation update.

[v.g.vassilev]

Landed in r276878.

[junbuml]

It appears that build fails due to this change :

llvm/tools/clang/lib/Sema/CodeCompleteConsumer.cpp:448:3: error: default label in switch which covers all enumeration values [-Werror,-Wcovered-switch-default]

default: llvm_unreachable("Unknown code completion result Kind.");
^

include/clang/Lex/Token.h

     PtrData = const_cast<void *>(D);
   }
 
+  /// getCompletionStem - For a code_completion token that
+  /// has a raw identifier as data, return the data pointer
+  StringRef getCompletionStem() const {
+    assert(is(tok::code_completion));
+    if (PtrData)
+      return StringRef(reinterpret_cast<const char *>(PtrData), getLength());
+    return StringRef();
+  }
+  void setCompletionStem(const char *Ptr) {
+    assert(is(tok::code_completion));
+    PtrData = const_cast<char*>(Ptr);
+  }
+
   /// getRawIdentifier - For a raw identifier token (i.e., an identifier
   /// lexed in raw mode), returns a reference to the text substring in the
   /// buffer if known.

include/clang/Sema/CodeCompleteConsumer.h

 class NamedDecl;
 class NestedNameSpecifier;
 class Sema;
+class Token;
 
 /// \brief The context in which code completion occurred, so that the
 /// code-completion consumer can process the results accordingly.

                                           CodeCompletionResult *Results,
                                           unsigned NumResults) { }
 
+  /// \name Code-completion callbacks
+  //@{
+  /// \brief Process the finalized code-completion results.
+  virtual void ProcessCodeCompleteResults(Token &Tok, Sema &S,
+                                          CodeCompletionContext Context,
+                                          CodeCompletionResult *Results,
+                                          unsigned NumResults) { }
   /// \param S the semantic-analyzer object for which code-completion is being
   /// done.
   ///

                                   CodeCompletionResult *Results,
                                   unsigned NumResults) override;
 
+  /// \brief Prints the finalized code-completion results.
+  void ProcessCodeCompleteResults(Token &Tok, Sema &S, CodeCompletionContext Context,
+                                  CodeCompletionResult *Results,
+                                  unsigned NumResults) override;
+
   void ProcessOverloadCandidates(Sema &S, unsigned CurrentArg,
                                  OverloadCandidate *Candidates,
                                  unsigned NumCandidates) override;

include/clang/Sema/Sema.h

     PCC_LocalDeclarationSpecifiers
   };
 
-  void CodeCompleteModuleImport(SourceLocation ImportLoc, ModuleIdPath Path);
-  void CodeCompleteOrdinaryName(Scope *S,
+  void CodeCompleteModuleImport(Token Tok, SourceLocation ImportLoc, ModuleIdPath Path);
+  void CodeCompleteOrdinaryName(Token Tok, Scope *S,
                                 ParserCompletionContext CompletionContext);
-  void CodeCompleteDeclSpec(Scope *S, DeclSpec &DS,
+  void CodeCompleteDeclSpec(Token Tok, Scope *S, DeclSpec &DS,
                             bool AllowNonIdentifiers,
                             bool AllowNestedNameSpecifiers);
 
   struct CodeCompleteExpressionData;
-  void CodeCompleteExpression(Scope *S,
+  void CodeCompleteExpression(Token Tok, Scope *S,
                               const CodeCompleteExpressionData &Data);
-  void CodeCompleteMemberReferenceExpr(Scope *S, Expr *Base,
+  void CodeCompleteMemberReferenceExpr(Token Tok, Scope *S, Expr *Base,
                                        SourceLocation OpLoc,
                                        bool IsArrow);
-  void CodeCompletePostfixExpression(Scope *S, ExprResult LHS);
-  void CodeCompleteTag(Scope *S, unsigned TagSpec);
-  void CodeCompleteTypeQualifiers(DeclSpec &DS);
-  void CodeCompleteCase(Scope *S);
-  void CodeCompleteCall(Scope *S, Expr *Fn, ArrayRef<Expr *> Args);
-  void CodeCompleteConstructor(Scope *S, QualType Type, SourceLocation Loc,
+  void CodeCompletePostfixExpression(Token Tok, Scope *S, ExprResult LHS);
+  void CodeCompleteTag(Token Tok, Scope *S, unsigned TagSpec);
+  void CodeCompleteTypeQualifiers(Token Tok, DeclSpec &DS);
+  void CodeCompleteCase(Token Tok, Scope *S);
+  void CodeCompleteCall(Token Tok, Scope *S, Expr *Fn, ArrayRef<Expr *> Args);
+  void CodeCompleteConstructor(Token Tok, Scope *S, QualType Type, SourceLocation Loc,
                                ArrayRef<Expr *> Args);
-  void CodeCompleteInitializer(Scope *S, Decl *D);
-  void CodeCompleteReturn(Scope *S);
-  void CodeCompleteAfterIf(Scope *S);
-  void CodeCompleteAssignmentRHS(Scope *S, Expr *LHS);
+  void CodeCompleteInitializer(Token Tok, Scope *S, Decl *D);
+  void CodeCompleteReturn(Token Tok, Scope *S);
+  void CodeCompleteAfterIf(Token Tok, Scope *S);
+  void CodeCompleteAssignmentRHS(Token Tok, Scope *S, Expr *LHS);
 
-  void CodeCompleteQualifiedId(Scope *S, CXXScopeSpec &SS,
+  void CodeCompleteQualifiedId(Token Tok, Scope *S, CXXScopeSpec &SS,
                                bool EnteringContext);
-  void CodeCompleteUsing(Scope *S);
-  void CodeCompleteUsingDirective(Scope *S);
-  void CodeCompleteNamespaceDecl(Scope *S);
-  void CodeCompleteNamespaceAliasDecl(Scope *S);
-  void CodeCompleteOperatorName(Scope *S);
-  void CodeCompleteConstructorInitializer(
+  void CodeCompleteUsing(Token Tok, Scope *S);
+  void CodeCompleteUsingDirective(Token Tok, Scope *S);
+  void CodeCompleteNamespaceDecl(Token Tok, Scope *S);
+  void CodeCompleteNamespaceAliasDecl(Token Tok, Scope *S);
+  void CodeCompleteOperatorName(Token Tok, Scope *S);
+  void CodeCompleteConstructorInitializer(Token Tok,
                                 Decl *Constructor,
                                 ArrayRef<CXXCtorInitializer *> Initializers);
 
-  void CodeCompleteLambdaIntroducer(Scope *S, LambdaIntroducer &Intro,
+  void CodeCompleteLambdaIntroducer(Token Tok, Scope *S, LambdaIntroducer &Intro,
                                     bool AfterAmpersand);
 
-  void CodeCompleteObjCAtDirective(Scope *S);
-  void CodeCompleteObjCAtVisibility(Scope *S);
-  void CodeCompleteObjCAtStatement(Scope *S);
-  void CodeCompleteObjCAtExpression(Scope *S);
-  void CodeCompleteObjCPropertyFlags(Scope *S, ObjCDeclSpec &ODS);
-  void CodeCompleteObjCPropertyGetter(Scope *S);
-  void CodeCompleteObjCPropertySetter(Scope *S);
-  void CodeCompleteObjCPassingType(Scope *S, ObjCDeclSpec &DS,
+  void CodeCompleteObjCAtDirective(Token Tok, Scope *S);
+  void CodeCompleteObjCAtVisibility(Token Tok, Scope *S);
+  void CodeCompleteObjCAtStatement(Token Tok, Scope *S);
+  void CodeCompleteObjCAtExpression(Token Tok, Scope *S);
+  void CodeCompleteObjCPropertyFlags(Token Tok, Scope *S, ObjCDeclSpec &ODS);
+  void CodeCompleteObjCPropertyGetter(Token Tok, Scope *S);
+  void CodeCompleteObjCPropertySetter(Token Tok, Scope *S);
+  void CodeCompleteObjCPassingType(Token Tok, Scope *S, ObjCDeclSpec &DS,
                                    bool IsParameter);
-  void CodeCompleteObjCMessageReceiver(Scope *S);
-  void CodeCompleteObjCSuperMessage(Scope *S, SourceLocation SuperLoc,
+  void CodeCompleteObjCMessageReceiver(Token Tok, Scope *S);
+  void CodeCompleteObjCSuperMessage(Token Tok, Scope *S, SourceLocation SuperLoc,
                                     ArrayRef<IdentifierInfo *> SelIdents,
                                     bool AtArgumentExpression);
-  void CodeCompleteObjCClassMessage(Scope *S, ParsedType Receiver,
+  void CodeCompleteObjCClassMessage(Token Tok, Scope *S, ParsedType Receiver,
                                     ArrayRef<IdentifierInfo *> SelIdents,
                                     bool AtArgumentExpression,
                                     bool IsSuper = false);
-  void CodeCompleteObjCInstanceMessage(Scope *S, Expr *Receiver,
+  void CodeCompleteObjCInstanceMessage(Token Tok, Scope *S, Expr *Receiver,
                                        ArrayRef<IdentifierInfo *> SelIdents,
                                        bool AtArgumentExpression,
                                        ObjCInterfaceDecl *Super = nullptr);
-  void CodeCompleteObjCForCollection(Scope *S,
+  void CodeCompleteObjCForCollection(Token Tok, Scope *S,
                                      DeclGroupPtrTy IterationVar);
-  void CodeCompleteObjCSelector(Scope *S,
+  void CodeCompleteObjCSelector(Token Tok, Scope *S,
                                 ArrayRef<IdentifierInfo *> SelIdents);
-  void CodeCompleteObjCProtocolReferences(IdentifierLocPair *Protocols,
+  void CodeCompleteObjCProtocolReferences(Token Tok, IdentifierLocPair *Protocols,
                                           unsigned NumProtocols);
-  void CodeCompleteObjCProtocolDecl(Scope *S);
-  void CodeCompleteObjCInterfaceDecl(Scope *S);
-  void CodeCompleteObjCSuperclass(Scope *S,
+  void CodeCompleteObjCProtocolDecl(Token Tok, Scope *S);
+  void CodeCompleteObjCInterfaceDecl(Token Tok, Scope *S);
+  void CodeCompleteObjCSuperclass(Token Tok, Scope *S,
                                   IdentifierInfo *ClassName,
                                   SourceLocation ClassNameLoc);
-  void CodeCompleteObjCImplementationDecl(Scope *S);
-  void CodeCompleteObjCInterfaceCategory(Scope *S,
+  void CodeCompleteObjCImplementationDecl(Token Tok, Scope *S);
+  void CodeCompleteObjCInterfaceCategory(Token Tok, Scope *S,
                                          IdentifierInfo *ClassName,
                                          SourceLocation ClassNameLoc);
-  void CodeCompleteObjCImplementationCategory(Scope *S,
+  void CodeCompleteObjCImplementationCategory(Token Tok, Scope *S,
                                               IdentifierInfo *ClassName,
                                               SourceLocation ClassNameLoc);
-  void CodeCompleteObjCPropertyDefinition(Scope *S);
-  void CodeCompleteObjCPropertySynthesizeIvar(Scope *S,
+  void CodeCompleteObjCPropertyDefinition(Token Tok, Scope *S);
+  void CodeCompleteObjCPropertySynthesizeIvar(Token Tok, Scope *S,
                                               IdentifierInfo *PropertyName);
-  void CodeCompleteObjCMethodDecl(Scope *S,
+  void CodeCompleteObjCMethodDecl(Token Tok, Scope *S,
                                   bool IsInstanceMethod,
                                   ParsedType ReturnType);
-  void CodeCompleteObjCMethodDeclSelector(Scope *S,
+  void CodeCompleteObjCMethodDeclSelector(Token Tok, Scope *S,
                                           bool IsInstanceMethod,
                                           bool AtParameterName,
                                           ParsedType ReturnType,
                                           ArrayRef<IdentifierInfo *> SelIdents);
-  void CodeCompletePreprocessorDirective(bool InConditional);
-  void CodeCompleteInPreprocessorConditionalExclusion(Scope *S);
-  void CodeCompletePreprocessorMacroName(bool IsDefinition);
-  void CodeCompletePreprocessorExpression();
-  void CodeCompletePreprocessorMacroArgument(Scope *S,
+  void CodeCompletePreprocessorDirective(Token Tok, bool InConditional);
+  void CodeCompleteInPreprocessorConditionalExclusion(Token Tok, Scope *S);
+  void CodeCompletePreprocessorMacroName(Token Tok, bool IsDefinition);
+  void CodeCompletePreprocessorExpression(Token Tok);
+  void CodeCompletePreprocessorMacroArgument(Token Tok, Scope *S,
                                              IdentifierInfo *Macro,
                                              MacroInfo *MacroInfo,
                                              unsigned Argument);
-  void CodeCompleteNaturalLanguage();
+  void CodeCompleteNaturalLanguage(Token Tok);
   void GatherGlobalCodeCompletions(CodeCompletionAllocator &Allocator,
                                    CodeCompletionTUInfo &CCTUInfo,
                   SmallVectorImpl<CodeCompletionResult> &Results);

lib/Lex/Lexer.cpp

 
   --CurPtr;   // Back up over the skipped character.
 
+  if (isCodeCompletionPoint(CurPtr)) {
+       // Return the code-completion token.
+      const char *IdStart = BufferPtr;
+      Result.startToken();
+      FormTokenWithChars(Result, CurPtr, tok::code_completion);
+      Result.setCompletionStem(IdStart);
+      cutOffLexing();
+      return true;
+  }
+
   // Fast path, no $,\,? in identifier found.  '\' might be an escaped newline
   // or UCN, and ? might be a trigraph for '\', an escaped newline or UCN.
   //

lib/Parse/ParseDecl.cpp

       }
 
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteInitializer(getCurScope(), ThisDecl);
+        Actions.CodeCompleteInitializer(Tok, getCurScope(), ThisDecl);
         Actions.FinalizeDeclaration(ThisDecl);
         cutOffParsing();
         return nullptr;

     }
 
     if (ParseExpressionList(Exprs, CommaLocs, [&] {
-          Actions.CodeCompleteConstructor(getCurScope(),
+          Actions.CodeCompleteConstructor(Tok, getCurScope(),
                  cast<VarDecl>(ThisDecl)->getType()->getCanonicalTypeInternal(),
                                           ThisDecl->getLocation(), Exprs);
        })) {

           = DSContext == DSC_top_level ||
             (DSContext == DSC_class && DS.isFriendSpecified());
 
-        Actions.CodeCompleteDeclSpec(getCurScope(), DS,
+        Actions.CodeCompleteDeclSpec(Tok, getCurScope(), DS,
                                      AllowNonIdentifiers,
                                      AllowNestedNameSpecifiers);
         return cutOffParsing();

       else if (CurParsedObjCImpl)
         CCC = Sema::PCC_ObjCImplementation;
 
-      Actions.CodeCompleteOrdinaryName(getCurScope(), CCC);
+      Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), CCC);
       return cutOffParsing();
     }
 

   // Parse the tag portion of this.
   if (Tok.is(tok::code_completion)) {
     // Code completion for an enum name.
-    Actions.CodeCompleteTag(getCurScope(), DeclSpec::TST_enum);
+    Actions.CodeCompleteTag(Tok, getCurScope(), DeclSpec::TST_enum);
     return cutOffParsing();
   }
 

 
     switch (Tok.getKind()) {
     case tok::code_completion:
-      Actions.CodeCompleteTypeQualifiers(DS);
+      Actions.CodeCompleteTypeQualifiers(Tok, DS);
       return cutOffParsing();
 
     case tok::kw_const:

lib/Parse/ParseDeclCXX.cpp

   ObjCDeclContextSwitch ObjCDC(*this);
     
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteNamespaceDecl(getCurScope());
+    Actions.CodeCompleteNamespaceDecl(Tok, getCurScope());
     cutOffParsing();
     return nullptr;
   }

   ConsumeToken(); // eat the '='.
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteNamespaceAliasDecl(getCurScope());
+    Actions.CodeCompleteNamespaceAliasDecl(Tok, getCurScope());
     cutOffParsing();
     return nullptr;
   }

   SourceLocation UsingLoc = ConsumeToken();
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteUsing(getCurScope());
+    Actions.CodeCompleteUsing(Tok, getCurScope());
     cutOffParsing();
     return nullptr;
   }

   SourceLocation NamespcLoc = ConsumeToken();
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteUsingDirective(getCurScope());
+    Actions.CodeCompleteUsingDirective(Tok, getCurScope());
     cutOffParsing();
     return nullptr;
   }

 
   if (Tok.is(tok::code_completion)) {
     // Code completion for a struct, class, or union name.
-    Actions.CodeCompleteTag(getCurScope(), TagType);
+    Actions.CodeCompleteTag(Tok, getCurScope(), TagType);
     return cutOffParsing();
   }
 

 
   do {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteConstructorInitializer(ConstructorDecl,
+      Actions.CodeCompleteConstructorInitializer(Tok, ConstructorDecl,
                                                  MemInitializers);
       return cutOffParsing();
     } else {

lib/Parse/ParseExpr.cpp

 /// \brief Parse an expr that doesn't include (top-level) commas.
 ExprResult Parser::ParseAssignmentExpression(TypeCastState isTypeCast) {
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Expression);
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Expression);
     cutOffParsing();
     return ExprError();
   }

     // Code completion for the right-hand side of an assignment expression
     // goes through a special hook that takes the left-hand side into account.
     if (Tok.is(tok::code_completion) && NextTokPrec == prec::Assignment) {
-      Actions.CodeCompleteAssignmentRHS(getCurScope(), LHS.get());
+      Actions.CodeCompleteAssignmentRHS(Tok, getCurScope(), LHS.get());
       cutOffParsing();
       return ExprError();
     }

     Res = ParseBlockLiteralExpression();
     break;
   case tok::code_completion: {
-    Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Expression);
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Expression);
     cutOffParsing();
     return ExprError();
   }

       if (InMessageExpression)
         return LHS;
         
-      Actions.CodeCompletePostfixExpression(getCurScope(), LHS);
+      Actions.CodeCompletePostfixExpression(Tok, getCurScope(), LHS);
       cutOffParsing();
       return ExprError();
         

       CommaLocsTy CommaLocs;
       
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteCall(getCurScope(), LHS.get(), None);
+        Actions.CodeCompleteCall(Tok, getCurScope(), LHS.get(), None);
         cutOffParsing();
         return ExprError();
       }

       if (OpKind == tok::l_paren || !LHS.isInvalid()) {
         if (Tok.isNot(tok::r_paren)) {
           if (ParseExpressionList(ArgExprs, CommaLocs, [&] {
-                Actions.CodeCompleteCall(getCurScope(), LHS.get(), ArgExprs);
+                Actions.CodeCompleteCall(Tok, getCurScope(), LHS.get(), ArgExprs);
              })) {
             (void)Actions.CorrectDelayedTyposInExpr(LHS);
             LHS = ExprError();

 
       if (Tok.is(tok::code_completion)) {
         // Code completion for a member access expression.
-        Actions.CodeCompleteMemberReferenceExpr(getCurScope(), LHS.get(),
+        Actions.CodeCompleteMemberReferenceExpr(Tok, getCurScope(), LHS.get(),
                                                 OpLoc, OpKind == tok::arrow);
         
         cutOffParsing();

   CastTy = ParsedType();
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteOrdinaryName(getCurScope(), 
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(),
                  ExprType >= CompoundLiteral? Sema::PCC_ParenthesizedExpression
                                             : Sema::PCC_Expression);
     cutOffParsing();

       if (Completer)
         Completer();
       else
-        Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Expression);
+        Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Expression);
       cutOffParsing();
       return true;
     }

 /// \endverbatim
 void Parser::ParseBlockId(SourceLocation CaretLoc) {
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Type);
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Type);
     return cutOffParsing();
   }
   

lib/Parse/ParseExprCXX.cpp

       if (Tok.is(tok::code_completion)) {
         // Code completion for a nested-name-specifier, where the code
         // code completion token follows the '::'.
-        Actions.CodeCompleteQualifiedId(getCurScope(), SS, EnteringContext);
+        Actions.CodeCompleteQualifiedId(Tok, getCurScope(), SS, EnteringContext);
         // Include code completion token into the range of the scope otherwise
         // when we try to annotate the scope tokens the dangling code completion
         // token will cause assertion in

         if (Tok.is(tok::code_completion) &&
             !(getLangOpts().ObjC1 && Intro.Default == LCD_None &&
               !Intro.Captures.empty())) {
-          Actions.CodeCompleteLambdaIntroducer(getCurScope(), Intro, 
+          Actions.CodeCompleteLambdaIntroducer(Tok, getCurScope(), Intro,
                                                /*AfterAmpersand=*/false);
           cutOffParsing();
           break;

       // If we're in Objective-C++ and we have a bare '[', then this is more
       // likely to be a message receiver.
       if (getLangOpts().ObjC1 && first)
-        Actions.CodeCompleteObjCMessageReceiver(getCurScope());
+        Actions.CodeCompleteObjCMessageReceiver(Tok, getCurScope());
       else
-        Actions.CodeCompleteLambdaIntroducer(getCurScope(), Intro, 
+        Actions.CodeCompleteLambdaIntroducer(Tok, getCurScope(), Intro,
                                              /*AfterAmpersand=*/false);
       cutOffParsing();
       break;

         ConsumeToken();
 
         if (Tok.is(tok::code_completion)) {
-          Actions.CodeCompleteLambdaIntroducer(getCurScope(), Intro, 
+          Actions.CodeCompleteLambdaIntroducer(Tok, getCurScope(), Intro,
                                                /*AfterAmpersand=*/true);
           cutOffParsing();
           break;

 
     if (Tok.isNot(tok::r_paren)) {
       if (ParseExpressionList(Exprs, CommaLocs, [&] {
-            Actions.CodeCompleteConstructor(getCurScope(),
+            Actions.CodeCompleteConstructor(Tok, getCurScope(),
                                       TypeRep.get()->getCanonicalTypeInternal(),
                                             DS.getLocEnd(), Exprs);
          })) {

                                SourceLocation Loc,
                                bool ConvertToBoolean) {
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Condition);
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Condition);
     cutOffParsing();
     return true;
   }

       
     case tok::code_completion: {
       // Code completion for the operator name.
-      Actions.CodeCompleteOperatorName(getCurScope());
+      Actions.CodeCompleteOperatorName(Tok, getCurScope());
       cutOffParsing();      
       // Don't try to parse any further.
       return true;

       if (ParseExpressionList(ConstructorArgs, CommaLocs, [&] {
             ParsedType TypeRep = Actions.ActOnTypeName(getCurScope(),
                                                        DeclaratorInfo).get();
-            Actions.CodeCompleteConstructor(getCurScope(),
+            Actions.CodeCompleteConstructor(Tok, getCurScope(),
                                       TypeRep.get()->getCanonicalTypeInternal(),
                                             DeclaratorInfo.getLocEnd(),
                                             ConstructorArgs);

lib/Parse/ParseObjc.cpp

   SourceLocation AtLoc = ConsumeToken(); // the "@"
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCAtDirective(getCurScope());
+    Actions.CodeCompleteObjCAtDirective(Tok, getCurScope());
     cutOffParsing();
     return DeclGroupPtrTy();
   }

 
   // Code completion after '@interface'.
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCInterfaceDecl(getCurScope());
+    Actions.CodeCompleteObjCInterfaceDecl(Tok, getCurScope());
     cutOffParsing();
     return nullptr;
   }

     SourceLocation categoryLoc;
     IdentifierInfo *categoryId = nullptr;
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCInterfaceCategory(getCurScope(), nameId, nameLoc);
+      Actions.CodeCompleteObjCInterfaceCategory(Tok, getCurScope(), nameId, nameLoc);
       cutOffParsing();
       return nullptr;
     }

 
     // Code completion of superclass names.
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCSuperclass(getCurScope(), nameId, nameLoc);
+      Actions.CodeCompleteObjCSuperclass(Tok, getCurScope(), nameId, nameLoc);
       cutOffParsing();
       return nullptr;
     }

 
     // Code completion within an Objective-C interface.
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteOrdinaryName(getCurScope(), 
+      Actions.CodeCompleteOrdinaryName(Tok, getCurScope(),
                             CurParsedObjCImpl? Sema::PCC_ObjCImplementation
                                              : Sema::PCC_ObjCInterface);
       return cutOffParsing();

     // Otherwise, we have an @ directive, eat the @.
     SourceLocation AtLoc = ConsumeToken(); // the "@"
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCAtDirective(getCurScope());
+      Actions.CodeCompleteObjCAtDirective(Tok, getCurScope());
       return cutOffParsing();
     }
 

   // We break out of the big loop in two cases: when we see @end or when we see
   // EOF.  In the former case, eat the @end.  In the later case, emit an error.
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCAtDirective(getCurScope());
+    Actions.CodeCompleteObjCAtDirective(Tok, getCurScope());
     return cutOffParsing();
   } else if (Tok.isObjCAtKeyword(tok::objc_end)) {
     ConsumeToken(); // the "end" identifier

 
   while (1) {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCPropertyFlags(getCurScope(), DS);
+      Actions.CodeCompleteObjCPropertyFlags(Tok, getCurScope(), DS);
       return cutOffParsing();
     }
     const IdentifierInfo *II = Tok.getIdentifierInfo();

 
       if (Tok.is(tok::code_completion)) {
         if (IsSetter)
-          Actions.CodeCompleteObjCPropertySetter(getCurScope());
+          Actions.CodeCompleteObjCPropertySetter(Tok, getCurScope());
         else
-          Actions.CodeCompleteObjCPropertyGetter(getCurScope());
+          Actions.CodeCompleteObjCPropertyGetter(Tok, getCurScope());
         return cutOffParsing();
       }
 

 
   while (1) {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCPassingType(getCurScope(), DS, 
+      Actions.CodeCompleteObjCPassingType(Tok, getCurScope(), DS,
                           Context == Declarator::ObjCParameterContext);
       return cutOffParsing();
     }

   ParsingDeclRAIIObject PD(*this, ParsingDeclRAIIObject::NoParent);
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCMethodDecl(getCurScope(), mType == tok::minus, 
+    Actions.CodeCompleteObjCMethodDecl(Tok, getCurScope(), mType == tok::minus,
                                        /*ReturnType=*/ ParsedType());
     cutOffParsing();
     return nullptr;

     MaybeParseGNUAttributes(methodAttrs);
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCMethodDecl(getCurScope(), mType == tok::minus, 
+    Actions.CodeCompleteObjCMethodDecl(Tok, getCurScope(), mType == tok::minus,
                                        ReturnType);
     cutOffParsing();
     return nullptr;

     // Code completion for the next piece of the selector.
     if (Tok.is(tok::code_completion)) {
       KeyIdents.push_back(SelIdent);
-      Actions.CodeCompleteObjCMethodDeclSelector(getCurScope(), 
+      Actions.CodeCompleteObjCMethodDeclSelector(Tok, getCurScope(),
                                                  mType == tok::minus,
                                                  /*AtParameterName=*/true,
                                                  ReturnType, KeyIdents);

 
     // Code completion for the next piece of the selector.
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCMethodDeclSelector(getCurScope(), 
+      Actions.CodeCompleteObjCMethodDeclSelector(Tok, getCurScope(),
                                                  mType == tok::minus,
                                                  /*AtParameterName=*/false,
                                                  ReturnType, KeyIdents);

 
   while (1) {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCProtocolReferences(ProtocolIdents.data(), 
+      Actions.CodeCompleteObjCProtocolReferences(Tok, ProtocolIdents.data(),
                                                  ProtocolIdents.size());
       cutOffParsing();
       return true;

     // Set the default visibility to private.
     if (TryConsumeToken(tok::at)) { // parse objc-visibility-spec
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteObjCAtVisibility(getCurScope());
+        Actions.CodeCompleteObjCAtVisibility(Tok, getCurScope());
         return cutOffParsing();
       }
       

     }
 
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteOrdinaryName(getCurScope(), 
+      Actions.CodeCompleteOrdinaryName(Tok, getCurScope(),
                                        Sema::PCC_ObjCInstanceVariableList);
       return cutOffParsing();
     }

   ConsumeToken(); // the "protocol" identifier
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCProtocolDecl(getCurScope());
+    Actions.CodeCompleteObjCProtocolDecl(Tok, getCurScope());
     cutOffParsing();
     return DeclGroupPtrTy();
   }

 
   // Code completion after '@implementation'.
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCImplementationDecl(getCurScope());
+    Actions.CodeCompleteObjCImplementationDecl(Tok, getCurScope());
     cutOffParsing();
     return DeclGroupPtrTy();
   }

     IdentifierInfo *categoryId = nullptr;
 
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCImplementationCategory(getCurScope(), nameId, nameLoc);
+      Actions.CodeCompleteObjCImplementationCategory(Tok, getCurScope(), nameId, nameLoc);
       cutOffParsing();
       return DeclGroupPtrTy();
     }

 
   while (true) {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCPropertyDefinition(getCurScope());
+      Actions.CodeCompleteObjCPropertyDefinition(Tok, getCurScope());
       cutOffParsing();
       return nullptr;
     }

     if (TryConsumeToken(tok::equal)) {
       // property '=' ivar-name
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteObjCPropertySynthesizeIvar(getCurScope(), propertyId);
+        Actions.CodeCompleteObjCPropertySynthesizeIvar(Tok, getCurScope(), propertyId);
         cutOffParsing();
         return nullptr;
       }

   ConsumeToken(); // consume dynamic
   while (true) {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteObjCPropertyDefinition(getCurScope());
+      Actions.CodeCompleteObjCPropertyDefinition(Tok, getCurScope());
       cutOffParsing();
       return nullptr;
     }

 
 StmtResult Parser::ParseObjCAtStatement(SourceLocation AtLoc) {
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCAtStatement(getCurScope());
+    Actions.CodeCompleteObjCAtStatement(Tok, getCurScope());
     cutOffParsing();
     return StmtError();
   }

 ExprResult Parser::ParseObjCAtExpression(SourceLocation AtLoc) {
   switch (Tok.getKind()) {
   case tok::code_completion:
-    Actions.CodeCompleteObjCAtExpression(getCurScope());
+    Actions.CodeCompleteObjCAtExpression(Tok, getCurScope());
     cutOffParsing();
     return ExprError();
 

   SourceLocation LBracLoc = ConsumeBracket(); // consume '['
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCMessageReceiver(getCurScope());
+    Actions.CodeCompleteObjCMessageReceiver(Tok, getCurScope());
     cutOffParsing();
     return ExprError();
   }

 
   if (Tok.is(tok::code_completion)) {
     if (SuperLoc.isValid())
-      Actions.CodeCompleteObjCSuperMessage(getCurScope(), SuperLoc, None,
+      Actions.CodeCompleteObjCSuperMessage(Tok, getCurScope(), SuperLoc, None,
                                            false);
     else if (ReceiverType)
-      Actions.CodeCompleteObjCClassMessage(getCurScope(), ReceiverType, None,
+      Actions.CodeCompleteObjCClassMessage(Tok, getCurScope(), ReceiverType, None,
                                            false);
     else
-      Actions.CodeCompleteObjCInstanceMessage(getCurScope(), ReceiverExpr,
+      Actions.CodeCompleteObjCInstanceMessage(Tok, getCurScope(), ReceiverExpr,
                                               None, false);
     cutOffParsing();
     return ExprError();

       
       if (Tok.is(tok::code_completion)) {
         if (SuperLoc.isValid())
-          Actions.CodeCompleteObjCSuperMessage(getCurScope(), SuperLoc, 
+          Actions.CodeCompleteObjCSuperMessage(Tok, getCurScope(), SuperLoc,
                                                KeyIdents,
                                                /*AtArgumentEpression=*/true);
         else if (ReceiverType)
-          Actions.CodeCompleteObjCClassMessage(getCurScope(), ReceiverType,
+          Actions.CodeCompleteObjCClassMessage(Tok, getCurScope(), ReceiverType,
                                                KeyIdents,
                                                /*AtArgumentEpression=*/true);
         else
-          Actions.CodeCompleteObjCInstanceMessage(getCurScope(), ReceiverExpr,
+          Actions.CodeCompleteObjCInstanceMessage(Tok, getCurScope(), ReceiverExpr,
                                                   KeyIdents,
                                                   /*AtArgumentEpression=*/true);
 

       // Code completion after each argument.
       if (Tok.is(tok::code_completion)) {
         if (SuperLoc.isValid())
-          Actions.CodeCompleteObjCSuperMessage(getCurScope(), SuperLoc, 
+          Actions.CodeCompleteObjCSuperMessage(Tok, getCurScope(), SuperLoc,
                                                KeyIdents,
                                                /*AtArgumentEpression=*/false);
         else if (ReceiverType)
-          Actions.CodeCompleteObjCClassMessage(getCurScope(), ReceiverType,
+          Actions.CodeCompleteObjCClassMessage(Tok, getCurScope(), ReceiverType,
                                                KeyIdents,
                                                /*AtArgumentEpression=*/false);
         else
-          Actions.CodeCompleteObjCInstanceMessage(getCurScope(), ReceiverExpr,
+          Actions.CodeCompleteObjCInstanceMessage(Tok, getCurScope(), ReceiverExpr,
                                                   KeyIdents,
                                                 /*AtArgumentEpression=*/false);
         cutOffParsing();

     ConsumeParen();
   
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteObjCSelector(getCurScope(), KeyIdents);
+    Actions.CodeCompleteObjCSelector(Tok, getCurScope(), KeyIdents);
     cutOffParsing();
     return ExprError();
   }

         break;
       
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteObjCSelector(getCurScope(), KeyIdents);
+        Actions.CodeCompleteObjCSelector(Tok, getCurScope(), KeyIdents);
         cutOffParsing();
         return ExprError();
       }

lib/Parse/ParseStmt.cpp

     }
 
   case tok::code_completion:
-    Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Statement);
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Statement);
     cutOffParsing();
     return StmtError();
 

     ColonLoc = SourceLocation();
 
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteCase(getCurScope());
+      Actions.CodeCompleteCase(Tok, getCurScope());
       cutOffParsing();
       return StmtError();
     }

     // Pop the 'else' scope if needed.
     InnerScope.Exit();
   } else if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteAfterIf(getCurScope());
+    Actions.CodeCompleteAfterIf(Tok, getCurScope());
     cutOffParsing();
     return StmtError();
   } else if (InnerStatementTrailingElseLoc.isValid()) {

   Decl *SecondVar = nullptr;
 
   if (Tok.is(tok::code_completion)) {
-    Actions.CodeCompleteOrdinaryName(getCurScope(),
+    Actions.CodeCompleteOrdinaryName(Tok, getCurScope(),
                                      C99orCXXorObjC? Sema::PCC_ForInit
                                                    : Sema::PCC_Expression);
     cutOffParsing();

       ConsumeToken(); // consume 'in'
 
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteObjCForCollection(getCurScope(), DG);
+        Actions.CodeCompleteObjCForCollection(Tok, getCurScope(), DG);
         cutOffParsing();
         return StmtError();
       }

       ConsumeToken(); // consume 'in'
 
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteObjCForCollection(getCurScope(), DeclGroupPtrTy());
+        Actions.CodeCompleteObjCForCollection(Tok, getCurScope(), DeclGroupPtrTy());
         cutOffParsing();
         return StmtError();
       }

   ExprResult R;
   if (Tok.isNot(tok::semi)) {
     if (Tok.is(tok::code_completion)) {
-      Actions.CodeCompleteReturn(getCurScope());
+      Actions.CodeCompleteReturn(Tok, getCurScope());
       cutOffParsing();
       return StmtError();
     }

lib/Parse/Parser.cpp

     SingleDecl = ParseObjCMethodDefinition();
     break;
   case tok::code_completion:
-      Actions.CodeCompleteOrdinaryName(getCurScope(), 
+      Actions.CodeCompleteOrdinaryName(Tok, getCurScope(),
                              CurParsedObjCImpl? Sema::PCC_ObjCImplementation
                                               : Sema::PCC_Namespace);
     cutOffParsing();

 
   for (Scope *S = getCurScope(); S; S = S->getParent()) {
     if (S->getFlags() & Scope::FnScope) {
-      Actions.CodeCompleteOrdinaryName(getCurScope(),
+      Actions.CodeCompleteOrdinaryName(Tok, getCurScope(),
                                        Sema::PCC_RecoveryInFunction);
       cutOffParsing();
       return PrevTokLocation;
     }
     
     if (S->getFlags() & Scope::ClassScope) {
-      Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Class);
+      Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Class);
       cutOffParsing();
       return PrevTokLocation;
     }
   }
   
-  Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Namespace);
+  Actions.CodeCompleteOrdinaryName(Tok, getCurScope(), Sema::PCC_Namespace);
   cutOffParsing();
   return PrevTokLocation;
 }

 // Code-completion pass-through functions
 
 void Parser::CodeCompleteDirective(bool InConditional) {
-  Actions.CodeCompletePreprocessorDirective(InConditional);
+  Actions.CodeCompletePreprocessorDirective(Tok, InConditional);
 }
 
 void Parser::CodeCompleteInConditionalExclusion() {
-  Actions.CodeCompleteInPreprocessorConditionalExclusion(getCurScope());
+  Actions.CodeCompleteInPreprocessorConditionalExclusion(Tok, getCurScope());
 }
 
 void Parser::CodeCompleteMacroName(bool IsDefinition) {
-  Actions.CodeCompletePreprocessorMacroName(IsDefinition);
+  Actions.CodeCompletePreprocessorMacroName(Tok, IsDefinition);
 }
 
 void Parser::CodeCompletePreprocessorExpression() { 
-  Actions.CodeCompletePreprocessorExpression();
+  Actions.CodeCompletePreprocessorExpression(Tok);
 }
 
 void Parser::CodeCompleteMacroArgument(IdentifierInfo *Macro,
                                        MacroInfo *MacroInfo,
                                        unsigned ArgumentIndex) {
-  Actions.CodeCompletePreprocessorMacroArgument(getCurScope(), Macro, MacroInfo,
+  Actions.CodeCompletePreprocessorMacroArgument(Tok, getCurScope(), Macro, MacroInfo,
                                                 ArgumentIndex);
 }
 
 void Parser::CodeCompleteNaturalLanguage() {
-  Actions.CodeCompleteNaturalLanguage();
+  Actions.CodeCompleteNaturalLanguage(Tok);
 }
 
 bool Parser::ParseMicrosoftIfExistsCondition(IfExistsCondition& Result) {

   do {
     if (!Tok.is(tok::identifier)) {
       if (Tok.is(tok::code_completion)) {
-        Actions.CodeCompleteModuleImport(ImportLoc, Path);
+        Actions.CodeCompleteModuleImport(Tok, ImportLoc, Path);
         cutOffParsing();
         return DeclGroupPtrTy();
       }

lib/Sema/CodeCompleteConsumer.cpp

 #include "clang/AST/DeclTemplate.h"
 #include "clang/Sema/Scope.h"
 #include "clang/Sema/Sema.h"
+#include "clang/Lex/Token.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/Twine.h"

 
 CodeCompleteConsumer::~CodeCompleteConsumer() { }
 
-void 
+void
 PrintingCodeCompleteConsumer::ProcessCodeCompleteResults(Sema &SemaRef,
                                                  CodeCompletionContext Context,
                                                  CodeCompletionResult *Results,
-                                                         unsigned NumResults) {
+  Token &&Tok = Token();
+  ProcessCodeCompleteResults(Tok, SemaRef, Context, Results, NumResults);
+}
+
+void
+PrintingCodeCompleteConsumer::ProcessCodeCompleteResults(Token &Tok, Sema &SemaRef,
+ CodeCompletionContext Context,
+ CodeCompletionResult *Results,
+ unsigned NumResults) {
   std::stable_sort(Results, Results + NumResults);
-  
+  StringRef Filter;
+  if (Tok.is(tok::code_completion)) {
+    Filter = Tok.getCompletionStem();
+  }
   // Print the results.
   for (unsigned I = 0; I != NumResults; ++I) {
-    OS << "COMPLETION: ";
     switch (Results[I].Kind) {
       case CodeCompletionResult::RK_Declaration:
+      if (!Filter.empty()) {
+        if (!(*Results[I].Declaration).getIdentifier()) {
+          break;
+        }
+        else {
+          StringRef CompletionName = (*Results[I].Declaration).getName();
+          if (!CompletionName.startswith(Filter))
+            break;
+        }
+      }
+      OS << "COMPLETION: ";
       OS << *Results[I].Declaration;
       if (Results[I].Hidden)
         OS << " (Hidden)";
-      if (CodeCompletionString *CCS 
-            = Results[I].CreateCodeCompletionString(SemaRef, getAllocator(),
-                                                    CCTUInfo,
-                                                    includeBriefComments())) {
+      if (CodeCompletionString *CCS
         OS << " : " << CCS->getAsString();
         if (const char *BriefComment = CCS->getBriefComment())
           OS << " : " << BriefComment;
       }
-        
       OS << '\n';
       break;
-      
       case CodeCompletionResult::RK_Keyword:
+      if (!Filter.empty()) {
+        StringRef CompletionName = Results[I].Keyword;
+        if (!CompletionName.startswith(Filter))
+          break;
+      }
+      OS << "COMPLETION: ";
       OS << Results[I].Keyword << '\n';
       break;
-        
-    case CodeCompletionResult::RK_Macro: {
+      case CodeCompletionResult::RK_Macro:
+      if (!Filter.empty()) {
+        StringRef CompletionName = Results[I].Macro->getName();
+        if (!CompletionName.startswith(Filter))
+          break;
+      }
       OS << Results[I].Macro->getName();
-      if (CodeCompletionString *CCS 
-            = Results[I].CreateCodeCompletionString(SemaRef, getAllocator(),
-                                                    CCTUInfo,
-                                                    includeBriefComments())) {
+      if (CodeCompletionString *CCS
+        OS << "COMPLETION: ";
         OS << " : " << CCS->getAsString();
       }
       OS << '\n';
       break;
-    }
-        
-    case CodeCompletionResult::RK_Pattern: {
-      OS << "Pattern : " 
-         << Results[I].Pattern->getAsString() << '\n';
+
+      case CodeCompletionResult::RK_Pattern:
+      if (!Filter.empty()) {
+        StringRef CompletionName = Results[I].Pattern->getAsString();
+        if (!CompletionName.startswith(Filter))
+          break;
+      }
+      OS << "COMPLETION: ";
+      OS << "Pattern : "<< Results[I].Pattern->getAsString() << '\n';
       break;
     }
-    }
   }
 }
 

lib/Sema/SemaCodeComplete.cpp

   Results.ExitScope();
 }
 
-static void HandleCodeCompleteResults(Sema *S,
+static void HandleCodeCompleteResults(Token Tok, Sema *S,
                                       CodeCompleteConsumer *CodeCompleter,
                                       CodeCompletionContext Context,
                                       CodeCompletionResult *Results,
                                       unsigned NumResults) {
-  if (CodeCompleter)
-    CodeCompleter->ProcessCodeCompleteResults(*S, Context, Results, NumResults);
+  if (CodeCompleter) {
+    CodeCompleter->ProcessCodeCompleteResults(Tok, *S, Context, Results, NumResults);
+  }
 }
 
 static enum CodeCompletionContext::Kind mapCodeCompletionContext(Sema &S, 

   }
 }
 
-void Sema::CodeCompleteModuleImport(SourceLocation ImportLoc, 
+void Sema::CodeCompleteModuleImport(Token Tok, SourceLocation ImportLoc,
                                     ModuleIdPath Path) {
   typedef CodeCompletionResult Result;
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),

       }
     }
   }
-  Results.ExitScope();    
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
-                            Results.data(),Results.size());
+  Results.ExitScope();
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
+                            Results.data(), Results.size());
 }
 
-void Sema::CodeCompleteOrdinaryName(Scope *S, 
+void Sema::CodeCompleteOrdinaryName(Token Tok, Scope *S,
                                     ParserCompletionContext CompletionContext) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

   if (CodeCompleter->includeMacros())
     AddMacroResults(PP, Results, false);
   
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 

                                        bool IsSuper,
                                        ResultBuilder &Results);
 
-void Sema::CodeCompleteDeclSpec(Scope *S, DeclSpec &DS,
+void Sema::CodeCompleteDeclSpec(Token Tok, Scope *S, DeclSpec &DS,
                                 bool AllowNonIdentifiers,
                                 bool AllowNestedNameSpecifiers) {
   typedef CodeCompletionResult Result;

   // Note that we intentionally suppress macro results here, since we do not
   // encourage using macros to produce the names of entities.
 
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(), Results.size());
 }

 
 /// \brief Perform code-completion in an expression context when we know what
 /// type we're looking for.
-void Sema::CodeCompleteExpression(Scope *S, 
+void Sema::CodeCompleteExpression(Token Tok, Scope *S,
                                   const CodeCompleteExpressionData &Data) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

 
   if (CodeCompleter->includeMacros())
     AddMacroResults(PP, Results, false, PreferredTypeIsPointer);
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                 CodeCompletionContext(CodeCompletionContext::CCC_Expression, 
                                       Data.PreferredType),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompletePostfixExpression(Scope *S, ExprResult E) {
+void Sema::CodeCompletePostfixExpression(Token Tok, Scope *S, ExprResult E) {
   if (E.isInvalid())
-    CodeCompleteOrdinaryName(S, PCC_RecoveryInFunction);
+    CodeCompleteOrdinaryName(Tok, S, PCC_RecoveryInFunction);
   else if (getLangOpts().ObjC1)
-    CodeCompleteObjCInstanceMessage(S, E.get(), None, false);
+    CodeCompleteObjCInstanceMessage(Tok, S, E.get(), None, false);
 }
 
 /// \brief The set of properties that have already been added, referenced by

   }
 }
 
-void Sema::CodeCompleteMemberReferenceExpr(Scope *S, Expr *Base,
+void Sema::CodeCompleteMemberReferenceExpr(Token Tok, Scope *S, Expr *Base,
                                            SourceLocation OpLoc,
                                            bool IsArrow) {
   if (!Base || !CodeCompleter)

   Results.ExitScope();
 
   // Hand off the results found for code completion.
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteTag(Scope *S, unsigned TagSpec) {
+void Sema::CodeCompleteTag(Token Tok, Scope *S, unsigned TagSpec) {
   if (!CodeCompleter)
     return;
 

     LookupVisibleDecls(S, LookupNestedNameSpecifierName, Consumer);
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteTypeQualifiers(DeclSpec &DS) {
+void Sema::CodeCompleteTypeQualifiers(Token Tok, DeclSpec &DS) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_TypeQualifiers);

       !(DS.getTypeQualifiers() & DeclSpec::TQ_atomic))
     Results.AddResult("_Atomic");
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(), Results.size());
 }
 
-void Sema::CodeCompleteCase(Scope *S) {
+void Sema::CodeCompleteCase(Token Tok, Scope *S) {
   if (getCurFunction()->SwitchStack.empty() || !CodeCompleter)
     return;
 

   if (!type->isEnumeralType()) {
     CodeCompleteExpressionData Data(type);
     Data.IntegralConstantExpression = true;
-    CodeCompleteExpression(S, Data);
+    CodeCompleteExpression(Tok, S, Data);
     return;
   }
   

     kind = CodeCompletionContext::CCC_OtherWithMacros;
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             kind,
                             Results.data(),Results.size());
 }

   return ParamType;
 }
 
-static void CodeCompleteOverloadResults(Sema &SemaRef, Scope *S,
+static void CodeCompleteOverloadResults(Token Tok, Sema &SemaRef, Scope *S,
                                     MutableArrayRef<ResultCandidate> Candidates,
                                         unsigned CurrentArg,
                                  bool CompleteExpressionWithCurrentArg = true) {

     ParamType = getParamType(SemaRef, Candidates, CurrentArg);
 
   if (ParamType.isNull())
-    SemaRef.CodeCompleteOrdinaryName(S, Sema::PCC_Expression);
+    SemaRef.CodeCompleteOrdinaryName(Tok, S, Sema::PCC_Expression);
   else
-    SemaRef.CodeCompleteExpression(S, ParamType);
+    SemaRef.CodeCompleteExpression(Tok, S, ParamType);
 
   if (!Candidates.empty())
     SemaRef.CodeCompleter->ProcessOverloadCandidates(SemaRef, CurrentArg,

                                                      Candidates.size());
 }
 
-void Sema::CodeCompleteCall(Scope *S, Expr *Fn, ArrayRef<Expr *> Args) {
+void Sema::CodeCompleteCall(Token Tok, Scope *S, Expr *Fn, ArrayRef<Expr *> Args) {
   if (!CodeCompleter)
     return;
 

   // Ignore type-dependent call expressions entirely.
   if (!Fn || Fn->isTypeDependent() || anyNullArguments(Args) ||
       Expr::hasAnyTypeDependentArguments(Args)) {
-    CodeCompleteOrdinaryName(S, PCC_Expression);
+    CodeCompleteOrdinaryName(Tok, S, PCC_Expression);
     return;
   }
 

   }
 
   mergeCandidatesWithResults(*this, Results, CandidateSet, Loc);
-  CodeCompleteOverloadResults(*this, S, Results, Args.size(),
+  CodeCompleteOverloadResults(Tok, *this, S, Results, Args.size(),
                               !CandidateSet.empty());
 }
 
-void Sema::CodeCompleteConstructor(Scope *S, QualType Type, SourceLocation Loc,
+void Sema::CodeCompleteConstructor(Token Tok, Scope *S, QualType Type, SourceLocation Loc,
                                    ArrayRef<Expr *> Args) {
   if (!CodeCompleter)
     return;

 
   SmallVector<ResultCandidate, 8> Results;
   mergeCandidatesWithResults(*this, Results, CandidateSet, Loc);
-  CodeCompleteOverloadResults(*this, S, Results, Args.size());
+  CodeCompleteOverloadResults(Tok, *this, S, Results, Args.size());
 }
 
-void Sema::CodeCompleteInitializer(Scope *S, Decl *D) {
+void Sema::CodeCompleteInitializer(Token Tok, Scope *S, Decl *D) {
   ValueDecl *VD = dyn_cast_or_null<ValueDecl>(D);
   if (!VD) {
-    CodeCompleteOrdinaryName(S, PCC_Expression);
+    CodeCompleteOrdinaryName(Tok, S, PCC_Expression);
     return;
   }
   
-  CodeCompleteExpression(S, VD->getType());
+  CodeCompleteExpression(Tok, S, VD->getType());
 }
 
-void Sema::CodeCompleteReturn(Scope *S) {
+void Sema::CodeCompleteReturn(Token Tok, Scope *S) {
   QualType ResultType;
   if (isa<BlockDecl>(CurContext)) {
     if (BlockScopeInfo *BSI = getCurBlock())

     ResultType = Method->getReturnType();
 
   if (ResultType.isNull())
-    CodeCompleteOrdinaryName(S, PCC_Expression);
+    CodeCompleteOrdinaryName(Tok, S, PCC_Expression);
   else
-    CodeCompleteExpression(S, ResultType);
+    CodeCompleteExpression(Tok, S, ResultType);
 }
 
-void Sema::CodeCompleteAfterIf(Scope *S) {
+void Sema::CodeCompleteAfterIf(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         mapCodeCompletionContext(*this, PCC_Statement));

   if (CodeCompleter->includeMacros())
     AddMacroResults(PP, Results, false);
   
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteAssignmentRHS(Scope *S, Expr *LHS) {
+void Sema::CodeCompleteAssignmentRHS(Token Tok, Scope *S, Expr *LHS) {
   if (LHS)
-    CodeCompleteExpression(S, static_cast<Expr *>(LHS)->getType());
+    CodeCompleteExpression(Tok, S, static_cast<Expr *>(LHS)->getType());
   else
-    CodeCompleteOrdinaryName(S, PCC_Expression);
+    CodeCompleteOrdinaryName(Tok, S, PCC_Expression);
 }
 
-void Sema::CodeCompleteQualifiedId(Scope *S, CXXScopeSpec &SS,
+void Sema::CodeCompleteQualifiedId(Token Tok, Scope *S, CXXScopeSpec &SS,
                                    bool EnteringContext) {
   if (!SS.getScopeRep() || !CodeCompleter)
     return;

   CodeCompletionDeclConsumer Consumer(Results, CurContext);
   LookupVisibleDecls(Ctx, LookupOrdinaryName, Consumer);
 
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteUsing(Scope *S) {
+void Sema::CodeCompleteUsing(Token Tok, Scope *S) {
   if (!CodeCompleter)
     return;
   

                      CodeCompleter->includeGlobals());
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_PotentiallyQualifiedName,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteUsingDirective(Scope *S) {
+void Sema::CodeCompleteUsingDirective(Token Tok, Scope *S) {
   if (!CodeCompleter)
     return;
   

   LookupVisibleDecls(S, LookupOrdinaryName, Consumer,
                      CodeCompleter->includeGlobals());
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Namespace,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteNamespaceDecl(Scope *S)  {
+void Sema::CodeCompleteNamespaceDecl(Token Tok, Scope *S)  {
   if (!CodeCompleter)
     return;
   

     Results.ExitScope();
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteNamespaceAliasDecl(Scope *S)  {
+void Sema::CodeCompleteNamespaceAliasDecl(Token Tok, Scope *S)  {
   if (!CodeCompleter)
     return;
   

   CodeCompletionDeclConsumer Consumer(Results, CurContext);
   LookupVisibleDecls(S, LookupOrdinaryName, Consumer,
                      CodeCompleter->includeGlobals());
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteOperatorName(Scope *S) {
+void Sema::CodeCompleteOperatorName(Token Tok, Scope *S) {
   if (!CodeCompleter)
     return;
 

   AddTypeSpecifierResults(getLangOpts(), Results);
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Type,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteConstructorInitializer(
+void Sema::CodeCompleteConstructorInitializer(Token Tok,
                               Decl *ConstructorD,
                               ArrayRef <CXXCtorInitializer *> Initializers) {
   PrintingPolicy Policy = getCompletionPrintingPolicy(*this);

   }
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(), Results.size());
 }
 

   return DC->isFileContext();
 }
 
-void Sema::CodeCompleteLambdaIntroducer(Scope *S, LambdaIntroducer &Intro,
+void Sema::CodeCompleteLambdaIntroducer(Token Tok, Scope *S, LambdaIntroducer &Intro,
                                         bool AfterAmpersand) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

   
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(), Results.size());
 }
 

   }
 }
 
-void Sema::CodeCompleteObjCAtDirective(Scope *S) {
+void Sema::CodeCompleteObjCAtDirective(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);

   else
     AddObjCTopLevelResults(Results, false);
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }

     Results.AddResult(Result(OBJC_AT_KEYWORD_NAME(NeedAt,"package")));
 }
 
-void Sema::CodeCompleteObjCAtVisibility(Scope *S) {
+void Sema::CodeCompleteObjCAtVisibility(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);
   Results.EnterNewScope();
   AddObjCVisibilityResults(getLangOpts(), Results, false);
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCAtStatement(Scope *S) {
+void Sema::CodeCompleteObjCAtStatement(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);

   AddObjCStatementResults(Results, false);
   AddObjCExpressionResults(Results, false);
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCAtExpression(Scope *S) {
+void Sema::CodeCompleteObjCAtExpression(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);
   Results.EnterNewScope();
   AddObjCExpressionResults(Results, false);
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }

   return false;
 }
 
-void Sema::CodeCompleteObjCPropertyFlags(Scope *S, ObjCDeclSpec &ODS) { 
+void Sema::CodeCompleteObjCPropertyFlags(Token Tok, Scope *S, ObjCDeclSpec &ODS) {
   if (!CodeCompleter)
     return;
   

     Results.AddResult(CodeCompletionResult(Getter.TakeString()));
   }
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }

 }
 
 
-void Sema::CodeCompleteObjCPropertyGetter(Scope *S) {
+void Sema::CodeCompleteObjCPropertyGetter(Token Tok, Scope *S) {
   // Try to find the interface where getters might live.
   ObjCInterfaceDecl *Class = dyn_cast_or_null<ObjCInterfaceDecl>(CurContext);
   if (!Class) {

   AddObjCMethods(Class, true, MK_ZeroArgSelector, None, CurContext, Selectors,
                  /*AllowSameLength=*/true, Results);
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter,
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCPropertySetter(Scope *S) {
+void Sema::CodeCompleteObjCPropertySetter(Token Tok, Scope *S) {
   // Try to find the interface where setters might live.
   ObjCInterfaceDecl *Class
     = dyn_cast_or_null<ObjCInterfaceDecl>(CurContext);

                  Selectors, /*AllowSameLength=*/true, Results);
 
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter,
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCPassingType(Scope *S, ObjCDeclSpec &DS,
+void Sema::CodeCompleteObjCPassingType(Token Tok, Scope *S, ObjCDeclSpec &DS,
                                        bool IsParameter) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

   if (CodeCompleter->includeMacros())
     AddMacroResults(PP, Results, false);
 
-  HandleCodeCompleteResults(this, CodeCompleter,
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Type,
                             Results.data(), Results.size());
 }

   return SuperMethod;
 }
                                    
-void Sema::CodeCompleteObjCMessageReceiver(Scope *S) {
+void Sema::CodeCompleteObjCMessageReceiver(Token Tok, Scope *S) {
   typedef CodeCompletionResult Result;
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

   
   if (CodeCompleter->includeMacros())
     AddMacroResults(PP, Results, false);
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(), Results.size());
   
 }
 
-void Sema::CodeCompleteObjCSuperMessage(Scope *S, SourceLocation SuperLoc,
+void Sema::CodeCompleteObjCSuperMessage(Token Tok, Scope *S, SourceLocation SuperLoc,
                                         ArrayRef<IdentifierInfo *> SelIdents,
                                         bool AtArgumentExpression) {
   ObjCInterfaceDecl *CDecl = nullptr;

       // We are inside an instance method, which means that the message
       // send [super ...] is actually calling an instance method on the
       // current object.
-      return CodeCompleteObjCInstanceMessage(S, nullptr, SelIdents,
+      return CodeCompleteObjCInstanceMessage(Tok, S, nullptr, SelIdents,
                                              AtArgumentExpression,
                                              CDecl);
     }

       id.setIdentifier(Super, SuperLoc);
       ExprResult SuperExpr = ActOnIdExpression(S, SS, TemplateKWLoc, id,
                                                false, false);
-      return CodeCompleteObjCInstanceMessage(S, (Expr *)SuperExpr.get(),
+      return CodeCompleteObjCInstanceMessage(Tok, S, (Expr *)SuperExpr.get(),
                                              SelIdents,
                                              AtArgumentExpression);
     }

   ParsedType Receiver;
   if (CDecl)
     Receiver = ParsedType::make(Context.getObjCInterfaceType(CDecl));
-  return CodeCompleteObjCClassMessage(S, Receiver, SelIdents, 
+  return CodeCompleteObjCClassMessage(Tok, S, Receiver, SelIdents,
                                       AtArgumentExpression,
                                       /*IsSuper=*/true);
 }

   Results.ExitScope();  
 }
 
-void Sema::CodeCompleteObjCClassMessage(Scope *S, ParsedType Receiver,
+void Sema::CodeCompleteObjCClassMessage(Token Tok, Scope *S, ParsedType Receiver,
                                         ArrayRef<IdentifierInfo *> SelIdents,
                                         bool AtArgumentExpression,
                                         bool IsSuper) {

     QualType PreferredType = getPreferredArgumentTypeForMessageSend(Results, 
                                                               SelIdents.size());
     if (PreferredType.isNull())
-      CodeCompleteOrdinaryName(S, PCC_Expression);
+      CodeCompleteOrdinaryName(Tok, S, PCC_Expression);
     else
-      CodeCompleteExpression(S, PreferredType);
+      CodeCompleteExpression(Tok, S, PreferredType);
     return;
   }
 
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(), Results.size());
 }
 
-void Sema::CodeCompleteObjCInstanceMessage(Scope *S, Expr *Receiver,
+void Sema::CodeCompleteObjCInstanceMessage(Token Tok, Scope *S, Expr *Receiver,
                                            ArrayRef<IdentifierInfo *> SelIdents,
                                            bool AtArgumentExpression,
                                            ObjCInterfaceDecl *Super) {

   if (ReceiverType->isObjCIdType() || ReceiverType->isObjCClassType()) {
     if (ObjCInterfaceDecl *IFace = GetAssumedMessageSendExprType(RecExpr)) {
       if (ReceiverType->isObjCClassType())
-        return CodeCompleteObjCClassMessage(S, 
+        return CodeCompleteObjCClassMessage(Tok, S,
                        ParsedType::make(Context.getObjCInterfaceType(IFace)),
                                             SelIdents,
                                             AtArgumentExpression, Super);

     QualType PreferredType = getPreferredArgumentTypeForMessageSend(Results, 
                                                               SelIdents.size());
     if (PreferredType.isNull())
-      CodeCompleteOrdinaryName(S, PCC_Expression);
+      CodeCompleteOrdinaryName(Tok, S, PCC_Expression);
     else
-      CodeCompleteExpression(S, PreferredType);
+      CodeCompleteExpression(Tok, S, PreferredType);
     return;
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             Results.getCompletionContext(),
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCForCollection(Scope *S, 
+void Sema::CodeCompleteObjCForCollection(Token Tok, Scope *S,
                                          DeclGroupPtrTy IterationVar) {
   CodeCompleteExpressionData Data;
   Data.ObjCCollection = true;

     }
   }
   
-  CodeCompleteExpression(S, Data);
+  CodeCompleteExpression(Tok, S, Data);
 }
 
-void Sema::CodeCompleteObjCSelector(Scope *S,
+void Sema::CodeCompleteObjCSelector(Token Tok, Scope *S,
                                     ArrayRef<IdentifierInfo *> SelIdents) {
   // If we have an external source, load the entire class method
   // pool from the AST file.

   }
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_SelectorName,
                             Results.data(), Results.size());
 }

   }
 }
 
-void Sema::CodeCompleteObjCProtocolReferences(IdentifierLocPair *Protocols,
+void Sema::CodeCompleteObjCProtocolReferences(Token Tok, IdentifierLocPair *Protocols,
                                               unsigned NumProtocols) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

     Results.ExitScope();
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCProtocolName,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCProtocolDecl(Scope *) {
+void Sema::CodeCompleteObjCProtocolDecl(Token Tok, Scope *) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_ObjCProtocolName);

     Results.ExitScope();
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCProtocolName,
                             Results.data(),Results.size());
 }

   }
 }
 
-void Sema::CodeCompleteObjCInterfaceDecl(Scope *S) { 
+void Sema::CodeCompleteObjCInterfaceDecl(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);

   
   Results.ExitScope();
 
-  HandleCodeCompleteResults(this, CodeCompleter,
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCInterfaceName,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCSuperclass(Scope *S, IdentifierInfo *ClassName,
+void Sema::CodeCompleteObjCSuperclass(Token Tok, Scope *S, IdentifierInfo *ClassName,
                                       SourceLocation ClassNameLoc) { 
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

   
   Results.ExitScope();
 
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCInterfaceName,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCImplementationDecl(Scope *S) { 
+void Sema::CodeCompleteObjCImplementationDecl(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);

   
   Results.ExitScope();
 
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCInterfaceName,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCInterfaceCategory(Scope *S, 
+void Sema::CodeCompleteObjCInterfaceCategory(Token Tok, Scope *S,
                                              IdentifierInfo *ClassName,
                                              SourceLocation ClassNameLoc) {
   typedef CodeCompletionResult Result;

                           CurContext, nullptr, false);
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCCategoryName,
                             Results.data(),Results.size());  
 }
 
-void Sema::CodeCompleteObjCImplementationCategory(Scope *S, 
+void Sema::CodeCompleteObjCImplementationCategory(Token Tok, Scope *S,
                                                   IdentifierInfo *ClassName,
                                                   SourceLocation ClassNameLoc) {
   typedef CodeCompletionResult Result;

     = LookupSingleName(TUScope, ClassName, ClassNameLoc, LookupOrdinaryName);
   ObjCInterfaceDecl *Class = dyn_cast_or_null<ObjCInterfaceDecl>(CurClass);
   if (!Class)
-    return CodeCompleteObjCInterfaceCategory(S, ClassName, ClassNameLoc);
+    return CodeCompleteObjCInterfaceCategory(Tok, S, ClassName, ClassNameLoc);
     
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),

   }
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_ObjCCategoryName,
                             Results.data(),Results.size());  
 }
 
-void Sema::CodeCompleteObjCPropertyDefinition(Scope *S) {
+void Sema::CodeCompleteObjCPropertyDefinition(Token Tok, Scope *S) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_Other);

                       AddedProperties, Results);
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());  
 }
 
-void Sema::CodeCompleteObjCPropertySynthesizeIvar(Scope *S, 
+void Sema::CodeCompleteObjCPropertySynthesizeIvar(Token Tok, Scope *S,
                                                   IdentifierInfo *PropertyName) {
   typedef CodeCompletionResult Result;
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),

   
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }

   }
 }
 
-void Sema::CodeCompleteObjCMethodDecl(Scope *S, 
+void Sema::CodeCompleteObjCMethodDecl(Token Tok, Scope *S,
                                       bool IsInstanceMethod,
                                       ParsedType ReturnTy) {
   // Determine the return type of the method we're declaring, if

   }
 
   if (!SearchDecl) {
-    HandleCodeCompleteResults(this, CodeCompleter, 
+    HandleCodeCompleteResults(Tok, this, CodeCompleter,
                               CodeCompletionContext::CCC_Other,
                               nullptr, 0);
     return;

   
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompleteObjCMethodDeclSelector(Scope *S, 
+void Sema::CodeCompleteObjCMethodDeclSelector(Token Tok, Scope *S,
                                               bool IsInstanceMethod,
                                               bool AtParameterName,
                                               ParsedType ReturnTy,

   }
   
   Results.ExitScope();
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_Other,
                             Results.data(),Results.size());
 }
 
-void Sema::CodeCompletePreprocessorDirective(bool InConditional) {
+void Sema::CodeCompletePreprocessorDirective(Token Tok, bool InConditional) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_PreprocessorDirective);

   // FIXME: we don't support #assert or #unassert, so don't suggest them.
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_PreprocessorDirective,
                             Results.data(), Results.size());
 }
 
-void Sema::CodeCompleteInPreprocessorConditionalExclusion(Scope *S) {
-  CodeCompleteOrdinaryName(S,
+void Sema::CodeCompleteInPreprocessorConditionalExclusion(Token Tok, Scope *S) {
+  CodeCompleteOrdinaryName(Tok, S,
                            S->getFnParent()? Sema::PCC_RecoveryInFunction 
                                            : Sema::PCC_Namespace);
 }
 
-void Sema::CodeCompletePreprocessorMacroName(bool IsDefinition) {
+void Sema::CodeCompletePreprocessorMacroName(Token Tok, bool IsDefinition) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         IsDefinition? CodeCompletionContext::CCC_MacroName

     // FIXME: Can we detect when the user just wrote an include guard above?
   }
   
-  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+  HandleCodeCompleteResults(Tok, this, CodeCompleter, Results.getCompletionContext(),
                             Results.data(), Results.size()); 
 }
 
-void Sema::CodeCompletePreprocessorExpression() {
+void Sema::CodeCompletePreprocessorExpression(Token Tok) {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),
                         CodeCompleter->getCodeCompletionTUInfo(),
                         CodeCompletionContext::CCC_PreprocessorExpression);

   Results.AddResult(Builder.TakeString());
   Results.ExitScope();
   
-  HandleCodeCompleteResults(this, CodeCompleter, 
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_PreprocessorExpression,
                             Results.data(), Results.size()); 
 }
 
-void Sema::CodeCompletePreprocessorMacroArgument(Scope *S,
+void Sema::CodeCompletePreprocessorMacroArgument(Token Tok, Scope *S,
                                                  IdentifierInfo *Macro,
                                                  MacroInfo *MacroInfo,
                                                  unsigned Argument) {

   // for the expanded tokens.
 }
 
-void Sema::CodeCompleteNaturalLanguage() {
-  HandleCodeCompleteResults(this, CodeCompleter,
+void Sema::CodeCompleteNaturalLanguage(Token Tok) {
+  HandleCodeCompleteResults(Tok, this, CodeCompleter,
                             CodeCompletionContext::CCC_NaturalLanguage,
                             nullptr, 0);
 }

test/CodeCompletion/Filtering/filter-function-name.cpp

+void myFunction();
+
+int main() {
+  myFunc
+  // RUN: %clang_cc1 -code-completion-at=%s:4:11 %s -o - | FileCheck -check-prefix=CHECK-CC1 %s
+  // CHECK-CC1: COMPLETION: myFunction : [#void#]myFunction()
+}
+

test/CodeCompletion/Filtering/filter-member-access.cpp

+class MyClass {
+private:
+  int anInteger;
+  double aDouble;
+
+public:
+  MyClass(int i, double d) {
+  anInteger = i;
+  aDouble = d;
+  } 
+  int getInt() { return anInteger; }
+  double getDouble() { return aDouble; }
+  void setInt(int value) { anInteger = value; }
+  void setDouble(double value) { aDouble = value; }
+};
+
+int main() {
+  MyClass* objectMyClass = new MyClass(5, 1.0);
+  objectMyClass->set
+  // RUN: %clang_cc1 -code-completion-at=%s:19:23 %s -o - | FileCheck -check-prefix=CHECK-CC1 %s
+  // CHECK-CC1: COMPLETION: setDouble : [#void#]setDouble(<#double value#>)
+  // CHECK-CC1-NEXT: COMPLETION: setInt : [#void#]setInt(<#int value#>)
+  return 0;
+}
 No newline at end of file

test/CodeCompletion/Filtering/filter-namespace.cpp

+namespace outerspace {
+  namespace innerspace { }
+}
+
+using namespace outerspace::inner
+// RUN: %clang_cc1 -code-completion-at=%s:5:34 %s -o - | FileCheck -check-prefix=CHECK-CC1 %s
+// CHECK-CC1: COMPLETION: innerspace : innerspace

test/CodeCompletion/Filtering/filter-ordinary-name.cpp

+int main() {
+  int myVariableOne;
+  int myVariableTwo;
+  myVar
+  // RUN: %clang_cc1 -code-completion-at=%s:4:10 %s -o - | FileCheck -check-prefix=CHECK-CC1 %s
+  // CHECK-CC1: COMPLETION: myVariableOne : [#int#]myVariableOne
+  // CHECK-CC1-NEXT: COMPLETION: myVariableTwo : [#int#]myVariableTwo
+}
 No newline at end of file