[CodeComplete] Fix completion in the middle of ident in ctor lists.

[ilya-biryukov]

Summary

The example that was broken before (^ designate completion points):

class Foo {
  Foo() : fie^ld^() {} // no completions were provided here.
  int field;
};

To fix it we don't cut off lexing after an identifier followed by code
completion token is lexed. Instead we skip the rest of identifier and
continue lexing.
This is consistent with behavior of completion when completion token is
right before the identifier.

[ilya-biryukov]

+Alex, in case he might know someone who can review it.

[aaron.ballman]

lib/Lex/Lexer.cpp

1667–1668

You should continue to assert that CurPtr < BufferEnd in this loop, no?

[ilya-biryukov]

• Fixed failing assert on the end of file. Added a test for that.

[ilya-biryukov]

lib/Lex/Lexer.cpp

1667–1668

AFAIK, all buffers end with '\0', so lexer usually loops without doing bound checking, e.g. a similar loop is at Lexer.cpp:1623.

It turned out the completion token is not added when completion point is at the end of the buffer, though, fixed that.

[aaron.ballman]

LGTM, modulo a tiny commenting nit.

lib/Lex/Lexer.cpp

1667

when completion -> when the completion

[ilya-biryukov]

• Fix the comment

lib/Lex/Lexer.cpp

FinishIdentifier:
     if (II->isHandleIdentifierCase())
       return PP->HandleIdentifier(Result);
 
     if (II->getTokenID() == tok::identifier && isCodeCompletionPoint(CurPtr)
         && II->getPPKeywordID() == tok::pp_not_keyword
         && II->getObjCKeywordID() == tok::objc_not_keyword) {
       // Return the code-completion token.
       Result.setKind(tok::code_completion);
-      cutOffLexing();
+      // Skip the code-completion char and all immediate identifier characters.
+      // This ensures we get consistent behavior when completing at any point in
+      // an identifier (i.e. at the start, in the middle, at the end). Note that
+      // only simple cases (i.e. [a-zA-Z0-9_]) are supported to keep the code
+      // simpler.
+      assert(*CurPtr == 0 && "Completion character must be 0");
+      ++CurPtr;
+      // Note that code completion token is not added as a separate character
+      // when completion point is at the end of the buffer. Therefore, we need
+      // to check if the buffer has ended.
+      if (CurPtr < BufferEnd) {
+        while (isIdentifierBody(*CurPtr))
+          ++CurPtr;
+      }
+      BufferPtr = CurPtr;
       return true;
     }
     return true;
   }
 
   // Otherwise, $,\,? in identifier found.  Enter slower path.
 
   C = getCharAndSize(CurPtr, Size);

test/CodeCompletion/ctor-initializer.cpp

struct A {
   int member1, member2;
 };
 
 struct B {
   B() : member2() {}
   // RUN: %clang_cc1 -fsyntax-only -std=c++98 -code-completion-at=%s:57:9 %s -o - | FileCheck -check-prefix=CHECK-CC7 %s
   // RUN: %clang_cc1 -fsyntax-only -std=c++14 -code-completion-at=%s:57:9 %s -o - | FileCheck -check-prefix=CHECK-CC7 %s
   // CHECK-CC7: COMPLETION: Pattern : member1(<#args#>
+  // Check in the middle and at the end of identifier too.
+  // RUN: %clang_cc1 -fsyntax-only -std=c++98 -code-completion-at=%s:57:13 %s -o - | FileCheck -check-prefix=CHECK-CC8 %s
+  // RUN: %clang_cc1 -fsyntax-only -std=c++98 -code-completion-at=%s:57:16 %s -o - | FileCheck -check-prefix=CHECK-CC8 %s
+  // CHECK-CC8: COMPLETION: Pattern : member2(<#args#>
   int member1, member2;
 };

test/CodeCompletion/end-of-file.cpp

+// Check that clang does not crash when completing at the last char in the
+// buffer.
+// NOTE: This file must *NOT* have newline at the end.
+// RUN: %clang_cc1 -code-completion-at=%s:7:2 %s | FileCheck %s
+// CHECK: COMPLETION: foo
+using foo = int***;
+f
 No newline at end of file