[CodeComplete] Fix completion in the middle of ident in ctor lists.

[ilya-biryukov]

Summary

The example that was broken before (^ designate completion points):

class Foo {
  Foo() : fie^ld^() {} // no completions were provided here.
  int field;
};

To fix it we don't cut off lexing after an identifier followed by code
completion token is lexed. Instead we skip the rest of identifier and
continue lexing.
This is consistent with behavior of completion when completion token is
right before the identifier.

[ilya-biryukov]

+Alex, in case he might know someone who can review it.

[aaron.ballman]

lib/Lex/Lexer.cpp

1665–1666

You should continue to assert that CurPtr < BufferEnd in this loop, no?

[ilya-biryukov]

• Fixed failing assert on the end of file. Added a test for that.

[ilya-biryukov]

lib/Lex/Lexer.cpp

1665–1666

AFAIK, all buffers end with '\0', so lexer usually loops without doing bound checking, e.g. a similar loop is at Lexer.cpp:1623.

It turned out the completion token is not added when completion point is at the end of the buffer, though, fixed that.

[aaron.ballman]

LGTM, modulo a tiny commenting nit.

lib/Lex/Lexer.cpp

1665

when completion -> when the completion

[ilya-biryukov]

• Fix the comment

lib/Lex/Lexer.cpp

FinishIdentifier:
     // completion, it writes IdentifierInfo into Result, and callers rely on it.
 
     // If the completion point is at the end of an identifier, we want to treat
     // the identifier as incomplete even if it resolves to a macro or a keyword.
     // This allows e.g. 'class^' to complete to 'classifier'.
     if (isCodeCompletionPoint(CurPtr)) {
       // Return the code-completion token.
       Result.setKind(tok::code_completion);
-      cutOffLexing();
+      // Skip the code-completion char and all immediate identifier characters.
+      // This ensures we get consistent behavior when completing at any point in
+      // an identifier (i.e. at the start, in the middle, at the end). Note that
+      // only simple cases (i.e. [a-zA-Z0-9_]) are supported to keep the code
+      // simpler.
+      assert(*CurPtr == 0 && "Completion character must be 0");
+      ++CurPtr;
+      // Note that code completion token is not added as a separate character
+      // when the completion point is at the end of the buffer. Therefore, we need
+      // to check if the buffer has ended.
+      if (CurPtr < BufferEnd) {
+        while (isIdentifierBody(*CurPtr))
+          ++CurPtr;
+      }
+      BufferPtr = CurPtr;
       return true;
     }
 
     // Finally, now that we know we have an identifier, pass this off to the
     // preprocessor, which may macro expand it or something.
     if (II->isHandleIdentifierCase())
       return PP->HandleIdentifier(Result);
 

test/CodeCompletion/ctor-initializer.cpp

struct A {
   int member1, member2;
 };
 
 struct B {
   B() : member2() {}
   // RUN: %clang_cc1 -fsyntax-only -std=c++98 -code-completion-at=%s:57:9 %s -o - | FileCheck -check-prefix=CHECK-CC7 %s
   // RUN: %clang_cc1 -fsyntax-only -std=c++14 -code-completion-at=%s:57:9 %s -o - | FileCheck -check-prefix=CHECK-CC7 %s
   // CHECK-CC7: COMPLETION: Pattern : member1(<#args#>
+  // Check in the middle and at the end of identifier too.
+  // RUN: %clang_cc1 -fsyntax-only -std=c++98 -code-completion-at=%s:57:13 %s -o - | FileCheck -check-prefix=CHECK-CC8 %s
+  // RUN: %clang_cc1 -fsyntax-only -std=c++98 -code-completion-at=%s:57:16 %s -o - | FileCheck -check-prefix=CHECK-CC8 %s
+  // CHECK-CC8: COMPLETION: Pattern : member2(<#args#>
   int member1, member2;
 };

test/CodeCompletion/end-of-file.cpp

+// Check that clang does not crash when completing at the last char in the
+// buffer.
+// NOTE: This file must *NOT* have newline at the end.
+// RUN: %clang_cc1 -code-completion-at=%s:7:2 %s | FileCheck %s
+// CHECK: COMPLETION: foo
+using foo = int***;
+f
 No newline at end of file