[libclang] Optionally add code completion results for arrow instead of dot

[yvvan]

Summary

Follow up for https://reviews.llvm.org/D41537 - libclang part is extracted into this review

[yvvan]

The base revision has changed - some minor changes were also required here.

[yvvan]

Output range of token being replaced instead of it's text

As a side-effect introduce a call to get a Token starting from specified location.

[yvvan]

Update according to c++ part changes

[nik]

Generates build errors here:

In file included from /d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c:4:0:
/d2/llvm/qtc/source/tools/clang/include/clang-c/Index.h:1332:62: warning: comma at end of enumerator list [-Wpedantic]
/d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c: In function 'tokens_spelling_at_range':
/d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c:2263:3: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
/d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c:2263:3: note: use option -std=c99, -std=gnu99, -std=c11 or -std=gnu11 to compile your code
/d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c: In function 'print_completion_result':
/d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c:2365:3: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
At top level:
/d2/llvm/qtc/source/tools/clang/tools/c-index-test/c-index-test.c:2257:13: warning: 'tokens_spelling_at_range' defined but not used [-Wunused-function]

include/clang-c/Index.h

85

s/string/index

88

Currently there is not more, so I would avoid the "etc.".

92

s/tokens/token

93

• Please use proper documentation style, e.g. make use of "\brief", "\param", "\returns". Currently one has to somewhat guess that the returned string is related to the replacement_range parameter.

• Is this a behavior change now? What happens to old code not aware of this function?

95

Please avoid double negation.

97

s/editors/clients/

98

Do you mean replacement_range instead of RemoveRange?

113

The vec_ptr-> example should go to its own line.

test/Index/complete-arrow-dot.cpp

21

That's pretty verbose, consider to use wildcards/regexes instead just checking for "requires fix-it:".

tools/c-index-test/c-index-test.c

41

Keep terminology of the API - s/CORRECTIONS/FIXITS

[yvvan]

Address review comments

include/clang-c/Index.h

88

It might change without Index.h being updated.

93

'\brief' is not used anymore.

Is this a behavior change now?

No, without CXCodeComplete_IncludeFixIts set you won't event have extra completions. And existing ones won't get any fix-its (because why would they?)

98

Yes, it's repalcement_range here

113

I've fixed it the same way it's done in CodeCompleteConsumer.h

[nik]

include/clang-c/Index.h

85

Indicate that calling this makes only sense if CXCodeComplete_IncludeFixIts was set.

86

"Tokenize the source code" sounds as if more than one token will be handled.

Suggestion: Reduce to "Get the raw lexical token starting with the given location".

90

Suggestion: CXCodeComplete_IncludeCompletionsWithFixIts

93

Still \param and \returns are useful since there the range is an output argument.

95

Clarify that you refer to clang_codeCompleteAt here.

Suggestion: By default, clang_codeCompleteAt()....

97

Add empty line before "For the client..." for readability.

98

...of the cursor after applying the fixits

106

new line

110

new line

112

"one of the ...", same below.

[nik]

• Sometimes you refer to "fixits", sometimes "fix-its" and some times "FixIts". Unify to what is already there.
• The term "completion items" is new so far. Use "completions" for consistency.

[yvvan]

Address review comments

[nik]

include/clang-c/Index.h

87

s/completion items/completions/g

126

Please document the parameters. It's more important here than for any other function that was introduced.

[yvvan]

Address comments about Index.h new functions parameters

[yvvan]

Diff with unix line endings

include/clang-c/Index.h

  * The version constants for the libclang API.
  * CINDEX_VERSION_MINOR should increase when there are API additions.
  * CINDEX_VERSION_MAJOR is intended for "major" source/ABI breaking changes.
  *
  * The policy about the libclang API was always to keep it source and ABI
  * compatible, thus CINDEX_VERSION_MAJOR is expected to remain stable.
  */
 #define CINDEX_VERSION_MAJOR 0
-#define CINDEX_VERSION_MINOR 48
+#define CINDEX_VERSION_MINOR 49
 
 #define CINDEX_VERSION_ENCODE(major, minor) ( \
       ((major) * 10000)                       \
     + ((minor) *     1))
 
 #define CINDEX_VERSION CINDEX_VERSION_ENCODE( \
     CINDEX_VERSION_MAJOR,                     \
     CINDEX_VERSION_MINOR )

 /**
  * An "index" that consists of a set of translation units that would
  * typically be linked together into an executable or library.
  */
 typedef void *CXIndex;
 
 /**
  * An opaque type representing target information for a given translation
  * unit.
  */
 typedef struct CXTargetInfoImpl *CXTargetInfo;
 
 /**
  * A single translation unit, which resides in an index.
  */
 typedef struct CXTranslationUnitImpl *CXTranslationUnit;
 
 /**
  * Opaque pointer representing client data that will be passed through
  * to various callbacks and visitors.
  */
 typedef void *CXClientData;
 
 /**
  * Provides the contents of a file that has not yet been saved to disk.
  *
  * Each CXUnsavedFile instance provides the name of a file on the
  * system along with the current contents of that file that have not
  * yet been saved to disk.
  */
 struct CXUnsavedFile {
   /**
    * The file whose contents have not yet been saved.
    *
    * This file must already exist in the file system.
    */
   const char *Filename;
 
   /**
    * A buffer containing the unsaved contents of this file.
    */
   const char *Contents;
 
   /**
    * The length of the unsaved contents of this buffer.
    */
   unsigned long Length;
 };
 
 /**
  * Describes the availability of a particular entity, which indicates
  * whether the use of this entity will result in a warning or error due to
  * it being deprecated or unavailable.
  */
 enum CXAvailabilityKind {
   /**
    * The entity is available.
    */
enum CXTranslationUnit_Flags {
   CXTranslationUnit_KeepGoing = 0x200,
 
   /**
    * Sets the preprocessor in a mode for parsing a single file only.
    */
   CXTranslationUnit_SingleFileParse = 0x400,
 
   /**
-   * \brief Used in combination with CXTranslationUnit_SkipFunctionBodies to
+   * Used in combination with CXTranslationUnit_SkipFunctionBodies to
    * constrain the skipping of function bodies to the preamble.
    *
    * The function bodies of the main file are not skipped.
    */
   CXTranslationUnit_LimitSkipFunctionBodiesToPreamble = 0x800
 };
 
 /**

  * Describes a single preprocessing token.
  */
 typedef struct {
   unsigned int_data[4];
   void *ptr_data;
 } CXToken;
 
 /**
+ * Get the raw lexical token starting with the given location.
+ *
+ * \param TU the translation unit whose text is being tokenized.
+ *
+ * \param Location the source location with which the token starts.
+ *
+ * \returns The token starting with the given location or NULL if no such token
+ * exist. The returned pointer must be freed with clang_disposeTokens before the
+ * translation unit is destroyed.
+ */
+CINDEX_LINKAGE CXToken *clang_getToken(CXTranslationUnit TU,
+                                       CXSourceLocation Location);
+
+/**
  * Determine the kind of the given token.
  */
 CINDEX_LINKAGE CXTokenKind clang_getTokenKind(CXToken);
 
 /**
  * Determine the spelling of the given token.
  *
  * The spelling of a token is the textual representation of that token, e.g.,
typedef struct {
   /**
    * The number of code-completion results stored in the
    * \c Results array.
    */
   unsigned NumResults;
 } CXCodeCompleteResults;
 
 /**
+ * Retrieve the number of fix-its for the given completion index.
+ *
+ * Calling this makes sense only if CXCodeComplete_IncludeCompletionsWithFixIts
+ * option was set.
+ *
+ * \param results The structure keeping all completion results
+ *
+ * \param completion_index The index of the completion
+ *
+ * \return The number of fix-its which must be applied before the completion at
+ * completion_index can be applied
+ */
+CINDEX_LINKAGE unsigned
+clang_getCompletionNumFixIts(CXCodeCompleteResults *results,
+                             unsigned completion_index);
+
+/**
+ * Fix-its that *must* be applied before inserting the text for the
+ * corresponding completion.
+ *
+ * By default, clang_codeCompleteAt() only returns completions with empty
+ * fix-its. Extra completions with non-empty fix-its should be explicitly
+ * requested by setting CXCodeComplete_IncludeCompletionsWithFixIts.
+ *
+ * For the clients to be able to compute position of the cursor after applying
+ * fix-its, the following conditions are guaranteed to hold for
+ * replacement_range of the stored fix-its:
+ *  - Ranges in the fix-its are guaranteed to never contain the completion
+ *  point (or identifier under completion point, if any) inside them, except
+ *  at the start or at the end of the range.
+ *  - If a fix-it range starts or ends with completion point (or starts or
+ *  ends after the identifier under completion point), it will contain at
+ *  least one character. It allows to unambiguously recompute completion
+ *  point after applying the fix-it.
+ *
+ * The intuition is that provided fix-its change code around the identifier we
+ * complete, but are not allowed to touch the identifier itself or the
+ * completion point. One example of completions with corrections are the ones
+ * replacing '.' with '->' and vice versa:
+ *
+ * std::unique_ptr<std::vector<int>> vec_ptr;
+ * In 'vec_ptr.^', one of the completions is 'push_back', it requires
+ * replacing '.' with '->'.
+ * In 'vec_ptr->^', one of the completions is 'release', it requires
+ * replacing '->' with '.'.
+ *
+ * \param results The structure keeping all completion results
+ *
+ * \param completion_index The index of the completion
+ *
+ * \param fixit_index The index of the fix-it for the completion at
+ * completion_index
+ *
+ * \param replacement_range The fix-it range that must be replaced before the
+ * completion at completion_index can be applied
+ *
+ * \returns The fix-it string that must replace the code at replacement_range
+ * before the completion at completion_index can be applied
+ */
+CINDEX_LINKAGE CXString clang_getCompletionFixIt(
+    CXCodeCompleteResults *results, unsigned completion_index,
+    unsigned fixit_index, CXSourceRange *replacement_range);
+
+/**
  * Flags that can be passed to \c clang_codeCompleteAt() to
  * modify its behavior.
  *
  * The enumerators in this enumeration can be bitwise-OR'd together to
  * provide multiple options to \c clang_codeCompleteAt().
  */
 enum CXCodeComplete_Flags {
   /**
enum CXCodeComplete_Flags {
    */
   CXCodeComplete_IncludeBriefComments = 0x04,
 
   /**
    * Whether to speed up completion by omitting top- or namespace-level entities
    * defined in the preamble. There's no guarantee any particular entity is
    * omitted. This may be useful if the headers are indexed externally.
    */
-  CXCodeComplete_SkipPreamble = 0x08
+  CXCodeComplete_SkipPreamble = 0x08,
+
+  /**
+   * Whether to include completions with small
+   * fix-its, e.g. change '.' to '->' on member access, etc.
+   */
+  CXCodeComplete_IncludeCompletionsWithFixIts = 0x10
 };
 
 /**
  * Bits that represent the context under which completion is occurring.
  *
  * The enumerators in this enumeration may be bitwise-OR'd together if multiple
  * contexts are occurring simultaneously.
  */

include/clang/Sema/CodeCompleteConsumer.h

public:
   ResultKind Kind;
 
   /// The cursor kind that describes this result.
   CXCursorKind CursorKind;
 
   /// The availability of this result.
   CXAvailabilityKind Availability = CXAvailability_Available;
 
-  /// FixIts that *must* be applied before inserting the text for the
-  /// corresponding completion item.
+  /// Fix-its that *must* be applied before inserting the text for the
+  /// corresponding completion.
   ///
-  /// Completion items with non-empty fixits will not be returned by default,
-  /// they should be explicitly requested by setting
-  /// CompletionOptions::IncludeFixIts. For the editors to be able to
-  /// compute position of the cursor for the completion item itself, the
-  /// following conditions are guaranteed to hold for RemoveRange of the stored
-  /// fixits:
-  ///  - Ranges in the fixits are guaranteed to never contain the completion
+  /// By default, CodeCompletionBuilder only returns completions with empty
+  /// fix-its. Extra completions with non-empty fix-its should be explicitly
+  /// requested by setting CompletionOptions::IncludeFixIts.
+  ///
+  /// For the clients to be able to compute position of the cursor after
+  /// applying fix-its, the following conditions are guaranteed to hold for
+  /// RemoveRange of the stored fix-its:
+  ///  - Ranges in the fix-its are guaranteed to never contain the completion
   ///  point (or identifier under completion point, if any) inside them, except
   ///  at the start or at the end of the range.
-  ///  - If a fixit range starts or ends with completion point (or starts or
+  ///  - If a fix-it range starts or ends with completion point (or starts or
   ///  ends after the identifier under completion point), it will contain at
   ///  least one character. It allows to unambiguously recompute completion
-  ///  point after applying the fixit.
-  /// The intuition is that provided fixits change code around the identifier we
-  /// complete, but are not allowed to touch the identifier itself or the
-  /// completion point. One example of completion items with corrections are the
-  /// ones replacing '.' with '->' and vice versa:
+  ///  point after applying the fix-it.
+  ///
+  /// The intuition is that provided fix-its change code around the identifier
+  /// we complete, but are not allowed to touch the identifier itself or the
+  /// completion point. One example of completions with corrections are the ones
+  /// replacing '.' with '->' and vice versa:
+  ///
   /// std::unique_ptr<std::vector<int>> vec_ptr;
-  /// In 'vec_ptr.^', one of completion items is 'push_back', it requires
+  /// In 'vec_ptr.^', one of the completions is 'push_back', it requires
   /// replacing '.' with '->'.
-  /// In 'vec_ptr->^', one of completion items is 'release', it requires
+  /// In 'vec_ptr->^', one of the completions is 'release', it requires
   /// replacing '->' with '.'.
   std::vector<FixItHint> FixIts;
 
   /// Whether this result is hidden by another name.
   bool Hidden : 1;
 
   /// Whether this result was found via lookup into a base class.
   bool QualifierIsInformative : 1;

test/Index/complete-arrow-dot.cpp

+struct X {
+  void doSomething();
+
+  int field;
+};
+
+void X::doSomething() {
+  // RUN: c-index-test -code-completion-at=%s:10:8 %s | FileCheck %s
+  // RUN: env CINDEXTEST_COMPLETION_INCLUDE_FIXITS=1 c-index-test -code-completion-at=%s:10:8 %s | FileCheck -check-prefix=CHECK-WITH-CORRECTION %s
+  this.;
+}
+
+void doSomething() {
+  // RUN: c-index-test -code-completion-at=%s:17:6 %s | FileCheck -check-prefix=CHECK-ARROW-TO-DOT %s
+  // RUN: env CINDEXTEST_COMPLETION_INCLUDE_FIXITS=1 c-index-test -code-completion-at=%s:17:6 %s | FileCheck -check-prefix=CHECK-ARROW-TO-DOT-WITH-CORRECTION %s
+  X x;
+  x->
+}
+
+// CHECK-NOT: CXXMethod:{ResultType void}{TypedText doSomething}{LeftParen (}{RightParen )} (34) (requires fix-it:{{.*}}
+// CHECK-NOT: FieldDecl:{ResultType int}{TypedText field} (35) (requires fix-it:{{.*}}
+// CHECK-NOT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder const X &}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-NOT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder X &&}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-NOT: StructDecl:{TypedText X}{Text ::} (75) (requires fix-it:{{.*}}
+// CHECK-NOT: CXXDestructor:{ResultType void}{TypedText ~X}{LeftParen (}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK: Completion contexts:
+// CHECK-NEXT: Dot member access
+
+// CHECK-WITH-CORRECTION: CXXMethod:{ResultType void}{TypedText doSomething}{LeftParen (}{RightParen )} (34) (requires fix-it:{{.*}}
+// CHECK-WITH-CORRECTION-NEXT: FieldDecl:{ResultType int}{TypedText field} (35) (requires fix-it:{{.*}}
+// CHECK-WITH-CORRECTION-NEXT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder const X &}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-WITH-CORRECTION-NEXT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder X &&}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-WITH-CORRECTION-NEXT: StructDecl:{TypedText X}{Text ::} (75) (requires fix-it:{{.*}}
+// CHECK-WITH-CORRECTION-NEXT: CXXDestructor:{ResultType void}{TypedText ~X}{LeftParen (}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-WITH-CORRECTION-NEXT: Completion contexts:
+// CHECK-WITH-CORRECTION-NEXT: Dot member access
+
+// CHECK-ARROW-TO-DOT-NOT: CXXMethod:{ResultType void}{TypedText doSomething}{LeftParen (}{RightParen )} (34) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-NOT: FieldDecl:{ResultType int}{TypedText field} (35) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-NOT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder const X &}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-NOT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder X &&}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-NOT: StructDecl:{TypedText X}{Text ::} (75) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-NOT: CXXDestructor:{ResultType void}{TypedText ~X}{LeftParen (}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT: Completion contexts:
+// CHECK-ARROW-TO-DOT-NEXT: Unknown
+
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION: CXXMethod:{ResultType void}{TypedText doSomething}{LeftParen (}{RightParen )} (34) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: FieldDecl:{ResultType int}{TypedText field} (35) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder const X &}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: CXXMethod:{ResultType X &}{TypedText operator=}{LeftParen (}{Placeholder X &&}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: StructDecl:{TypedText X}{Text ::} (75) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: CXXDestructor:{ResultType void}{TypedText ~X}{LeftParen (}{RightParen )} (79) (requires fix-it:{{.*}}
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: Completion contexts:
+// CHECK-ARROW-TO-DOT-WITH-CORRECTION-NEXT: Arrow member access

tools/c-index-test/c-index-test.c

 char *basename(const char* path)
 {
     char* base1 = (char*)strrchr(path, '/');
     char* base2 = (char*)strrchr(path, '\\');
     if (base1 && base2)
         return((base1 > base2) ? base1 + 1 : base2 + 1);
     else if (base1)
         return(base1 + 1);
     else if (base2)
         return(base2 + 1);
 
     return((char*)path);
 }
 char *dirname(char* path)
 {
     char* base1 = (char*)strrchr(path, '/');
     char* base2 = (char*)strrchr(path, '\\');
for (I = 0; I != N; ++I) {
     fprintf(file, "{%s %s}",
             clang_getCompletionChunkKindSpelling(Kind),
             cstr ? cstr : "");
     clang_disposeString(text);
   }
 
 }
 
-static void print_completion_result(CXCompletionResult *completion_result,
+static void print_line_column(CXSourceLocation location, FILE *file) {
+    unsigned line, column;
+    clang_getExpansionLocation(location, NULL, &line, &column, NULL);
+    fprintf(file, "%d:%d", line, column);
+}
+
+static void print_token_range(CXTranslationUnit translation_unit,
+                              CXSourceLocation start, FILE *file) {
+  CXToken *token = clang_getToken(translation_unit, start);
+
+  fprintf(file, "{");
+  if (token != NULL) {
+    CXSourceRange token_range = clang_getTokenExtent(translation_unit, *token);
+    print_line_column(clang_getRangeStart(token_range), file);
+    fprintf(file, "-");
+    print_line_column(clang_getRangeEnd(token_range), file);
+    clang_disposeTokens(translation_unit, token, 1);
+  }
+
+  fprintf(file, "}");
+}
+
+static void print_completion_result(CXTranslationUnit translation_unit,
+                                    CXCodeCompleteResults *completion_results,
+                                    unsigned index,
                                     FILE *file) {
+  CXCompletionResult *completion_result = completion_results->Results + index;
   CXString ks = clang_getCursorKindSpelling(completion_result->CursorKind);
   unsigned annotationCount;
   enum CXCursorKind ParentKind;
   CXString ParentName;
   CXString BriefComment;
   CXString Annotation;
   const char *BriefCommentCString;
   
static void print_completion_result(CXTranslationUnit translation_unit,
 
   BriefComment = clang_getCompletionBriefComment(
                                         completion_result->CompletionString);
   BriefCommentCString = clang_getCString(BriefComment);
   if (BriefCommentCString && *BriefCommentCString != '\0') {
     fprintf(file, "(brief comment: %s)", BriefCommentCString);
   }
   clang_disposeString(BriefComment);
-  
+  unsigned i;
+  for (i = 0; i < clang_getCompletionNumFixIts(completion_results, index);
+       ++i) {
+    CXSourceRange correction_range;
+    CXString FixIt = clang_getCompletionFixIt(completion_results, index, i,
+                                              &correction_range);
+    fprintf(file, " (requires fix-it: ");
+    print_token_range(translation_unit, clang_getRangeStart(correction_range),
+                      file);
+    fprintf(file, " to \"%s\")", clang_getCString(FixIt));
+    clang_disposeString(FixIt);
+  }
+
   fprintf(file, "\n");
 }
 
 void print_completion_contexts(unsigned long long contexts, FILE *file) {
   fprintf(file, "Completion contexts:\n");
   if (contexts == CXCompletionContext_Unknown) {
     fprintf(file, "Unknown\n");
   }
int perform_code_completion(int argc, const char **argv, int timing_only) {
   const char *InvocationPath;
 
   if (getenv("CINDEXTEST_CODE_COMPLETE_PATTERNS"))
     completionOptions |= CXCodeComplete_IncludeCodePatterns;
   if (getenv("CINDEXTEST_COMPLETION_BRIEF_COMMENTS"))
     completionOptions |= CXCodeComplete_IncludeBriefComments;
   if (getenv("CINDEXTEST_COMPLETION_SKIP_PREAMBLE"))
     completionOptions |= CXCodeComplete_SkipPreamble;
+  if (getenv("CINDEXTEST_COMPLETION_INCLUDE_FIXITS"))
+    completionOptions |= CXCodeComplete_IncludeCompletionsWithFixIts;
   
   if (timing_only)
     input += strlen("-code-completion-timing=");
   else
     input += strlen("-code-completion-at=");
 
   if ((errorCode = parse_file_line_column(input, &filename, &line, &column,
                                           0, 0)))
if (results) {
     enum CXCursorKind containerKind;
     CXString objCSelector;
     const char *selectorString;
     if (!timing_only) {      
       /* Sort the code-completion results based on the typed text. */
       clang_sortCodeCompletionResults(results->Results, results->NumResults);
 
       for (i = 0; i != n; ++i)
-        print_completion_result(results->Results + i, stdout);
+        print_completion_result(TU, results, i, stdout);
     }
     n = clang_codeCompleteGetNumDiagnostics(results);
     for (i = 0; i != n; ++i) {
       CXDiagnostic diag = clang_codeCompleteGetDiagnostic(results, i);
       PrintDiagnostic(diag);
       clang_disposeDiagnostic(diag);
     }
     
while (indent > 0) {
     --indent;
   }
 }
 
 static void printLocation(CXSourceLocation L) {
   CXFile File;
   CXString FileName;
   unsigned line, column, offset;
-  
   clang_getExpansionLocation(L, &File, &line, &column, &offset);
   FileName = clang_getFileName(File);
-  
   fprintf(stderr, "%s:%d:%d", clang_getCString(FileName), line, column);
   clang_disposeString(FileName);
 }
 
 static void printRanges(CXDiagnostic D, unsigned indent) {
   unsigned i, n = clang_getDiagnosticNumRanges(D);
   
   for (i = 0; i < n; ++i) {

tools/libclang/CIndex.cpp

if (Tok.isLiteral()) {
       CXTok.int_data[0] = CXToken_Punctuation;
       CXTok.ptr_data = nullptr;
     }
     CXTokens.push_back(CXTok);
     previousWasAt = Tok.is(tok::at);
   } while (Lex.getBufferLocation() < EffectiveBufferEnd);
 }
 
+CXToken *clang_getToken(CXTranslationUnit TU, CXSourceLocation Location) {
+  LOG_FUNC_SECTION {
+    *Log << TU << ' ' << Location;
+  }
+
+  if (isNotUsableTU(TU)) {
+    LOG_BAD_TU(TU);
+    return NULL;
+  }
+
+  ASTUnit *CXXUnit = cxtu::getASTUnit(TU);
+  if (!CXXUnit)
+    return NULL;
+
+  SourceLocation Begin = cxloc::translateSourceLocation(Location);
+  if (Begin.isInvalid())
+    return NULL;
+  SourceManager &SM = CXXUnit->getSourceManager();
+  std::pair<FileID, unsigned> DecomposedEnd = SM.getDecomposedLoc(Begin);
+  DecomposedEnd.second += Lexer::MeasureTokenLength(Begin, SM, CXXUnit->getLangOpts());
+
+  SourceLocation End = SM.getComposedLoc(DecomposedEnd.first, DecomposedEnd.second);
+
+  SmallVector<CXToken, 32> CXTokens;
+  getTokens(CXXUnit, SourceRange(Begin, End), CXTokens);
+
+  if (CXTokens.empty())
+    return NULL;
+
+  CXTokens.resize(1);
+  CXToken *Token = static_cast<CXToken *>(llvm::safe_malloc(sizeof(CXToken)));
+
+  memmove(Token, CXTokens.data(), sizeof(CXToken));
+  return Token;
+}
+
 void clang_tokenize(CXTranslationUnit TU, CXSourceRange Range,
                     CXToken **Tokens, unsigned *NumTokens) {
   LOG_FUNC_SECTION {
     *Log << TU << ' ' << Range;
   }
 
   if (Tokens)
     *Tokens = nullptr;

tools/libclang/CIndexCodeCompletion.cpp

 // code completion.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CIndexer.h"
 #include "CIndexDiagnostic.h"
 #include "CLog.h"
 #include "CXCursor.h"
+#include "CXSourceLocation.h"
 #include "CXString.h"
 #include "CXTranslationUnit.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/AST/Type.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Frontend/ASTUnit.h"
struct AllocatedCXCodeCompleteResults : public CXCodeCompleteResults {
 
   /// a boolean value indicating whether there is complete information
   /// about the container
   unsigned ContainerIsIncomplete;
   
   /// A string containing the Objective-C selector entered thus far for a
   /// message send.
   std::string Selector;
+
+  /// Vector of fix-its for each completion result that *must* be applied
+  /// before that result for the corresponding completion item.
+  std::vector<std::vector<FixItHint>> FixItsVector;
 };
 
 } // end anonymous namespace
 
+unsigned clang_getCompletionNumFixIts(CXCodeCompleteResults *results,
+                                      unsigned completion_index) {
+  AllocatedCXCodeCompleteResults *allocated_results = (AllocatedCXCodeCompleteResults *)results;
+
+  if (!allocated_results || allocated_results->FixItsVector.size() <= completion_index)
+    return 0;
+
+  return static_cast<unsigned>(allocated_results->FixItsVector[completion_index].size());
+}
+
+CXString clang_getCompletionFixIt(CXCodeCompleteResults *results,
+                                  unsigned completion_index,
+                                  unsigned fixit_index,
+                                  CXSourceRange *replacement_range) {
+  AllocatedCXCodeCompleteResults *allocated_results = (AllocatedCXCodeCompleteResults *)results;
+
+  if (!allocated_results || allocated_results->FixItsVector.size() <= completion_index) {
+    if (replacement_range)
+      *replacement_range = clang_getNullRange();
+    return cxstring::createNull();
+  }
+
+  ArrayRef<FixItHint> FixIts = allocated_results->FixItsVector[completion_index];
+  if (FixIts.size() <= fixit_index) {
+    if (replacement_range)
+      *replacement_range = clang_getNullRange();
+    return cxstring::createNull();
+  }
+
+  const FixItHint &FixIt = FixIts[fixit_index];
+  if (replacement_range) {
+    *replacement_range = cxloc::translateSourceRange(
+        *allocated_results->SourceMgr, allocated_results->LangOpts,
+        FixIt.RemoveRange);
+  }
+
+  return cxstring::createRef(FixIt.CodeToInsert.c_str());
+}
+
 /// Tracks the number of code-completion result objects that are 
 /// currently active.
 ///
 /// Used for debugging purposes only.
 static std::atomic<unsigned> CodeCompletionResultObjects;
 
 AllocatedCXCodeCompleteResults::AllocatedCXCodeCompleteResults(
     IntrusiveRefCntPtr<FileManager> FileMgr)
CaptureCompletionResults(const CodeCompleteOptions &Opts,
         TU(TranslationUnit) { }
     ~CaptureCompletionResults() override { Finish(); }
 
     void ProcessCodeCompleteResults(Sema &S, 
                                     CodeCompletionContext Context,
                                     CodeCompletionResult *Results,
                                     unsigned NumResults) override {
       StoredResults.reserve(StoredResults.size() + NumResults);
+      if (includeFixIts())
+        AllocatedResults.FixItsVector.reserve(NumResults);
       for (unsigned I = 0; I != NumResults; ++I) {
-        CodeCompletionString *StoredCompletion        
+        CodeCompletionString *StoredCompletion
           = Results[I].CreateCodeCompletionString(S, Context, getAllocator(),
                                                   getCodeCompletionTUInfo(),
                                                   includeBriefComments());
         
         CXCompletionResult R;
         R.CursorKind = Results[I].CursorKind;
         R.CompletionString = StoredCompletion;
         StoredResults.push_back(R);
+        if (includeFixIts())
+          AllocatedResults.FixItsVector.emplace_back(std::move(Results[I].FixIts));
       }
-      
       enum CodeCompletionContext::Kind contextKind = Context.getKind();
       
       AllocatedResults.ContextKind = contextKind;
       AllocatedResults.Contexts = getContextsForContextKind(contextKind, S);
       
       AllocatedResults.Selector = "";
       ArrayRef<IdentifierInfo *> SelIdents = Context.getSelIdents();
       for (ArrayRef<IdentifierInfo *>::iterator I = SelIdents.begin(),

 
 static CXCodeCompleteResults *
 clang_codeCompleteAt_Impl(CXTranslationUnit TU, const char *complete_filename,
                           unsigned complete_line, unsigned complete_column,
                           ArrayRef<CXUnsavedFile> unsaved_files,
                           unsigned options) {
   bool IncludeBriefComments = options & CXCodeComplete_IncludeBriefComments;
   bool SkipPreamble = options & CXCodeComplete_SkipPreamble;
+  bool IncludeFixIts = options & CXCodeComplete_IncludeCompletionsWithFixIts;
 
 #ifdef UDP_CODE_COMPLETION_LOGGER
 #ifdef UDP_CODE_COMPLETION_LOGGER_PORT
   const llvm::TimeRecord &StartTime =  llvm::TimeRecord::getCurrentTime();
 #endif
 #endif
-
   bool EnableLogging = getenv("LIBCLANG_CODE_COMPLETION_LOGGING") != nullptr;
 
   if (cxtu::isNotUsableTU(TU)) {
     LOG_BAD_TU(TU);
     return nullptr;
   }
 
   ASTUnit *AST = cxtu::getASTUnit(TU);
AllocatedCXCodeCompleteResults *Results = new AllocatedCXCodeCompleteResults(
       &AST->getFileManager());
   Results->Results = nullptr;
   Results->NumResults = 0;
   
   // Create a code-completion consumer to capture the results.
   CodeCompleteOptions Opts;
   Opts.IncludeBriefComments = IncludeBriefComments;
   Opts.LoadExternal = !SkipPreamble;
+  Opts.IncludeFixIts = IncludeFixIts;
   CaptureCompletionResults Capture(Opts, *Results, &TU);
 
   // Perform completion.
   std::vector<const char *> CArgs;
   for (const auto &Arg : TU->Arguments)
     CArgs.push_back(Arg.c_str());
   std::string CompletionInvocation =
       llvm::formatv("-code-completion-at={0}:{1}:{2}", complete_filename,

 namespace {
   struct OrderCompletionResults {
     bool operator()(const CXCompletionResult &XR, 
                     const CXCompletionResult &YR) const {
       CodeCompletionString *X
         = (CodeCompletionString *)XR.CompletionString;
       CodeCompletionString *Y
         = (CodeCompletionString *)YR.CompletionString;
-      
       SmallString<256> XBuffer;
       StringRef XText = GetTypedName(X, XBuffer);
       SmallString<256> YBuffer;
       StringRef YText = GetTypedName(Y, YBuffer);
       
       if (XText.empty() || YText.empty())
         return !XText.empty();
             

tools/libclang/libclang.exports

 clang_getCXXAccessSpecifier
 clang_getCanonicalCursor
 clang_getCanonicalType
 clang_getChildDiagnostics
 clang_getClangVersion
 clang_getCompletionAnnotation
 clang_getCompletionAvailability
 clang_getCompletionBriefComment
-clang_getCompletionChunkCompletionString
-clang_getCompletionChunkKind
-clang_getCompletionChunkText
-clang_getCompletionNumAnnotations
-clang_getCompletionParent
-clang_getCompletionPriority
+clang_getCompletionChunkCompletionString
+clang_getCompletionChunkKind
+clang_getCompletionChunkText
+clang_getCompletionNumFixIts
+clang_getCompletionFixIt
+clang_getCompletionNumAnnotations
+clang_getCompletionParent
+clang_getCompletionPriority
 clang_getCursor
 clang_getCursorAvailability
 clang_getCursorCompletionString
 clang_getCursorDefinition
 clang_getCursorDisplayName
 clang_getCursorPrintingPolicy
 clang_getCursorPrettyPrinted
 clang_getCursorExtent

 clang_getRange
 clang_getRangeEnd
 clang_getRangeStart
 clang_getRemappings
 clang_getRemappingsFromFileList
 clang_getResultType
 clang_getSkippedRanges
 clang_getSpecializedCursorTemplate
-clang_getSpellingLocation
-clang_getTUResourceUsageName
-clang_getTemplateCursorKind
-clang_getTokenExtent
-clang_getTokenKind
-clang_getTokenLocation
+clang_getSpellingLocation
+clang_getTUResourceUsageName
+clang_getTemplateCursorKind
+clang_getToken
+clang_getTokenExtent
+clang_getTokenKind
+clang_getTokenLocation
 clang_getTokenSpelling
 clang_getTranslationUnitCursor
 clang_getTranslationUnitSpelling
 clang_getTranslationUnitTargetInfo
 clang_getTypeDeclaration
 clang_getTypeKindSpelling
 clang_getTypeSpelling
 clang_getTypedefDeclUnderlyingType