[clang] Store code completion token range in preprocessor.

[kadircet]

Summary

This change is to support a new fature in clangd, tests will be send toclang-tools-extra with that change.

Unittests are included in: https://reviews.llvm.org/D50449

[ilya-biryukov]

Maybe split the commit message into multiple lines?

I suggest we wait for clangd changes to be reviewed and land them together, so that we have the tests and usages of this code.
Other than that LG

include/clang/Lex/Preprocessor.h

1139

NIT: parameters are are UpperCamelCase in LLVM naming conventions

[kadircet]

• Resolve discussions.

[ilya-biryukov]

LGTM, but let's land together with a dependent revision to hove some code that actually tests it.

include/clang/Lex/Preprocessor.h

313

NIT: s/taken/token

[ilya-biryukov]

NIT: maybe also note the number of the clangd revision in this change's description

[kadircet]

• Resolve discussions.
• Fix typo.

include/clang/Lex/Preprocessor.h

class Preprocessor {
 
   /// True if we hit the code-completion point.
   bool CodeCompletionReached = false;
 
   /// The code completion token containing the information
   /// on the stem that is to be code completed.
   IdentifierInfo *CodeCompletionII = nullptr;
 
+  /// Range for the code completion token.
+  SourceRange CodeCompletionTokenRange;
+
   /// The directory that the main file should be considered to occupy,
   /// if it does not correspond to a real file (as happens when building a
   /// module).
   const DirectoryEntry *MainFileDir = nullptr;
 
   /// The number of bytes that we will initially skip when entering the
   /// main file, along with a flag that indicates whether skipping this number
   /// of bytes will place the lexer at the start of a line.
public:
   /// completion point.
   void CodeCompleteNaturalLanguage();
 
   /// Set the code completion token for filtering purposes.
   void setCodeCompletionIdentifierInfo(IdentifierInfo *Filter) {
     CodeCompletionII = Filter;
   }
 
+  /// Set the code completion token range for detecting replacement range later
+  /// on.
+  void setCodeCompletionTokenRange(const SourceLocation Start,
+                                   const SourceLocation End) {
+    CodeCompletionTokenRange = {Start, End};
+  }
+  SourceRange getCodeCompletionTokenRange() const {
+    return CodeCompletionTokenRange;
+  }
+
   /// Get the code completion token for filtering purposes.
   StringRef getCodeCompletionFilter() {
     if (CodeCompletionII)
       return CodeCompletionII->getName();
     return {};
   }
 
   /// Retrieve the preprocessing record, or NULL if there is no

lib/Lex/Preprocessor.cpp

case CLK_LexAfterModuleImport:
       ReturnedToken = true;
       break;
     }
   } while (!ReturnedToken);
 
   if (Result.is(tok::code_completion) && Result.getIdentifierInfo()) {
     // Remember the identifier before code completion token.
     setCodeCompletionIdentifierInfo(Result.getIdentifierInfo());
+    setCodeCompletionTokenRange(Result.getLocation(), Result.getEndLoc());
     // Set IdenfitierInfo to null to avoid confusing code that handles both
     // identifiers and completion tokens.
     Result.setIdentifierInfo(nullptr);
   }
 
   LastTokenWasAt = Result.is(tok::at);
 }