[ClangRepl] support code completion at a REPL

[capfredf]

Summary

This patch enabled code completion for ClangREPL. The feature was built upon three
existing Clang components: a list completer for LineEditor,
a CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method.
with the first component serving as the main entry point of handling interactive inputs.

Because a completion point for a compiler instance has to be unchanged once it
is set, an incremental compiler instance is created for each code
completion. Such a compiler instance carries over AST context source from the
main interpreter compiler in order to obtain declarations or bindings from
previous input in the same REPL session.

The most important API codeComplete in Interpreter/CodeCompletion is a thin
wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as
a code completion point and a ReplCompletionConsumer, which communicates
completion results from SemaCodeCompletion back to the list completer for the
REPL.

In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels
were added so that SemaCodeCompletion can treat top level statements like
expression statements at the REPL. For example,

clang-repl> int foo = 42;
clang-repl> f<tab>

From a parser's persective, the cursor is at a top level. If we used code
completion without any changes, PCC_Namespace would be supplied to
Sema::CodeCompleteOrdinaryName, and thus the completion results would not
include foo.

Currently, the way we use PCC_TopLevelOrExpression and
CCC_TopLevelOrExpression is no different from the way we use PCC_Statement
and CCC_Statement respectively.

[v.g.vassilev]

Can you extend the commit log to include a description of how things are currently done? For example, it would be good to read about design and technical decisions, etc.

clang/lib/Interpreter/CodeCompletion.cpp

80

Why we drop the error on the floor?

85

I am not sure why we need to wrap this code. Can we teach the TopLevelStmtDecl to work with the code completion logic?

[capfredf]

clang/lib/Interpreter/CodeCompletion.cpp

85

I think the problem has less to do with the TopLevelStmtDecl. Usually, a TopLevelStmtDecl instance is a successful result from ParseXXXXX, but when CodeCompletion is triggered during parsing, nothing meaning comes out from ParseXXXX.

We need to wrap our input because we need Sema::CodeCompleteOrdinaryName to work on code from the REPL in a different completion context than the same code from a regular source file.

In a regular C++ file,

int foo = 42;
f_

Since top-level expressions are not supported, foo should not be an option.

But in a REPL session,

clang-repl> int foo = 42;
clang-repl> f_

we are allowed to use foo to make a statement like foo + 84;.

Since the argument for CompletionContext to CodeCompleteOrdinaryName is controlled by the parsing process, ie. there is no way to configure it from outside, I think faking context is a straightforward solution.

Alternatively, we can check if the compiler is in incremental mode and call CodeCompleteOrdinaryName with the context we want in parsing. Something like:

if (PP.isIncrementalProcessingEnabled() {
  Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Statement);
} else {
  Actions.CodeCompleteOrdinaryName(getCurScope(), Sema::PCC_Namespace);
}

I personally think both solutions are extensionally equivalent, but the alternative is a bit intrusive.

[capfredf]

changes per discussions

[capfredf]

update

[capfredf]

move the comment up

[capfredf]

move the comment up

[capfredf]

remove an unused decl

[capfredf]

clang/lib/Interpreter/CodeCompletion.cpp

85

I think the combination of checking isIncrementalProcessingEnabled and adding a new PCC might be better since it is really a different context at the REPL

[capfredf]

get rid of the wrapping

[v.g.vassilev]

@capfredf this seems to be heading in a good direction. Please find another round of comments.

@aaron.ballman, I could not find a person that's active in that area. Could you help with finding reviewers for this type of patch?

clang/include/clang/Interpreter/Interpreter.h

116

I could not find where this interface was used. If it is unused, let'd drop it.

clang/include/clang/Sema/Sema.h

13324

clang/lib/Interpreter/IncrementalParser.cpp

338

Seems that we have stray debug fragments? Can you remove them?

374–382

Why we had to comment out this line?

clang/lib/Interpreter/Interpreter.cpp

248

IIUC, setCodeCompletionConsumer takes the ownership of CConsumer. I'd suggest to drop CConsumer member.

[capfredf]

clang/include/clang/Interpreter/Interpreter.h

116

it is used at L86 of CodeCompletion.cpp

clang/lib/Interpreter/IncrementalParser.cpp

374–382

This is old code. FID is created in a different way. See below.
I will remove the commented code

[capfredf]

• rename
• Do not export ReplCompletionConsumer
• remove commented code

[capfredf]

@v.g.vassilev I moved the ReplCodeCompletionConsumer class out of the header file, since it is not needed elsewhere.

[v.g.vassilev]

clang/include/clang/Sema/CodeCompleteConsumer.h

342

clang/lib/Interpreter/CodeCompletion.cpp

103

Let's move this in Interpreter::createForCodeCompletion.

119

Please remove the braces around single statement blocks.

clang/lib/Interpreter/Interpreter.cpp

304

I still do not entirely understand why we can "just" ask the codecompletion infrastructure for the possible options at the current position. I know that's probably easier set than done but I'd like to entertain that idea for a while..

[capfredf]

clang/lib/Interpreter/Interpreter.cpp

304

I'm a bit confused. Can you expand on the suggestion?

[capfredf]

clang/lib/Interpreter/CodeCompletion.cpp

103

The reasons I moved this out of Interpreter::createForCodeCompletion are

1. Design-wise, we don't need to expose ReplCompletionConsumer to Interpreter.cpp. All createForCodeCompletion needs is a subclass instance of CodeCompletionConsumer.
2. ReplCompletionConsumer has more data members on the type-directed branch than the one on this branch. I'd rather create an instance here than pass all the necessary arguments to one layer down and do that there.

[capfredf]

Remove unnecessary braces

[capfredf]

handle cases where code completion in a function declaration

[capfredf]

clang/unittests/Interpreter/CodeCompletionTest.cpp

71

@v.g.vassilev The way I fixed this is a bit hacky. Do you have a better idea?

[v.g.vassilev]

clang/unittests/Interpreter/CodeCompletionTest.cpp

71

Now I think I understand what you were after. Instead of handling the error in the constructor we should handle it in the caller. Here is some information how we could do that: https://llvm.org/docs/ProgrammersManual.html#fallible-constructors

[capfredf]

changes per @v.g.vassilev's comments

[v.g.vassilev]

I think we are getting there. I added a few suggestions how to improve the current design.

clang/include/clang/Interpreter/CodeCompletion.h

16

I still am a bit uncomfortable with relying on a line editor api to address a generic question one could as the interpreter such as "What's visible at that code position"?

Can we not make the overloaded operators outside of the ReplListCompleter class. That means moving them in ClangRepl.cpp?

clang/include/clang/Interpreter/Interpreter.h

42

We should put that in its alphabetical order.

clang/lib/Interpreter/CodeCompletion.cpp

112

I do not believe we need a spe

clang/lib/Interpreter/IncrementalParser.cpp

324–325

Is there any other particular reason for this routine to be different rather than PP.SetCodeCompletionPoint(*FE, Line, Col)? That is, can we merge back the old code and only set the point of completion externally?

[v.g.vassilev]

clang/lib/Interpreter/Interpreter.cpp

304

I do not believe we need a special case for creating a CodeCompletionConsumer. I think we could rely on the logic in CompilerInstance::createCodeCompletionConsumer which can be triggered in IncrementalAction::ExecuteAction just like in ASTFrontendAction::ExecuteAction. We will need to copy some of the code over but then you could start the same interpreter infrastructure with slightly different flags.

clang/lib/Parse/ParseDecl.cpp

6652

Can we test this change separately outside of clang-repl? That would mean creating a tests and run it with clang -Xclang -fincremental-extensions

clang/lib/Parse/Parser.cpp

934

Likewise, it would be great if we have a test in clang-only mode.

[capfredf]

clang/lib/Interpreter/CodeCompletion.cpp

112

I am not sure I am following.
For a regular interpreter, create Interpreter -> create Incremental Parser -> IncrmentalASTAction triggered -> set a default code completion consumer(PrintCodeCompletionConsumer) to CI if no code consumer has been set -> Sema initialized.
Based on this logic, we need to provide our CodeConsumer so that IncrementalASTAction does not create the default, plus we need to a parent CI to a new code completion interpreter. I don't see how we can avoid a special code path for creation

clang/lib/Interpreter/IncrementalParser.cpp

324–325

The regular Parse does some lexing after parsing, which I don't think is necessary for code completion.

Also, this is more or less because, as a functional programmer, I prefer building things using small functions with explicit arguments.

I'll try if I can use the old parse with the completion point set and tested.

[capfredf]

changes per discussions

[capfredf]

use ASTUnit::codeComplete

[capfredf]

clang/include/clang/Frontend/ASTUnit.h

901 ↗

(On Diff #549658)

@v.g.vassilev Not sure if this is the right solution or idiom to extend this method.

we can make this high order function more general, like std::unique_ptr<SyntaxOnlyAction*><()>

[capfredf]

changes per discussions

[v.g.vassilev]

clang/lib/Interpreter/IncrementalParser.h

98

Can we not move this class definition locally to the CodeComplete interface?

[v.g.vassilev]

This looks mostly good to me. I noticed that @sammccall has done some work in that area in clangd and I was wondering if he has some thoughts about this patch.

[capfredf]

up

[capfredf]

up

[capfredf]

use auto

[v.g.vassilev]

clang/lib/Interpreter/CodeCompletion.cpp

2

I would propose to rename this file to InterpreterCodeCompletion.cpp and implement the Interpreter::codeComplete interface. Then we can move all of the content of ExternalSource.{h,cpp}, CodeCompletion.h and ExternalSource.h in it. That will increase the encapsulation of the code.

[capfredf]

move everything into InterpreterCodeCompletion.

the entry point for code completion is now a function, codeCompletion

[capfredf]

format

[sammccall]

This mostly looks good.

My main concern is adding code to the parser to suppress errors when code completing: this is unlikely to be the only such place. Existing consumers seem happy to ignore errors, if this doesn't work for clang-repl it'd be good to understand why.

clang/include/clang/Frontend/ASTUnit.h

901 ↗

(On Diff #549658)

There's no need for this to be a SyntaxOnlyAction, you can take FrontendAction here.

I don't think there's any need to provide a factory, it's safe to assume it will always create one action. (We have FrontendActionFactory in tooling, and it's a pain.)

891 ↗

(On Diff #549760)

This comment doesn't really say anything beyond echoing the name/type, either remove it or add some separate explanation?

e.g. "If Act is specified, it will be used to parse the file. This allows customizing the parse by overriding FrontendAction's lifecycle methods."

clang/include/clang/Interpreter/InterpreterCodeCompletion.h

30 ↗

(On Diff #549760)

nit: lowercase initial letter for functions

30 ↗

(On Diff #549760)

you might consider vector<string> instead.

this signature doesn't provide any way to return owned names, and so requires that the results are already allocated somewhere as contiguous strings.

This will prevents completing names like operator int, as well as some non-names that might prove useful.

This is an interactive operation, copying the strings is unlikely to add significant latency on human scale.

clang/include/clang/Sema/CodeCompleteConsumer.h

342

I don't think this name fits with the others, it describes the client rather than the grammatical/semantic context.

I would suggest maybe CCC_TopLevelOrExpression?

clang/lib/Interpreter/InterpreterCodeCompletion.cpp

72 ↗

(On Diff #549760)

(up to you, but default between cases seems surprising)

clang/lib/Parse/ParseDecl.cpp

6644

AIUI, all code completion actions should be ignoring errors, and we should always be happy to cut off parsing and give up as soon as the completion token is reached.

So why do we need special handling for incremental processing here?

(i.e. what behavior do non-repl users see here - if it's correct can clang-repl do the same, if it's wrong why don't we fix both?)

[capfredf]

address @sammccall 's comments

clang/include/clang/Frontend/ASTUnit.h

901 ↗

(On Diff #549658)

Ideally, this method requires a FrontendAction with hasCodeCompletionSupport returning true. The reason I chose SyntaxOnlyAction are 1. the old code used it 2. SyntaxOnlyAction::hasCodeCompletionSupport returns true. Certainly, right now SyntaxOnlyAction does not prevent its subclasses from overriding this method.

clang/include/clang/Sema/CodeCompleteConsumer.h

342

Nice. This one sounds better indeed.

[capfredf]

@sammccall Thank you very much for your valuable input.

[capfredf]

up

[v.g.vassilev]

@sammccall could you take another look? It seems quite ready to me to land.

[sammccall]

LGTM, thanks!

clang/include/clang/Sema/CodeCompleteConsumer.h

341

I think this is non-obvious & worthy of more explanation

/// Completion in a namespace or global scope, but also accept expressions.
/// This occurs in REPL inputs, which may declare things  or print expression
/// values.

[v.g.vassilev]

Thanks for working on this @capfredf. Let me know if I should land that for you.

[capfredf]

update the commit message

[capfredf]

fix code in Clangd

[v.g.vassilev]

clang/unittests/Interpreter/CodeCompletionTest.cpp

39

We should make this an out parameter instead of returning it by copy.

43

[sammccall]

clang-tools-extra/clangd/CodeComplete.cpp

845 ↗

(On Diff #552736)

This should be true rather than false, since both TopLevel and Expression are completed using the index.

(Though I realize that for now we'll never hit this context)

[capfredf]

fix potential memory issues

[capfredf]

fix memory issues

clang/include/clang/Interpreter/CodeCompletion.h

+//===------ CodeCompletion.h - Code Completion for ClangRepl -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the classes which performs code completion at the REPL.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_INTERPRETER_CODE_COMPLETION_H
+#define LLVM_CLANG_INTERPRETER_CODE_COMPLETION_H
+#include "llvm/LineEditor/LineEditor.h"
+
+namespace llvm {
+  class raw_ostream;
+  class StringRef;
+} // namespace llvm
+
+namespace clang {
+class Interpreter;
+class CodeCompletionResult;
+class IncrementalCompilerBuilder;
+
+struct ReplListCompleter {
+  IncrementalCompilerBuilder &CB;
+  Interpreter &MainInterp;
+  ReplListCompleter(IncrementalCompilerBuilder &CB, Interpreter &Interp);
+  ReplListCompleter(IncrementalCompilerBuilder &CB, Interpreter &Interp, llvm::raw_ostream& ErrStream) :
+    CB(CB), MainInterp(Interp), ErrStream(ErrStream) {};
+  std::vector<llvm::LineEditor::Completion> operator()(llvm::StringRef Buffer,
+                                                       size_t Pos) const;
+private:
+  llvm::raw_ostream& ErrStream;
+  std::vector<llvm::StringRef>
+  toCodeCompleteStrings(const std::vector<CodeCompletionResult> &Results) const;
+
+};
+
+} // namespace clang
+#endif

clang/include/clang/Interpreter/Interpreter.h

 namespace orc {
 class LLJIT;
 class ThreadSafeContext;
 } // namespace orc
 } // namespace llvm
 
 namespace clang {
 
+class CodeCompletionResult;
 class CompilerInstance;
 class IncrementalExecutor;
 class IncrementalParser;
+class CodeCompleteConsumer;
 
 /// Create a pre-configured \c CompilerInstance for incremental processing.
 class IncrementalCompilerBuilder {
 public:
   IncrementalCompilerBuilder() {}
 
   void SetCompilerArgs(const std::vector<const char *> &Args) {
     UserArgs = Args;
class Interpreter {
   std::unique_ptr<llvm::orc::ThreadSafeContext> TSCtx;
   std::unique_ptr<IncrementalParser> IncrParser;
   std::unique_ptr<IncrementalExecutor> IncrExecutor;
 
   // An optional parser for CUDA offloading
   std::unique_ptr<IncrementalParser> DeviceParser;
 
   Interpreter(std::unique_ptr<CompilerInstance> CI, llvm::Error &Err);
+  Interpreter(std::unique_ptr<CompilerInstance> CI, llvm::Error &Err,
+              CodeCompleteConsumer* CConsumer,
+              const CompilerInstance *ParentCI = nullptr);
 
   llvm::Error CreateExecutor();
   unsigned InitPTUSize = 0;
 
   // This member holds the last result of the value printing. It's a class
   // member because we might want to access it after more inputs. If no value
   // printing happens, it's in an invalid state.
   Value LastValue;
 
 public:
   ~Interpreter();
+
   static llvm::Expected<std::unique_ptr<Interpreter>>
   create(std::unique_ptr<CompilerInstance> CI);
+
   static llvm::Expected<std::unique_ptr<Interpreter>>
   createWithCUDA(std::unique_ptr<CompilerInstance> CI,
                  std::unique_ptr<CompilerInstance> DCI);
+
+  static llvm::Expected<std::unique_ptr<Interpreter>>
+  createForCodeCompletion(IncrementalCompilerBuilder &CB,
+                          const CompilerInstance *ParentCI,
+                          CodeCompleteConsumer* CConsumer);
+
   const ASTContext &getASTContext() const;
   ASTContext &getASTContext();
+  llvm::Error CodeComplete(llvm::StringRef Input, size_t Col, size_t Line = 1);
   const CompilerInstance *getCompilerInstance() const;
   llvm::Expected<llvm::orc::LLJIT &> getExecutionEngine();
 
   llvm::Expected<PartialTranslationUnit &> Parse(llvm::StringRef Code);
   llvm::Error Execute(PartialTranslationUnit &T);
   llvm::Error ParseAndExecute(llvm::StringRef Code, Value *V = nullptr);
   llvm::Expected<llvm::orc::ExecutorAddr> CompileDtorCall(CXXRecordDecl *CXXRD);
 

clang/include/clang/Sema/CodeCompleteConsumer.h

enum Kind {
     /// Code completion of an attribute name.
     CCC_Attribute,
 
     /// An unknown context, in which we are recovering from a parsing
     /// error and don't know which completions we should give.
     CCC_Recovery,
 
     /// Code completion in a @class forward declaration.
-    CCC_ObjCClassForwardDecl
+    CCC_ObjCClassForwardDecl,
+
+    /// Code completion at a top level in a REPL session.
+    CCC_ReplTopLevel,
   };
 
   using VisitedContextSet = llvm::SmallPtrSet<DeclContext *, 8>;
 
 private:
   Kind CCKind;
 
   /// Indicates whether we are completing a name of a using declaration, e.g.

clang/include/clang/Sema/Sema.h

enum ParserCompletionContext {
     PCC_RecoveryInFunction,
     /// Code completion occurs where only a type is permitted.
     PCC_Type,
     /// Code completion occurs in a parenthesized expression, which
     /// might also be a type cast.
     PCC_ParenthesizedExpression,
     /// Code completion occurs within a sequence of declaration
     /// specifiers within a function, method, or block.
-    PCC_LocalDeclarationSpecifiers
+    PCC_LocalDeclarationSpecifiers,
+    /// Code completion occurs at top-level in a REPL session
+    PCC_TopLevelStmtDecl,
   };
 
   void CodeCompleteModuleImport(SourceLocation ImportLoc, ModuleIdPath Path);
   void CodeCompleteOrdinaryName(Scope *S,
                                 ParserCompletionContext CompletionContext);
   void CodeCompleteDeclSpec(Scope *S, DeclSpec &DS,
                             bool AllowNonIdentifiers,
                             bool AllowNestedNameSpecifiers);

clang/lib/Frontend/ASTUnit.cpp

static void CalculateHiddenNames(const CodeCompletionContext &Context,
   case CodeCompletionContext::CCC_ArrowMemberAccess:
   case CodeCompletionContext::CCC_ObjCPropertyAccess:
   case CodeCompletionContext::CCC_Namespace:
   case CodeCompletionContext::CCC_Type:
   case CodeCompletionContext::CCC_Symbol:
   case CodeCompletionContext::CCC_SymbolOrNewName:
   case CodeCompletionContext::CCC_ParenthesizedExpression:
   case CodeCompletionContext::CCC_ObjCInterfaceName:
+  case CodeCompletionContext::CCC_ReplTopLevel:
     break;
 
   case CodeCompletionContext::CCC_EnumTag:
   case CodeCompletionContext::CCC_UnionTag:
   case CodeCompletionContext::CCC_ClassOrStructTag:
     OnlyTagNames = true;
     break;
 

clang/lib/Interpreter/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
    core
    native
    MC
    Option
    OrcJit
    OrcShared
    OrcTargetProcess
    Support
    Target
    TargetParser
   )
 
 add_clang_library(clangInterpreter
+  CodeCompletion.cpp
   DeviceOffload.cpp
+  ExternalSource.cpp
   IncrementalExecutor.cpp
   IncrementalParser.cpp
   Interpreter.cpp
   InterpreterUtils.cpp
   Value.cpp
 
   DEPENDS
   intrinsics_gen

clang/lib/Interpreter/CodeCompletion.cpp

+//===------ CodeCompletion.cpp - Code Completion for ClangRepl -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the classes which performs code completion at the REPL.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Interpreter/CodeCompletion.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Interpreter/Interpreter.h"
+#include "clang/Lex/PreprocessorOptions.h"
+#include "clang/Sema/CodeCompleteConsumer.h"
+#include "clang/Sema/CodeCompleteOptions.h"
+#include "clang/Sema/Sema.h"
+
+namespace clang {
+
+clang::CodeCompleteOptions getClangCompleteOpts() {
+  clang::CodeCompleteOptions Opts;
+  Opts.IncludeCodePatterns = true;
+  Opts.IncludeMacros = true;
+  Opts.IncludeGlobals = true;
+  Opts.IncludeBriefComments = true;
+  return Opts;
+}
+
+class ReplCompletionConsumer : public CodeCompleteConsumer {
+public:
+  ReplCompletionConsumer(std::vector<CodeCompletionResult> &Results)
+      : CodeCompleteConsumer(getClangCompleteOpts()),
+        CCAllocator(std::make_shared<GlobalCodeCompletionAllocator>()),
+        CCTUInfo(CCAllocator), Results(Results){};
+  void ProcessCodeCompleteResults(class Sema &S, CodeCompletionContext Context,
+                                  CodeCompletionResult *InResults,
+                                  unsigned NumResults) final;
+
+  clang::CodeCompletionAllocator &getAllocator() override {
+    return *CCAllocator;
+  }
+
+  clang::CodeCompletionTUInfo &getCodeCompletionTUInfo() override {
+    return CCTUInfo;
+  }
+
+private:
+  std::shared_ptr<GlobalCodeCompletionAllocator> CCAllocator;
+  CodeCompletionTUInfo CCTUInfo;
+  std::vector<CodeCompletionResult> &Results;
+};
+
+void ReplCompletionConsumer::ProcessCodeCompleteResults(
+    class Sema &S, CodeCompletionContext Context,
+    CodeCompletionResult *InResults, unsigned NumResults) {
+  for (unsigned I = 0; I < NumResults; ++I) {
+    auto &Result = InResults[I];
+    switch (Result.Kind) {
+    case CodeCompletionResult::RK_Declaration:
+      if (Result.Declaration->getIdentifier()) {
+        Results.push_back(Result);
+      }
+      break;
+    default:
+      break;
+    case CodeCompletionResult::RK_Keyword:
+      Results.push_back(Result);
+      break;
+    }
+  }
+}
+
+std::vector<StringRef> ReplListCompleter::toCodeCompleteStrings(
+    const std::vector<CodeCompletionResult> &Results) const {
+  std::vector<StringRef> CompletionStrings;
+  for (auto Res : Results) {
+    switch (Res.Kind) {
+    case CodeCompletionResult::RK_Declaration:
+      if (auto *ID = Res.Declaration->getIdentifier()) {
+        CompletionStrings.push_back(ID->getName());
+      }
+      break;
+    case CodeCompletionResult::RK_Keyword:
+      CompletionStrings.push_back(Res.Keyword);
+      break;
+    default:
+      break;
+    }
+  }
+  return CompletionStrings;
+}
+
+ReplListCompleter::ReplListCompleter(IncrementalCompilerBuilder &CB,
+                                     Interpreter &Interp)
+    : CB(CB), MainInterp(Interp), ErrStream(llvm::errs()){
+
+                                  };
+
+std::vector<llvm::LineEditor::Completion>
+ReplListCompleter::operator()(llvm::StringRef Buffer, size_t Pos) const {
+  std::vector<llvm::LineEditor::Completion> Comps;
+  std::vector<CodeCompletionResult> Results;
+  auto *CConsumer = new ReplCompletionConsumer(Results);
+  auto Interp = Interpreter::createForCodeCompletion(
+      CB, MainInterp.getCompilerInstance(), CConsumer);
+
+  if (auto Err = Interp.takeError()) {
+    // log the error and returns an empty vector;
+
+    llvm::logAllUnhandledErrors(std::move(Err), ErrStream, "error: ");
+    return {};
+  }
+
+  auto Lines = std::count(Buffer.begin(), Buffer.end(), '\n') + 1;
+
+  if (auto Err = (*Interp)->CodeComplete(Buffer, Pos + 1, Lines)) {
+    llvm::logAllUnhandledErrors(std::move(Err), ErrStream, "error: ");
+    return {};
+  }
+
+  size_t space_pos = Buffer.rfind(" ");
+  llvm::StringRef s;
+  if (space_pos == llvm::StringRef::npos) {
+    s = Buffer;
+  } else {
+    s = Buffer.substr(space_pos + 1);
+  }
+
+  for (auto c : toCodeCompleteStrings(Results)) {
+    if (c.startswith(s))
+      Comps.push_back(
+          llvm::LineEditor::Completion(c.substr(s.size()).str(), c.str()));
+  }
+  return Comps;
+}
+} // namespace clang

clang/lib/Interpreter/ExternalSource.h

+//==----- ExternalSource.h - External AST Source for Code Completion ------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines components that make declarations parsed and executed by
+// the interpreter visible to the context where code completion is being
+// triggered.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/AST/ExternalASTSource.h"
+
+namespace clang {
+class ASTContext;
+class FileManager;
+class ASTImporter;
+
+class ExternalSource : public clang::ExternalASTSource {
+  ASTContext &ChildASTCtxt;
+  TranslationUnitDecl *ChildTUDeclCtxt;
+  ASTContext &ParentASTCtxt;
+  TranslationUnitDecl *ParentTUDeclCtxt;
+
+  std::unique_ptr<ASTImporter> Importer;
+
+public:
+  ExternalSource(ASTContext &ChildASTCtxt, FileManager &ChildFM,
+                 ASTContext &ParentASTCtxt, FileManager &ParentFM);
+  bool FindExternalVisibleDeclsByName(const DeclContext *DC,
+                                      DeclarationName Name) override;
+  void
+  completeVisibleDeclsMap(const clang::DeclContext *childDeclContext) override;
+};
+} // namespace clang

clang/lib/Interpreter/ExternalSource.cpp

+//===--- ExternalSource.cpp - External AST Source for Code Completion ----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// The file implements classes that make declarations parsed and executed by the
+// interpreter visible to the context where code completion is being triggered.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ExternalSource.h"
+#include "clang/AST/ASTImporter.h"
+#include "clang/AST/DeclarationName.h"
+#include "clang/Basic/IdentifierTable.h"
+
+namespace clang {
+ExternalSource::ExternalSource(ASTContext &ChildASTCtxt, FileManager &ChildFM,
+                               ASTContext &ParentASTCtxt, FileManager &ParentFM)
+    : ChildASTCtxt(ChildASTCtxt),
+      ChildTUDeclCtxt(ChildASTCtxt.getTranslationUnitDecl()),
+      ParentASTCtxt(ParentASTCtxt),
+      ParentTUDeclCtxt(ParentASTCtxt.getTranslationUnitDecl()) {
+  ASTImporter *importer =
+      new ASTImporter(ChildASTCtxt, ChildFM, ParentASTCtxt, ParentFM,
+                      /*MinimalImport : ON*/ true);
+  Importer.reset(importer);
+}
+
+bool ExternalSource::FindExternalVisibleDeclsByName(const DeclContext *DC,
+                                                    DeclarationName Name) {
+  IdentifierTable &ParentIdTable = ParentASTCtxt.Idents;
+
+  auto ParentDeclName =
+      DeclarationName(&(ParentIdTable.get(Name.getAsString())));
+
+  DeclContext::lookup_result lookup_result =
+      ParentTUDeclCtxt->lookup(ParentDeclName);
+
+  if (!lookup_result.empty()) {
+    return true;
+  }
+  return false;
+}
+
+void ExternalSource::completeVisibleDeclsMap(
+    const DeclContext *ChildDeclContext) {
+  assert(ChildDeclContext && ChildDeclContext == ChildTUDeclCtxt &&
+         "No child decl context!");
+
+  if (!ChildDeclContext->hasExternalVisibleStorage())
+    return;
+
+  for (auto *DeclCtxt = ParentTUDeclCtxt; DeclCtxt != nullptr;
+       DeclCtxt = DeclCtxt->getPreviousDecl()) {
+    for (auto &IDeclContext : DeclCtxt->decls()) {
+      if (NamedDecl *Decl = llvm::dyn_cast<NamedDecl>(IDeclContext)) {
+        if (auto DeclOrErr = Importer->Import(Decl)) {
+          if (NamedDecl *importedNamedDecl =
+                  llvm::dyn_cast<NamedDecl>(*DeclOrErr)) {
+            SetExternalVisibleDeclsForName(ChildDeclContext,
+                                           importedNamedDecl->getDeclName(),
+                                           importedNamedDecl);
+          }
+
+        } else {
+          llvm::consumeError(DeclOrErr.takeError());
+        }
+      }
+    }
+    ChildDeclContext->setHasExternalLexicalStorage(false);
+  }
+}
+
+} // namespace clang

clang/lib/Interpreter/IncrementalParser.h

 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Error.h"
 
 #include <list>
 #include <memory>
 namespace llvm {
 class LLVMContext;
-}
+} // namespace llvm
 
 namespace clang {
 class ASTConsumer;
 class CodeGenerator;
 class CompilerInstance;
 class IncrementalAction;
 class Interpreter;
 class Parser;
protected:
   /// of code.
   std::list<PartialTranslationUnit> PTUs;
 
   IncrementalParser();
 
 public:
   IncrementalParser(Interpreter &Interp,
                     std::unique_ptr<CompilerInstance> Instance,
-                    llvm::LLVMContext &LLVMCtx, llvm::Error &Err);
+                    llvm::LLVMContext &LLVMCtx, llvm::Error &Err,
+                    const CompilerInstance *ParentCI = nullptr);
   virtual ~IncrementalParser();
 
   CompilerInstance *getCI() { return CI.get(); }
   CodeGenerator *getCodeGen() const;
 
   /// Parses incremental input by creating an in-memory file.
   ///\returns a \c PartialTranslationUnit which holds information about the
   /// \c TranslationUnitDecl and \c llvm::Module corresponding to the input.
   virtual llvm::Expected<PartialTranslationUnit &> Parse(llvm::StringRef Input);
+  llvm::Error ParseForCodeCompletion(llvm::StringRef Input, size_t Col, size_t Line);
 
   /// Uses the CodeGenModule mangled name cache and avoids recomputing.
   ///\returns the mangled name of a \c GD.
   llvm::StringRef GetMangledName(GlobalDecl GD) const;
 
   void CleanUpPTU(PartialTranslationUnit &PTU);
 
   std::list<PartialTranslationUnit> &getPTUs() { return PTUs; }
 
   std::unique_ptr<llvm::Module> GenModule();
 
 private:
   llvm::Expected<PartialTranslationUnit &> ParseOrWrapTopLevelDecl();
+
+  llvm::Expected<PartialTranslationUnit &> ParseForPTU(FileID FID,
+                                                       SourceLocation SrcLoc);
+
+  std::pair<FileID, SourceLocation> createSourceFile(llvm::StringRef SourceName,
+                                                     llvm::StringRef Input);
 };
 } // end namespace clang
 
 #endif // LLVM_CLANG_LIB_INTERPRETER_INCREMENTALPARSER_H

clang/lib/Interpreter/IncrementalParser.cpp

 //===--------- IncrementalParser.cpp - Incremental Compilation  -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the class which performs incremental code compilation.
 //
 //===----------------------------------------------------------------------===//
 
 #include "IncrementalParser.h"
+#include "ExternalSource.h"
 #include "clang/AST/DeclContextInternals.h"
 #include "clang/CodeGen/BackendUtil.h"
 #include "clang/CodeGen/CodeGenAction.h"
 #include "clang/CodeGen/ModuleBuilder.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendAction.h"
 #include "clang/FrontendTool/Utils.h"
 #include "clang/Interpreter/Interpreter.h"

 /// such as \p Preprocessor and \p Sema assuming no further input will come.
 ///
 /// \p IncrementalAction ensures it keep its underlying action's objects alive
 /// as long as the \p IncrementalParser needs them.
 ///
 class IncrementalAction : public WrapperFrontendAction {
 private:
   bool IsTerminating = false;
+  const CompilerInstance *ParentCI;
 
 public:
   IncrementalAction(CompilerInstance &CI, llvm::LLVMContext &LLVMCtx,
-                    llvm::Error &Err)
+                    llvm::Error &Err,
+                    const CompilerInstance *ParentCI = nullptr)
       : WrapperFrontendAction([&]() {
           llvm::ErrorAsOutParameter EAO(&Err);
           std::unique_ptr<FrontendAction> Act;
           switch (CI.getFrontendOpts().ProgramAction) {
           default:
             Err = llvm::createStringError(
                 std::errc::state_not_recoverable,
                 "Driver initialization failed. "
IncrementalAction(CompilerInstance &CI, llvm::LLVMContext &LLVMCtx,
             [[fallthrough]];
           case frontend::PrintPreprocessedInput:
             [[fallthrough]];
           case frontend::EmitLLVMOnly:
             Act.reset(new EmitLLVMOnlyAction(&LLVMCtx));
             break;
           }
           return Act;
-        }()) {}
+        }()),
+        ParentCI(ParentCI) {}
   FrontendAction *getWrapped() const { return WrappedAction.get(); }
   TranslationUnitKind getTranslationUnitKind() override {
     return TU_Incremental;
   }
   void ExecuteAction() override {
     CompilerInstance &CI = getCompilerInstance();
     assert(CI.hasPreprocessor() && "No PP!");
 
     // FIXME: Move the truncation aspect of this into Sema, we delayed this till
     // here so the source manager would be initialized.
     if (hasCodeCompletionSupport() &&
         !CI.getFrontendOpts().CodeCompletionAt.FileName.empty())
       CI.createCodeCompletionConsumer();
 
     // Use a code completion consumer?
     CodeCompleteConsumer *CompletionConsumer = nullptr;
     if (CI.hasCodeCompletionConsumer())
       CompletionConsumer = &CI.getCodeCompletionConsumer();
 
     Preprocessor &PP = CI.getPreprocessor();
     PP.EnterMainSourceFile();
 
+    if (ParentCI) {
+      ExternalSource *myExternalSource = new ExternalSource(
+          CI.getASTContext(), CI.getFileManager(), ParentCI->getASTContext(),
+          ParentCI->getFileManager());
+      llvm::IntrusiveRefCntPtr<ExternalASTSource> astContextExternalSource(
+          myExternalSource);
+      CI.getASTContext().setExternalSource(astContextExternalSource);
+      CI.getASTContext().getTranslationUnitDecl()->setHasExternalVisibleStorage(
+          true);
+    }
+
     if (!CI.hasSema())
       CI.createSema(getTranslationUnitKind(), CompletionConsumer);
   }
 
   // Do not terminate after processing the input. This allows us to keep various
   // clang objects alive and to incrementally grow the current TU.
   void EndSourceFile() override {
     // The WrappedAction can be nullptr if we issued an error in the ctor.
CodeGenerator *IncrementalParser::getCodeGen() const {
   return static_cast<CodeGenAction *>(WrappedAct)->getCodeGenerator();
 }
 
 IncrementalParser::IncrementalParser() {}
 
 IncrementalParser::IncrementalParser(Interpreter &Interp,
                                      std::unique_ptr<CompilerInstance> Instance,
                                      llvm::LLVMContext &LLVMCtx,
-                                     llvm::Error &Err)
+                                     llvm::Error &Err,
+                                     const CompilerInstance *ParentCI)
     : CI(std::move(Instance)) {
   llvm::ErrorAsOutParameter EAO(&Err);
-  Act = std::make_unique<IncrementalAction>(*CI, LLVMCtx, Err);
+  Act = std::make_unique<IncrementalAction>(*CI, LLVMCtx, Err, ParentCI);
   if (Err)
     return;
   CI->ExecuteAction(*Act);
   std::unique_ptr<ASTConsumer> IncrConsumer =
       std::make_unique<IncrementalASTConsumer>(Interp, CI->takeASTConsumer());
   CI->setASTConsumer(std::move(IncrConsumer));
   Consumer = &CI->getASTConsumer();
   P.reset(
IncrementalParser::ParseOrWrapTopLevelDecl() {
   LocalInstantiations.perform();
   GlobalInstantiations.perform();
 
   Consumer->HandleTranslationUnit(C);
 
   return LastPTU;
 }
 
-llvm::Expected<PartialTranslationUnit &>
-IncrementalParser::Parse(llvm::StringRef input) {
+llvm::Error IncrementalParser::ParseForCodeCompletion(llvm::StringRef input,
+                                               size_t Col, size_t Line) {
   Preprocessor &PP = CI->getPreprocessor();
   assert(PP.isIncrementalProcessingEnabled() && "Not in incremental mode!?");
 
   std::ostringstream SourceName;
-  SourceName << "input_line_" << InputCount++;
+  SourceName << "input_line_[Completion]";
+
+  auto [FID, SrcLoc] = createSourceFile(SourceName.str(), input);
+  auto FE = CI->getSourceManager().getFileEntryRefForID(FID);
+
+  PP.SetCodeCompletionPoint(*FE, Line, Col);
+
+  // NewLoc only used for diags.
+  if (PP.EnterSourceFile(FID, /*DirLookup=*/nullptr, SrcLoc))
+    return llvm::Error::success();;
+
+  auto PTU = ParseOrWrapTopLevelDecl();
+  if (auto Err = PTU.takeError()) {
+    return std::move(Err);
+  }
 
+  return llvm::Error::success();;
+}
+
+std::pair<FileID, SourceLocation>
+IncrementalParser::createSourceFile(llvm::StringRef SourceName,
+                                    llvm::StringRef Input) {
   // Create an uninitialized memory buffer, copy code in and append "\n"
-  size_t InputSize = input.size(); // don't include trailing 0
+  size_t InputSize = Input.size(); // don't include trailing 0
   // MemBuffer size should *not* include terminating zero
   std::unique_ptr<llvm::MemoryBuffer> MB(
       llvm::WritableMemoryBuffer::getNewUninitMemBuffer(InputSize + 1,
                                                         SourceName.str()));
   char *MBStart = const_cast<char *>(MB->getBufferStart());
-  memcpy(MBStart, input.data(), InputSize);
+  memcpy(MBStart, Input.data(), InputSize);
   MBStart[InputSize] = '\n';
 
   SourceManager &SM = CI->getSourceManager();
 
   // FIXME: Create SourceLocation, which will allow clang to order the overload
   // candidates for example
   SourceLocation NewLoc = SM.getLocForStartOfFile(SM.getMainFileID());
 
+
+  const clang::FileEntry *FE = SM.getFileManager().getVirtualFile(
+      SourceName.str(), InputSize, 0 /* mod time*/);
+  SM.overrideFileContents(FE, std::move(MB));
+
   // Create FileID for the current buffer.
-  FileID FID = SM.createFileID(std::move(MB), SrcMgr::C_User, /*LoadedID=*/0,
-                               /*LoadedOffset=*/0, NewLoc);
+  FileID FID = SM.createFileID(FE, NewLoc, SrcMgr::C_User);
+  return {FID, NewLoc};
+}
+
+llvm::Expected<PartialTranslationUnit &>
+IncrementalParser::ParseForPTU(FileID FID, SourceLocation SrcLoc) {
+  // Create an uninitialized memory buffer, copy code in and append "\n"
+  Preprocessor &PP = CI->getPreprocessor();
+  assert(PP.isIncrementalProcessingEnabled() && "Not in incremental mode!?");
 
   // NewLoc only used for diags.
-  if (PP.EnterSourceFile(FID, /*DirLookup=*/nullptr, NewLoc))
+  if (PP.EnterSourceFile(FID, /*DirLookup=*/nullptr, SrcLoc))
     return llvm::make_error<llvm::StringError>("Parsing failed. "
                                                "Cannot enter source file.",
                                                std::error_code());
 
   auto PTU = ParseOrWrapTopLevelDecl();
   if (!PTU)
-    return PTU.takeError();
+    return std::move(PTU.takeError());
+  return *PTU;
+}
+
+llvm::Expected<PartialTranslationUnit &>
+IncrementalParser::Parse(llvm::StringRef input) {
+  Preprocessor &PP = CI->getPreprocessor();
+  std::ostringstream SourceName;
+  SourceName << "input_line_" << InputCount++;
+
+  auto [FID, SrcLoc] = createSourceFile(SourceName.str(), input);
+  auto PTU = ParseForPTU(FID, SrcLoc);
+
+  if (!PTU) {
+    return std::move(PTU.takeError());
+  }
 
   if (PP.getLangOpts().DelayedTemplateParsing) {
     // Microsoft-specific:
     // Late parsed templates can leave unswallowed "macro"-like tokens.
     // They will seriously confuse the Parser when entering the next
     // source file. So lex until we are EOF.
     Token Tok;
     do {

clang/lib/Interpreter/Interpreter.cpp

 //===------ Interpreter.cpp - Incremental Compilation and Execution -------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the component which performs incremental code
 // compilation and execution.
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Interpreter/Interpreter.h"
 
 #include "DeviceOffload.h"
+#include "ExternalSource.h"
 #include "IncrementalExecutor.h"
 #include "IncrementalParser.h"
 
 #include "InterpreterUtils.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Mangle.h"
 #include "clang/AST/TypeVisitor.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/CodeGen/CodeGenAction.h"
 #include "clang/CodeGen/ModuleBuilder.h"
 #include "clang/CodeGen/ObjectFilePCHContainerOperations.h"
 #include "clang/Driver/Compilation.h"
 #include "clang/Driver/Driver.h"
 #include "clang/Driver/Job.h"
 #include "clang/Driver/Options.h"
 #include "clang/Driver/Tool.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/TextDiagnosticBuffer.h"
+#include "clang/Interpreter/CodeCompletion.h"
 #include "clang/Interpreter/Value.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Sema/Lookup.h"
 #include "llvm/ExecutionEngine/JITSymbol.h"
 #include "llvm/ExecutionEngine/Orc/LLJIT.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/ErrorHandling.h"
CreateCI(const llvm::opt::ArgStringList &Argv) {
   Clang->getTarget().adjust(Clang->getDiagnostics(), Clang->getLangOpts());
 
   // Don't clear the AST before backend codegen since we do codegen multiple
   // times, reusing the same AST.
   Clang->getCodeGenOpts().ClearASTBeforeBackend = false;
 
   Clang->getFrontendOpts().DisableFree = false;
   Clang->getCodeGenOpts().DisableFree = false;
-
   return std::move(Clang);
 }
 
 } // anonymous namespace
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
 IncrementalCompilerBuilder::create(std::vector<const char *> &ClangArgv) {
 
Interpreter::Interpreter(std::unique_ptr<CompilerInstance> CI,
                          llvm::Error &Err) {
   llvm::ErrorAsOutParameter EAO(&Err);
   auto LLVMCtx = std::make_unique<llvm::LLVMContext>();
   TSCtx = std::make_unique<llvm::orc::ThreadSafeContext>(std::move(LLVMCtx));
   IncrParser = std::make_unique<IncrementalParser>(*this, std::move(CI),
                                                    *TSCtx->getContext(), Err);
 }
 
+Interpreter::Interpreter(std::unique_ptr<CompilerInstance> CI, llvm::Error &Err,
+                         CodeCompleteConsumer* CConsumer,
+                         const CompilerInstance *ParentCI) {
+  llvm::ErrorAsOutParameter EAO(&Err);
+  auto LLVMCtx = std::make_unique<llvm::LLVMContext>();
+  TSCtx = std::make_unique<llvm::orc::ThreadSafeContext>(std::move(LLVMCtx));
+  CI->setCodeCompletionConsumer(CConsumer);
+  IncrParser = std::make_unique<IncrementalParser>(
+      *this, std::move(CI), *TSCtx->getContext(), Err, ParentCI);
+}
+
 Interpreter::~Interpreter() {
   if (IncrExecutor) {
     if (llvm::Error Err = IncrExecutor->cleanUp())
       llvm::report_fatal_error(
           llvm::Twine("Failed to clean up IncrementalExecutor: ") +
           toString(std::move(Err)));
   }
 }
Interpreter::create(std::unique_ptr<CompilerInstance> CI) {
   // FIXME: This is a ugly hack. Undo command checks its availability by looking
   // at the size of the PTU list. However we have parsed something in the
   // beginning of the REPL so we have to mark them as 'Irrevocable'.
   Interp->InitPTUSize = Interp->IncrParser->getPTUs().size();
   return std::move(Interp);
 }
 
 llvm::Expected<std::unique_ptr<Interpreter>>
+Interpreter::createForCodeCompletion(
+    IncrementalCompilerBuilder &CB, const CompilerInstance *ParentCI,
+    CodeCompleteConsumer* CConsumer) {
+  auto CI = CB.CreateCpp();
+  if (auto Err = CI.takeError()) {
+    return std::move(Err);
+  }
+
+  (*CI)->getPreprocessorOpts().SingleFileParseMode = true;
+
+  (*CI)->getLangOpts().SpellChecking = false;
+  (*CI)->getLangOpts().DelayedTemplateParsing = false;
+
+  llvm::Error Err = llvm::Error::success();
+  auto Interp = std::unique_ptr<Interpreter>(
+      new Interpreter(std::move(*CI), Err, CConsumer, ParentCI));
+
+  if (Err)
+    return std::move(Err);
+
+  Interp->InitPTUSize = Interp->IncrParser->getPTUs().size();
+  return std::move(Interp);
+}
+
+llvm::Expected<std::unique_ptr<Interpreter>>
 Interpreter::createWithCUDA(std::unique_ptr<CompilerInstance> CI,
                             std::unique_ptr<CompilerInstance> DCI) {
   // avoid writing fat binary to disk using an in-memory virtual file system
   llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> IMVFS =
       std::make_unique<llvm::vfs::InMemoryFileSystem>();
   llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem> OverlayVFS =
       std::make_unique<llvm::vfs::OverlayFileSystem>(
           llvm::vfs::getRealFileSystem());
Expr *Interpreter::SynthesizeExpr(Expr *E) {
 
   ExprResult Result = Builder.getCall();
   // It could fail, like printing an array type in C. (not supported)
   if (Result.isInvalid())
     return E;
   return Result.get();
 }
 
+llvm::Error Interpreter::CodeComplete(llvm::StringRef Input, size_t Col, size_t Line) {
+  return IncrParser->ParseForCodeCompletion(Input, Col, Line);
+}
+
 // Temporary rvalue struct that need special care.
 REPL_EXTERNAL_VISIBILITY void *
 __clang_Interpreter_SetValueWithAlloc(void *This, void *OutVal,
                                       void *OpaqueType) {
   Value &VRef = *(Value *)OutVal;
   VRef = Value(static_cast<Interpreter *>(This), OpaqueType);
   return VRef.getPtr();
 }

clang/lib/Parse/ParseDecl.cpp

 #include "clang/AST/ASTContext.h"
 #include "clang/AST/DeclTemplate.h"
 #include "clang/AST/PrettyDeclStackTrace.h"
 #include "clang/Basic/AddressSpaces.h"
 #include "clang/Basic/AttributeCommonInfo.h"
 #include "clang/Basic/Attributes.h"
 #include "clang/Basic/CharInfo.h"
 #include "clang/Basic/TargetInfo.h"
+#include "clang/Basic/TokenKinds.h"
 #include "clang/Parse/ParseDiagnostic.h"
 #include "clang/Parse/Parser.h"
 #include "clang/Parse/RAIIObjectsForParser.h"
 #include "clang/Sema/EnterExpressionEvaluationContext.h"
 #include "clang/Sema/Lookup.h"
 #include "clang/Sema/ParsedTemplate.h"
 #include "clang/Sema/Scope.h"
 #include "clang/Sema/SemaDiagnostic.h"
assert(D.isPastIdentifier() &&
          "Haven't past the location of the identifier yet?");
 
   // Don't parse attributes unless we have parsed an unparenthesized name.
   if (D.hasName() && !D.getNumTypeObjects())
     MaybeParseCXX11Attributes(D);
 
   while (true) {
     if (Tok.is(tok::l_paren)) {
+      if (PP.isIncrementalProcessingEnabled() && NextToken().is(tok::code_completion)) {
+        // In clang-repl, code completion for input like `void foo(<tab>` should not trigger a parsing error.
+        // So we make the declarator malformed and exits the loop.
+        ConsumeParen();
+        cutOffParsing();
+        D.SetIdentifier(nullptr, Tok.getLocation());
+        D.setInvalidType(true);
+        break;
+      }
+
       bool IsFunctionDeclaration = D.isFunctionDeclaratorAFunctionDeclaration();
       // Enter function-declaration scope, limiting any declarators to the
       // function prototype scope, including parameter declarators.
       ParseScope PrototypeScope(this,
                                 Scope::FunctionPrototypeScope|Scope::DeclScope|
                                 (IsFunctionDeclaration
                                    ? Scope::FunctionDeclarationScope : 0));
 

clang/lib/Parse/Parser.cpp

Parser::ParseExternalDeclaration(ParsedAttributes &Attrs,
   case tok::code_completion:
     cutOffParsing();
     if (CurParsedObjCImpl) {
       // Code-complete Objective-C methods even without leading '-'/'+' prefix.
       Actions.CodeCompleteObjCMethodDecl(getCurScope(),
                                          /*IsInstanceMethod=*/std::nullopt,
                                          /*ReturnType=*/nullptr);
     }
+
+    Sema::ParserCompletionContext PCC;
+    if (CurParsedObjCImpl) {
+      PCC = Sema::PCC_ObjCImplementation;
+    } else if (PP.isIncrementalProcessingEnabled()) {
+      PCC = Sema::PCC_TopLevelStmtDecl;
+    } else {
+      PCC = Sema::PCC_Namespace;
+    };
     Actions.CodeCompleteOrdinaryName(
         getCurScope(),
-        CurParsedObjCImpl ? Sema::PCC_ObjCImplementation : Sema::PCC_Namespace);
+        PCC);
     return nullptr;
   case tok::kw_import: {
     Sema::ModuleImportState IS = Sema::ModuleImportState::NotACXX20Module;
     if (getLangOpts().CPlusPlusModules) {
       llvm_unreachable("not expecting a c++20 import here");
       ProhibitAttributes(Attrs);
     }
     SingleDecl = ParseModuleImport(SourceLocation(), IS);

clang/lib/Sema/CodeCompleteConsumer.cpp

bool CodeCompletionContext::wantConstructorResults() const {
   switch (CCKind) {
   case CCC_Recovery:
   case CCC_Statement:
   case CCC_Expression:
   case CCC_ObjCMessageReceiver:
   case CCC_ParenthesizedExpression:
   case CCC_Symbol:
   case CCC_SymbolOrNewName:
+  case CCC_ReplTopLevel:
     return true;
 
   case CCC_TopLevel:
   case CCC_ObjCInterface:
   case CCC_ObjCImplementation:
   case CCC_ObjCIvarList:
   case CCC_ClassStructUnion:
   case CCC_DotMemberAccess:
StringRef clang::getCompletionKindString(CodeCompletionContext::Kind Kind) {
   case CCKind::CCC_IncludedFile:
     return "IncludedFile";
   case CCKind::CCC_Attribute:
     return "Attribute";
   case CCKind::CCC_Recovery:
     return "Recovery";
   case CCKind::CCC_ObjCClassForwardDecl:
     return "ObjCClassForwardDecl";
+  case CCKind::CCC_ReplTopLevel:
+    return "ReplTopLevel";
   }
   llvm_unreachable("Invalid CodeCompletionContext::Kind!");
 }
 
 //===----------------------------------------------------------------------===//
 // Code completion string implementation
 //===----------------------------------------------------------------------===//
 

clang/lib/Sema/SemaCodeComplete.cpp

explicit ResultBuilder(Sema &SemaRef, CodeCompletionAllocator &Allocator,
         ObjCImplementation(nullptr) {
     // If this is an Objective-C instance method definition, dig out the
     // corresponding implementation.
     switch (CompletionContext.getKind()) {
     case CodeCompletionContext::CCC_Expression:
     case CodeCompletionContext::CCC_ObjCMessageReceiver:
     case CodeCompletionContext::CCC_ParenthesizedExpression:
     case CodeCompletionContext::CCC_Statement:
+    case CodeCompletionContext::CCC_ReplTopLevel:
     case CodeCompletionContext::CCC_Recovery:
       if (ObjCMethodDecl *Method = SemaRef.getCurMethodDecl())
         if (Method->isInstanceMethod())
           if (ObjCInterfaceDecl *Interface = Method->getClassInterface())
             ObjCImplementation = Interface->getImplementation();
       break;
 
     default:
static void AddFunctionSpecifiers(Sema::ParserCompletionContext CCC,
   case Sema::PCC_Template:
     if (LangOpts.CPlusPlus || LangOpts.C99)
       Results.AddResult(Result("inline"));
     break;
 
   case Sema::PCC_ObjCInstanceVariableList:
   case Sema::PCC_Expression:
   case Sema::PCC_Statement:
+  case Sema::PCC_TopLevelStmtDecl:
   case Sema::PCC_ForInit:
   case Sema::PCC_Condition:
   case Sema::PCC_RecoveryInFunction:
   case Sema::PCC_Type:
   case Sema::PCC_ParenthesizedExpression:
   case Sema::PCC_LocalDeclarationSpecifiers:
     break;
   }
static bool WantTypesInContext(Sema::ParserCompletionContext CCC,
   case Sema::PCC_ObjCInstanceVariableList:
   case Sema::PCC_Template:
   case Sema::PCC_MemberTemplate:
   case Sema::PCC_Statement:
   case Sema::PCC_RecoveryInFunction:
   case Sema::PCC_Type:
   case Sema::PCC_ParenthesizedExpression:
   case Sema::PCC_LocalDeclarationSpecifiers:
+  case Sema::PCC_TopLevelStmtDecl:
     return true;
 
   case Sema::PCC_Expression:
   case Sema::PCC_Condition:
     return LangOpts.CPlusPlus;
 
   case Sema::PCC_ObjCInterface:
   case Sema::PCC_ObjCImplementation:
case Sema::PCC_ObjCImplementation:
     AddFunctionSpecifiers(CCC, SemaRef.getLangOpts(), Results);
     break;
 
   case Sema::PCC_ObjCInstanceVariableList:
     AddObjCVisibilityResults(SemaRef.getLangOpts(), Results, true);
     break;
 
   case Sema::PCC_RecoveryInFunction:
+  case Sema::PCC_TopLevelStmtDecl:
   case Sema::PCC_Statement: {
     if (SemaRef.getLangOpts().CPlusPlus11)
       AddUsingAliasResult(Builder, Results);
 
     AddTypedefResult(Results);
 
     if (SemaRef.getLangOpts().CPlusPlus && Results.includeCodePatterns() &&
         SemaRef.getLangOpts().CXXExceptions) {
mapCodeCompletionContext(Sema &S, Sema::ParserCompletionContext PCC) {
   case Sema::PCC_Type:
     return CodeCompletionContext::CCC_Type;
 
   case Sema::PCC_ParenthesizedExpression:
     return CodeCompletionContext::CCC_ParenthesizedExpression;
 
   case Sema::PCC_LocalDeclarationSpecifiers:
     return CodeCompletionContext::CCC_Type;
+  case Sema::PCC_TopLevelStmtDecl:
+    return CodeCompletionContext::CCC_ReplTopLevel;
   }
 
   llvm_unreachable("Invalid ParserCompletionContext!");
 }
 
 /// If we're in a C++ virtual member function, add completion results
 /// that invoke the functions we override, since it's common to invoke the
 /// overridden function as well as adding new functionality.
void Sema::CodeCompleteOrdinaryName(Scope *S,
   case PCC_Template:
   case PCC_MemberTemplate:
   case PCC_Type:
   case PCC_LocalDeclarationSpecifiers:
     Results.setFilter(&ResultBuilder::IsOrdinaryNonValueName);
     break;
 
   case PCC_Statement:
+  case PCC_TopLevelStmtDecl:
   case PCC_ParenthesizedExpression:
   case PCC_Expression:
   case PCC_ForInit:
   case PCC_Condition:
     if (WantTypesInContext(CompletionContext, getLangOpts()))
       Results.setFilter(&ResultBuilder::IsOrdinaryName);
     else
       Results.setFilter(&ResultBuilder::IsOrdinaryNonTypeName);
void Sema::CodeCompleteOrdinaryName(Scope *S,
 
   AddOrdinaryNameResults(CompletionContext, S, *this, Results);
   Results.ExitScope();
 
   switch (CompletionContext) {
   case PCC_ParenthesizedExpression:
   case PCC_Expression:
   case PCC_Statement:
+  case PCC_TopLevelStmtDecl:
   case PCC_RecoveryInFunction:
     if (S->getFnParent())
       AddPrettyFunctionResults(getLangOpts(), Results);
     break;
 
   case PCC_Namespace:
   case PCC_Class:
   case PCC_ObjCInterface:

clang/tools/clang-repl/ClangRepl.cpp

 //===--- tools/clang-repl/ClangRepl.cpp - clang-repl - the Clang REPL -----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //  This file implements a REPL tool on top of clang.
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendDiagnostic.h"
+#include "clang/Interpreter/CodeCompletion.h"
 #include "clang/Interpreter/Interpreter.h"
 
 #include "llvm/ExecutionEngine/Orc/LLJIT.h"
 #include "llvm/LineEditor/LineEditor.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ManagedStatic.h" // llvm_shutdown
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetSelect.h"
for (const std::string &input : OptInputs) {
     if (auto Err = Interp->ParseAndExecute(input))
       llvm::logAllUnhandledErrors(std::move(Err), llvm::errs(), "error: ");
   }
 
   bool HasError = false;
 
   if (OptInputs.empty()) {
     llvm::LineEditor LE("clang-repl");
-    // FIXME: Add LE.setListCompleter
     std::string Input;
+    LE.setListCompleter(clang::ReplListCompleter(CB, *Interp));
     while (std::optional<std::string> Line = LE.readLine()) {
       llvm::StringRef L = *Line;
       L = L.trim();
       if (L.endswith("\\")) {
         // FIXME: Support #ifdef X \ ...
         Input += L.drop_back(1);
         LE.setPrompt("clang-repl...   ");
         continue;
       }
 
       Input += L;
-
       if (Input == R"(%quit)") {
         break;
-      } else if (Input == R"(%undo)") {
+      }
+      if (Input == R"(%undo)") {
         if (auto Err = Interp->Undo()) {
           llvm::logAllUnhandledErrors(std::move(Err), llvm::errs(), "error: ");
           HasError = true;
         }
       } else if (Input.rfind("%lib ", 0) == 0) {
         if (auto Err = Interp->LoadDynamicLibrary(Input.data() + 5)) {
           llvm::logAllUnhandledErrors(std::move(Err), llvm::errs(), "error: ");
           HasError = true;

clang/tools/libclang/CIndexCodeCompletion.cpp

switch (kind) {
     case CodeCompletionContext::CCC_ObjCInterface:
     case CodeCompletionContext::CCC_ObjCImplementation:
     case CodeCompletionContext::CCC_ObjCClassForwardDecl:
     case CodeCompletionContext::CCC_NewName:
     case CodeCompletionContext::CCC_MacroName:
     case CodeCompletionContext::CCC_PreprocessorExpression:
     case CodeCompletionContext::CCC_PreprocessorDirective:
     case CodeCompletionContext::CCC_Attribute:
+    case CodeCompletionContext::CCC_ReplTopLevel:
     case CodeCompletionContext::CCC_TypeQualifiers: {
       //Only Clang results should be accepted, so we'll set all of the other
       //context bits to 0 (i.e. the empty set)
       contexts = CXCompletionContext_Unexposed;
       break;
     }
     case CodeCompletionContext::CCC_Recovery: {
       //We don't know what the current context is, so we'll return unknown

clang/unittests/Interpreter/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   ${LLVM_TARGETS_TO_BUILD}
   Core
   OrcJIT
   Support
   TargetParser
   )
 
 add_clang_unittest(ClangReplInterpreterTests
   IncrementalProcessingTest.cpp
   InterpreterTest.cpp
+  CodeCompletionTest.cpp
   )
 target_link_libraries(ClangReplInterpreterTests PUBLIC
   clangAST
   clangBasic
   clangInterpreter
   clangFrontend
   clangSema
   )
 
 # Exceptions on Windows are not yet supported.
 if(NOT WIN32)
   add_subdirectory(ExceptionTests)
 endif()
 
 export_executable_symbols(ClangReplInterpreterTests)

clang/unittests/Interpreter/CodeCompletionTest.cpp

+#include "clang/Interpreter/CodeCompletion.h"
+#include "clang/Interpreter/Interpreter.h"
+
+#include "llvm/LineEditor/LineEditor.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/raw_ostream.h"
+
+
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+using namespace clang;
+namespace {
+auto CB = clang::IncrementalCompilerBuilder();
+
+static std::unique_ptr<Interpreter> createInterpreter() {
+  auto CI = cantFail(CB.CreateCpp());
+  return cantFail(clang::Interpreter::create(std::move(CI)));
+}
+
+TEST(CodeCompletionTest, Sanity) {
+  auto Interp = createInterpreter();
+  if (auto R = Interp->ParseAndExecute("int foo = 12;")) {
+    consumeError(std::move(R));
+    return;
+  }
+  auto Completer = ReplListCompleter(CB, *Interp);
+  std::vector<llvm::LineEditor::Completion> comps =
+      Completer(std::string("f"), 1);
+  EXPECT_EQ((size_t)2, comps.size()); // foo and float
+  EXPECT_EQ(comps[0].TypedText, std::string("oo"));
+}
+
+TEST(CodeCompletionTest, SanityNoneValid) {
+  auto Interp = createInterpreter();
+  if (auto R = Interp->ParseAndExecute("int foo = 12;")) {
+    consumeError(std::move(R));
+    return;
+  }
+  auto Completer = ReplListCompleter(CB, *Interp);
+  std::vector<llvm::LineEditor::Completion> comps =
+      Completer(std::string("babanana"), 8);
+  EXPECT_EQ((size_t)0, comps.size()); // foo and float
+}
+
+TEST(CodeCompletionTest, TwoDecls) {
+  auto Interp = createInterpreter();
+  if (auto R = Interp->ParseAndExecute("int application = 12;")) {
+    consumeError(std::move(R));
+    return;
+  }
+  if (auto R = Interp->ParseAndExecute("int apple = 12;")) {
+    consumeError(std::move(R));
+    return;
+  }
+  auto Completer = ReplListCompleter(CB, *Interp);
+  std::vector<llvm::LineEditor::Completion> comps =
+      Completer(std::string("app"), 3);
+  EXPECT_EQ((size_t)2, comps.size());
+}
+
+
+TEST(CodeCompletionTest, CompFunDeclsNoError) {
+  auto Interp = createInterpreter();
+  std::string out;
+  auto rs = llvm::raw_string_ostream(out);
+  auto Completer = ReplListCompleter(CB, *Interp, rs);
+  std::vector<llvm::LineEditor::Completion> comps =
+    Completer(std::string("void app("), 9);
+  EXPECT_EQ((size_t)0, out.length());
+}
+
+} // anonymous namespace