[CodeComplete] Add completions for filenames in #include directives.

[sammccall]

Summary

The dir component ("somedir" in #include <somedir/fo...>) is considered fixed.
We append "foo" to each directory on the include path, and then list its files.

Completions are of the forms:
<somedir/foo.h>
^-text--^^-TT-^ (TypedText)
and
<somedir/foodir/
^-text--^^-TT--^

The filter is set to the filename part ("fo"), so fuzzy matching can be
applied to the filename only.

No fancy scoring/priorities are set, and no information is added to
CodeCompleteResult to make smart scoring possible. Could be in future.

Note directory iteration is currently stat-happy on unix but D51921 will fix.

[ilya-biryukov]

Amazing, can't wait for it to land!

lib/Lex/Lexer.cpp

1931 ↗

(On Diff #165432)

It's also valid to use \ for path separators on Windows, maybe also handle those?
Clang also supports this when running e.g. clang-cl /c foo.cpp.

2065 ↗

(On Diff #165432)

Does that mean we're losing completion at the end of file?
Not sure if it's a big deal. The only common pattern I can come up with is starting with an empty file and writing:

#include "^

Should we handle that? WDYT?

2075 ↗

(On Diff #165432)

This code looks exactly the same between two functions.
Maybe extract it into a helper function?

lib/Sema/SemaCodeComplete.cpp

8021 ↗

(On Diff #165432)

Could we avoid adding /, > and " if it's currently missing?
Otherwise it'll be annoying for editors that auto-add closing quotes and completions when editing existing includes, i.e.

// It's cool if we complete bar.h" (with closing quote) here:
#include "foo/^
// It's annoying if we complete an extra closing quote here:
#include "foo/^"

8028 ↗

(On Diff #165432)

Maybe leave this out?
When completing std::^ the completion is vector, not std::vector.
In the same spirit, for includes I would expect completions for #include <foo/^> to be bar.h, not <foo/bar.h>.

8046 ↗

(On Diff #165432)

The size of /usr/include on my machine is ~830 files and dirs, which is a little close to a limit.
Not sure if the number of files grow with time or with a number of installed packages, but maybe we want a slightly higher limit to make sure it's won stop showing some results from this important include dir anytime soon?

8057 ↗

(On Diff #165432)

Maybe do case-insensitive matching?
A corner case, but still...

8076 ↗

(On Diff #165432)

NIT: Maybe return from each branch and put llvm_unreachable at the end of the function?
To get an error in case the new enum value is added in addition to the compiler warning.

Feel free to ignore if you like the current version more, the compiler warning is not likely to go unnoticed anyway.

[sammccall]

Thanks a lot, great comments!
I haven't made changes yet (I will) but there's a few questions to work out first...

lib/Lex/Lexer.cpp

1931 ↗

(On Diff #165432)

Ah yuck, that's true (for some value of "valid"). Pre-C++11 it was explicitly UB...

So if a codebase consistently uses backslashes, we're going to break that style at least sometimes: we can't know whether to complete #include <foo as <foobar/ or <foobar\.

Any objection to just normalizing slashes for the whole include as part of completion?
e.g. #include <foo\ba would complete as #include <foo/bar/ and #include <foo/baz.h>?

Anything else is going to be fiddly, probably only help a few users a little bit, and leave edge cases.

2065 ↗

(On Diff #165432)

Ah no, it doesn't, though I thought so at first.

First: the address of C is CurPtr-1 (getAndAdvanceChar is in spirit return *CurPtr++).

Second: the buffer is null-terminated one-past-the-end (*BufferEnd == 0) and also has an embedded null at the completion point. So when completing at EOF it looks like:

# i n c l u d e < a \0 \0
                    ^C ^CurPtr
^BufferStart    ^BufPtr^BufferEnd

and we hit the case below. We only hit this case if we never saw a completion point.

lib/Sema/SemaCodeComplete.cpp

8021 ↗

(On Diff #165432)

Don't editors that add quotes/parens typically avoid duplicating them when they're typed/completed?

This isn't actually easy to do, because after completing file.h you want the cursor after the closing quote, and after completing dir/ you want the cursor inside the closing quote. I don't see a way to do this with CodeCompletionString other than generating the quote in the former case but not the latter. (At least not one that doesn't break something else...)

8028 ↗

(On Diff #165432)

I tried a few iterations here and this was the only one that didn't seem totally odd. Basically it's about the fact that these are quoted strings.

Ergonomically, having the closing quote completed after a filename *feels* right - you can hit enter afterwards and go to the next line.
If the closing quote is inserted, it has to be displayed (can't be avoided, also would be confusing).

So if you make the completion range just the filename, you end up with a completion list like:

#include <foo/ba^`
              bar.h>
              baz/
              backwards.h>

vs the current

#include <foo/ba^`
         <foo/bar.h>
         <foo/baz/
         <foo/backwards.h>

So I see your point here but I'm not sure the other behavior is actually better.

8057 ↗

(On Diff #165432)

This matches Driver's behavior (I'd reuse it but I don't think Sema can/should depend on Driver). Clang doesn't think "HPP" is a valid c++ header filename.
I'll add a comment.

[sammccall]

Unify common completion code from angled/quoted strings in Lexer.
Handle #include paths with \ on windows (normalize them to /)
Document why we picked particular extensions for headers.
Increase per-dir listing limit to 2500.

1. Updating D52076: [CodeComplete] Add completions for filenames in #include directives. #
2. Enter a brief description of the changes included in this update.
3. The first line is used as subject, next lines as comment. #
4. If you intended to create a new revision, use:
5. $ arc diff --create

:

[sammccall]

Addressed the comments I was sure about.
A couple of open questions in SemaCodeComplete about the formatting of the completions, but want to know what you think.

lib/Sema/SemaCodeComplete.cpp

8046 ↗

(On Diff #165432)

Wow, I only have 300.

Bumped the limit to 2500. This is based on handwavy advice on the internet that bad filesystems get slow in the low thousands.

8076 ↗

(On Diff #165432)

That won't give an error, that will give undefined behavior! (If a new value is added and the warning is missed)
The current code will just skip over the directory in that case, which I think is fine. (We have -Werror buildbots)

(Unless you mean return a dummy value, which is a little to unusual for my taste)

[ilya-biryukov]

Sorry for the late response.

Please see the comments about adding closing quotes and slashes. It does not seem to work in the clients that auto-add closing quotes or when completing inside existing includes.

lib/Lex/Lexer.cpp

1931 ↗

(On Diff #165432)

If we normalize, we probably want a config option for Windows users to be friendly to them.
No good layer to add the option, though (except a compiler flag, but that looks weird), so I don't think that's directly addressable.

Also, do we want to actually change slashes before the path component we are completing in the same include directive?

1. If we do, I suspect we might have problems with the existing language clients. We ran into problems when trying to add . to -> fix-its.
2. If we don't, the completion label would look differently from what's written. That's probably not a big deal.

2065 ↗

(On Diff #165432)

Ah, I see. Thanks for explanation. LG

lib/Sema/SemaCodeComplete.cpp

8021 ↗

(On Diff #165432)

Don't editors that add quotes/parens typically avoid duplicating them when they're typed/completed?

They do that on typing, but not on completions, which are treated literally, e.g. they'll an extra quote is actually added.
I've checked in VSCode and it produces double quotes in those cases. Not sure about the other editors that have this.
That looks like a corner case my intuition is that other editors won't do what we want either.

8028 ↗

(On Diff #165432)

Ergonomically, having the closing quote completed after a filename *feels* right - you can hit enter afterwards and go to the next line.

It definitely gives the best experience in Vim (which does not add closing quotes automatically) when writing new code.
The results are confusing in editors that insert quotes, e.g.

#include "foo/bar/^" --> #include "foo/bar/baz.h""

And when completing in existing code even in Vim:

#include "foo/bar^/baz.h" --> #include "foo/baraba//baz.h"
#include "foo/bar/baz.h^" --> #include "foo/bar/baz.h""

The added ergonomics is definitely nice in some cases, but the added confusion in other (not the most frequent, but still quite common) cases doesn't look nice.
WDYT?

8076 ↗

(On Diff #165432)

Agree, dummy values are definitely worse.
I thought it deterministically crashes when compiled without optimizations, so you get a runtime error in addition to the compilation warning.

But yeah, -Werror buildbots will quickly catch it anyway, so it's not a big deal.

[sammccall]

Address comments.
Only complete last segment for better compatibility.

[sammccall]

After experimenting, editor support for replacing non-identifier text before the cursor is... spotty at best.
So switched to just replacing the last path component.

(This makes me a bit sad because the completions now have closing quotes but not opening ones, which looks messy)

lib/Lex/Lexer.cpp

1931 ↗

(On Diff #165432)

So we can't reliably do the right thing here. e.g. consider completing a directory after #include < at the top of the file - which slash style?
I'm not sure that sometimes respecting \-style separators is better than never.
It's hard to see how we add an option for this, and unclear that it will help many people.

(Also this is totally nonstandard and the insertions we're suggesting will work. People can reasonably expect *some* friction from their tools if they're particular about an unusual style.)

Also, do we want to actually change slashes before the path component we are completing in the same include directive?

In the latest patch, these aren't part of the replacement range.

lib/Sema/SemaCodeComplete.cpp

8021 ↗

(On Diff #165432)

Added logic (to the lexer) to eat the closing quote.
This makes VSCode do the right thing. Some naive editors may still get this wrong though.

8028 ↗

(On Diff #165432)

(see above, we're now eating the closing quote)

8076 ↗

(On Diff #165432)

I was confused about this too, it works the way you describe for debug builds.

For NDEBUG, llvm_unreachable deliberately invokes UB (via intrinsic) to allow the compiler to optimize out the code handling this case.

[ilya-biryukov]

lib/Lex/Lexer.cpp

2086 ↗

(On Diff #165759)

This will probably break for backslash includes.
We should probably also handle backslashes here when LangOpts.MSVCCompat is true.

[sammccall]

Handle completion after a backslash in MSVC-compat mode.

[ilya-biryukov]

LGTM

lib/Sema/SemaCodeComplete.cpp

8057 ↗

(On Diff #165432)

IIUC, Driver uses this to infer the language based on file extension (when no explicit -x flag was provided).
However, this does not stop anyone from using uppercase file extensions in practice (and I assume inferring on headers is also pretty rare, since all compiles start with source files anyway).
But this should rare in any case, I don't think it's a blocker.

[sammccall]

lib/Sema/SemaCodeComplete.cpp

8057 ↗

(On Diff #165432)

Oh true, headers will work in practice with any extension, so Driver might be too restrictive.
Made these case insensitive.

cfe/trunk/include/clang-c/Index.h

enum CXCompletionContext {
   CXCompletionContext_MacroName = 1 << 20,
 
   /**
    * Natural language completions should be included in the results.
    */
   CXCompletionContext_NaturalLanguage = 1 << 21,
 
   /**
+   * #include file completions should be included in the results.
+   */
+  CXCompletionContext_IncludedFile = 1 << 22,
+
+  /**
    * The current context is unknown, so set all contexts.
    */
-  CXCompletionContext_Unknown = ((1 << 22) - 1)
+  CXCompletionContext_Unknown = ((1 << 23) - 1)
 };
 
 /**
  * Returns a default set of code-completion options that can be
  * passed to\c clang_codeCompleteAt().
  */
 CINDEX_LINKAGE unsigned clang_defaultCodeCompleteOptions(void);
 

cfe/trunk/include/clang/Lex/CodeCompletionHandler.h

 //===--- CodeCompletionHandler.h - Preprocessor code completion -*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 //  This file defines the CodeCompletionHandler interface, which provides
 //  code-completion callbacks for the preprocessor.
 //
 //===----------------------------------------------------------------------===//
 #ifndef LLVM_CLANG_LEX_CODECOMPLETIONHANDLER_H
 #define LLVM_CLANG_LEX_CODECOMPLETIONHANDLER_H
 
+#include "llvm/ADT/StringRef.h"
+
 namespace clang {
 
 class IdentifierInfo;
 class MacroInfo;
 
 /// Callback handler that receives notifications when performing code
 /// completion within the preprocessor.
 class CodeCompletionHandler {
public:
   ///
   /// There will be another callback invocation after the macro arguments are
   /// parsed, so this callback should generally be used to note that the next
   /// callback is invoked inside a macro argument.
   virtual void CodeCompleteMacroArgument(IdentifierInfo *Macro,
                                          MacroInfo *MacroInfo,
                                          unsigned ArgumentIndex) { }
 
+  /// Callback invoked when performing code completion inside the filename
+  /// part of an #include directive. (Also #import, #include_next, etc).
+  /// \p Dir is the directory relative to the include path, e.g. "a" for <a/b.
+  virtual void CodeCompleteIncludedFile(llvm::StringRef Dir, bool IsAngled) {}
+
   /// Callback invoked when performing code completion in a part of the
   /// file where we expect natural language, e.g., a comment, string, or
   /// \#error directive.
   virtual void CodeCompleteNaturalLanguage() { }
 };
 
 }
 
 #endif // LLVM_CLANG_LEX_CODECOMPLETIONHANDLER_H

cfe/trunk/include/clang/Lex/Lexer.h

private:
 
   bool lexEditorPlaceholder(Token &Result, const char *CurPtr);
 
   bool isCodeCompletionPoint(const char *CurPtr) const;
   void cutOffLexing() { BufferPtr = BufferEnd; }
 
   bool isHexaLiteral(const char *Start, const LangOptions &LangOpts);
 
+  void codeCompleteIncludedFile(const char *PathStart,
+                                const char *CompletionPoint, bool IsAngled);
+
   /// Read a universal character name.
   ///
   /// \param StartPtr The position in the source buffer after the initial '\'.
   ///                 If the UCN is syntactically well-formed (but not 
   ///                 necessarily valid), this parameter will be updated to
   ///                 point to the character after the UCN.
   /// \param SlashLoc The position in the source buffer of the '\'.
   /// \param Result   The token being formed. Pass \c nullptr to suppress

cfe/trunk/include/clang/Lex/Preprocessor.h

CodeCompletionHandler *getCodeCompletionHandler() const {
     return CodeComplete;
   }
 
   /// Clear out the code completion handler.
   void clearCodeCompletionHandler() {
     CodeComplete = nullptr;
   }
 
+  /// Hook used by the lexer to invoke the "included file" code
+  /// completion point.
+  void CodeCompleteIncludedFile(llvm::StringRef Dir, bool IsAngled);
+
   /// Hook used by the lexer to invoke the "natural language" code
   /// completion point.
   void CodeCompleteNaturalLanguage();
 
   /// Set the code completion token for filtering purposes.
   void setCodeCompletionIdentifierInfo(IdentifierInfo *Filter) {
     CodeCompletionII = Filter;
   }

cfe/trunk/include/clang/Parse/Parser.h

private:
   //===--------------------------------------------------------------------===//
   // Preprocessor code-completion pass-through
   void CodeCompleteDirective(bool InConditional) override;
   void CodeCompleteInConditionalExclusion() override;
   void CodeCompleteMacroName(bool IsDefinition) override;
   void CodeCompletePreprocessorExpression() override;
   void CodeCompleteMacroArgument(IdentifierInfo *Macro, MacroInfo *MacroInfo,
                                  unsigned ArgumentIndex) override;
+  void CodeCompleteIncludedFile(llvm::StringRef Dir, bool IsAngled) override;
   void CodeCompleteNaturalLanguage() override;
 };
 
 }  // end namespace clang
 
 #endif

cfe/trunk/include/clang/Sema/CodeCompleteConsumer.h

enum Kind {
 
     /// Code completion where the name of an Objective-C class is
     /// expected.
     CCC_ObjCInterfaceName,
 
     /// Code completion where an Objective-C category name is expected.
     CCC_ObjCCategoryName,
 
+    /// Code completion inside the filename part of a #include directive.
+    CCC_IncludedFile,
+
     /// An unknown context, in which we are recovering from a parsing
     /// error and don't know which completions we should give.
     CCC_Recovery
   };
 
   using VisitedContextSet = llvm::SmallPtrSet<DeclContext *, 8>;
 
 private:

cfe/trunk/include/clang/Sema/Sema.h

public:
   void CodeCompletePreprocessorDirective(bool InConditional);
   void CodeCompleteInPreprocessorConditionalExclusion(Scope *S);
   void CodeCompletePreprocessorMacroName(bool IsDefinition);
   void CodeCompletePreprocessorExpression();
   void CodeCompletePreprocessorMacroArgument(Scope *S,
                                              IdentifierInfo *Macro,
                                              MacroInfo *MacroInfo,
                                              unsigned Argument);
+  void CodeCompleteIncludedFile(llvm::StringRef Dir, bool IsAngled);
   void CodeCompleteNaturalLanguage();
   void CodeCompleteAvailabilityPlatformName();
   void GatherGlobalCodeCompletions(CodeCompletionAllocator &Allocator,
                                    CodeCompletionTUInfo &CCTUInfo,
                   SmallVectorImpl<CodeCompletionResult> &Results);
   //@}
 
   //===--------------------------------------------------------------------===//

cfe/trunk/lib/Frontend/ASTUnit.cpp

static void CalculateHiddenNames(const CodeCompletionContext &Context,
   case CodeCompletionContext::CCC_NaturalLanguage:
   case CodeCompletionContext::CCC_SelectorName:
   case CodeCompletionContext::CCC_TypeQualifiers:
   case CodeCompletionContext::CCC_Other:
   case CodeCompletionContext::CCC_OtherWithMacros:
   case CodeCompletionContext::CCC_ObjCInstanceMessage:
   case CodeCompletionContext::CCC_ObjCClassMessage:
   case CodeCompletionContext::CCC_ObjCCategoryName:
+  case CodeCompletionContext::CCC_IncludedFile:
     // We're looking for nothing, or we're looking for names that cannot
     // be hidden.
     return;
   }
 
   using Result = CodeCompletionResult;
   for (unsigned I = 0; I != NumResults; ++I) {
     if (Results[I].Kind != Result::RK_Declaration)

cfe/trunk/lib/Lex/Lexer.cpp

const char *Lexer::LexUDSuffix(Token &Result, const char *CurPtr,
 
   return CurPtr;
 }
 
 /// LexStringLiteral - Lex the remainder of a string literal, after having lexed
 /// either " or L" or u8" or u" or U".
 bool Lexer::LexStringLiteral(Token &Result, const char *CurPtr,
                              tok::TokenKind Kind) {
+  const char *AfterQuote = CurPtr;
   // Does this string contain the \0 character?
   const char *NulCharacter = nullptr;
 
   if (!isLexingRawMode() &&
       (Kind == tok::utf8_string_literal ||
        Kind == tok::utf16_string_literal ||
        Kind == tok::utf32_string_literal))
     Diag(BufferPtr, getLangOpts().CPlusPlus
if (C == '\n' || C == '\r' ||             // Newline.
       if (!isLexingRawMode() && !LangOpts.AsmPreprocessor)
         Diag(BufferPtr, diag::ext_unterminated_char_or_string) << 1;
       FormTokenWithChars(Result, CurPtr-1, tok::unknown);
       return true;
     }
 
     if (C == 0) {
       if (isCodeCompletionPoint(CurPtr-1)) {
+        if (ParsingFilename)
+          codeCompleteIncludedFile(AfterQuote, CurPtr - 1, /*IsAngled=*/false);
+        else
           PP->CodeCompleteNaturalLanguage();
-        FormTokenWithChars(Result, CurPtr-1, tok::unknown);
+        FormTokenWithChars(Result, CurPtr - 1, tok::unknown);
         cutOffLexing();
         return true;
       }
 
       NulCharacter = CurPtr-1;
     }
     C = getAndAdvanceChar(CurPtr, Result);
   }
bool Lexer::LexAngledStringLiteral(Token &Result, const char *CurPtr) {
   const char *AfterLessPos = CurPtr;
   char C = getAndAdvanceChar(CurPtr, Result);
   while (C != '>') {
     // Skip escaped characters.  Escaped newlines will already be processed by
     // getAndAdvanceChar.
     if (C == '\\')
       C = getAndAdvanceChar(CurPtr, Result);
 
-    if (C == '\n' || C == '\r' ||             // Newline.
-        (C == 0 && (CurPtr-1 == BufferEnd ||  // End of file.
-                    isCodeCompletionPoint(CurPtr-1)))) {
+    if (C == '\n' || C == '\r' ||                // Newline.
+        (C == 0 && (CurPtr - 1 == BufferEnd))) { // End of file.
       // If the filename is unterminated, then it must just be a lone <
       // character.  Return this as such.
       FormTokenWithChars(Result, AfterLessPos, tok::less);
       return true;
     }
 
     if (C == 0) {
+      if (isCodeCompletionPoint(CurPtr - 1)) {
+        codeCompleteIncludedFile(AfterLessPos, CurPtr - 1, /*IsAngled=*/true);
+        cutOffLexing();
+        FormTokenWithChars(Result, CurPtr - 1, tok::unknown);
+        return true;
+      }
       NulCharacter = CurPtr-1;
     }
     C = getAndAdvanceChar(CurPtr, Result);
   }
 
   // If a nul character existed in the string, warn about it.
   if (NulCharacter && !isLexingRawMode())
     Diag(NulCharacter, diag::null_in_char_or_string) << 1;
 
   // Update the location of token as well as BufferPtr.
   const char *TokStart = BufferPtr;
   FormTokenWithChars(Result, CurPtr, tok::angle_string_literal);
   Result.setLiteralData(TokStart);
   return true;
 }
 
+void Lexer::codeCompleteIncludedFile(const char *PathStart,
+                                     const char *CompletionPoint,
+                                     bool IsAngled) {
+  // Completion only applies to the filename, after the last slash.
+  StringRef PartialPath(PathStart, CompletionPoint - PathStart);
+  auto Slash = PartialPath.find_last_of(LangOpts.MSVCCompat ? "/\\" : "/");
+  StringRef Dir =
+      (Slash == StringRef::npos) ? "" : PartialPath.take_front(Slash);
+  const char *StartOfFilename =
+      (Slash == StringRef::npos) ? PathStart : PathStart + Slash + 1;
+  // Code completion filter range is the filename only, up to completion point.
+  PP->setCodeCompletionIdentifierInfo(&PP->getIdentifierTable().get(
+      StringRef(StartOfFilename, CompletionPoint - StartOfFilename)));
+  // We should replace the characters up to the closing quote, if any.
+  while (CompletionPoint < BufferEnd) {
+    char Next = *(CompletionPoint + 1);
+    if (Next == 0 || Next == '\r' || Next == '\n')
+      break;
+    ++CompletionPoint;
+    if (Next == (IsAngled ? '>' : '"'))
+      break;
+  }
+  PP->setCodeCompletionTokenRange(
+      FileLoc.getLocWithOffset(StartOfFilename - BufferStart),
+      FileLoc.getLocWithOffset(CompletionPoint - BufferStart));
+  PP->CodeCompleteIncludedFile(Dir, IsAngled);
+}
+
 /// LexCharConstant - Lex the remainder of a character constant, after having
 /// lexed either ' or L' or u8' or u' or U'.
 bool Lexer::LexCharConstant(Token &Result, const char *CurPtr,
                             tok::TokenKind Kind) {
   // Does this character contain the \0 character?
   const char *NulCharacter = nullptr;
 
   if (!isLexingRawMode()) {

cfe/trunk/lib/Lex/Preprocessor.cpp

bool Preprocessor::SetCodeCompletionPoint(const FileEntry *File,
   char *NewPos = std::copy(Buffer->getBufferStart(), Position, NewBuf);
   *NewPos = '\0';
   std::copy(Position, Buffer->getBufferEnd(), NewPos+1);
   SourceMgr.overrideFileContents(File, std::move(NewBuffer));
 
   return false;
 }
 
+void Preprocessor::CodeCompleteIncludedFile(llvm::StringRef Dir,
+                                            bool IsAngled) {
+  if (CodeComplete)
+    CodeComplete->CodeCompleteIncludedFile(Dir, IsAngled);
+  setCodeCompletionReached();
+}
+
 void Preprocessor::CodeCompleteNaturalLanguage() {
   if (CodeComplete)
     CodeComplete->CodeCompleteNaturalLanguage();
   setCodeCompletionReached();
 }
 
 /// getSpelling - This method is used to get the spelling of a token into a
 /// SmallVector. Note that the returned StringRef may not point to the

cfe/trunk/lib/Parse/Parser.cpp

 
 void Parser::CodeCompleteMacroArgument(IdentifierInfo *Macro,
                                        MacroInfo *MacroInfo,
                                        unsigned ArgumentIndex) {
   Actions.CodeCompletePreprocessorMacroArgument(getCurScope(), Macro, MacroInfo,
                                                 ArgumentIndex);
 }
 
+void Parser::CodeCompleteIncludedFile(llvm::StringRef Dir, bool IsAngled) {
+  Actions.CodeCompleteIncludedFile(Dir, IsAngled);
+}
+
 void Parser::CodeCompleteNaturalLanguage() {
   Actions.CodeCompleteNaturalLanguage();
 }
 
 bool Parser::ParseMicrosoftIfExistsCondition(IfExistsCondition& Result) {
   assert((Tok.is(tok::kw___if_exists) || Tok.is(tok::kw___if_not_exists)) &&
          "Expected '__if_exists' or '__if_not_exists'");
   Result.IsIfExists = Tok.is(tok::kw___if_exists);

cfe/trunk/lib/Sema/CodeCompleteConsumer.cpp

bool CodeCompletionContext::wantConstructorResults() const {
   case CCC_SelectorName:
   case CCC_TypeQualifiers:
   case CCC_Other:
   case CCC_OtherWithMacros:
   case CCC_ObjCInstanceMessage:
   case CCC_ObjCClassMessage:
   case CCC_ObjCInterfaceName:
   case CCC_ObjCCategoryName:
+  case CCC_IncludedFile:
     return false;
   }
 
   llvm_unreachable("Invalid CodeCompletionContext::Kind!");
 }
 
 StringRef clang::getCompletionKindString(CodeCompletionContext::Kind Kind) {
   using CCKind = CodeCompletionContext::Kind;
StringRef clang::getCompletionKindString(CodeCompletionContext::Kind Kind) {
   case CCKind::CCC_ObjCInstanceMessage:
     return "ObjCInstanceMessage";
   case CCKind::CCC_ObjCClassMessage:
     return "ObjCClassMessage";
   case CCKind::CCC_ObjCInterfaceName:
     return "ObjCInterfaceName";
   case CCKind::CCC_ObjCCategoryName:
     return "ObjCCategoryName";
+  case CCKind::CCC_IncludedFile:
+    return "IncludedFile";
   case CCKind::CCC_Recovery:
     return "Recovery";
   }
   llvm_unreachable("Invalid CodeCompletionContext::Kind!");
 }
 
 //===----------------------------------------------------------------------===//
 // Code completion string implementation
bool PrintingCodeCompleteConsumer::isResultFilteredOut(StringRef Filter,
   case CodeCompletionResult::RK_Declaration:
     return !(Result.Declaration->getIdentifier() &&
             Result.Declaration->getIdentifier()->getName().startswith(Filter));
   case CodeCompletionResult::RK_Keyword:
     return !StringRef(Result.Keyword).startswith(Filter);
   case CodeCompletionResult::RK_Macro:
     return !Result.Macro->getName().startswith(Filter);
   case CodeCompletionResult::RK_Pattern:
-    return !StringRef(Result.Pattern->getAsString()).startswith(Filter);
+    return !(Result.Pattern->getTypedText() &&
+             StringRef(Result.Pattern->getTypedText()).startswith(Filter));
   }
   llvm_unreachable("Unknown code completion result Kind.");
 }
 
 void
 PrintingCodeCompleteConsumer::ProcessCodeCompleteResults(Sema &SemaRef,
                                                  CodeCompletionContext Context,
                                                  CodeCompletionResult *Results,

cfe/trunk/lib/Sema/SemaCodeComplete.cpp

 #include "clang/Sema/SemaInternal.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/SmallBitVector.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/ADT/Twine.h"
+#include "llvm/ADT/iterator_range.h"
+#include "llvm/Support/Path.h"
 #include <list>
 #include <map>
 #include <vector>
 
 using namespace clang;
 using namespace sema;
 
 namespace {
void Sema::CodeCompletePreprocessorMacroArgument(Scope *S,
                                                  unsigned Argument) {
   // FIXME: In the future, we could provide "overload" results, much like we
   // do for function calls.
 
   // Now just ignore this. There will be another code-completion callback
   // for the expanded tokens.
 }
 
+// This handles completion inside an #include filename, e.g. #include <foo/ba
+// We look for the directory "foo" under each directory on the include path,
+// list its files, and reassemble the appropriate #include.
+void Sema::CodeCompleteIncludedFile(llvm::StringRef Dir, bool Angled) {
+  // RelDir should use /, but unescaped \ is possible on windows!
+  // Our completions will normalize to / for simplicity, this case is rare.
+  std::string RelDir = llvm::sys::path::convert_to_slash(Dir);
+  // We need the native slashes for the actual file system interactions.
+  SmallString<128> NativeRelDir = StringRef(RelDir);
+  llvm::sys::path::native(NativeRelDir);
+  auto FS = getSourceManager().getFileManager().getVirtualFileSystem();
+
+  ResultBuilder Results(*this, CodeCompleter->getAllocator(),
+                        CodeCompleter->getCodeCompletionTUInfo(),
+                        CodeCompletionContext::CCC_IncludedFile);
+  llvm::DenseSet<StringRef> SeenResults; // To deduplicate results.
+
+  // Helper: adds one file or directory completion result.
+  auto AddCompletion = [&](StringRef Filename, bool IsDirectory) {
+    SmallString<64> TypedChunk = Filename;
+    // Directory completion is up to the slash, e.g. <sys/
+    TypedChunk.push_back(IsDirectory ? '/' : Angled ? '>' : '"');
+    auto R = SeenResults.insert(TypedChunk);
+    if (R.second) { // New completion
+      const char *InternedTyped = Results.getAllocator().CopyString(TypedChunk);
+      *R.first = InternedTyped; // Avoid dangling StringRef.
+      CodeCompletionBuilder Builder(CodeCompleter->getAllocator(),
+                                    CodeCompleter->getCodeCompletionTUInfo());
+      Builder.AddTypedTextChunk(InternedTyped);
+      // The result is a "Pattern", which is pretty opaque.
+      // We may want to include the real filename to allow smart ranking.
+      Results.AddResult(CodeCompletionResult(Builder.TakeString()));
+    }
+  };
+
+  // Helper: scans IncludeDir for nice files, and adds results for each.
+  auto AddFilesFromIncludeDir = [&](StringRef IncludeDir, bool IsSystem) {
+    llvm::SmallString<128> Dir = IncludeDir;
+    if (!NativeRelDir.empty())
+      llvm::sys::path::append(Dir, NativeRelDir);
+
+    std::error_code EC;
+    unsigned Count = 0;
+    for (auto It = FS->dir_begin(Dir, EC);
+         !EC && It != vfs::directory_iterator(); It.increment(EC)) {
+      if (++Count == 2500) // If we happen to hit a huge directory,
+        break;             // bail out early so we're not too slow.
+      StringRef Filename = llvm::sys::path::filename(It->path());
+      switch (It->type()) {
+      case llvm::sys::fs::file_type::directory_file:
+        AddCompletion(Filename, /*IsDirectory=*/true);
+        break;
+      case llvm::sys::fs::file_type::regular_file:
+        // Only files that really look like headers. (Except in system dirs).
+        if (!IsSystem) {
+          // Header extensions from Types.def, which we can't depend on here.
+          if (!(Filename.endswith_lower(".h") ||
+                Filename.endswith_lower(".hh") ||
+                Filename.endswith_lower(".hpp") ||
+                Filename.endswith_lower(".inc")))
+            break;
+        }
+        AddCompletion(Filename, /*IsDirectory=*/false);
+        break;
+      default:
+        break;
+      }
+    }
+  };
+
+  // Helper: adds results relative to IncludeDir, if possible.
+  auto AddFilesFromDirLookup = [&](const DirectoryLookup &IncludeDir,
+                                   bool IsSystem) {
+    llvm::SmallString<128> Dir;
+    switch (IncludeDir.getLookupType()) {
+    case DirectoryLookup::LT_HeaderMap:
+      // header maps are not (currently) enumerable.
+      break;
+    case DirectoryLookup::LT_NormalDir:
+      AddFilesFromIncludeDir(IncludeDir.getDir()->getName(), IsSystem);
+      break;
+    case DirectoryLookup::LT_Framework:
+      AddFilesFromIncludeDir(IncludeDir.getFrameworkDir()->getName(), IsSystem);
+      break;
+    }
+  };
+
+  // Finally with all our helpers, we can scan the include path.
+  // Do this in standard order so deduplication keeps the right file.
+  // (In case we decide to add more details to the results later).
+  const auto &S = PP.getHeaderSearchInfo();
+  using llvm::make_range;
+  if (!Angled) {
+    // The current directory is on the include path for "quoted" includes.
+    auto *CurFile = PP.getCurrentFileLexer()->getFileEntry();
+    if (CurFile && CurFile->getDir())
+      AddFilesFromIncludeDir(CurFile->getDir()->getName(), false);
+    for (const auto &D : make_range(S.quoted_dir_begin(), S.quoted_dir_end()))
+      AddFilesFromDirLookup(D, false);
+  }
+  for (const auto &D : make_range(S.angled_dir_begin(), S.angled_dir_end()))
+    AddFilesFromDirLookup(D, true);
+  for (const auto &D : make_range(S.system_dir_begin(), S.system_dir_end()))
+    AddFilesFromDirLookup(D, true);
+
+  HandleCodeCompleteResults(this, CodeCompleter, Results.getCompletionContext(),
+                            Results.data(), Results.size());
+}
+
 void Sema::CodeCompleteNaturalLanguage() {
   HandleCodeCompleteResults(this, CodeCompleter,
                             CodeCompletionContext::CCC_NaturalLanguage,
                             nullptr, 0);
 }
 
 void Sema::CodeCompleteAvailabilityPlatformName() {
   ResultBuilder Results(*this, CodeCompleter->getAllocator(),

cfe/trunk/test/CodeCompletion/included-files.cpp

+// RUN: rm -rf %t && mkdir %t && cp %s %t/main.cc && mkdir %t/a
+// RUN: touch %t/foo.h && touch %t/foo.cc && touch %t/a/foosys %t/a/foosys.h
+
+// Quoted string shows header-ish files from CWD, and all from system.
+#include "foo.h"
+// RUN: %clang -fsyntax-only -isystem %t/a -Xclang -code-completion-at=%t/main.cc:5:13 %t/main.cc | FileCheck -check-prefix=CHECK-1 %s
+// CHECK-1-NOT: foo.cc"
+// CHECK-1: foo.h"
+// CHECK-1: foosys"
+
+// Quoted string with dir shows header-ish files in that subdir.
+#include "a/foosys"
+// RUN: %clang -fsyntax-only -isystem %t/a -Xclang -code-completion-at=%t/main.cc:12:13 %t/main.cc | FileCheck -check-prefix=CHECK-2 %s
+// CHECK-2-NOT: foo.h"
+// CHECK-2: foosys.h"
+// CHECK-2-NOT: foosys"
+
+// Angled string showes all files, but only in system dirs.
+#include <foosys>
+// RUN: %clang -fsyntax-only -isystem %t/a -Xclang -code-completion-at=%t/main.cc:19:13 %t/main.cc | FileCheck -check-prefix=CHECK-3 %s
+// CHECK-3-NOT: foo.cc>
+// CHECK-3-NOT: foo.h>
+// CHECK-3: foosys>
+
+// Backslash handling.
+#include "a\foosys"
+// RUN: %clang -fsyntax-only -isystem %t/a -Xclang -code-completion-at=%t/main.cc:26:13 %t/main.cc -fms-compatibility | FileCheck -check-prefix=CHECK-4 %s
+// CHECK-4: foosys.h"
+

cfe/trunk/tools/libclang/CIndexCodeCompletion.cpp

switch (kind) {
     case CodeCompletionContext::CCC_MacroNameUse: {
       contexts = CXCompletionContext_MacroName;
       break;
     }
     case CodeCompletionContext::CCC_NaturalLanguage: {
       contexts = CXCompletionContext_NaturalLanguage;
       break;
     }
+    case CodeCompletionContext::CCC_IncludedFile: {
+      contexts = CXCompletionContext_IncludedFile;
+      break;
+    }
     case CodeCompletionContext::CCC_SelectorName: {
       contexts = CXCompletionContext_ObjCSelectorName;
       break;
     }
     case CodeCompletionContext::CCC_ParenthesizedExpression: {
       contexts = CXCompletionContext_AnyType |
                  CXCompletionContext_ObjCInterface |
                  CXCompletionContext_AnyValue;