Make file and directory name completion work properly on Windows.

[zturner]

Summary

There were a couple of problems with this function on Windows. Different separators and differences in how tilde expressions are resolved for starters, but in addition there was no clear indication of what the function's inputs or outputs were supposed to be, and there were no tests to demonstrate its use.

To more easily paper over the differences between Windows paths, non-Windows paths, and tilde expressions, I've ported this function to use LLVM-based directory iteration (in fact, I would like to eliminate all of LLDB's directory iteration code entirely since LLVM's is cleaner / more efficient (i.e. it invokes fewer stat calls)). and llvm's portable path manipulation library.

Since file and directory completion assumes you are referring to files and directories on your local machine, it's safe to assume the path syntax properties of the host in doing so, so LLVM's APIs are perfect for this.

I've also added a fairly robust set of unit tests. Since you can't really predict what users will be on your machine, or what their home directories will be, I added an interface called TildeExpressionResolver, and in the unit test I've mocked up a fake implementation that acts like a unix password database. This allows us to configure some fake users and home directories in the test, so we can exercise all of those hard-to-test codepaths that normally otherwise depend on the host.

After this patch, file and directory completion works on Windows, and the same tests pass without my patch which suggests that I have not broken anything.

Note that I think there was a bug in the original implementation regarding symlinks which I've tried to fix. Previously, if you tried to complete /foo/bar/baz, and it finds /foo/bar/bazel which is a symlink, then it treats this as a directory as long as /foo/bar is a directory, which doesn't make any sense.

To fix this, if /foo/bar/bazel is a symlink, I stat it again with symlink-following on (which the previous code did not do), and check if the result is a directory.

There's no good way to test this, however.

[jingham]

What is the motivation behind of the "2" versions of the functions you added?

[zturner]

In D30789#696870, @jingham wrote:

What is the motivation behind of the "2" versions of the functions you added?

Ahh yea. I can call them something else if you have a better name. But basically in a unit test I don't have an instance of a CommandInterpreter object, and creating one is non-trivial (it requires a Debugger instance, which requires calling Debugger::CreateInstance(), which depends on a lot of other system initialization having been done.

But none of the methods I was trying to test use the CommandInterpreter anyway, so I just needed some kind of interface into the completion that didn't require me to pass one in.

[amccarth]

lldb/source/Commands/CommandCompletions.cpp

108 ↗

(On Diff #91212)

I know you're not inventing this API, but since you're changing it, can you do something about the bool and bool & parameters? It makes reading the call sites difficult without referring back here to see what those mean.

112 ↗

(On Diff #91212)

This leaves the caller with no way to say the don't want a tilde resolver. I didn't expect that from reading the API. Perhaps a comment on the API could clarify that you're going to get this default behavior if you specify nullptr.

[labath]

Seems very reasonable. A couple of details I noticed are in the inline comments.

As for the name, I'd suggest just dropping the 2 and letting overload resolution handle that. If we don't need the interpreter argument, then maybe we should aim for removing that overload in the future.

lldb/include/lldb/Utility/TildeExpressionResolver.h

18 ↗

(On Diff #91212)

Is there a reason why one these is a struct and the other a class?

btw, if you move the destructor definition into the .cpp file, it will also serve as a key method.

lldb/source/Commands/CommandCompletions.cpp

112 ↗

(On Diff #91212)

I agree with Adrian.
What do you think about an API like:
DiskFilesOrDirectories(..., const TildeExpressionResolver &Resolver = StandardTildeExpressionResolver()) ?

116 ↗

(On Diff #91212)

in PosixApi.h you define PATH_MAX as 32768, which sees a bit too much for a stack object. As this doesn't actually impact correctness, should we use a more down-to-earth value here?

167 ↗

(On Diff #91212)

This comment could use some reformatting.

lldb/source/Utility/TildeExpressionResolver.cpp

44 ↗

(On Diff #91212)

return 0;

49 ↗

(On Diff #91212)

Is this function allowed to be called with an empty Expr? The assert above seems to indicate that is the case, but then this will be undefined.

[zturner]

Addressed suggestions from labath@

[zturner]

lldb/source/Commands/CommandCompletions.cpp

112 ↗

(On Diff #91212)

I don't actually think there *should* be a way to specify no resolver. That sounds like YAGNI. The only real use cases are "Do it for real" or "do it with a mock implementation". I've updated this to take a reference, which should indicate that it is required, and now the version that delegates to this function allocates a real instance in its own stack frame and passes it in. Hopefully this makes things more clear.

[labath]

lgtm

lldb/trunk/include/lldb/Interpreter/CommandCompletions.h

 
 // Other libraries and framework includes
 // Project includes
 #include "lldb/Core/FileSpecList.h"
 #include "lldb/Core/SearchFilter.h"
 #include "lldb/Utility/RegularExpression.h"
 #include "lldb/lldb-private.h"
 
+#include "llvm/ADT/Twine.h"
+
 namespace lldb_private {
+struct TildeExpressionResolver;
 class CommandCompletions {
 public:
   //----------------------------------------------------------------------
   // This is the command completion callback that is used to complete the
   // argument of the option
   // it is bound to (in the OptionDefinition table below).  Return the total
   // number of matches.
   //----------------------------------------------------------------------
public:
   //----------------------------------------------------------------------
   // These are the generic completer functions:
   //----------------------------------------------------------------------
   static int DiskFiles(CommandInterpreter &interpreter,
                        llvm::StringRef partial_file_name, int match_start_point,
                        int max_return_elements, SearchFilter *searcher,
                        bool &word_complete, StringList &matches);
 
+  static int DiskFiles(const llvm::Twine &partial_file_name,
+                       StringList &matches, TildeExpressionResolver &Resolver);
+
   static int DiskDirectories(CommandInterpreter &interpreter,
                              llvm::StringRef partial_file_name,
                              int match_start_point, int max_return_elements,
                              SearchFilter *searcher, bool &word_complete,
                              StringList &matches);
 
+  static int DiskDirectories(const llvm::Twine &partial_file_name,
+                             StringList &matches,
+                             TildeExpressionResolver &Resolver);
+
   static int SourceFiles(CommandInterpreter &interpreter,
                          llvm::StringRef partial_file_name,
                          int match_start_point, int max_return_elements,
                          SearchFilter *searcher, bool &word_complete,
                          StringList &matches);
 
   static int Modules(CommandInterpreter &interpreter,
                      llvm::StringRef partial_file_name, int match_start_point,

lldb/trunk/include/lldb/Utility/TildeExpressionResolver.h

+//===--------------------- TildeExpressionResolver.h ------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_UTILITY_TILDE_EXPRESSION_RESOLVER_H
+#define LLDB_UTILITY_TILDE_EXPRESSION_RESOLVER_H
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
+
+namespace lldb_private {
+class TildeExpressionResolver {
+public:
+  virtual ~TildeExpressionResolver();
+
+  /// \brief Resolve a Tilde Expression contained according to bash rules.
+  ///
+  /// \param Expr Contains the tilde expression to resolve.  A valid tilde
+  ///             expression must begin with a tilde and contain only non
+  ///             separator characters.
+  ///
+  /// \param Output Contains the resolved tilde expression, or the original
+  ///               input if the tilde expression could not be resolved.
+  ///
+  /// \returns true if \p Expr was successfully resolved, false otherwise.
+  virtual bool ResolveExact(llvm::StringRef Expr,
+                            llvm::SmallVectorImpl<char> &Output) = 0;
+
+  /// \brief Auto-complete a tilde expression with all matching values.
+  ///
+  /// \param Expr Contains the tilde expression prefix to resolve.  See
+  ///             ResolveExact() for validity rules.
+  ///
+  /// \param Output Contains all matching home directories, each one
+  ///               itself unresolved (i.e. you need to call ResolveExact
+  ///               on each item to turn it into a real path).
+  ///
+  /// \returns true if there were any matches, false otherwise.
+  virtual bool ResolvePartial(llvm::StringRef Expr,
+                              llvm::StringSet<> &Output) = 0;
+};
+
+class StandardTildeExpressionResolver : public TildeExpressionResolver {
+public:
+  bool ResolveExact(llvm::StringRef Expr,
+                    llvm::SmallVectorImpl<char> &Output) override;
+  bool ResolvePartial(llvm::StringRef Expr, llvm::StringSet<> &Output) override;
+};
+}
+
+#endif // #ifndef LLDB_UTILITY_TILDE_EXPRESSION_RESOLVER_H

lldb/trunk/source/Commands/CommandCompletions.cpp

 #include <sys/stat.h>
 #if defined(__APPLE__) || defined(__linux__)
 #include <pwd.h>
 #endif
 
 // C++ Includes
 // Other libraries and framework includes
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/StringSet.h"
 
 // Project includes
 #include "lldb/Core/FileSpecList.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/PluginManager.h"
 #include "lldb/Host/FileSpec.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Interpreter/Args.h"
 #include "lldb/Interpreter/CommandCompletions.h"
 #include "lldb/Interpreter/CommandInterpreter.h"
 #include "lldb/Interpreter/OptionValueProperties.h"
 #include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/Variable.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/CleanUp.h"
+#include "lldb/Utility/TildeExpressionResolver.h"
 
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
 
 using namespace lldb_private;
 
 CommandCompletions::CommonCompletionElement
     CommandCompletions::g_common_completions[] = {
         {eCustomCompletion, nullptr},
         {eSourceFileCompletion, CommandCompletions::SourceFiles},
         {eDiskFileCompletion, CommandCompletions::DiskFiles},
if (searcher == nullptr) {
     SearchFilterForUnconstrainedSearches null_searcher(target_sp);
     completer.DoCompletion(&null_searcher);
   } else {
     completer.DoCompletion(searcher);
   }
   return matches.GetSize();
 }
 
-typedef struct DiskFilesOrDirectoriesBaton {
-  const char *remainder;
-  char *partial_name_copy;
-  bool only_directories;
-  bool *saw_directory;
-  StringList *matches;
-  char *end_ptr;
-  size_t baselen;
-} DiskFilesOrDirectoriesBaton;
-
-FileSpec::EnumerateDirectoryResult
-DiskFilesOrDirectoriesCallback(void *baton, llvm::sys::fs::file_type file_type,
-                               const FileSpec &spec) {
-  const char *name = spec.GetFilename().AsCString();
-
-  const DiskFilesOrDirectoriesBaton *parameters =
-      (DiskFilesOrDirectoriesBaton *)baton;
-  char *end_ptr = parameters->end_ptr;
-  char *partial_name_copy = parameters->partial_name_copy;
-  const char *remainder = parameters->remainder;
-
-  // Omit ".", ".." and any . files if the match string doesn't start with .
-  if (name[0] == '.') {
-    if (name[1] == '\0')
-      return FileSpec::eEnumerateDirectoryResultNext;
-    else if (name[1] == '.' && name[2] == '\0')
-      return FileSpec::eEnumerateDirectoryResultNext;
-    else if (remainder[0] != '.')
-      return FileSpec::eEnumerateDirectoryResultNext;
-  }
-
-  // If we found a directory, we put a "/" at the end of the name.
-
-  if (remainder[0] == '\0' || strstr(name, remainder) == name) {
-    if (strlen(name) + parameters->baselen >= PATH_MAX)
-      return FileSpec::eEnumerateDirectoryResultNext;
+static int DiskFilesOrDirectories(const llvm::Twine &partial_name,
+                                  bool only_directories, bool &saw_directory,
+                                  StringList &matches,
+                                  TildeExpressionResolver &Resolver) {
+  matches.Clear();
+
+  llvm::SmallString<256> CompletionBuffer;
+  llvm::SmallString<256> Storage;
+  partial_name.toVector(CompletionBuffer);
 
-    strcpy(end_ptr, name);
+  if (CompletionBuffer.size() >= PATH_MAX)
+    return 0;
 
-    namespace fs = llvm::sys::fs;
-    bool isa_directory = false;
-    if (file_type == fs::file_type::directory_file)
-      isa_directory = true;
-    else if (file_type == fs::file_type::symlink_file)
-      isa_directory = fs::is_directory(partial_name_copy);
+  namespace path = llvm::sys::path;
 
-    if (isa_directory) {
-      *parameters->saw_directory = true;
-      size_t len = strlen(parameters->partial_name_copy);
-      partial_name_copy[len] = '/';
-      partial_name_copy[len + 1] = '\0';
-    }
-    if (parameters->only_directories && !isa_directory)
-      return FileSpec::eEnumerateDirectoryResultNext;
-    parameters->matches->AppendString(partial_name_copy);
-  }
+  llvm::StringRef SearchDir;
+  llvm::StringRef PartialItem;
 
-  return FileSpec::eEnumerateDirectoryResultNext;
+  if (CompletionBuffer.startswith("~")) {
+    llvm::StringRef Buffer(CompletionBuffer);
+    size_t FirstSep = Buffer.find_if(path::is_separator);
+
+    llvm::StringRef Username = Buffer.take_front(FirstSep);
+    llvm::StringRef Remainder;
+    if (FirstSep != llvm::StringRef::npos)
+      Remainder = Buffer.drop_front(FirstSep + 1);
+
+    llvm::SmallString<PATH_MAX> Resolved;
+    if (!Resolver->ResolveExact(Username, Resolved)) {
+      // We couldn't resolve it as a full username.  If there were no slashes
+      // then this might be a partial username.   We try to resolve it as such
+      // but after that, we're done regardless of any matches.
+      if (FirstSep == llvm::StringRef::npos) {
+        llvm::StringSet<> MatchSet;
+        saw_directory = Resolver->ResolvePartial(Username, MatchSet);
+        for (const auto &S : MatchSet) {
+          Resolved = S.getKey();
+          path::append(Resolved, path::get_separator());
+          matches.AppendString(Resolved);
+        }
+        saw_directory = (matches.GetSize() > 0);
       }
-
-static int DiskFilesOrDirectories(llvm::StringRef partial_file_name,
-                                  bool only_directories, bool &saw_directory,
-                                  StringList &matches) {
-  // I'm going to  use the "glob" function with GLOB_TILDE for user directory
-  // expansion.
-  // If it is not defined on your host system, you'll need to implement it
-  // yourself...
-
-  size_t partial_name_len = partial_file_name.size();
-
-  if (partial_name_len >= PATH_MAX)
       return matches.GetSize();
+    }
 
-  // This copy of the string will be cut up into the directory part, and the
-  // remainder.  end_ptr below will point to the place of the remainder in this
-  // string.  Then when we've resolved the containing directory, and opened it,
-  // we'll read the directory contents and overwrite the partial_name_copy
-  // starting from end_ptr with each of the matches.  Thus we will preserve the
-  // form the user originally typed.
-
-  char partial_name_copy[PATH_MAX];
-  memcpy(partial_name_copy, partial_file_name.data(), partial_name_len);
-  partial_name_copy[partial_name_len] = '\0';
-
-  // We'll need to save a copy of the remainder for comparison, which we do
-  // here.
-  char remainder[PATH_MAX];
-
-  // end_ptr will point past the last / in partial_name_copy, or if there is no
-  // slash to the beginning of the string.
-  char *end_ptr;
-
-  end_ptr = strrchr(partial_name_copy, '/');
-
-  // This will store the resolved form of the containing directory
-  llvm::SmallString<64> containing_part;
-
-  if (end_ptr == nullptr) {
-    // There's no directory.  If the thing begins with a "~" then this is a bare
-    // user name.
-    if (*partial_name_copy == '~') {
-      // Nothing here but the user name.  We could just put a slash on the end,
-      // but for completeness sake we'll resolve the user name and only put a
-      // slash
-      // on the end if it exists.
-      llvm::SmallString<64> resolved_username(partial_name_copy);
-      FileSpec::ResolveUsername(resolved_username);
-
-      // Not sure how this would happen, a username longer than PATH_MAX?
-      // Still...
-      if (resolved_username.size() == 0) {
-        // The user name didn't resolve, let's look in the password database for
-        // matches.
-        // The user name database contains duplicates, and is not in
-        // alphabetical order, so
-        // we'll use a set to manage that for us.
-        FileSpec::ResolvePartialUsername(partial_name_copy, matches);
-        if (matches.GetSize() > 0)
-          saw_directory = true;
-        return matches.GetSize();
-      } else {
-        // The thing exists, put a '/' on the end, and return it...
-        // FIXME: complete user names here:
-        partial_name_copy[partial_name_len] = '/';
-        partial_name_copy[partial_name_len + 1] = '\0';
-        matches.AppendString(partial_name_copy);
-        saw_directory = true;
-        return matches.GetSize();
-      }
-    } else {
-      // The containing part is the CWD, and the whole string is the remainder.
-      containing_part = ".";
-      strcpy(remainder, partial_name_copy);
-      end_ptr = partial_name_copy;
-    }
-  } else {
-    if (end_ptr == partial_name_copy) {
-      // We're completing a file or directory in the root volume.
-      containing_part = "/";
-    } else {
-      containing_part.append(partial_name_copy, end_ptr);
-    }
-    // Push end_ptr past the final "/" and set remainder.
-    end_ptr++;
-    strcpy(remainder, end_ptr);
+    // If there was no trailing slash, then we're done as soon as we resolve the
+    // expression to the correct directory.  Otherwise we need to continue
+    // looking for matches within that directory.
+    if (FirstSep == llvm::StringRef::npos) {
+      // Make sure it ends with a separator.
+      path::append(CompletionBuffer, path::get_separator());
+      matches.AppendString(CompletionBuffer);
+      return 1;
+
+    // We want to keep the form the user typed, so we special case this to
+    // search in the fully resolved directory, but CompletionBuffer keeps the
+    // unmodified form that the user typed.
+    Storage = Resolved;
+    SearchDir = Resolved;
+    SearchDir = path::parent_path(CompletionBuffer);
   }
 
-  // Look for a user name in the containing part, and if it's there, resolve it
-  // and stick the
-  // result back into the containing_part:
+  size_t FullPrefixLen = CompletionBuffer.size();
 
-  if (*partial_name_copy == '~') {
-    FileSpec::ResolveUsername(containing_part);
-    // User name doesn't exist, we're not getting any further...
-    if (containing_part.empty())
-      return matches.GetSize();
-  }
+  PartialItem = path::filename(CompletionBuffer);
+  if (PartialItem == ".")
+    PartialItem = llvm::StringRef();
+
+  assert(!SearchDir.empty());
+  assert(!PartialItem.contains(path::get_separator()));
+
+  // SearchDir now contains the directory to search in, and Prefix contains the
+  // text we want to match against items in that directory.
+
+  std::error_code EC;
+  fs::directory_iterator Iter(SearchDir, EC, false);
+  fs::directory_iterator End;
+  for (; Iter != End && !EC; Iter.increment(EC)) {
+    auto &Entry = *Iter;
+
+    auto Name = path::filename(Entry.path());
+
+    // Omit ".", ".."
+    if (Name == "." || Name == ".." || !Name.startswith(PartialItem))
+      continue;
+
+    // We have a match.
+
+    fs::file_status st;
+    if (EC = Entry.status(st))
+      continue;
 
-  // Okay, containing_part is now the directory we want to open and look for
-  // files:
+    // If it's a symlink, then we treat it as a directory as long as the target
+    // is a directory.
+    bool is_dir = fs::is_directory(st);
+    if (fs::is_symlink_file(st)) {
+      fs::file_status target_st;
+      if (!fs::status(Entry.path(), target_st))
+        is_dir = fs::is_directory(target_st);
+    }
+    if (only_directories && !is_dir)
+      continue;
 
-  size_t baselen = end_ptr - partial_name_copy;
+    // Shrink it back down so that it just has the original prefix the user
+    // typed and remove the part of the name which is common to the located
+    // item and what the user typed.
+    CompletionBuffer.resize(FullPrefixLen);
+    Name = Name.drop_front(PartialItem.size());
+    CompletionBuffer.append(Name);
 
-  DiskFilesOrDirectoriesBaton parameters;
-  parameters.remainder = remainder;
-  parameters.partial_name_copy = partial_name_copy;
-  parameters.only_directories = only_directories;
-  parameters.saw_directory = &saw_directory;
-  parameters.matches = &matches;
-  parameters.end_ptr = end_ptr;
-  parameters.baselen = baselen;
+    if (is_dir) {
+      saw_directory = true;
+      path::append(CompletionBuffer, path::get_separator());
+    }
 
-  FileSpec::EnumerateDirectory(containing_part.c_str(), true, true, true,
-                               DiskFilesOrDirectoriesCallback, &parameters);
+    matches.AppendString(CompletionBuffer);
+  }
 
   return matches.GetSize();
 }
 
 int CommandCompletions::DiskFiles(CommandInterpreter &interpreter,
                                   llvm::StringRef partial_file_name,
                                   int match_start_point,
                                   int max_return_elements,
                                   SearchFilter *searcher, bool &word_complete,
                                   StringList &matches) {
-  int ret_val =
-      DiskFilesOrDirectories(partial_file_name, false, word_complete, matches);
-  word_complete = !word_complete;
+  word_complete = false;
+  StandardTildeExpressionResolver Resolver;
+  return DiskFiles(partial_file_name, matches, Resolver);
+}
+
+int CommandCompletions::DiskFiles(const llvm::Twine &partial_file_name,
+                                  StringList &matches,
+                                  TildeExpressionResolver &Resolver) {
+  bool word_complete;
+  int ret_val = DiskFilesOrDirectories(partial_file_name, false, word_complete,
+                                       matches, Resolver);
   return ret_val;
 }
 
 int CommandCompletions::DiskDirectories(
     CommandInterpreter &interpreter, llvm::StringRef partial_file_name,
     int match_start_point, int max_return_elements, SearchFilter *searcher,
     bool &word_complete, StringList &matches) {
-  int ret_val =
-      DiskFilesOrDirectories(partial_file_name, true, word_complete, matches);
   word_complete = false;
+  StandardTildeExpressionResolver Resolver;
+  return DiskDirectories(partial_file_name, matches, Resolver);
+}
+
+int CommandCompletions::DiskDirectories(const llvm::Twine &partial_file_name,
+                                        StringList &matches,
+                                        TildeExpressionResolver &Resolver) {
+  bool word_complete;
+  int ret_val = DiskFilesOrDirectories(partial_file_name, true, word_complete,
+                                       matches, Resolver);
   return ret_val;
 }
 
 int CommandCompletions::Modules(CommandInterpreter &interpreter,
                                 llvm::StringRef partial_file_name,
                                 int match_start_point, int max_return_elements,
                                 SearchFilter *searcher, bool &word_complete,
                                 StringList &matches) {

lldb/trunk/source/Utility/CMakeLists.txt

add_lldb_library(lldbUtility
   Stream.cpp
   StreamCallback.cpp
   StreamGDBRemote.cpp
   StreamString.cpp
   StringExtractor.cpp
   StringExtractorGDBRemote.cpp
   StringLexer.cpp
   TaskPool.cpp
+  TildeExpressionResolver.cpp
   UserID.cpp
   UriParser.cpp
   UUID.cpp
   VASprintf.cpp
   VMRange.cpp
 
   LINK_LIBS
   # lldbUtility cannot have any dependencies
 
   LINK_COMPONENTS
     Support
   )

lldb/trunk/source/Utility/TildeExpressionResolver.cpp

+//===--------------------- TildeExpressionResolver.cpp ----------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Utility/TildeExpressionResolver.h"
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+
+using namespace lldb_private;
+using namespace llvm;
+
+namespace fs = llvm::sys::fs;
+namespace path = llvm::sys::path;
+
+TildeExpressionResolver::~TildeExpressionResolver() {}
+
+bool StandardTildeExpressionResolver::ResolveExact(
+    StringRef Expr, SmallVectorImpl<char> &Output) {
+  // We expect the tilde expression to be ONLY the expression itself, and
+  // contain
+  // no separators.
+  assert(!llvm::any_of(Expr, path::is_separator));
+  assert(Expr.empty() || Expr[0] == '~');
+
+  return !fs::real_path(Expr, Output, true);
+}
+
+bool StandardTildeExpressionResolver::ResolvePartial(StringRef Expr,
+                                                     StringSet<> &Output) {
+  // We expect the tilde expression to be ONLY the expression itself, and
+  // contain no separators.
+  assert(!llvm::any_of(Expr, path::is_separator));
+  assert(Expr.empty() || Expr[0] == '~');
+
+  Output.clear();
+#if defined(LLVM_ON_WIN32)
+  return false;
+#else
+  if (Expr.empty())
+    return false;
+
+  SmallString<32> Buffer = "~";
+  setpwent();
+  struct passwd *user_entry;
+  Expr = Expr.drop_front();
+
+  while ((user_entry = getpwent()) != NULL) {
+    StringRef ThisName(user_entry->pw_name);
+    if (!ThisName.startswith(Expr))
+      continue;
+
+    Buffer.resize(1);
+    Buffer.append(ThisName);
+    Buffer.append(path::get_separator()) Output.insert(Buffer);
+  }
+
+  return true;
+#endif
+}

lldb/trunk/unittests/Interpreter/CMakeLists.txt

 add_lldb_unittest(InterpreterTests
   TestArgs.cpp
+  TestCompletion.cpp
 
   LINK_LIBS
     lldbInterpreter
   )
 
 target_link_libraries(InterpreterTests
   ${PYTHON_LIBRARY}
   )

lldb/trunk/unittests/Interpreter/TestCompletion.cpp

+//===-- TestCompletion.cpp --------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "gtest/gtest.h"
+
+#include "lldb/Core/StringList.h"
+#include "lldb/Interpreter/CommandCompletions.h"
+#include "lldb/Utility/TildeExpressionResolver.h"
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace fs = llvm::sys::fs;
+namespace path = llvm::sys::path;
+using namespace llvm;
+using namespace lldb_private;
+
+#define ASSERT_NO_ERROR(x)                                                     \
+  if (std::error_code ASSERT_NO_ERROR_ec = x) {                                \
+    SmallString<128> MessageStorage;                                           \
+    raw_svector_ostream Message(MessageStorage);                               \
+    Message << #x ": did not return errc::success.\n"                          \
+            << "error number: " << ASSERT_NO_ERROR_ec.value() << "\n"          \
+            << "error message: " << ASSERT_NO_ERROR_ec.message() << "\n";      \
+    GTEST_FATAL_FAILURE_(MessageStorage.c_str());                              \
+  } else {                                                                     \
+  }
+
+namespace {
+
+class MockTildeExpressionResolver : public TildeExpressionResolver {
+  StringRef CurrentUser;
+  StringMap<StringRef> UserDirectories;
+
+public:
+  explicit MockTildeExpressionResolver(StringRef CurrentUser, StringRef HomeDir)
+      : CurrentUser(CurrentUser) {
+    UserDirectories.insert(std::make_pair(CurrentUser, HomeDir));
+  }
+
+  void AddKnownUser(StringRef User, StringRef HomeDir) {
+    assert(UserDirectories.find(User) == UserDirectories.end());
+    UserDirectories.insert(std::make_pair(User, HomeDir));
+  }
+
+  void Clear() {
+    CurrentUser = StringRef();
+    UserDirectories.clear();
+  }
+
+  void SetCurrentUser(StringRef User) {
+    assert(UserDirectories.find(User) != UserDirectories.end());
+    CurrentUser = User;
+  }
+
+  bool ResolveExact(StringRef Expr, SmallVectorImpl<char> &Output) override {
+    Output.clear();
+
+    assert(!llvm::any_of(Expr, llvm::sys::path::is_separator));
+    assert(Expr.empty() || Expr[0] == '~');
+    Expr = Expr.drop_front();
+    if (Expr.empty()) {
+      auto Dir = UserDirectories[CurrentUser];
+      Output.append(Dir.begin(), Dir.end());
+      return true;
+    }
+
+    for (const auto &User : UserDirectories) {
+      if (User.getKey() != Expr)
+        continue;
+      Output.append(User.getValue().begin(), User.getValue().end());
+      return true;
+    }
+    return false;
+  }
+
+  bool ResolvePartial(StringRef Expr, StringSet<> &Output) override {
+    Output.clear();
+
+    assert(!llvm::any_of(Expr, llvm::sys::path::is_separator));
+    assert(Expr.empty() || Expr[0] == '~');
+    Expr = Expr.drop_front();
+
+    SmallString<16> QualifiedName = "~";
+    for (const auto &User : UserDirectories) {
+      if (!User.getKey().startswith(Expr))
+        continue;
+      QualifiedName.resize(1);
+      QualifiedName.append(User.getKey().begin(), User.getKey().end());
+      Output.insert(QualifiedName);
+    }
+
+    return !Output.empty();
+  }
+};
+
+class CompletionTest : public testing::Test {
+protected:
+  /// Unique temporary directory in which all created filesystem entities must
+  /// be placed. It is removed at the end of the test suite.
+  static SmallString<128> BaseDir;
+
+  static SmallString<128> DirFoo;
+  static SmallString<128> DirFooA;
+  static SmallString<128> DirFooB;
+  static SmallString<128> DirFooC;
+  static SmallString<128> DirBar;
+  static SmallString<128> DirBaz;
+  static SmallString<128> DirTestFolder;
+
+  static SmallString<128> FileAA;
+  static SmallString<128> FileAB;
+  static SmallString<128> FileAC;
+  static SmallString<128> FileFoo;
+  static SmallString<128> FileBar;
+  static SmallString<128> FileBaz;
+
+  static void SetUpTestCase() {
+    ASSERT_NO_ERROR(fs::createUniqueDirectory("FsCompletion", BaseDir));
+    const char *DirNames[] = {"foo", "fooa", "foob",       "fooc",
+                              "bar", "baz",  "test_folder"};
+    const char *FileNames[] = {"aa%%%%.tmp",  "ab%%%%.tmp",  "ac%%%%.tmp",
+                               "foo%%%%.tmp", "bar%%%%.tmp", "baz%%%%.tmp"};
+    SmallString<128> *Dirs[] = {&DirFoo, &DirFooA, &DirFooB,      &DirFooC,
+                                &DirBar, &DirBaz,  &DirTestFolder};
+    for (auto Dir : llvm::zip(DirNames, Dirs)) {
+      auto &Path = *std::get<1>(Dir);
+      Path = BaseDir;
+      path::append(Path, std::get<0>(Dir));
+      ASSERT_NO_ERROR(fs::create_directory(Path));
+    }
+
+    SmallString<128> *Files[] = {&FileAA,  &FileAB,  &FileAC,
+                                 &FileFoo, &FileBar, &FileBaz};
+    for (auto File : llvm::zip(FileNames, Files)) {
+      auto &Path = *std::get<1>(File);
+      Path = BaseDir;
+      path::append(Path, std::get<0>(File));
+      int FD;
+      ASSERT_NO_ERROR(fs::createUniqueFile(Path, FD, Path));
+      ::close(FD);
+    }
+  }
+
+  static void TearDownTestCase() {
+    ASSERT_NO_ERROR(fs::remove_directories(BaseDir));
+  }
+
+  static bool HasEquivalentFile(const Twine &Path, const StringList &Paths) {
+    for (int I = 0; I < Paths.GetSize(); ++I) {
+      if (fs::equivalent(Path, Paths[I]))
+        return true;
+    }
+    return false;
+  }
+
+  static bool ContainsExactString(const Twine &Str, const StringList &Paths) {
+    SmallString<16> Storage;
+    StringRef Rendered = Str.toStringRef(Storage);
+    for (int I = 0; I < Paths.GetSize(); ++I) {
+      if (Paths[I] == Rendered)
+        return true;
+    }
+    return false;
+  }
+};
+
+SmallString<128> CompletionTest::BaseDir;
+
+SmallString<128> CompletionTest::DirFoo;
+SmallString<128> CompletionTest::DirFooA;
+SmallString<128> CompletionTest::DirFooB;
+SmallString<128> CompletionTest::DirFooC;
+SmallString<128> CompletionTest::DirBar;
+SmallString<128> CompletionTest::DirBaz;
+SmallString<128> CompletionTest::DirTestFolder;
+
+SmallString<128> CompletionTest::FileAA;
+SmallString<128> CompletionTest::FileAB;
+SmallString<128> CompletionTest::FileAC;
+SmallString<128> CompletionTest::FileFoo;
+SmallString<128> CompletionTest::FileBar;
+SmallString<128> CompletionTest::FileBaz;
+}
+
+TEST_F(CompletionTest, DirCompletionAbsolute) {
+  // All calls to DiskDirectories() return only directories, even when
+  // there are files which also match.  The tests below all check this
+  // by asserting an exact result count, and verifying against known
+  // folders.
+
+  StringList Results;
+  // When a directory is specified that doesn't end in a slash, it searches
+  // for that directory, not items under it.
+  int Count = CommandCompletions::DiskDirectories2(BaseDir, Results);
+  ASSERT_EQ(1, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(BaseDir, Results));
+
+  // When the same directory ends with a slash, it finds all children.
+  Count = CommandCompletions::DiskDirectories2(Twine(BaseDir) + "/", Results);
+  ASSERT_EQ(7, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(DirFoo, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooA, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooB, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooC, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirBar, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirBaz, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirTestFolder, Results));
+
+  // When a partial name matches, it returns all matches.  If it matches both
+  // a full name AND some partial names, it returns all of them.
+  Count =
+      CommandCompletions::DiskDirectories2(Twine(BaseDir) + "/foo", Results);
+  ASSERT_EQ(4, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(DirFoo, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooA, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooB, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooC, Results));
+
+  // If it matches only partial names, it still works as expected.
+  Count = CommandCompletions::DiskDirectories2(Twine(BaseDir) + "/b", Results);
+  ASSERT_EQ(2, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(DirBar, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirBaz, Results));
+}
+
+TEST_F(CompletionTest, FileCompletionAbsolute) {
+  // All calls to DiskFiles() return both files and directories  The tests below
+  // all check this by asserting an exact result count, and verifying against
+  // known folders.
+
+  StringList Results;
+  // When an item is specified that doesn't end in a slash but exactly matches
+  // one item, it returns that item.
+  int Count = CommandCompletions::DiskFiles2(Twine(BaseDir) + "/fooa", Results);
+  ASSERT_EQ(1, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(DirFooA, Results));
+
+  // The previous check verified a directory match.  But it should work for
+  // files too.
+  Count = CommandCompletions::DiskFiles2(Twine(BaseDir) + "/aa", Results);
+  ASSERT_EQ(1, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(FileAA, Results));
+
+  // When it ends with a slash, it should find all files and directories.
+  Count = CommandCompletions::DiskFiles2(Twine(BaseDir) + "/", Results);
+  ASSERT_EQ(13, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(DirFoo, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooA, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooB, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooC, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirBar, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirBaz, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirTestFolder, Results));
+
+  EXPECT_TRUE(HasEquivalentFile(FileAA, Results));
+  EXPECT_TRUE(HasEquivalentFile(FileAB, Results));
+  EXPECT_TRUE(HasEquivalentFile(FileAC, Results));
+  EXPECT_TRUE(HasEquivalentFile(FileFoo, Results));
+  EXPECT_TRUE(HasEquivalentFile(FileBar, Results));
+  EXPECT_TRUE(HasEquivalentFile(FileBaz, Results));
+
+  // When a partial name matches, it returns all file & directory matches.
+  Count = CommandCompletions::DiskFiles2(Twine(BaseDir) + "/foo", Results);
+  ASSERT_EQ(5, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(HasEquivalentFile(DirFoo, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooA, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooB, Results));
+  EXPECT_TRUE(HasEquivalentFile(DirFooC, Results));
+  EXPECT_TRUE(HasEquivalentFile(FileFoo, Results));
+}
+
+TEST_F(CompletionTest, DirCompletionUsername) {
+  MockTildeExpressionResolver Resolver("James", BaseDir);
+  Resolver.AddKnownUser("Kirk", DirFooB);
+  Resolver.AddKnownUser("Lars", DirFooC);
+  Resolver.AddKnownUser("Jason", DirFoo);
+  Resolver.AddKnownUser("Larry", DirFooA);
+
+  // Just resolving current user's home directory by itself should return the
+  // directory.
+  StringList Results;
+  int Count = CommandCompletions::DiskDirectories2("~", Results, &Resolver);
+  ASSERT_EQ(1, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(ContainsExactString(Twine("~") + path::get_separator(), Results));
+
+  // With a slash appended, it should return all items in the directory.
+  Count = CommandCompletions::DiskDirectories2("~/", Results, &Resolver);
+  ASSERT_EQ(7, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~/foo") + path::get_separator(), Results));
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~/fooa") + path::get_separator(), Results));
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~/foob") + path::get_separator(), Results));
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~/fooc") + path::get_separator(), Results));
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~/bar") + path::get_separator(), Results));
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~/baz") + path::get_separator(), Results));
+  EXPECT_TRUE(ContainsExactString(
+      Twine("~/test_folder") + path::get_separator(), Results));
+
+  // With ~username syntax it should return one match if there is an exact
+  // match.
+  // It shouldn't translate to the actual directory, it should keep the form the
+  // user typed.
+  Count = CommandCompletions::DiskDirectories2("~Lars", Results, &Resolver);
+  ASSERT_EQ(1, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~Lars") + path::get_separator(), Results));
+
+  // But with a username that is not found, no results are returned.
+  Count = CommandCompletions::DiskDirectories2("~Dave", Results, &Resolver);
+  ASSERT_EQ(0, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+
+  // And if there are multiple matches, it should return all of them.
+  Count = CommandCompletions::DiskDirectories2("~La", Results, &Resolver);
+  ASSERT_EQ(2, Count);
+  ASSERT_EQ(Count, Results.GetSize());
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~Lars") + path::get_separator(), Results));
+  EXPECT_TRUE(
+      ContainsExactString(Twine("~Larry") + path::get_separator(), Results));
+}