[GSoC] Shell autocompletion for clang

[yamaguchi]

Summary

This is a first patch for GSoC project, bash-completion for clang.
To use this on bash, please run source clang/utils/bash-autocomplete.sh.
bash-autocomplete.sh is code for bash-completion.

Simple flag completion and path completion is available in this patch.

[yamaguchi]

Add reviewers.

[jroelofs]

clang/include/clang/Driver/Options.td

472

Please add a HelpText so it shows up in the docs.

clang/test/Driver/autocomplete.c

2

count and FileCheck are preferred over adding new uses of grep.

clang/utils/bash-autocomplete.sh

2

Please add a commment explaining that this file should be sourced in the user's .bashrc. You also need cmake goop to install this under share/clang.

[yamaguchi]

clang/include/clang/Driver/Options.td

472

I think this flag should not be show up with --help, because this --autocomplete flag is used only for clang internal.

[ruiu]

clang/utils/bash-autocomplete.sh

7–11

s/cur"/cur" / for consistency. Or s/$( /$(/ if you prefer, but $( foo) looks odd.

13

Remove blank line.

14–15

What is the meaning of && here? It seems you can remove it.

llvm/lib/Option/OptTable.cpp

195

Remove unnecessary ().

[yamaguchi]

clang/utils/bash-autocomplete.sh

14–15

syntax error will occur without &&, because for example,
This is correct:
_fun() { echo "hoge"; } && echo "huga"
But this is wrong:
_fun() { echo "hoge"; } echo "huga"

[yamaguchi]

Update diff.

[yamaguchi]

Update diff.
Remove &&, because breaking after shell function was enough.

[yamaguchi]

Update diff.
Add comment in bash-autocomplete.sh.

[jroelofs]

clang/include/clang/Driver/Options.td

472

ok, good point.

[ruiu]

I'm fine with these nits. (Am I supposed to sign off?)

clang/test/Driver/autocomplete.c

4

You want to change this to -std={{.*}}-stdlib= to make robust for changes that add new options which are between the top in the asciibetical order.

6

Maybe just foo suffices. % doesn't seem make sense to me.

clang/utils/bash-autocomplete.sh

14

Fix indentation.

[teemperor]

@ruiu We wanted to push this patch with the follow-up one about the argument completion, so no need to do this now :).

[ruiu]

Why do you want to not submit this alone but wait for another patch? In-tree incremental development is a normal thing, and the feature this patch provides is already useful without any further changes, so I don't see a reason to wait.

[teemperor]

Because the small follow up patch is relevant for the review of this one. If people disagree with the plan there to extend Options.td or OptTable for this feature (and due to the limitations of Options.td in regards to dynamic arguments, this is already controversial), then we have to completely revisit this code which assumes we can do everything in OptTable. I fully agree with incremental development in tree, but for those first two patches I don't see "consensus on the end goal to define the first increment" yet (to quote the LLVM dev policy on this). Sorry, I should have mentioned this fact in the previous mail, but typing on the smartphone is cumbersome.

[ruiu]

If you are adding code to handle dynamic arguments, then yes, we probably need to define what is the first incremental step towards the goal after discussing how to handle dynamic arguments. But this patch seems somewhat orthogonal to that, no? This doesn't do anything about dynamic arguments and makes sense without a follow-up patch. It adds only 12 lines of code to OptTable.cpp, so "completely revising" it shouldn't be a big deal at all. So I believe this is something we usually submit incrementally.

[v.g.vassilev]

@teemperor, @ruiu, thanks for the quick review and comments! I second the incremental development argument. IMHO several dependent phabricator patches can be considered incremental development.

@ruiu, please feel free give green light if this patch looks good to you.

I'd feel much more comfortable if this patch lands closer to actual google summer of code coding period which is in the end of the month. Let's aim, for landing it next week.

[yamaguchi]

Update diff.

[ruiu]

LGTM, but you want to wait until next week for Vassil.

I think you made a good start for your GSoC project, Yuka. Great work!

clang/CMakeLists.txt

if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY)
 
   install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/include/clang
     DESTINATION include
     FILES_MATCHING
     PATTERN "CMakeFiles" EXCLUDE
     PATTERN "*.inc"
     PATTERN "*.h"
     )
+
+  install(PROGRAMS utils/bash-autocomplete.sh
+    DESTINATION share/clang
+    )
 endif()
 
 add_definitions( -D_GNU_SOURCE )
 
 option(CLANG_BUILD_TOOLS
   "Build the Clang tools. If OFF, just generate build targets." ON)
 
 option(CLANG_ENABLE_ARCMT "Build ARCMT." ON)

clang/include/clang/Driver/Options.td

 def Z_Joined : Joined<["-"], "Z">;
 def all__load : Flag<["-"], "all_load">;
 def allowable__client : Separate<["-"], "allowable_client">;
 def ansi : Flag<["-", "--"], "ansi">;
 def arch__errors__fatal : Flag<["-"], "arch_errors_fatal">;
 def arch : Separate<["-"], "arch">, Flags<[DriverOption]>;
 def arch__only : Separate<["-"], "arch_only">;
 def a : Joined<["-"], "a">;
+def autocomplete : Joined<["--"], "autocomplete=">;
 def bind__at__load : Flag<["-"], "bind_at_load">;
 def bundle__loader : Separate<["-"], "bundle_loader">;
 def bundle : Flag<["-"], "bundle">;
 def b : JoinedOrSeparate<["-"], "b">, Flags<[Unsupported]>;
 def cl_opt_disable : Flag<["-"], "cl-opt-disable">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. This option disables all optimizations. By default optimizations are enabled.">;
 def cl_strict_aliasing : Flag<["-"], "cl-strict-aliasing">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. This option is added for compatibility with OpenCL 1.0.">;

clang/lib/Driver/Driver.cpp

if (Arg *A = C.getArgs().getLastArg(options::OPT_print_file_name_EQ)) {
     return false;
   }
 
   if (Arg *A = C.getArgs().getLastArg(options::OPT_print_prog_name_EQ)) {
     llvm::outs() << GetProgramPath(A->getValue(), TC) << "\n";
     return false;
   }
 
+  if (Arg *A = C.getArgs().getLastArg(options::OPT_autocomplete)) {
+    // Print out all options that start with a given argument. This is used for
+    // shell autocompletion.
+    llvm::outs() << llvm::join(Opts->findByPrefix(A->getValue()), " ") << '\n';
+    return false;
+  }
+
   if (C.getArgs().hasArg(options::OPT_print_libgcc_file_name)) {
     ToolChain::RuntimeLibType RLT = TC.GetRuntimeLibType(C.getArgs());
     switch (RLT) {
     case ToolChain::RLT_CompilerRT:
       llvm::outs() << TC.getCompilerRT(C.getArgs(), "builtins") << "\n";
       break;
     case ToolChain::RLT_Libgcc:
       llvm::outs() << GetFilePath("libgcc.a", TC) << "\n";

clang/test/Driver/autocomplete.c

+// RUN: %clang --autocomplete=-fsyn | FileCheck %s -check-prefix=FSYN
+// FSYN: -fsyntax-only
+// RUN: %clang --autocomplete=-s | FileCheck %s -check-prefix=STD
+// STD: -std= -stdlib=
+// RUN: %clang --autocomplete=foo | not FileCheck %s -check-prefix=NONE
+// NONE: {{%[a-z]}}

clang/utils/bash-autocomplete.sh

+# Please add "source /path/to/bash-autocomplete.sh" to your .bashrc to use this.
+_clang()
+{
+    local cur prev words cword flags
+    _init_completion -n : || return
+
+    flags=$( clang --autocomplete="$cur" )
+    if [[ "$flags" == "" || "$cur" == "" ]]; then
+        _filedir
+    else
+        COMPREPLY=( $( compgen -W "$flags" -- "$cur" ) )
+    fi
+} 
+    complete -F _clang clang

llvm/include/llvm/Option/OptTable.h

public:
   }
 
   /// \brief Get the meta-variable name to use when describing
   /// this options values in the help text.
   const char *getOptionMetaVar(OptSpecifier id) const {
     return getInfo(id).MetaVar;
   }
 
+  /// Find flags from OptTable which starts with Cur.
+  ///
+  /// \param [in] Cur - String prefix that all returned flags need
+  //  to start with.
+  ///
+  /// \return The vector of flags which start with Cur.
+  std::vector<std::string> findByPrefix(StringRef Cur) const;
+
   /// \brief Parse a single argument; returning the new argument and
   /// updating Index.
   ///
   /// \param [in,out] Index - The current parsing position in the argument
   /// string list; on return this will be the index of the next argument
   /// string to parse.
   /// \param [in] FlagsToInclude - Only parse options with any of these flags.
   /// Zero is the default which includes all flags.

llvm/lib/Option/OptTable.cpp

if (Str.startswith(Prefix)) {
           : Rest.startswith(I->Name);
       if (Matched)
         return Prefix.size() + StringRef(I->Name).size();
     }
   }
   return 0;
 }
 
+std::vector<std::string> OptTable::findByPrefix(StringRef Cur) const {
+  std::vector<std::string> Ret;
+  for (const Info &In : OptionInfos.slice(FirstSearchableIndex)) {
+    if (!In.Prefixes)
+      continue;
+    for (int I = 0; In.Prefixes[I]; I++) {
+      std::string S = std::string(In.Prefixes[I]) + std::string(In.Name);
+      if (StringRef(S).startswith(Cur))
+        Ret.push_back(S);
+    }
+  }
+  return Ret;
+}
+
 Arg *OptTable::ParseOneArg(const ArgList &Args, unsigned &Index,
                            unsigned FlagsToInclude,
                            unsigned FlagsToExclude) const {
   unsigned Prev = Index;
   const char *Str = Args.getArgString(Index);
 
   // Anything that doesn't start with PrefixesUnion is an input, as is '-'
   // itself.