[GSoC] Flag value completion for clang

[yamaguchi]

Summary

This is patch for GSoC project, bash-completion for clang.

To use this on bash, please run source clang/utils/bash-autocomplete.sh.
bash-autocomplete.sh is code for bash-completion.

In this patch, Options.td was mainly changed in order to add value class in Options.inc. It adds initial support for auto-completing flag values in bash. The general idea is to make Options.td aware of all possible values for all flags, and then generate a lookup data structure in the clang driver.

From the bash side we call clang with --autocomplete like this clang --autocomplete=-stdlib=,l and then the clang driver returns all possible values which starts with "l" in -stdlib's Values class.

[yamaguchi]

Update diff using arc.

[teemperor]

Good progress and good catch with also checking lld :)

clang/lib/Driver/Driver.cpp

1233

Split, S and V aren't that meaningful names, which makes the code is a bit hard to follow. Maybe replace it with something like ListOfPassedFlags, PassedFlags and SuggestedCompletions. Also please add some comment why you're suddenly splitting that string by ",". E.g. give an example of what a possible flag value is like -Xclang,-std= and that this is the format you expect to receive by the shell.

lld/COFF/DriverUtils.cpp

659–660

Minor thing: Please make sure the trailing '\' are still aligned after your change. Here and on line 665.

llvm/include/llvm/Option/OptParser.td

95

'Value' -> 'Values' as it contains multiple values.

Is there a reason why were not using list<string> as a type (like with the prefixes)? It seems the only downside of using list<string> is that the generated Options.inc is handling this in a bit funky way, but I think we can live with this.

llvm/include/llvm/Option/OptTable.h

117

Please add documentation :) Also findByValue is not saying that much. Maybe something like suggestValueCompletions.

llvm/lib/Option/OptTable.cpp

194

Ret -> Result.

198

Hmm, will this work with "--" flags? Also, same as above with naming S and C.

206

Please add brackets around the for body. the if is fine, but the for is not one line (even though it works, so it looks a bit out of place.

211

The whole function deserves some comments on why you do the things the way you do. Don't hesitate to give examples and describe the ideas behind all the checks/splitting/merging strings.

Especially the first three if checks in this function are not obvious why they are here.

[yamaguchi]

Update diff.

[yamaguchi]

llvm/lib/Option/OptTable.cpp

198

@teemperor
Do you think we should add some sample value completions to flags start with "--" (such as --target) and flags which don't contain "=" (such as -mllvm <value>) ?
This part of code is rather ad hoc right now for -stdlib.

[v.g.vassilev]

clang/include/clang/Driver/Options.h

42–44

Value seems too generic, shall we add ARGVALUE instead?

clang/include/clang/Driver/Options.td

2204

Values seems too generic, can we use ArgValues instead?

clang/utils/bash-autocomplete.sh

6–7

Do the changes in bash-autocomplete.sh depend on argument value completion? Shouldn't that go in a separate review?

llvm/include/llvm/Option/OptTable.h

117

We need three backslashes (///) to enable doxygen.

[yamaguchi]

clang/utils/bash-autocomplete.sh

6–7

Bash parses foo=bar to [foo, =, bar].
So it is somehow necessary to handle multiple argument in order to do value completion.
Eg. when "-std=foo" was given, bash parses it to ["-std", "=", "foo"], but we want all of the array, not just one element of it.

[yamaguchi]

Update patch. Changed Values to ArgValues.

[ruiu]

Overall looking good. Good work! A few minor comments.

clang/include/clang/Driver/Options.td

2204

I'd keep it as Values, as everything is essentially related to command line arguments, and Args seems redundant. For example, we didn't name ArgFlags but just Flags, and HelpText instead of ArgHelpText, etc.

clang/lib/Driver/Driver.cpp

1232

nit: insert a blank line before a meaningful code block so that code doesn't look too dense.

1233

make a list of flag values.

1234

You are using PassedFlags only once. Do you want to replace it with StringRef(A->getValue()).split(...)?

clang/test/Driver/autocomplete.c

7

Why do you want to pass the arguments in the reverse order?

[v.g.vassilev]

clang/include/clang/Driver/Options.td

2204

My reasoning for asking this is that I wanted to hint about the relationship between the values (as this is a very broad term) and arguments. I'd read ArgValues as possible values for an argument without having to dig into context.

That said, I don't think switching back to Values is a significant issue, if @ruiu feels strongly about it , please follow his suggestion and land the patch.

clang/lib/Driver/Driver.cpp

1234

PassedFlags looks more readable to casual readers like me ;)

[yamaguchi]

Update patch and add support for clang -stdlib=[tab] case.
In this case we expect to see all possible values.
(Eg. libc++ libstdc++ platform for clang -stdlib=[tab]).

[yamaguchi]

clang/test/Driver/autocomplete.c

7

In the reverse order, current flag is always exists at first, so I thought it is more simple. (Eg. clang -fsyntax-only -o foo -std=c will be passed as c,=,-std,foo,-i,-fsyntax-only)

[yamaguchi]

llvm/lib/Option/OptTable.cpp

198

I had add support for this in this update as well, thanks.

[ruiu]

As to the order of Command variable contents, I think I'm not convinced. You can access the last element as Command[Command.size() - 1] (or maybe Command.back()), no? It is slightly awkward than Command[0], but that's not horribly hard to handle, and passing arguments in the reverse order seems more counter-intuitive to me.

clang/include/clang/Driver/Options.td

2204

I think I feel fairly strongly prefer "Value" over "ArgValue". If this is a one-time name, I agree that we should use "ArgValue", but this is not the case. We are going to add bunch of "Values" to options that take arguments, and I expect that the verbose name will start feeling too verbose. So please switch back to "Values".

clang/utils/bash-autocomplete.sh

7–28

Don't you want to make it local?

8

If you don't write do on the same line as for, you don't need to write ; at end of a line. I.e. write it in either

for ...
do
   command...
done

or

for ...; do
  command...
done

This is because you need a newline between for and do, and ; works as a newline.

10

nit: you don't need double double-quotes. Instead, `"$arg${COMP_WORDS[$i]}," should just work.

15–16

Alternatively, COMPRELY=( $( compgen -W "$flags" -- "") ) should work.

17

Use elif so that you don't have to write two fis at end. In a bash script, if ... else if is usually written as

if ...; then
elif ...; then
elif ...; then
fi

If you use else if instead of elif, and if you indent the code properly, it'll look like

if ...; then
else
  if ...; then
  fi
fi

This is why you needed two fis.

[ruiu]

clang/utils/bash-autocomplete.sh

10

On second thought, I think what you actually want to do is to concatenate the elements of the array with ,, without adding , at end of the string. For example, if COMP_WORDS contains "a", "b" and "c", you want to create "a,b,c" instead of "a,b,c," or ",a,b,c".

There are surprisingly large number of different ways of doing this, but it seems this is one of the easiest ways.

arg="$(IFS=,; echo "${COMP_WORDS[*]}")"

IFS is a special shell variable containing a word splitting words. By setting the variable, COMP_WORDS[*] is expanded to (for example) "a,b,c" instead of the default "a b c".

[yamaguchi]

Update patch. Rui's comment about the bash part is very reasonable.

[yamaguchi]

Cleaned suggestValueCompletions and bash-autocomplete.sh, and added tests.

We have passed flags and values from bash to clang like clang --autocomplete=l,=,-stdlib, but I've changed this to pass clang --autocomplte=-stdlib=,l instead. This is more simple.

Added support and test for -meabi flag.
This flag is corner case, because it doesn't take = suffix like -stdlib=.

When clang -stdlib=[tab] is pushed, we would give possible values (libstd++,..), and when clang -stdlib= [tab] is pushed, we just give files under current directory.
And when clang -meabi [tab] is pushed, we give possible values (gnu,..).

[yamaguchi]

Made trivial change.

[yamaguchi]

Made trivial bug fix.

[yamaguchi]

Made trivial change.

[yamaguchi]

Made a trivial change in optionMatches.

[ruiu]

Yuka,

This is beautiful. Overall looking pretty good. Some minor stylistic comments.

clang/include/clang/Driver/Options.h

43

Since you are now using Values as a variable name, you want to rename ARGVALUE VALUES.

clang/include/clang/Driver/Options.td

1709–1710

You wrote Values before HelpText here but in the opposite order for stdlib_EQ. This is not a strict rule or anything, but it is better to be consistent in the option order.

clang/lib/Driver/Driver.cpp

1233

Move this inside the first if so that it is clear that this comment does not describe the entire if and else but only the if part.

You could improve the comment: If the flag is in the form of "--autocomplete=-foo", we were requested to print out all option names that start with "-foo". For example, "--autocomplete=-fsyn" is expanded to "-fsyntax-only".

1238

Likewise: If the flag is in the form of "--autocomplete=-foo,bar", we were requested to print out all option values for "-foo" that start with "bar". For example, "--autocomplete=-stdlib=,l" is expanded to "libc++" and "libstdc++".

clang/utils/bash-autocomplete.sh

7–28

It is better to describe what you are doing here and why. How about this:

bash autocompletion always separates '=' as a token even if there's no space before/after it. On the other hand, '=' is just a regular character for clang options that contain '='. For example, "-stdlib=" is defined as is, instead of "-stdlib" and "=". So, we need to partially undo bash tokenization here for impedance matching.

llvm/lib/Option/OptTable.cpp

189

returns -> Returns

[yamaguchi]

Update code comment according to Rui's comment.

[yamaguchi]

Add support and test for -cl-std=,-fno-sanitize-coverage=,-ffp-contract=,-flto=,-fveclib=,-fshow-overloads=,-fvisibility=,-mfloat-abi=,-mthread-model.

[v.g.vassilev]

LGTM. My comments can be addressed in a separate patch.

clang/include/clang/Driver/Options.td

496

I think we should have a way to express that the values are case insensitive, instead of enumerating all possible combinations.

1341

We should be able to express which value is the default value if nothing is specified. A non-very readable solution would be to put the default always as the first value.

[teemperor]

LGTM.

[ruiu]

LGTM

[ruiu]

clang/include/clang/Driver/Options.td

496

Looks like this is not really a case-insensitive option, as it does not allow "cL2.0" for example. The enumerated values are exact list of values that the option accepts.

[yamaguchi]

Landed in rL305805.

clang/include/clang/Driver/Options.h

enum ClangFlags {
   CC1Option = (1 << 10),
   CC1AsOption = (1 << 11),
   NoDriverOption = (1 << 12),
   Ignored = (1 << 13)
 };
 
 enum ID {
     OPT_INVALID = 0, // This is not an option ID.
-#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
-               HELPTEXT, METAVAR) OPT_##ID,
+#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
+  OPT_##ID,
 #include "clang/Driver/Options.inc"
     LastOption
 #undef OPTION
   };
 }
 
 std::unique_ptr<llvm::opt::OptTable> createDriverOptTable();
 }
 }
 
 #endif

clang/include/clang/Driver/Options.td

def cl_unsafe_math_optimizations : Flag<["-"], "cl-unsafe-math-optimizations">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. Allow unsafe floating-point optimizations.  Also implies -cl-no-signed-zeros and -cl-mad-enable.">;
 def cl_fast_relaxed_math : Flag<["-"], "cl-fast-relaxed-math">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. Sets -cl-finite-math-only and -cl-unsafe-math-optimizations, and defines __FAST_RELAXED_MATH__.">;
 def cl_mad_enable : Flag<["-"], "cl-mad-enable">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. Allow use of less precise MAD computations in the generated binary.">;
 def cl_no_signed_zeros : Flag<["-"], "cl-no-signed-zeros">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. Allow use of less precise no signed zeros computations in the generated binary.">;
 def cl_std_EQ : Joined<["-"], "cl-std=">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL language standard to compile for.">;
 def cl_denorms_are_zero : Flag<["-"], "cl-denorms-are-zero">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. Allow denormals to be flushed to zero.">;
 def cl_fp32_correctly_rounded_divide_sqrt : Flag<["-"], "cl-fp32-correctly-rounded-divide-sqrt">, Group<opencl_Group>, Flags<[CC1Option]>,
   HelpText<"OpenCL only. Specify that single precision floating-point divide and sqrt used in the program source are correctly rounded.">;
 def client__name : JoinedOrSeparate<["-"], "client_name">;
 def combine : Flag<["-", "--"], "combine">, Flags<[DriverOption, Unsupported]>;
 def compatibility__version : JoinedOrSeparate<["-"], "compatibility_version">;
 def coverage : Flag<["-", "--"], "coverage">;

 def fshort_enums : Flag<["-"], "fshort-enums">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Allocate to an enum type only as many bytes as it needs for the declared range of possible values">;
 def fshort_wchar : Flag<["-"], "fshort-wchar">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Force wchar_t to be a short unsigned int">;
 def fno_short_wchar : Flag<["-"], "fno-short-wchar">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Force wchar_t to be an unsigned int">;
 def fshow_overloads_EQ : Joined<["-"], "fshow-overloads=">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Which overload candidates to show when overload resolution fails: "
            "best|all; defaults to all">;
 def fshow_column : Flag<["-"], "fshow-column">, Group<f_Group>, Flags<[CC1Option]>;
 def fshow_source_location : Flag<["-"], "fshow-source-location">, Group<f_Group>;
 def fspell_checking : Flag<["-"], "fspell-checking">, Group<f_Group>;
 def fspell_checking_limit_EQ : Joined<["-"], "fspell-checking-limit=">, Group<f_Group>;
 def fsigned_bitfields : Flag<["-"], "fsigned-bitfields">, Group<f_Group>;
 def fsigned_char : Flag<["-"], "fsigned-char">, Group<f_Group>;
 def fno_signed_char : Flag<["-"], "fno-signed-char">, Group<f_Group>,
     Flags<[CC1Option]>, HelpText<"Char is unsigned">;

 def mstackrealign : Flag<["-"], "mstackrealign">, Group<m_Group>, Flags<[CC1Option]>,
   HelpText<"Force realign the stack at entry to every function">;
 def mstack_alignment : Joined<["-"], "mstack-alignment=">, Group<m_Group>, Flags<[CC1Option]>,
   HelpText<"Set the stack alignment">;
 def mstack_probe_size : Joined<["-"], "mstack-probe-size=">, Group<m_Group>, Flags<[CC1Option]>,
   HelpText<"Set the stack probe size">;
 def mthread_model : Separate<["-"], "mthread-model">, Group<m_Group>, Flags<[CC1Option]>,
   HelpText<"The thread model to use, e.g. posix, single (posix by default)">;
-def meabi : Separate<["-"], "meabi">, Group<m_Group>, Flags<[CC1Option]>,
+def meabi : Separate<["-"], "meabi">, Group<m_Group>, Flags<[CC1Option]>, Values<"default,4,5,gnu">,
   HelpText<"Set EABI type, e.g. 4, 5 or gnu (default depends on triple)">;
 
 def mmmx : Flag<["-"], "mmmx">, Group<m_x86_Features_Group>;
 def mno_3dnowa : Flag<["-"], "mno-3dnowa">, Group<m_x86_Features_Group>;
 def mno_3dnow : Flag<["-"], "mno-3dnow">, Group<m_x86_Features_Group>;
 def mno_constant_cfstrings : Flag<["-"], "mno-constant-cfstrings">, Group<m_Group>;
 def mno_global_merge : Flag<["-"], "mno-global-merge">, Group<m_Group>, Flags<[CC1Option]>,
   HelpText<"Disable merging of globals">;
 def mno_mmx : Flag<["-"], "mno-mmx">, Group<m_x86_Features_Group>;

 def specs : Separate<["-", "--"], "specs">, Flags<[Unsupported]>;
 def static_libgcc : Flag<["-"], "static-libgcc">;
 def static_libstdcxx : Flag<["-"], "static-libstdc++">;
 def static : Flag<["-", "--"], "static">, Flags<[NoArgumentUnused]>;
 def std_default_EQ : Joined<["-"], "std-default=">;
 def std_EQ : Joined<["-", "--"], "std=">, Flags<[CC1Option]>,
   Group<CompileOnly_Group>, HelpText<"Language standard to compile for">;
 def stdlib_EQ : Joined<["-", "--"], "stdlib=">, Flags<[CC1Option]>,
-  HelpText<"C++ standard library to use">;
+  HelpText<"C++ standard library to use">, Values<"libc++,libstdc++,platform">;
 def sub__library : JoinedOrSeparate<["-"], "sub_library">;
 def sub__umbrella : JoinedOrSeparate<["-"], "sub_umbrella">;
 def system_header_prefix : Joined<["--"], "system-header-prefix=">,
   Group<clang_i_Group>, Flags<[CC1Option]>, MetaVarName<"<prefix>">,
   HelpText<"Treat all #include paths starting with <prefix> as including a "
            "system header.">;
 def : Separate<["--"], "system-header-prefix">, Alias<system_header_prefix>;
 def no_system_header_prefix : Joined<["--"], "no-system-header-prefix=">,

clang/lib/Driver/Driver.cpp

bool Driver::HandleImmediateArgs(const Compilation &C) {
   if (Arg *A = C.getArgs().getLastArg(options::OPT_print_prog_name_EQ)) {
     llvm::outs() << GetProgramPath(A->getValue(), TC) << "\n";
     return false;
   }
 
   if (Arg *A = C.getArgs().getLastArg(options::OPT_autocomplete)) {
     // Print out all options that start with a given argument. This is used for
     // shell autocompletion.
-    llvm::outs() << llvm::join(Opts->findByPrefix(A->getValue()), " ") << '\n';
+    StringRef PassedFlags = A->getValue();
+    std::vector<std::string> SuggestedCompletions;
+
+    // When the flag was passed like "--autocomplete=-fsyn"
+    // we want to autocomplete "-fsyn" as "-fsyntax-only"
+    if (PassedFlags.find(',') == StringRef::npos) {
+      SuggestedCompletions = Opts->findByPrefix(PassedFlags);
+    } else {
+      // When flags were passed like "--autocomplete=-stdlib=,l"
+      // we want to autocomplete appropriate value which starts with "l"
+      // for "-stdlib=".
+      StringRef Option, Arg;
+      std::tie(Option, Arg) = PassedFlags.split(',');
+      SuggestedCompletions = Opts->suggestValueCompletions(Option, Arg);
+    }
+
+    llvm::outs() << llvm::join(SuggestedCompletions, " ") << '\n';
     return false;
   }
 
   if (C.getArgs().hasArg(options::OPT_print_libgcc_file_name)) {
     ToolChain::RuntimeLibType RLT = TC.GetRuntimeLibType(C.getArgs());
     switch (RLT) {
     case ToolChain::RLT_CompilerRT:
       llvm::outs() << TC.getCompilerRT(C.getArgs(), "builtins") << "\n";

clang/lib/Driver/DriverOptions.cpp

 using namespace clang::driver::options;
 using namespace llvm::opt;
 
 #define PREFIX(NAME, VALUE) static const char *const NAME[] = VALUE;
 #include "clang/Driver/Options.inc"
 #undef PREFIX
 
 static const OptTable::Info InfoTable[] = {
-#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
-               HELPTEXT, METAVAR)   \
-  { PREFIX, NAME, HELPTEXT, METAVAR, OPT_##ID, Option::KIND##Class, PARAM, \
-    FLAGS, OPT_##GROUP, OPT_##ALIAS, ALIASARGS },
+#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
+  {PREFIX, NAME,  HELPTEXT,    METAVAR,     OPT_##ID,  Option::KIND##Class,    \
+   PARAM,  FLAGS, OPT_##GROUP, OPT_##ALIAS, ALIASARGS, ARGVALUE},
 #include "clang/Driver/Options.inc"
 #undef OPTION
 };
 
 namespace {
 
 class DriverOptTable : public OptTable {
 public:

clang/test/Driver/autocomplete.c

 // RUN: %clang --autocomplete=-fsyn | FileCheck %s -check-prefix=FSYN
 // FSYN: -fsyntax-only
 // RUN: %clang --autocomplete=-s | FileCheck %s -check-prefix=STD
 // STD: -std={{.*}}-stdlib=
 // RUN: %clang --autocomplete=foo | not FileCheck %s -check-prefix=NONE
 // NONE: foo
+// RUN: %clang --autocomplete=-stdlib=,l | FileCheck %s -check-prefix=STDLIB
+// STDLIB: libc++ libstdc++
+// RUN: %clang --autocomplete=-stdlib=, | FileCheck %s -check-prefix=STDLIBALL
+// STDLIBALL: libc++ libstdc++ platform
+// RUN: %clang --autocomplete=-meabi,d | FileCheck %s -check-prefix=MEABI
+// MEABI: default
+// RUN: %clang --autocomplete=-meabi, | FileCheck %s -check-prefix=MEABIALL
+// MEABIALL: default 4 5 gnu

clang/utils/bash-autocomplete.sh

 # Please add "source /path/to/bash-autocomplete.sh" to your .bashrc to use this.
 _clang()
 {
-  local cur prev words cword flags
+  local cur prev words cword arg
   _init_completion -n : || return
 
-  flags=$( clang --autocomplete="$cur" )
-  if [[ "$flags" == "" || "$cur" == "" ]]; then
+  local w1="${COMP_WORDS[$cword - 1]}"
+  local w2="${COMP_WORDS[$cword - 2]}"
+  if [[ "$cur" == -* ]]; then
+    # -foo<tab>
+    arg="$cur"
+  elif [[ "$w1" == -*  && "$cur" == '=' ]]; then
+    # -foo=<tab>
+    arg="$w1=,"
+  elif [[ "$w1" == -* ]]; then
+    # -foo <tab> or -foo bar<tab>
+    arg="$w1,$cur"
+  elif [[ "$w2" == -* && "$w1" == '=' ]]; then
+    # -foo=bar<tab>
+    arg="$w2=,$cur"
+  else
+    _filedir
+  fi
+
+  flags=$( clang --autocomplete="$arg" )
+  if [[ "$cur" == "=" ]]; then
+    COMPREPLY=( $( compgen -W "$flags" -- "") )
+  elif [[ "$flags" == "" || "$cur" == "" ]]; then
     _filedir
   else
     COMPREPLY=( $( compgen -W "$flags" -- "$cur" ) )
   fi
 }
 complete -F _clang clang

lld/COFF/Driver.h

 std::unique_ptr<MemoryBuffer>
 convertResToCOFF(const std::vector<MemoryBufferRef> &MBs);
 
 void runMSVCLinker(std::string Rsp, ArrayRef<StringRef> Objects);
 
 // Create enum with OPT_xxx values for each option in Options.td
 enum {
   OPT_INVALID = 0,
-#define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11) OPT_##ID,
+#define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11, _12) OPT_##ID,
 #include "Options.inc"
 #undef OPTION
 };
 
 } // namespace coff
 } // namespace lld
 
 #endif

lld/COFF/DriverUtils.cpp

 // Create OptTable
 
 // Create prefix string literals used in Options.td
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "Options.inc"
 #undef PREFIX
 
 // Create table mapping all options defined in Options.td
 static const llvm::opt::OptTable::Info infoTable[] = {
-#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X6, X7, X8, X9, X10)    \
-  {                                                                    \
-    X1, X2, X9, X10, OPT_##ID, llvm::opt::Option::KIND##Class, X8, X7, \
-    OPT_##GROUP, OPT_##ALIAS, X6                                       \
-  },
+#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X6, X7, X8, X9, X10, X11)       \
+  {X1, X2, X9,          X10,         OPT_##ID, llvm::opt::Option::KIND##Class, \
+   X8, X7, OPT_##GROUP, OPT_##ALIAS, X6,       X11},
 #include "Options.inc"
 #undef OPTION
 };
 
 class COFFOptTable : public llvm::opt::OptTable {
 public:
   COFFOptTable() : OptTable(infoTable, true) {}
 };

lld/ELF/Driver.h

 public:
   ELFOptTable();
   llvm::opt::InputArgList parse(ArrayRef<const char *> Argv);
 };
 
 // Create enum with OPT_xxx values for each option in Options.td
 enum {
   OPT_INVALID = 0,
-#define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11) OPT_##ID,
+#define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11, _12) OPT_##ID,
 #include "Options.inc"
 #undef OPTION
 };
 
 void printHelp(const char *Argv0);
 std::string createResponseFile(const llvm::opt::InputArgList &Args);
 
 llvm::Optional<std::string> findFromSearchPaths(StringRef Path);
 llvm::Optional<std::string> searchLibrary(StringRef Path);
 
 } // namespace elf
 } // namespace lld
 
 #endif

lld/ELF/DriverUtils.cpp

 
 // Create prefix string literals used in Options.td
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "Options.inc"
 #undef PREFIX
 
 // Create table mapping all options defined in Options.td
 static const opt::OptTable::Info OptInfo[] = {
-#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X6, X7, X8, X9, X10)            \
+#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X6, X7, X8, X9, X10, X11)       \
   {X1, X2, X9,          X10,         OPT_##ID, opt::Option::KIND##Class,       \
-   X8, X7, OPT_##GROUP, OPT_##ALIAS, X6},
+   X8, X7, OPT_##GROUP, OPT_##ALIAS, X6,       X11},
 #include "Options.inc"
 #undef OPTION
 };
 
 ELFOptTable::ELFOptTable() : OptTable(OptInfo) {}
 
 // Parse -color-diagnostics={auto,always,never} or -no-color-diagnostics.
 static bool getColorDiagnostics(opt::InputArgList &Args) {

lld/lib/Driver/DarwinLdDriver.cpp

 
 using namespace lld;
 
 namespace {
 
 // Create enum with OPT_xxx values for each option in DarwinLdOptions.td
 enum {
   OPT_INVALID = 0,
-#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
-               HELP, META) \
-          OPT_##ID,
+#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
+               HELP, META, ARGVALUE)                                           \
 #include "DarwinLdOptions.inc"
 #undef OPTION
 };
 
 // Create prefix string literals used in DarwinLdOptions.td
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "DarwinLdOptions.inc"
 #undef PREFIX
 
 // Create table mapping all options defined in DarwinLdOptions.td
 static const llvm::opt::OptTable::Info infoTable[] = {
-#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
-               HELPTEXT, METAVAR)   \
-  { PREFIX, NAME, HELPTEXT, METAVAR, OPT_##ID, llvm::opt::Option::KIND##Class, \
-    PARAM, FLAGS, OPT_##GROUP, OPT_##ALIAS, ALIASARGS },
+#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
+  {PREFIX,      NAME,      HELPTEXT,                                           \
+   METAVAR,     OPT_##ID,  llvm::opt::Option::KIND##Class,                     \
+   PARAM,       FLAGS,     OPT_##GROUP,                                        \
+   OPT_##ALIAS, ALIASARGS, ARGVALUE},
 #include "DarwinLdOptions.inc"
 #undef OPTION
 };
 
 // Create OptTable class for parsing actual command line arguments
 class DarwinLdOptTable : public llvm::opt::OptTable {
 public:
   DarwinLdOptTable() : OptTable(infoTable) {}

llvm/include/llvm/Option/OptParser.td

class Option<list<string> prefixes, string name, OptionKind kind> {
   string EnumName = ?; // Uses the def name if undefined.
   list<string> Prefixes = prefixes;
   string Name = name;
   OptionKind Kind = kind;
   // Used by MultiArg option kind.
   int NumArgs = 0;
   string HelpText = ?;
   string MetaVarName = ?;
+  string Values = ?;
   list<OptionFlag> Flags = [];
   OptionGroup Group = ?;
   Option Alias = ?;
   list<string> AliasArgs = [];
 }
 
 // Helpers for defining options.
 

 
 class Alias<Option alias> { Option Alias = alias; }
 class AliasArgs<list<string> aliasargs> { list<string> AliasArgs = aliasargs; }
 class EnumName<string name> { string EnumName = name; }
 class Flags<list<OptionFlag> flags> { list<OptionFlag> Flags = flags; }
 class Group<OptionGroup group> { OptionGroup Group = group; }
 class HelpText<string text> { string HelpText = text; }
 class MetaVarName<string name> { string MetaVarName = name; }
+class Values<string value> { string Values = value; }
 
 // Predefined options.
 
 // FIXME: Have generator validate that these appear in correct position (and
 // aren't duplicated).
 def INPUT : Option<[], "<input>", KIND_INPUT>;
 def UNKNOWN : Option<[], "<unknown>", KIND_UNKNOWN>;

llvm/include/llvm/Option/OptTable.h

struct Info {
     const char *MetaVar;
     unsigned ID;
     unsigned char Kind;
     unsigned char Param;
     unsigned short Flags;
     unsigned short GroupID;
     unsigned short AliasID;
     const char *AliasArgs;
+    const char *Values;
   };
 
 private:
   /// \brief The static option information table.
   ArrayRef<Info> OptionInfos;
   bool IgnoreCase;
 
   unsigned TheInputOptionID;
public:
   }
 
   /// \brief Get the meta-variable name to use when describing
   /// this options values in the help text.
   const char *getOptionMetaVar(OptSpecifier id) const {
     return getInfo(id).MetaVar;
   }
 
+  /// Find possible value for given flags. This is used for shell
+  /// autocompletion.
+  ///
+  /// \param [in] Option - Key flag like "-stdlib=" when "-stdlib=l"
+  /// was passed to clang.
+  ///
+  /// \param [in] Arg - Value which we want to autocomplete like "l"
+  /// when "-stdlib=l" was passed to clang.
+  ///
+  /// \return The vector of possible values.
+  std::vector<std::string> suggestValueCompletions(StringRef Option,
+                                                   StringRef Arg) const;
+
   /// Find flags from OptTable which starts with Cur.
   ///
   /// \param [in] Cur - String prefix that all returned flags need
   //  to start with.
   ///
   /// \return The vector of flags which start with Cur.
   std::vector<std::string> findByPrefix(StringRef Cur) const;
 

llvm/include/llvm/Option/Option.h

 class Option {
 public:
   enum OptionClass {
     GroupClass = 0,
     InputClass,
     UnknownClass,
     FlagClass,
     JoinedClass,
+    ValuesClass,
     SeparateClass,
     RemainingArgsClass,
     RemainingArgsJoinedClass,
     CommaJoinedClass,
     MultiArgClass,
     JoinedOrSeparateClass,
     JoinedAndSeparateClass
   };
RenderStyleKind getRenderStyle() const {
     case UnknownClass:
       return RenderValuesStyle;
     case JoinedClass:
     case JoinedAndSeparateClass:
       return RenderJoinedStyle;
     case CommaJoinedClass:
       return RenderCommaJoinedStyle;
     case FlagClass:
+    case ValuesClass:
     case SeparateClass:
     case MultiArgClass:
     case JoinedOrSeparateClass:
     case RemainingArgsClass:
     case RemainingArgsJoinedClass:
       return RenderSeparateStyle;
     }
     llvm_unreachable("Unexpected kind!");

llvm/lib/Option/OptTable.cpp

if (Str.startswith(Prefix)) {
           : Rest.startswith(I->Name);
       if (Matched)
         return Prefix.size() + StringRef(I->Name).size();
     }
   }
   return 0;
 }
 
+// returns false if one of the Prefixes + In.Names matches C
+static bool optionMatches(const OptTable::Info &In, std::string C) {
+  if (!(In.Values && In.Prefixes))
+    return false;
+
+  for (size_t I = 0; In.Prefixes[I]; I++)
+    if (C == std::string(In.Prefixes[I]) + In.Name)
+      return true;
+  return false;
+}
+
+// This function is for flag value completion.
+// Eg. When "-stdlib=" and "l" was passed to this function, it will return
+// appropiriate values for stdlib, which starts with l.
+std::vector<std::string>
+OptTable::suggestValueCompletions(StringRef Option, StringRef Arg) const {
+  // Search all options and return possible values.
+  for (const Info &In : OptionInfos.slice(FirstSearchableIndex)) {
+    if (!optionMatches(In, Option))
+      continue;
+
+    SmallVector<StringRef, 8> Candidates;
+    StringRef(In.Values).split(Candidates, ",", -1, false);
+
+    std::vector<std::string> Result;
+    for (StringRef Val : Candidates)
+      if (Val.startswith(Arg))
+        Result.push_back(Val);
+    return Result;
+  }
+  return {};
+}
+
 std::vector<std::string> OptTable::findByPrefix(StringRef Cur) const {
   std::vector<std::string> Ret;
   for (const Info &In : OptionInfos.slice(FirstSearchableIndex)) {
     if (!In.Prefixes)
       continue;
     for (int I = 0; In.Prefixes[I]; I++) {
       std::string S = std::string(In.Prefixes[I]) + std::string(In.Name);
       if (StringRef(S).startswith(Cur))
else {
         Name += " <value>";
       }
     }
     break;
 
   case Option::FlagClass:
     break;
 
+  case Option::ValuesClass:
+    break;
+
   case Option::SeparateClass: case Option::JoinedOrSeparateClass:
   case Option::RemainingArgsClass: case Option::RemainingArgsJoinedClass:
     Name += ' ';
     LLVM_FALLTHROUGH;
   case Option::JoinedClass: case Option::CommaJoinedClass:
   case Option::JoinedAndSeparateClass:
     if (const char *MetaVarName = Opts.getOptionMetaVar(Id))
       Name += MetaVarName;

llvm/lib/Option/Option.cpp

void Option::print(raw_ostream &O) const {
   O << "<";
   switch (getKind()) {
 #define P(N) case N: O << #N; break
     P(GroupClass);
     P(InputClass);
     P(UnknownClass);
     P(FlagClass);
     P(JoinedClass);
+    P(ValuesClass);
     P(SeparateClass);
     P(CommaJoinedClass);
     P(MultiArgClass);
     P(JoinedOrSeparateClass);
     P(JoinedAndSeparateClass);
     P(RemainingArgsClass);
     P(RemainingArgsJoinedClass);
 #undef P

llvm/lib/ToolDrivers/llvm-lib/LibDriver.cpp

 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 
 namespace {
 
 enum {
   OPT_INVALID = 0,
-#define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11) OPT_##ID,
+#define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11, _12) OPT_##ID,
 #include "Options.inc"
 #undef OPTION
 };
 
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "Options.inc"
 #undef PREFIX
 
 static const llvm::opt::OptTable::Info infoTable[] = {
-#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X6, X7, X8, X9, X10)    \
-  {                                                                    \
-    X1, X2, X9, X10, OPT_##ID, llvm::opt::Option::KIND##Class, X8, X7, \
-    OPT_##GROUP, OPT_##ALIAS, X6                                       \
-  },
+#define OPTION(X1, X2, ID, KIND, GROUP, ALIAS, X6, X7, X8, X9, X10, X11)       \
+  {X1, X2, X9,          X10,         OPT_##ID, llvm::opt::Option::KIND##Class, \
+   X8, X7, OPT_##GROUP, OPT_##ALIAS, X6,       X11},
 #include "Options.inc"
 #undef OPTION
 };
 
 class LibOptTable : public llvm::opt::OptTable {
 public:
   LibOptTable() : OptTable(infoTable, true) {}
 };

llvm/tools/llvm-cvtres/llvm-cvtres.cpp

 using namespace llvm;
 using namespace object;
 
 namespace {
 
 enum ID {
   OPT_INVALID = 0, // This is not an option ID.
 #define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
-               HELPTEXT, METAVAR)                                              \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
   OPT_##ID,
 #include "Opts.inc"
 #undef OPTION
 };
 
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "Opts.inc"
 #undef PREFIX
 
 static const opt::OptTable::Info InfoTable[] = {
 #define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
-               HELPTEXT, METAVAR)                                              \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
   {                                                                            \
-      PREFIX,      NAME,     HELPTEXT,                                         \
-      METAVAR,     OPT_##ID, opt::Option::KIND##Class,                         \
-      PARAM,       FLAGS,    OPT_##GROUP,                                      \
-      OPT_##ALIAS, ALIASARGS},
+      PREFIX,      NAME,      HELPTEXT,                                        \
+      METAVAR,     OPT_##ID,  opt::Option::KIND##Class,                        \
+      PARAM,       FLAGS,     OPT_##GROUP,                                     \
+      OPT_##ALIAS, ALIASARGS, ARGVALUE},
 #include "Opts.inc"
 #undef OPTION
 };
 
 class CvtResOptTable : public opt::OptTable {
 public:
   CvtResOptTable() : OptTable(InfoTable, true) {}
 };

llvm/unittests/Option/OptionParsingTest.cpp

 #include "llvm/Option/Option.h"
 #include "gtest/gtest.h"
 
 using namespace llvm;
 using namespace llvm::opt;
 
 enum ID {
   OPT_INVALID = 0, // This is not an option ID.
-#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
-               HELPTEXT, METAVAR) OPT_##ID,
+#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
+  OPT_##ID,
 #include "Opts.inc"
   LastOption
 #undef OPTION
 };
 
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "Opts.inc"
 #undef PREFIX
 
 enum OptionFlags {
   OptFlag1 = (1 << 4),
   OptFlag2 = (1 << 5),
   OptFlag3 = (1 << 6)
 };
 
 static const OptTable::Info InfoTable[] = {
-#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM, \
-               HELPTEXT, METAVAR)   \
-  { PREFIX, NAME, HELPTEXT, METAVAR, OPT_##ID, Option::KIND##Class, PARAM, \
-    FLAGS, OPT_##GROUP, OPT_##ALIAS, ALIASARGS },
+#define OPTION(PREFIX, NAME, ID, KIND, GROUP, ALIAS, ALIASARGS, FLAGS, PARAM,  \
+               HELPTEXT, METAVAR, ARGVALUE)                                    \
+  {PREFIX, NAME,  HELPTEXT,    METAVAR,     OPT_##ID,  Option::KIND##Class,    \
+   PARAM,  FLAGS, OPT_##GROUP, OPT_##ALIAS, ALIASARGS, ARGVALUE},
 #include "Opts.inc"
 #undef OPTION
 };
 
 namespace {
 class TestOptTable : public OptTable {
 public:
   TestOptTable(bool IgnoreCase = false)

llvm/utils/TableGen/OptParserEmitter.cpp

for (unsigned i = 0, e = Groups.size(); i != e; ++i) {
     if (!isa<UnsetInit>(R.getValueInit("HelpText"))) {
       OS << ",\n";
       OS << "       ";
       write_cstring(OS, R.getValueAsString("HelpText"));
     } else
       OS << ", nullptr";
 
     // The option meta-variable name (unused).
+    OS << ", nullptr";
+
+    // The option Values (unused for groups).
     OS << ", nullptr)\n";
   }
   OS << "\n";
 
   OS << "//////////\n";
   OS << "// Options\n\n";
   for (unsigned i = 0, e = Opts.size(); i != e; ++i) {
     const Record &R = *Opts[i];
for (unsigned i = 0, e = Opts.size(); i != e; ++i) {
 
     // The option meta-variable name.
     OS << ", ";
     if (!isa<UnsetInit>(R.getValueInit("MetaVarName")))
       write_cstring(OS, R.getValueAsString("MetaVarName"));
     else
       OS << "nullptr";
 
+    // The option Values. Used for shell autocompletion.
+    OS << ", ";
+    if (!isa<UnsetInit>(R.getValueInit("Values")))
+      write_cstring(OS, R.getValueAsString("Values"));
+    else
+      OS << "nullptr";
+
     OS << ")\n";
   }
   OS << "#endif // OPTION\n";
 }
 } // end namespace llvm