Added support of configuration files

[sepavloff]

Summary

Added support of configuration files

Configuration files store set of options and allow to apply all of them
in a simple way, just by selection of particular file. This change extends
CommandLine Library with functions that read such files.

[ABataev]

lib/Support/CommandLine.cpp

984

What if Dir is nullptr?

1005

What if Argv is empty?

1012

'cfg_option' does not follow LLVM coding standard. And 'x'

1028–1029

Use llvm::sys::path::replace_extension(CfgFile, ".cfg")

1064

Remove 'else', unconditional 'return CfgFileSearch::Ignored;'

1196

What if Cur == Source.begin()?

[sepavloff]

lib/Support/CommandLine.cpp

984

It would be programmer's error. Added assert.

1005

It would be programmers's error - Argv must represents argv from main. Added assert.

1012

Fixed.

1028–1029

That would change semantics. A user can use something like testing.64 as a config name, it would be silently converted to testing.cfg which is not the desired behavior.

1064

Fixed.

1196

This is an error. Rewrote this place, added unit tests for this function.

[sepavloff]

Updated patch according to reviewer's notes.

[sepavloff]

Updated patch

Based on feedback on mail list added three function instead of cl::findConfigFile.
It allows a user to make more flexible config file search. Also made some cosmetic
changes.

[sepavloff]

Updated documentation

[mgorny]

lib/Support/CommandLine.cpp

1075

As I've mentioned on D24933, I think it'd be actually better to have the default dirs stored in LLVM rather than provided by the tools. This way, we can get them consistent in all LLVM tools that are going to use the configuration files, and avoid repeating them in every one of them.

[sepavloff]

Fixed comments and small error in unit test.

[ABataev]

include/llvm/Support/CommandLine.h

1793

\brief tag is not required

1802

s/TokenizeConfigFile/tokenizeConfigFile/g

1818

No \brief

1833

No \brief

1844

No \brief

1857

No \brief

1871

No \brief

lib/Support/CommandLine.cpp

716–717

Is this sequence expected on Linux too?

1034

Use llvm::sys::path::extension(CfgFile) != ".cfg" instead

[sepavloff]

Addressed reviewer's notes.

[sepavloff]

lib/Support/CommandLine.cpp

716–717

Config file can be prepared on Windows machine and then be used on Linux. Processing Windows EOLs on Linux host saves users from unexpected behavior in this case.

[sepavloff]

Updated patch

Several small changes:

• Improved documentation,
• cl::readConfigFile got new argument, number of inserted argument. It may be useful if the option read from config file need to be validated.
• function cl::findDefaultCfgFile is renamed to cl::searchForFile. In fact it is general purpose function, not config file specific.

[sepavloff]

Updated patch

• Enhanced documentation,
• Search for config files specified by --config is allowed in binary directory as well,
• Does not add ".cfg" to config files automatically.

[sepavloff]

Updated patch

Simplified the patch:

• removed suport of reading config file name from environment variable.

It is more convenient to read options from such variable rather than the
name of config file. Config file still can be specified by environment
variable using option --config.

• API function do not return error code anymore, they set string argument

to the error text.

Also the function readConfigFile now returns boolean value to indicate
if config file was successfully read.

[sepavloff]

Simplified patch

Configuration files are no more considered a feature of CommandLine
Library. Now the patch contains only minimal set of function necessary
for operations on config files. All other functionality is moved to
clang sources.

[sepavloff]

Updated patch

Support and tests for comments and line continuation are removed from
this patch. These features are now presented in D36045. Special
function for reading config files is still required as nested @file
must be resolved relative to including file, not current directory.

[sepavloff]

Updated patch

Processing comments in config files is now made in the function
tokenizeConfigFile.

[sepavloff]

Could this patch be reviewed?
It is the only dependency for D24933 (Enable configuration files in clang), which is now accepted.

[hfinkel]

lib/Support/CommandLine.cpp

715

This seems like a breaking change. Can you add a parameter to control this so that this only happens when called from cl::tokenizeConfigFile.

[sepavloff]

Updated patch

Treatment of trailing backslashes is moved to tokenizeConfigFile.
Now all operations related to config file support are separated from
the rest of code.

[hfinkel]

LGTM

docs/CommandLine.rst

 Unices have a relatively low limit on command-line length. It is therefore
 customary to use the so-called 'response files' to circumvent this
 restriction. These files are mentioned on the command-line (using the "@file")
 syntax. The program reads these files and inserts the contents into argv,
 thereby working around the command-line length limits. Response files are
 enabled by an optional fourth argument to `cl::ParseEnvironmentOptions`_ and
 `cl::ParseCommandLineOptions`_.
 
+.. _configuration files:
+
+Configuration files
+^^^^^^^^^^^^^^^^^^^
+
+Set of options may be grouped into **configurations**, which makes it easier to
+specify necessary options. For instance, a build tool can produce an executable
+either for debugging or product delivery. Both modes may require to specify many
+options that define include paths, set of libraries, compiler options and so on.
+These options can be grouped info *debug* and *release* configurations, and be
+applied just by choosing the configuration name.
+
+A configuration is a collection of options stored in a file. Options may be
+specified on one or several lines. Long options may be split between several
+lines by trailing backslash. Lines composed of whitespace characters only are
+ignored as well as lines in which the first non-blank character is `#`. Example
+of configuration file:
+
+::
+
+    # Comment
+    -option_1 -option2
+
+    # Next option
+    -option_3
+    # Option split between lines
+    -option_4="long long\
+    argument"
+
+When a configuration is selected, all options contained in it are inserted into
+the set of command line arguments as if they were specified at the beginning
+of command line.
+
+Configuration file may be specified in several ways:
+
+    - using command line option `--config`,
+    - by setting up environment variable, specific to the application,
+    - as a default configuration.
+
+The function **cl::findConfigFileFromArgs** is used to find configuration
+specified in command line. It searches command line for option `--config`,
+which must be followed by is either a full path to configuration file, or just a
+configuration file name, for instance:
+
+::
+
+    --config /home/user/cfgs/testing.txt
+    --config debug.cfg
+
+If the argument contains a directory separator, it is considered as a file path,
+options are read from that file. Otherwise the argument is treated as a file
+name, which is searched for in the directories provided in call to
+**cl::findConfigFileFromArgs**. For instance, if the function is called:
+
+.. code-block:: c++
+
+    const char *Dirs[] = { "~/.llvm", "/etc/llvm" };
+    llvm::SmallString<128> ConfigFile;
+    auto Res = llvm::cl::findConfigFileFromArgs(ConfigFile, argv, Dirs, false);
+
+then in the invocation:
+
+::
+
+    appl --config debug.cfg
+
+file `debug.cfg` is searched for in the directories `~/.llvm` and `/etc/llvm`. If
+the function call is changed to:
+
+.. code-block:: c++
+
+    const char *Dirs[] = { "~/.llvm", "/etc/llvm" };
+    llvm::SmallString<128> ConfigFile;
+    auto Res = llvm::cl::findConfigFileFromArgs(ConfigFile, argv, Dirs, true);
+
+`debug.cfg` is searched for in the directory there the executable file `appl`
+resides, after it has not been found in the directories specified by `Dirs`.
+
+The function **cl::findConfigFileFromEnv** may be used to find config file
+specified by environment variable. If it is called:
+
+.. code-block:: c++
+
+    llvm::SmallString<128> ConfigFile;
+    auto SRes = llvm::cl::findConfigFileFromEnv(ConfigFile, "APPLCFG");
+
+then in the invocation:
+
+::
+
+    APPLCFG=/home/user/test.cfg
+    appl
+
+the file `/home/user/test.cfg` will be used as configuration file.
+
+Default configuration file may be found using the function **cl::searchForFile**.
+The code:
+
+.. code-block:: c++
+
+    llvm::SmallString<128> ConfigFile;
+    const char *Dirs[] = { "~/.llvm", "/etc/llvm" };
+    auto SRes = llvm::cl::searchForFile(ConfigFile, Dirs, argv[0], "appl.cfg");
+
+in the case of invocation:
+
+::
+
+    /export/tools/appl
+
+sequentially searches for the files:
+
+::
+
+    ~/.llvm/appl.cfg
+    /etc/llvm/appl.cfg
+    /export/tools/appl.cfg
+
+To prevent from search in binary directory, use empty string as the third parameter:
+
+.. code-block:: c++
+
+    llvm::SmallString<128> ConfigFile;
+    const char *Dirs[] = { "~/.llvm", "/etc/llvm" };
+    auto SRes = llvm::cl::searchForFile(ConfigFile, Dirs, StringRef(),"appl.cfg");
+
+The following code demonstrates how an application could process config files:
+
+.. code-block:: c++
+
+    // This array must contain argument specified in application invocation, these
+    // are same as specified by argument of `main`.
+    SmallVector<const char *, 256> argv;
+
+    // Directories that are searched for configuration files.
+    static const char * const Dirs[] = { "~/.llvm", "/etc/llvm" };
+
+    // Variable that is assigned the path to the found file.
+    llvm::SmallString<128> ConfigFile;
+
+    // Look for the configuration file specigied in command line.
+    auto SRes = llvm::cl::findConfigFileFromArgs(ConfigFile, argv, Dirs, true);
+    if (llvm::cl::checkConfigFileSearchResult(SRes, ConfigFile, Dirs, argv[0]))
+      return 1;
+
+    // If command line does not contain option '--config', look for envirinment
+    // variable, "APPLCFG" in our case.
+    if (SRes == llvm::cl::SearchResult::NotSpecified) {
+      SRes = llvm::cl::findConfigFileFromEnv(ConfigFile, "APPLCFG");
+      if (llvm::cl::checkConfigFileSearchResult(SRes, ConfigFile, Dirs, argv[0]))
+        return 1;
+    }
+
+    // Finally try to find default config file.
+    if (SRes == llvm::cl::SearchResult::NotSpecified) {
+      if (llvm::cl::searchForFile(ConfigFile, Dirs, ProgName, "appl.cfg"))
+        SRes = llvm::cl::SearchResult::Successful;
+    }
+
+    // If config file is found, read it.
+    unsigned NumOpts;
+    if (SRes == llvm::cl::SearchResult::Successful)
+      llvm::cl::readConfigFile(ConfigFile, Saver, argv, NumOpts);
+
+
 Top-Level Classes and Functions
 -------------------------------
 
 Despite all of the built-in flexibility, the CommandLine option library really
 only consists of one function `cl::ParseCommandLineOptions`_) and three main
 classes: `cl::opt`_, `cl::list`_, and `cl::alias`_.  This section describes
 these three classes in detail.
 

include/llvm/Support/CommandLine.h

void TokenizeWindowsCommandLine(StringRef Source, StringSaver &Saver,
                                 bool MarkEOLs = false);
 
 /// \brief String tokenization function type.  Should be compatible with either
 /// Windows or Unix command line tokenizers.
 typedef void (*TokenizerCallback)(StringRef Source, StringSaver &Saver,
                                   SmallVectorImpl<const char *> &NewArgv,
                                   bool MarkEOLs);
 
+/// Tokenizes content of configuration file.
+///
+/// \param [in] Source The string representing content of config file.
+/// \param [in] Saver Delegates back to the caller for saving parsed strings.
+/// \param [out] NewArgv All parsed strings are appended to NewArgv.
+/// \param [in] MarkEOLs Added for compatibility with TokenizerCallback.
+///
+/// It works like TokenizeGNUCommandLine with ability to skip comment lines.
+///
+void tokenizeConfigFile(StringRef Source, StringSaver &Saver,
+                        SmallVectorImpl<const char *> &NewArgv,
+                        bool MarkEOLs = false);
+
+/// Looks for the specified file in well-known directories.
+///
+/// \param CfgFileName [out] File path, if the file was found.
+/// \param Dirs [in] Directories used for the search.
+/// \param ProgramFullPath [in] Path to the current executable or empty string.
+/// \param FileName [in] Name of the file to search for.
+/// \return True if file was found.
+///
+/// Looks for file specified by FileName sequentially in directories specified
+/// by Dirs. If not found and ProgramFullPath is not empty, searches directory
+/// where the current executable resides.
+///
+bool searchForFile(SmallVectorImpl<char> &CfgFileName,
+                   ArrayRef<const char *> Dirs, StringRef ProgramFullPath,
+                   StringRef FileName);
+
+/// Enumerates possible results of configuration file search.
+///
+enum class SearchResult {
+  Successful,   ///< File is found.
+  NotSpecified, ///< File is not specified, no diagnostic required.
+  NoArgument,   ///< Option '--config' is not followed by argument.
+  Multiple,     ///< Several options '--config' are found.
+  NotFoundCfg,  ///< Configuration specified by --config was not found.
+  NotFoundOpt,  ///< File specified by '--config' does not exist.
+  NotFoundEnv   ///< File specified by environmental variable does not exist.
+};
+
+/// Tries to find configuration file specified by command line arguments.
+///
+/// \param CfgFileName [out] Configuration file name, if it was specified by
+/// command line.
+/// \param Argv [in, out] Command line options supplied to the executable.
+/// \param Dirs [in] Directories used to search configuration file.
+/// \param SearchInBinDir [in] If True, also search directory were executable
+///                            resides.
+/// \return Code of the search result.
+///
+/// If option '--config' is specified, its argument is assigned to CfgFileName.
+/// Both the option and the argument are removed from Argv. If the resulted
+/// CfgFileName contains a directory separator, it is treated as file path, no
+/// search is made. Otherwise CfgFileName is appended extension '.cfg', if it
+/// does not have such yet, and the obtained file name is searched for in the
+/// directories:
+/// - specified by Dir,
+/// - the directory where executable resides, if SearchInBinDir is True.
+/// If The file is found, its full path replaces the value in CfgFileName.
+///
+/// The function returns values (enumerators of SearchResult):
+/// - Successful    - file is specified and found,
+/// - NotSpecified  - no option '--config' was found
+/// - NoArgument,
+/// - Multiple      - if option '--config' is used erroneously.
+/// - NotFoundCfg   - config was specified but not found.
+///
+SearchResult findConfigFileFromArgs(SmallVectorImpl<char> &CfgFileName,
+                                    SmallVectorImpl<const char *> &Argv,
+                                    ArrayRef<const char *> Dirs,
+                                    bool SearchInBinDir);
+
+/// Tries to find configuration file specified by the given environment
+/// variable.
+///
+/// \param CfgFileName [out] Configuration file name specified by the variable.
+/// \param VarName [in] Environment variable that may contain full path to
+///                     configuration file.
+/// \return Code of the search result.
+///
+/// If environment variable VarName exists and is not empty, tries to treat its
+/// value as config file path.
+///
+/// The function returns values (enumerators of SearchResult):
+/// - Successful    - file is specified and found,
+/// - NotSpecified  - variable VarName is not found,
+/// - NotFoundCfg   - the file specified by VarName is not found.
+///
+SearchResult findConfigFileFromEnv(SmallVectorImpl<char> &CfgFileName,
+                                   StringRef VarName);
+
+/// Report error occurred in config file search.
+///
+/// \param Res [in] Code of search result.
+/// \param CfgFileName [in] If set it is configuration name or full path to
+///                         the configuration file.
+/// \param Dirs [in] Directories used to search for the configuration file.
+/// \param SearchInBinDir [in] If True, directory were executable resides also
+///                            was searched.
+/// \param ProgramFullPath [in] Path to the current executable.
+/// \return true if the specified code represents an error.
+///
+bool checkConfigFileSearchResult(SearchResult Res,
+                                 StringRef CfgFileName,
+                                 ArrayRef<const char *> Dirs,
+                                 bool SearchInBinDir,
+                                 StringRef ProgramFullPath);
+
+/// Tries to read command line options from configuration file.
+///
+/// \param CfgFileName [in] Path to configuration file.
+/// \param Saver [in] Objects that saves allocated strings.
+/// \param Argv [out] Command line into which options are read.
+/// \param Num [out] Variable that is assigned number of options read from
+///                  the config file.
+///
+/// Inserts options read from configuration file into Argv starting from index
+/// 1 (Argv[0] must contain executable path).
+///
+void readConfigFile(SmallVectorImpl<char> &CfgFileName, StringSaver &Saver,
+                    SmallVectorImpl<const char *> &Argv, unsigned &Num);
+
 /// \brief Expand response files on a command line recursively using the given
 /// StringSaver and tokenization strategy.  Argv should contain the command line
 /// before expansion and will be modified in place. If requested, Argv will
 /// also be populated with nullptrs indicating where each response file line
 /// ends, which is useful for the "/link" argument that needs to consume all
 /// remaining arguments only until the next end of line, when in a response
 /// file.
 ///
 /// \param [in] Saver Delegates back to the caller for saving parsed strings.
 /// \param [in] Tokenizer Tokenization strategy. Typically Unix or Windows.
 /// \param [in,out] Argv Command line into which to expand response files.
 /// \param [in] MarkEOLs Mark end of lines and the end of the response file
 /// with nullptrs in the Argv vector.
 /// \param [in] RelativeNames true if names of nested response files must be
 /// resolved relative to including file.
 /// \return true if all @files were expanded successfully or there were none.
+///
 bool ExpandResponseFiles(StringSaver &Saver, TokenizerCallback Tokenizer,
                          SmallVectorImpl<const char *> &Argv,
                          bool MarkEOLs = false, bool RelativeNames = false);
 
 /// \brief Mark all options not part of this category as cl::ReallyHidden.
 ///
 /// \param Category the category of options to keep displaying
 ///

lib/Support/CommandLine.cpp

if (Token.empty()) {
       }
       if (I == E)
         break;
     }
 
     // Backslash escapes the next character.
     if (I + 1 < E && Src[I] == '\\') {
       ++I; // Skip the escape.
+      if (Src[I] == '\n')
+        continue; // Ignore backlash followed by '\n'.
+      if (Src[I] == '\r' && I + 1 < E && Src[I + 1] == '\n') {
+        ++I;
+        continue; // Ignore backlash followed by \r\n.
+      }
       Token.push_back(Src[I]);
       continue;
     }
 
     // Consume a quoted string.
     if (isQuote(Src[I])) {
       char Quote = Src[I++];
       while (I != E && Src[I] != Quote) {
if (!ExpandResponseFile(Arg + 1, Saver, Tokenizer, ExpandedArgv,
       continue;
     }
     Argv.erase(Argv.begin() + I);
     Argv.insert(Argv.begin() + I, ExpandedArgv.begin(), ExpandedArgv.end());
   }
   return AllExpanded;
 }
 
+/// Search the given directories for the specified file.
+///
+/// \param FilePath [out] Full file path, if it was found.
+/// \param FileName [in] The file name to search for.
+/// \param Directories [in] List of directories to search.
+/// \return True if the file was found.
+///
+static bool searchDirectoriesForFile(SmallVectorImpl<char> &FilePath,
+                                     std::string FileName,
+                                     ArrayRef<const char *> Directories) {
+  for (const char *Dir : Directories) {
+    assert(Dir);
+    FilePath.clear();
+    llvm::sys::path::append(FilePath, Dir, FileName);
+    llvm::sys::path::native(FilePath);
+    if (llvm::sys::fs::is_regular_file(FilePath))
+      return true;
+  }
+
+  return false;
+}
+
+bool cl::searchForFile(SmallVectorImpl<char> &CfgFileName,
+                       ArrayRef<const char *> Dirs, StringRef ProgramFullPath,
+                       StringRef FileName) {
+  CfgFileName.clear();
+  if (searchDirectoriesForFile(CfgFileName, FileName, Dirs))
+    return true;
+
+  // If not found, try searching the directory where executable resides.
+  CfgFileName.clear();
+  if (!ProgramFullPath.empty()) {
+    llvm::sys::path::append(CfgFileName,
+                            llvm::sys::path::parent_path(ProgramFullPath),
+                            FileName);
+    if (llvm::sys::fs::is_regular_file(CfgFileName))
+      return true;
+  }
+
+  return false;
+}
+
+cl::SearchResult cl::findConfigFileFromArgs(SmallVectorImpl<char> &CfgFileName,
+                                            SmallVectorImpl<const char *> &Argv,
+                                            ArrayRef<const char *> Dirs,
+                                            bool SearchInBinDir) {
+  assert(!Argv.empty());
+  CfgFileName.clear();
+
+  // If command line contains option '--config', try to load the configuration
+  // specified by it.
+  auto CfgOption = std::find_if(Argv.begin(), Argv.end(), [](const char *X) {
+    return strcmp(X, "--config") == 0;
+  });
+  if (CfgOption == Argv.end())
+    return SearchResult::NotSpecified;
+
+  // '--config' must be followed by config name.
+  if (CfgOption + 1 == Argv.end())
+    return SearchResult::NoArgument;
+  // Only one '--config' is allowed.
+  if (std::find_if(CfgOption + 1, Argv.end(),
+                   [](const char *X) {
+    return strcmp(X, "--config") == 0;
+  }) != Argv.end())
+    return SearchResult::Multiple;
+  std::string CfgFile = *(CfgOption + 1);
+
+  // Remove '--config' and its parameter from command line options.
+  Argv.erase(CfgOption, CfgOption + 2);
+  CfgFileName.append(CfgFile.begin(), CfgFile.end());
+
+  // If argument contains directory separator, treat it as a path to
+  // configuration file.
+  if (llvm::sys::path::has_parent_path(CfgFile)) {
+    if (!llvm::sys::fs::is_regular_file(CfgFile))
+      return SearchResult::NotFoundOpt;
+    if (llvm::sys::path::is_relative(CfgFile))
+      llvm::sys::fs::current_path(CfgFileName);
+    else
+      CfgFileName.clear();
+    llvm::sys::path::append(CfgFileName, CfgFile);
+    return SearchResult::Successful;
+  }
+
+  // Look configuration file in well-known directories, if they are specified.
+  if (searchForFile(CfgFileName, Dirs, SearchInBinDir ? Argv[0] : StringRef(),
+                    CfgFile))
+    return SearchResult::Successful;
+
+  // Keep configuration name for error report.
+  CfgFileName.clear();
+  CfgFileName.append(CfgFile.begin(), CfgFile.end());
+  return SearchResult::NotFoundCfg;
+}
+
+cl::SearchResult cl::findConfigFileFromEnv(SmallVectorImpl<char> &CfgFileName,
+                                           StringRef VarName) {
+  CfgFileName.clear();
+  if (auto CfgPath = sys::Process::GetEnv(VarName)) {
+    CfgFileName.append(CfgPath->begin(), CfgPath->end());
+    if (!CfgFileName.empty()) {
+      if (!llvm::sys::fs::is_regular_file(CfgFileName))
+        return SearchResult::NotFoundEnv;
+      return SearchResult::Successful;
+    }
+  }
+  return SearchResult::NotSpecified;
+}
+
+bool cl::checkConfigFileSearchResult(SearchResult Res,
+                                     StringRef CfgFile,
+                                     ArrayRef<const char *> Dirs,
+                                     bool SearchInBinDir,
+                                     StringRef ProgramFullPath) {
+  switch (Res) {
+  case SearchResult::Successful:
+  case SearchResult::NotSpecified:
+    return false;
+  case SearchResult::NoArgument:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Option '--config' must be followed by "
+      "configuration name or full path to configuration file\n";
+    break;
+  case SearchResult::Multiple:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: More than one option '--config' is specified\n";
+    break;
+  case SearchResult::NotFoundCfg:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Configuration '" << CfgFile << "' specified by "
+      "option '--config' cannot be found in directories:\n";
+    for (const char *Dir : Dirs) {
+      SmallString<128> DirStr;
+      llvm::sys::path::native(Dir, DirStr);
+      errs() << "    " << DirStr << "\n";
+    }
+    if (SearchInBinDir) {
+      SmallString<128> BinDir(ProgramFullPath);
+      llvm::sys::path::remove_filename(BinDir);
+      errs() << "    " << BinDir << "\n";
+    }
+    break;
+  case SearchResult::NotFoundOpt:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Configuration file '" << CfgFile << "' specified "
+      "by option '--config' cannot be found\n";
+    break;
+  case SearchResult::NotFoundEnv:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Configuration file '" << CfgFile << "' specified "
+      "by environment variable cannot be found\n";
+    break;
+  }
+  return true;
+}
+
+void cl::readConfigFile(SmallVectorImpl<char> &CfgFile, StringSaver &Saver,
+                        SmallVectorImpl<const char *> &Argv, unsigned &Num) {
+  CfgFile.push_back(0);
+  SmallVector<const char *, 8> NewArgs;
+  ExpandResponseFile(CfgFile.data(), Saver, tokenizeConfigFile, NewArgs, false,
+                     true);
+  ExpandResponseFiles(Saver, tokenizeConfigFile, NewArgs, false);
+  Num = NewArgs.size();
+  Argv.insert(Argv.begin() + 1, NewArgs.begin(), NewArgs.end());
+}
+
 /// ParseEnvironmentOptions - An alternative entry point to the
 /// CommandLine library, which allows you to read the program's name
 /// from the caller (as PROGNAME) and its command-line arguments from
 /// an environment variable (whose name is given in ENVVAR).
 ///
 void cl::ParseEnvironmentOptions(const char *progName, const char *envVar,
                                  const char *Overview) {
   // Check args.
void cl::ParseEnvironmentOptions(const char *progName, const char *envVar,
 
   // Parse the value of the environment variable into a "command line"
   // and hand it off to ParseCommandLineOptions().
   TokenizeGNUCommandLine(*envValue, Saver, newArgv);
   int newArgc = static_cast<int>(newArgv.size());
   ParseCommandLineOptions(newArgc, &newArgv[0], StringRef(Overview));
 }
 
+void cl::tokenizeConfigFile(StringRef Source, StringSaver &Saver,
+                            SmallVectorImpl<const char *> &NewArgv,
+                            bool MarkEOLs) {
+  for (const char *Cur = Source.begin(); Cur != Source.end();) {
+    // Check for comment line.
+    if (isWhitespace(*Cur)) {
+      while (Cur != Source.end() && isWhitespace(*Cur))
+        ++Cur;
+      continue;
+    }
+    if (*Cur == '#') {
+      while (Cur != Source.end() && *Cur != '\n')
+        ++Cur;
+      continue;
+    }
+    // Find end of current line.
+    const char *Start = Cur;
+    for (const char *End = Source.end(); Cur != End; ++Cur) {
+      if (*Cur == '\\') {
+        if (Cur + 1 != End) {
+          ++Cur;
+          if (Cur != End && *Cur == '\r' && (Cur + 1 != End) && Cur[1] == '\n')
+            ++Cur;
+        }
+      } else if (*Cur == '\n')
+        break;
+    }
+    // Tokenize line.
+    StringRef Line(Start, Cur - Start);
+    cl::TokenizeGNUCommandLine(Line, Saver, NewArgv, MarkEOLs);
+  }
+}
+
 bool cl::ParseCommandLineOptions(int argc, const char *const *argv,
                                  StringRef Overview, bool IgnoreErrors) {
   return GlobalParser->ParseCommandLineOptions(argc, argv, Overview,
                                                IgnoreErrors);
 }
 
 void CommandLineParser::ResetAllOptionOccurrences() {
   // So that we can parse different command lines multiple times in succession

unittests/Support/CommandLineTest.cpp

TEST(CommandLineTest, TokenizeWindowsCommandLine) {
   const char Input[] = "a\\b c\\\\d e\\\\\"f g\" h\\\"i j\\\\\\\"k \"lmn\" o pqr "
                       "\"st \\\"u\" \\v";
   const char *const Output[] = { "a\\b", "c\\\\d", "e\\f g", "h\"i", "j\\\"k",
                                  "lmn", "o", "pqr", "st \"u", "\\v" };
   testCommandLineTokenizer(cl::TokenizeWindowsCommandLine, Input, Output,
                            array_lengthof(Output));
 }
 
+TEST(CommandLineTest, TokenizeConfigFile1) {
+  const char *Input = "\\";
+  const char *const Output[] = { "\\" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile2) {
+  const char *Input = "\\abc";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile3) {
+  const char *Input = "abc\\";
+  const char *const Output[] = { "abc\\" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile4) {
+  const char *Input = "abc\\\n123";
+  const char *const Output[] = { "abc123" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile5) {
+  const char *Input = "abc\\\r\n123";
+  const char *const Output[] = { "abc123" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile6) {
+  const char *Input = "abc\\\n";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile7) {
+  const char *Input = "abc\\\r\n";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile8) {
+  SmallVector<const char *, 0> Actual;
+  BumpPtrAllocator A;
+  StringSaver Saver(A);
+  cl::tokenizeConfigFile("\\\n", Saver, Actual, /*MarkEOLs=*/false);
+  EXPECT_TRUE(Actual.empty());
+}
+
+TEST(CommandLineTest, TokenizeConfigFile9) {
+  SmallVector<const char *, 0> Actual;
+  BumpPtrAllocator A;
+  StringSaver Saver(A);
+  cl::tokenizeConfigFile("\\\r\n", Saver, Actual, /*MarkEOLs=*/false);
+  EXPECT_TRUE(Actual.empty());
+}
+
+TEST(CommandLineTest, TokenizeConfigFile10) {
+  const char *Input = "\\\nabc";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile11) {
+  const char *Input = "\\\r\nabc";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::tokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
 TEST(CommandLineTest, AliasesWithArguments) {
   static const size_t ARGC = 3;
   const char *const Inputs[][ARGC] = {
     { "-tool", "-actual=x", "-extra" },
     { "-tool", "-actual", "x" },
     { "-tool", "-alias=x", "-extra" },
     { "-tool", "-alias", "x" }
   };
TEST(CommandLineTest, ResponseFiles) {
   EXPECT_STREQ(Argv[8], "-flag_2");
 
   llvm::sys::fs::remove(IncludedFileName2);
   llvm::sys::fs::remove(IncDir);
   llvm::sys::fs::remove(IncludedFileName);
   llvm::sys::fs::remove(TestDir);
 }
 
+TEST(CommandLineTest, ConfigFileErrors) {
+  llvm::SmallString<128> CfgFileName;
+  llvm::SmallVector<const char *, 1> Argv = { "test/test" };
+  ArrayRef<const char *> Dirs;
+  cl::SearchResult Result;
+
+  // Config in not specified.
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::NotSpecified);
+
+  // Missed argument of --config
+  Argv = { "--config" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::NoArgument);
+
+  // Multiple options --config
+  Argv = { "--config", "a", "--config", "b" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::Multiple);
+
+  // Inexistent file specified by full path.
+  Argv = { "--config", "/abcd.cfg" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundOpt);
+  EXPECT_TRUE(Argv.empty());
+  EXPECT_TRUE(CfgFileName.equals("/abcd.cfg"));
+
+  // Inexistent configuration.
+  Argv = { "--config", "abcd.cfg" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundCfg);
+  EXPECT_TRUE(Argv.empty());
+  EXPECT_TRUE(CfgFileName.equals("abcd.cfg"));
+
+  // Inexistent configuration, configuration without extension.
+  Argv = { "--config", "abcd" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundCfg);
+  EXPECT_TRUE(Argv.empty());
+  EXPECT_TRUE(CfgFileName.equals("abcd"));
+}
+
+TEST(CommandLineTest, ReadConfigFile) {
+  llvm::SmallString<128> CfgFileName;
+  llvm::SmallVector<const char *, 1> Argv;
+  const char *ProgramFullPath = "/test/test";
+  ArrayRef<const char *> Dirs;
+  cl::SearchResult Result;
+
+  llvm::SmallString<128> TestCfg;
+  std::error_code EC =
+    llvm::sys::fs::createTemporaryFile("unittest", "cfg", TestCfg);
+  EXPECT_TRUE(!EC);
+  std::string Directory = llvm::sys::path::parent_path(TestCfg);
+  std::string FileName = llvm::sys::path::filename(TestCfg);
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  std::ofstream Cfg(TestCfg.c_str());
+  Cfg << "# Comment\n";
+  Cfg << "-option_1 -option_2\n";
+  Cfg << "-option_3=abcd\n";
+  Cfg.close();
+
+  Argv = { ProgramFullPath, "-flag_1", "--config", FileName.c_str() };
+  Dirs = ArrayRef<const char *>(Directory.c_str());
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_TRUE(CfgFileName.equals(TestCfg));
+  EXPECT_EQ(Argv.size(), 2);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-flag_1");
+
+  unsigned NumOpts = 0;
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv, NumOpts);
+
+  EXPECT_EQ(Argv.size(), 5);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-option_1");
+  EXPECT_STREQ(Argv[2], "-option_2");
+  EXPECT_STREQ(Argv[3], "-option_3=abcd");
+  EXPECT_STREQ(Argv[4], "-flag_1");
+  EXPECT_EQ(NumOpts, 3);
+
+  // Reading from file with lines concatenated by trailing \.
+  Cfg.open(TestCfg.c_str(), std::ofstream::trunc);
+  EXPECT_TRUE(Cfg.is_open());
+
+  Cfg << "\n\n# Comment\n";
+  Cfg << "-option_\\\n";
+  Cfg << "1 -option_3=abcd\n";
+  Cfg.close();
+
+  Argv = { ProgramFullPath, "-flag_1", "--config", FileName.c_str(),
+           "-flag_2" };
+  Dirs = ArrayRef<const char *>(Directory.c_str());
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_TRUE(CfgFileName.equals(TestCfg));
+  EXPECT_EQ(Argv.size(), 3);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-flag_1");
+  EXPECT_STREQ(Argv[2], "-flag_2");
+
+  NumOpts = 0;
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv, NumOpts);
+
+  EXPECT_EQ(Argv.size(), 5);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-option_1");
+  EXPECT_STREQ(Argv[2], "-option_3=abcd");
+  EXPECT_STREQ(Argv[3], "-flag_1");
+  EXPECT_STREQ(Argv[4], "-flag_2");
+  EXPECT_EQ(NumOpts, 2);
+
+  llvm::sys::fs::remove(TestCfg);
+}
+
+TEST(CommandLineTest, ReadConfigFileFromBinDir) {
+  // Create test directory
+  llvm::SmallString<128> TestDir;
+  std::error_code EC =
+    llvm::sys::fs::createUniqueDirectory("unittest", TestDir);
+  EXPECT_TRUE(!EC);
+
+  // Pretend executable is in the test directory.
+  llvm::SmallString<128> ProgramFullPath;
+  llvm::sys::path::append(ProgramFullPath, TestDir, "testtool");
+
+  const char * ConfigFileName = "test.cfg";
+
+  // Create config file in the binary directory.
+  llvm::SmallString<128> BinCfg;
+  llvm::sys::path::append(BinCfg, TestDir, ConfigFileName);
+  std::ofstream Cfg(BinCfg.c_str());
+  EXPECT_TRUE(Cfg.is_open());
+  Cfg << "-option_1 -option_2";
+  Cfg.close();
+
+  // Create directory for config files.
+  llvm::SmallString<128> TestCfgDir;
+  llvm::sys::path::append(TestCfgDir, TestDir, "stddir");
+  EC = llvm::sys::fs::create_directory(TestCfgDir);
+  EXPECT_TRUE(!EC);
+
+  // Create config file there.
+  llvm::SmallString<128> TestCfg;
+  llvm::sys::path::append(TestCfg, TestCfgDir, ConfigFileName);
+  std::ofstream Cfg2(TestCfg.c_str());
+  EXPECT_TRUE(Cfg2.is_open());
+  Cfg2 << "-option_3=abcd";
+  Cfg2.close();
+
+  // Working variables shared between test cases.
+  llvm::SmallString<128> CfgFileName;
+  llvm::SmallVector<const char *, 1> Argv;
+  ArrayRef<const char *> Dirs;
+  cl::SearchResult Result;
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  // Case 1: search directory has higher priority than binary.
+  Argv = { ProgramFullPath.c_str(), "--config", ConfigFileName };
+  Dirs = ArrayRef<const char *>(TestCfgDir.c_str());
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, true);
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_TRUE(strcmp(CfgFileName.c_str(), TestCfg.c_str()) == 0);
+
+  // Case 2: without search directories config file in binary directory must
+  // be found.
+  Argv = { ProgramFullPath.c_str(), "--config", ConfigFileName };
+  Dirs = ArrayRef<const char *>();
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, true);
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_TRUE(strcmp(CfgFileName.c_str(), BinCfg.c_str()) == 0);
+
+  // Case 3: Config file is not found if search in binary directory is
+  // suppressed.
+  Argv = { ProgramFullPath.c_str(), "--config", ConfigFileName };
+  Dirs = ArrayRef<const char *>();
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs, false);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundCfg);
+
+  llvm::sys::fs::remove(TestCfg);
+  llvm::sys::fs::remove(TestCfgDir);
+  llvm::sys::fs::remove(BinCfg);
+  llvm::sys::fs::remove(TestDir);
+}
+
+TEST(CommandLineTest, ReadDefaultConfigFile) {
+  llvm::SmallString<128> TestDir;
+  std::error_code EC =
+      llvm::sys::fs::createUniqueDirectory("unittest", TestDir);
+  EXPECT_TRUE(!EC);
+
+  llvm::SmallString<128> TestTool;
+  llvm::sys::path::append(TestTool, TestDir, "testtool");
+  std::ofstream Tool(TestTool.c_str());
+  EXPECT_TRUE(Tool.is_open());
+  Tool << std::endl;
+  Tool.close();
+
+  llvm::SmallString<128> TestCfg;
+  llvm::sys::path::append(TestCfg, TestDir, "test.cfg");
+  std::ofstream Cfg(TestCfg.c_str());
+  EXPECT_TRUE(Cfg.is_open());
+  Cfg << "-option_1 -option_2\n"
+         "-option_3=abcd\n";
+  Cfg.close();
+
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  llvm::SmallVector<const char *, 1> Argv = { TestTool.c_str(), "-flag_1" };
+  llvm::SmallString<128> CfgFileName;
+  bool Result = cl::searchForFile(CfgFileName, ArrayRef<const char *>(),
+                                  Argv[0], "test.cfg");
+
+  EXPECT_TRUE(Result);
+  EXPECT_STREQ(CfgFileName.c_str(), TestCfg.c_str());
+
+  unsigned NumOpts = 0;
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv, NumOpts);
+
+  EXPECT_EQ(Argv.size(), 5);
+  EXPECT_STREQ(Argv[0], TestTool.c_str());
+  EXPECT_STREQ(Argv[1], "-option_1");
+  EXPECT_STREQ(Argv[2], "-option_2");
+  EXPECT_STREQ(Argv[3], "-option_3=abcd");
+  EXPECT_STREQ(Argv[4], "-flag_1");
+  EXPECT_EQ(NumOpts, 3);
+
+  llvm::sys::fs::remove(TestCfg);
+  llvm::sys::fs::remove(TestTool);
+  llvm::sys::fs::remove(TestDir);
+}
+
+TEST(CommandLineTest, ReadDefaultConfigFileFromStdDir) {
+  llvm::SmallString<128> TestDir;
+  std::error_code EC =
+    llvm::sys::fs::createUniqueDirectory("unittest", TestDir);
+  EXPECT_TRUE(!EC);
+
+  llvm::SmallString<128> TestTool;
+  llvm::sys::path::append(TestTool, TestDir, "testtool");
+  std::ofstream Tool(TestTool.c_str());
+  EXPECT_TRUE(Tool.is_open());
+  Tool << std::endl;
+  Tool.close();
+
+  llvm::SmallString<128> TestCfgDir;
+  llvm::sys::path::append(TestCfgDir, TestDir, "stddir");
+  EC = llvm::sys::fs::create_directory(TestCfgDir);
+  EXPECT_TRUE(!EC);
+
+  llvm::SmallString<128> TestCfg;
+  llvm::sys::path::append(TestCfg, TestCfgDir, "test.cfg");
+  std::ofstream Cfg(TestCfg.c_str());
+  EXPECT_TRUE(Cfg.is_open());
+  Cfg << "-option_3=abcd\n";
+  Cfg.close();
+
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  llvm::SmallVector<const char *, 1> Argv = { TestTool.c_str(), "-flag_1" };
+  const char *TestCfgDirStr = TestCfgDir.c_str();
+  ArrayRef<const char *> Dirs(TestCfgDirStr);
+  llvm::SmallString<128> CfgFileName;
+  bool Result = cl::searchForFile(CfgFileName, Dirs, Argv[0], "test.cfg");
+
+  EXPECT_TRUE(Result);
+  EXPECT_STREQ(CfgFileName.c_str(), TestCfg.c_str());
+
+  unsigned NumOpts = 0;
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv, NumOpts);
+
+  EXPECT_EQ(Argv.size(), 3);
+  EXPECT_STREQ(Argv[0], TestTool.c_str());
+  EXPECT_STREQ(Argv[1], "-option_3=abcd");
+  EXPECT_STREQ(Argv[2], "-flag_1");
+  EXPECT_EQ(NumOpts, 1);
+
+  llvm::sys::fs::remove(TestCfg);
+  llvm::sys::fs::remove(TestCfgDir);
+  llvm::sys::fs::remove(TestTool);
+  llvm::sys::fs::remove(TestDir);
+}
+
 }  // anonymous namespace