Added support of configuration files

[sepavloff]

Summary

Added support of configuration files

Configuration files store set of options and allow to apply all of them
in a simple way, just by selection of particular file. This change extends
CommandLine Library with functions that read such files.

[ABataev]

lib/Support/CommandLine.cpp

956

What if Dir is nullptr?

977

What if Argv is empty?

984

'cfg_option' does not follow LLVM coding standard. And 'x'

1000–1001

Use llvm::sys::path::replace_extension(CfgFile, ".cfg")

1036

Remove 'else', unconditional 'return CfgFileSearch::Ignored;'

1161

What if Cur == Source.begin()?

[sepavloff]

lib/Support/CommandLine.cpp

956

It would be programmer's error. Added assert.

977

It would be programmers's error - Argv must represents argv from main. Added assert.

984

Fixed.

1000–1001

That would change semantics. A user can use something like testing.64 as a config name, it would be silently converted to testing.cfg which is not the desired behavior.

1036

Fixed.

1161

This is an error. Rewrote this place, added unit tests for this function.

[sepavloff]

Updated patch according to reviewer's notes.

[sepavloff]

Updated patch

Based on feedback on mail list added three function instead of cl::findConfigFile.
It allows a user to make more flexible config file search. Also made some cosmetic
changes.

[sepavloff]

Updated documentation

[mgorny]

lib/Support/CommandLine.cpp

1047

As I've mentioned on D24933, I think it'd be actually better to have the default dirs stored in LLVM rather than provided by the tools. This way, we can get them consistent in all LLVM tools that are going to use the configuration files, and avoid repeating them in every one of them.

[sepavloff]

Fixed comments and small error in unit test.

[ABataev]

include/llvm/Support/CommandLine.h

1780

\brief tag is not required

1789

s/TokenizeConfigFile/tokenizeConfigFile/g

1805

No \brief

1820

No \brief

1831

No \brief

1844

No \brief

1858

No \brief

lib/Support/CommandLine.cpp

710–711

Is this sequence expected on Linux too?

1006

Use llvm::sys::path::extension(CfgFile) != ".cfg" instead

[sepavloff]

Addressed reviewer's notes.

[sepavloff]

lib/Support/CommandLine.cpp

710–711

Config file can be prepared on Windows machine and then be used on Linux. Processing Windows EOLs on Linux host saves users from unexpected behavior in this case.

[sepavloff]

Updated patch

Several small changes:

• Improved documentation,
• cl::readConfigFile got new argument, number of inserted argument. It may be useful if the option read from config file need to be validated.
• function cl::findDefaultCfgFile is renamed to cl::searchForFile. In fact it is general purpose function, not config file specific.

[sepavloff]

Updated patch

• Enhanced documentation,
• Search for config files specified by --config is allowed in binary directory as well,
• Does not add ".cfg" to config files automatically.

[sepavloff]

Updated patch

Simplified the patch:

• removed suport of reading config file name from environment variable.

It is more convenient to read options from such variable rather than the
name of config file. Config file still can be specified by environment
variable using option --config.

• API function do not return error code anymore, they set string argument

to the error text.

Also the function readConfigFile now returns boolean value to indicate
if config file was successfully read.

[sepavloff]

Simplified patch

Configuration files are no more considered a feature of CommandLine
Library. Now the patch contains only minimal set of function necessary
for operations on config files. All other functionality is moved to
clang sources.

[sepavloff]

Updated patch

Support and tests for comments and line continuation are removed from
this patch. These features are now presented in D36045. Special
function for reading config files is still required as nested @file
must be resolved relative to including file, not current directory.

[sepavloff]

Updated patch

Processing comments in config files is now made in the function
tokenizeConfigFile.

[sepavloff]

Could this patch be reviewed?
It is the only dependency for D24933 (Enable configuration files in clang), which is now accepted.

[hfinkel]

lib/Support/CommandLine.cpp

709

This seems like a breaking change. Can you add a parameter to control this so that this only happens when called from cl::tokenizeConfigFile.

[sepavloff]

Updated patch

Treatment of trailing backslashes is moved to tokenizeConfigFile.
Now all operations related to config file support are separated from
the rest of code.

[hfinkel]

LGTM

docs/CommandLine.rst

 Unices have a relatively low limit on command-line length. It is therefore
 customary to use the so-called 'response files' to circumvent this
 restriction. These files are mentioned on the command-line (using the "@file")
 syntax. The program reads these files and inserts the contents into argv,
 thereby working around the command-line length limits. Response files are
 enabled by an optional fourth argument to `cl::ParseEnvironmentOptions`_ and
 `cl::ParseCommandLineOptions`_.
 
+.. _configuration files:
+
+Configuration files
+^^^^^^^^^^^^^^^^^^^
+
+Set of options may be groupped into **configurations**, which makes it easier to
+specify necessary options. For instance, a build tool may be run to produce a
+program for debugging or for product delivery. Both modes may require to specify
+many options that set include paths, set of libraries, compiler options and so
+on. These options can be groupped info *debug* and *release* configurations, and
+be specified just by choosing one of them.
+
+A configuration is a collection of options stored in a file. Options may be
+specified on one or several lines. Long options may be split between several
+lines by trailing backslash. Lines composed of whitespace characters only are
+ignored as well as lines in which the first non-blank character is `#`. Example
+of configuration file:
+
+::
+
+    # Comment
+    -option_1 -option2
+
+    # Next option
+    -option_3
+    # Option split between lines
+    -option_4="long long\
+    argument"
+
+Particular configuration file may be specified in several ways:
+
+    - Using command line option `--config`, followed by configuration file name,
+    - By setting up environment variable, specific to the application.
+    - As a default configuration.
+
+There are separate functions that search for configuration file name specified by
+each way.
+
+The function **cl::findConfigFileFromArgs** is used to find configuration
+specified in command line. It searches command line for the option `--config`,
+which must be followed by is either a full path to configuration file, or just a
+configuration file name, for instance:
+
+::
+
+    --config /home/user/cfgs/testing.txt
+    --config debug
+    --config debug.cfg
+
+If the argument contains a directory separator, it is considered as a file name,
+options are read from that file. Otherwise the argument is appended extension
+`cfg` if it is not specified yet, and the obtained file name is searched for
+in the directories specified in the application source code. For instance, in
+the invocation:
+
+::
+
+    appl --config debug
+
+file `debug.cfg` is searched for in the directories `~/.llvm` and `/etc/llvm`,
+if they were specified in call to **cl::findConfigFileFromArgs**.
+
+The function **cl::findConfigFileFromEnv** may be used to find config file
+specified by environment variable. If neither `--config` nor environment variable
+are specified, the application may looks for a default configuration file using
+function **cl::findDefaultCfgFile**. It searches for the given file in the
+specified set of well-known directories. If the file was not found there, it is
+looked for in the directory where application executable resides. For example,
+if a tool `appl` was run by the command:
+
+::
+
+    /export/tools/appl
+
+the cl::findConfigFileFromEnv looks for the files:
+
+::
+
+    ~/.llvm/appl.cfg
+    /etc/llvm/appl.cfg
+    /export/tools/appl.cfg
+
+if the directories `~/.llvm` and `/etc/llvm` were specified in call to
+**cl::findDefaultCfgFile**.
+
+To use configuration files, application calls appropriate library functions as
+in the example:
+
+.. code-block:: c++
+
+    // This array must contain argument specified in application invocation, these
+    // are same as specified by argument of `main`.
+    SmallVector<const char *, 256> argv;
+
+    // Directories that are searched for configuration files.
+    static const char * const SearchDirs[] = { "~/.llvm", "/etc/llvm" };
+
+    // Variable that is assigned the path to the found file.
+    llvm::SmallString<128> ConfigFile;
+
+    // Look for the configuration file specigied in command line.
+    auto SRes = llvm::cl::findConfigFileFromArgs(ConfigFile, argv, SearchDirs);
+    if (llvm::cl::checkConfigFileSearchResult(SRes, ConfigFile, SearchDirs, argv[0]))
+      return 1;
+
+    // If command line does not contain option '--config', look for envirinment
+    // variable, "APPLCFG" in our case.
+    if (SRes == llvm::cl::SearchResult::NotSpecified) {
+      SRes = llvm::cl::findConfigFileFromEnv(ConfigFile, "APPLCFG");
+      if (llvm::cl::checkConfigFileSearchResult(SRes, ConfigFile, SearchDirs, argv[0]))
+        return 1;
+    }
+
+    // Finally try to find default config file.
+    SRes = llvm::cl::findDefaultCfgFile(ConfigFile, SearchDirs, ProgName,
+                                        "appl.cfg");
+    if (llvm::cl::checkConfigFileSearchResult(SRes, ConfigFile, SearchDirs, argv[0]))
+      return 1;
+
+    // If file is found, read it.
+    if (SRes == llvm::cl::CfgFileSearch::Successful)
+      llvm::cl::readConfigFile(ConfigFile, Saver, argv);
+
+
 Top-Level Classes and Functions
 -------------------------------
 
 Despite all of the built-in flexibility, the CommandLine option library really
 only consists of one function `cl::ParseCommandLineOptions`_) and three main
 classes: `cl::opt`_, `cl::list`_, and `cl::alias`_.  This section describes
 these three classes in detail.
 

include/llvm/Support/CommandLine.h

void TokenizeWindowsCommandLine(StringRef Source, StringSaver &Saver,
                                 bool MarkEOLs = false);
 
 /// \brief String tokenization function type.  Should be compatible with either
 /// Windows or Unix command line tokenizers.
 typedef void (*TokenizerCallback)(StringRef Source, StringSaver &Saver,
                                   SmallVectorImpl<const char *> &NewArgv,
                                   bool MarkEOLs);
 
+/// \brief Tokenizes content of configuration file.
+///
+/// \param [in] Source The string representing content of config file.
+/// \param [in] Saver Delegates back to the caller for saving parsed strings.
+/// \param [in] MarkEOLs Added for compatibility with TokenizerCallback.
+/// \param [out] NewArgv All parsed strings are appended to NewArgv.
+///
+/// It works like TokenizeGNUCommandLine with ability to skip comment lines.
+///
+void TokenizeConfigFile(StringRef Source, StringSaver &Saver,
+                        SmallVectorImpl<const char *> &NewArgv,
+                        bool MarkEOLs = false);
+
+/// Enumerates possible results of configuration file search.
+///
+enum class SearchResult {
+  Successful,   ///< File is found.
+  NotSpecified, ///< File is not specified, no diagnostic required.
+  NoArgument,   ///< Option '--config' is not followed by argument.
+  Multiple,     ///< Several options '--config' are found.
+  NotFoundCfg,  ///< Configuration specified by --config was not found.
+  NotFoundOpt,  ///< File specified by '--config' does not exist.
+  NotFoundEnv   ///< File specified by environmental variable does not exist.
+};
+
+/// \brief Tries to find configuration file specified by command line arguments.
+///
+/// \param CfgFileName [out] Configuration file name, if it was specified by
+/// command line.
+/// \param Argv [in, out] Command line option supplied to the executable.
+/// \param Dirs [in] Directories used to search configuration file.
+/// \return Error code of the search.
+///
+/// If option '--config' is specified, its argument is assigned to CfgName. Both
+/// the option and the argument are removed from Argv.
+///
+SearchResult findConfigFileFromArgs(SmallVectorImpl<char> &CfgFileName,
+                                     SmallVectorImpl<const char *> &Argv,
+                                     ArrayRef<const char *> Dirs);
+
+/// \brief Tries to find configuration file specified by the given environment
+/// variable.
+///
+/// \param CfgFileName [out] Configuration file name specified by the variable.
+/// \param VarName [in] Environment variable that may contain full path to
+///                     configuration file.
+/// \return Error code of the search.
+///
+SearchResult findConfigFileFromEnv(SmallVectorImpl<char> &CfgFileName,
+                                    StringRef VarName);
+
+/// \brief Tries to find default configuration file.
+///
+/// \param CfgFileName [out] Configuration file name, if the file was found.
+/// \param Dirs [in] Directories used to search configuration file.
+/// \param ProgramFullPath [in] Path to the tool executable.
+/// \param FileName [in] Name of the configuration file.
+/// \return Error code of the search.
+///
+SearchResult findDefaultCfgFile(SmallVectorImpl<char> &CfgFileName,
+                                ArrayRef<const char *> Dirs,
+                                StringRef ProgramFullPath,
+                                StringRef FileName);
+
+/// \brief Report error occurred in config file search.
+///
+/// \param Res [in] Code of search result.
+/// \param CfgFileName [in] If set it is configuration name or full path to
+///                         configuration file.
+/// \param Dirs [in] Directories used to search configuration file.
+/// \param ProgramFullPath [in] Path to the tool executable.
+/// \param ToolName [in] Name of tool that reads options.
+/// \return true if the specified code represents an error.
+///
+bool checkConfigFileSearchResult(SearchResult Res,
+                                 StringRef CfgFileName,
+                                 ArrayRef<const char *> Dirs,
+                                 StringRef ProgramFullPath);
+
+/// \brief Tries to read command line options from configuration file.
+///
+/// \param Saver [in] Objects that saves allocated strings.
+/// \param Argv [out] Command line into which options are read.
+/// \param ToolName [in] Name of tool that reads options.
+///
+/// Configuration file provides a facility for a tool to set up some command
+/// line options before the options actually specified in command line. Options
+/// specified in configuration file are automatically processed every time the
+/// tool executable is run.
+///
+/// The default configuration file is a file named <ToolName>.cfg (for instance
+/// 'clang.cfg') placed in the same directory as the tool executable. If
+/// environment variable <ToolName>CFG (for instance CLANGCFG) is set, its value
+/// is considered as full path to configuration file, default file is not used
+/// in this case.
+///
+void readConfigFile(SmallVectorImpl<char> &CfgFileName, StringSaver &Saver,
+                    SmallVectorImpl<const char *> &Argv);
+
 /// \brief Expand response files on a command line recursively using the given
 /// StringSaver and tokenization strategy.  Argv should contain the command line
 /// before expansion and will be modified in place. If requested, Argv will
 /// also be populated with nullptrs indicating where each response file line
 /// ends, which is useful for the "/link" argument that needs to consume all
 /// remaining arguments only until the next end of line, when in a response
 /// file.
 ///

lib/Support/CommandLine.cpp

 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Config/config.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Process.h"
 #include "llvm/Support/StringSaver.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cstdlib>
if (Token.empty()) {
       }
       if (I == E)
         break;
     }
 
     // Backslash escapes the next character.
     if (I + 1 < E && Src[I] == '\\') {
       ++I; // Skip the escape.
+      if (Src[I] == '\n')
+        continue; // Ignore backlash followed by '\n'.
+      if (Src[I] == '\r' && I + 1 < E && Src[I + 1] == '\n') {
+        ++I;
+        continue; // Ignore backlash followed by \r\n.
+      }
       Token.push_back(Src[I]);
       continue;
     }
 
     // Consume a quoted string.
     if (isQuote(Src[I])) {
       char Quote = Src[I++];
       while (I != E && Src[I] != Quote) {
if (!ExpandResponseFile(Arg + 1, Saver, Tokenizer, ExpandedArgv,
       continue;
     }
     Argv.erase(Argv.begin() + I);
     Argv.insert(Argv.begin() + I, ExpandedArgv.begin(), ExpandedArgv.end());
   }
   return AllExpanded;
 }
 
+static bool findFileInDirectories(SmallVectorImpl<char> &FilePath,
+                                  std::string FileName,
+                                  ArrayRef<const char *> Directories) {
+  for (const char *Dir : Directories) {
+    assert(Dir);
+    FilePath.clear();
+    if (Dir[0] == '~') {
+      assert(llvm::sys::path::is_separator(Dir[1]));
+      if (llvm::sys::path::home_directory(FilePath)) {
+        llvm::sys::path::append(FilePath, Dir + 2, FileName);
+        if (llvm::sys::fs::is_regular_file(FilePath))
+          return true;
+      }
+      continue;
+    }
+    llvm::sys::path::append(FilePath, Dir, FileName);
+    if (llvm::sys::fs::is_regular_file(FilePath))
+      return true;
+  }
+
+  return false;
+}
+
+cl::SearchResult cl::findConfigFileFromArgs(SmallVectorImpl<char> &CfgFileName,
+                                            SmallVectorImpl<const char *> &Argv,
+                                            ArrayRef<const char *> Dirs) {
+  assert(!Argv.empty());
+  CfgFileName.clear();
+
+  // If command line contains option '--config', try to load the configuration
+  // specified by it.
+  auto CfgOption = std::find_if(Argv.begin(), Argv.end(), [](const char *X) {
+    return strcmp(X, "--config") == 0;
+  });
+  if (CfgOption == Argv.end())
+    return SearchResult::NotSpecified;
+
+  // '--config' must be followed by config name.
+  if (CfgOption + 1 == Argv.end())
+    return SearchResult::NoArgument;
+  // Only one '--config' is allowed.
+  if (std::find_if(CfgOption + 1, Argv.end(),
+                    [](const char *X) {
+                      return strcmp(X, "--config") == 0;
+                    }) != Argv.end())
+    return SearchResult::Multiple;
+  std::string CfgFile = *(CfgOption + 1);
+
+  // Remove '--config' and its parameter from command line options.
+  Argv.erase(CfgOption, CfgOption + 2);
+
+  // If argument contains directory separator, treat it as a path to
+  // configuration file. Otherwise it is a configuration name and must be
+  // resolved to file path.
+  if (!llvm::sys::path::has_parent_path(CfgFile)) {
+    if (!StringRef(CfgFile).endswith(".cfg"))
+      CfgFile += ".cfg";
+
+    // Look configuration file in well-known directories, if they are
+    // specified.
+    if (findFileInDirectories(CfgFileName, CfgFile, Dirs))
+      return SearchResult::Successful;
+
+    // Keep configuration name for error report.
+    CfgFileName.clear();
+    CfgFileName.append(CfgFile.begin(), CfgFile.end());
+    return SearchResult::NotFoundCfg;
+  }
+
+  // Configuration was specified by path.
+  CfgFileName.append(CfgFile.begin(), CfgFile.end());
+
+  assert(!CfgFileName.empty());
+  if (!llvm::sys::fs::is_regular_file(CfgFileName))
+    return SearchResult::NotFoundOpt;
+  return SearchResult::Successful;
+}
+
+
+cl::SearchResult cl::findConfigFileFromEnv(SmallVectorImpl<char> &CfgFileName,
+                                           StringRef VarName) {
+  CfgFileName.clear();
+  if (auto CfgPath = sys::Process::GetEnv(VarName)) {
+    CfgFileName.append(CfgPath->begin(), CfgPath->end());
+    if (!CfgFileName.empty()) {
+      if (!llvm::sys::fs::is_regular_file(CfgFileName))
+        return SearchResult::NotFoundEnv;
+      return SearchResult::Successful;
+    }
+    // Treat empty variable as if it was is not set.
+  }
+  return SearchResult::NotSpecified;
+}
+
+
+cl::SearchResult cl::findDefaultCfgFile(SmallVectorImpl<char> &CfgFileName,
+                                        ArrayRef<const char *> Dirs,
+                                        StringRef ProgramFullPath,
+                                        StringRef FileName) {
+  CfgFileName.clear();
+  if (findFileInDirectories(CfgFileName, FileName, Dirs))
+    return SearchResult::Successful;
+
+  // If not found, try searching the directory where executable resides.
+  CfgFileName.clear();
+  llvm::sys::path::append(CfgFileName,
+                          llvm::sys::path::parent_path(ProgramFullPath),
+                          FileName);
+  if (llvm::sys::fs::is_regular_file(CfgFileName))
+    return SearchResult::Successful;
+  return SearchResult::NotSpecified;
+}
+
+
+bool cl::checkConfigFileSearchResult(SearchResult Res,
+                                     StringRef CfgFile,
+                                     ArrayRef<const char *> Dirs,
+                                     StringRef ProgramFullPath) {
+  switch (Res) {
+  case SearchResult::Successful:
+  case SearchResult::NotSpecified:
+    return false;
+  case SearchResult::NoArgument:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Option '--config' must be followed by "
+      "configuration name or full path to configuration file\n";
+    break;
+  case SearchResult::Multiple:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: More than one option '--config' is specified\n";
+    break;
+  case SearchResult::NotFoundCfg:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Configuration '" << CfgFile << "' specified by "
+      "option '--config' cannot be found in directories:\n";
+    for (const char *Dir : Dirs)
+      errs() << "    " << Dir << "\n";
+    break;
+  case SearchResult::NotFoundOpt:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Configuration file '" << CfgFile << "' specified "
+      "by option '--config' cannot be found\n";
+    break;
+  case SearchResult::NotFoundEnv:
+    errs() << ProgramFullPath <<
+      ": CommandLine Error: Configuration file '" << CfgFile << "' specified "
+      "by environment variable cannot be found\n";
+    break;
+  }
+  return true;
+}
+
+void cl::readConfigFile(SmallVectorImpl<char> &CfgFile, StringSaver &Saver,
+                        SmallVectorImpl<const char *> &Argv) {
+  CfgFile.push_back(0);
+  SmallVector<const char *, 8> NewArgs;
+  ExpandResponseFile(CfgFile.data(), Saver, TokenizeConfigFile, NewArgs);
+  ExpandResponseFiles(Saver, TokenizeConfigFile, NewArgs, false);
+  Argv.insert(Argv.begin() + 1, NewArgs.begin(), NewArgs.end());
+}
+
 /// ParseEnvironmentOptions - An alternative entry point to the
 /// CommandLine library, which allows you to read the program's name
 /// from the caller (as PROGNAME) and its command-line arguments from
 /// an environment variable (whose name is given in ENVVAR).
 ///
 void cl::ParseEnvironmentOptions(const char *progName, const char *envVar,
                                  const char *Overview) {
   // Check args.
void cl::ParseEnvironmentOptions(const char *progName, const char *envVar,
 
   // Parse the value of the environment variable into a "command line"
   // and hand it off to ParseCommandLineOptions().
   TokenizeGNUCommandLine(*envValue, Saver, newArgv);
   int newArgc = static_cast<int>(newArgv.size());
   ParseCommandLineOptions(newArgc, &newArgv[0], Overview);
 }
 
+void cl::TokenizeConfigFile(StringRef Source, StringSaver &Saver,
+                            SmallVectorImpl<const char *> &NewArgv,
+                            bool MarkEOLs) {
+  for (const char *Cur = Source.begin(); Cur != Source.end();) {
+    // Check for comment line.
+    if (isWhitespace(*Cur)) {
+      while (Cur != Source.end() && isWhitespace(*Cur))
+        ++Cur;
+      continue;
+    }
+    if (*Cur == '#') {
+      while (Cur != Source.end() && *Cur != '\n')
+        ++Cur;
+      continue;
+    }
+    // Find end of current line.
+    const char *Start = Cur;
+    for (const char *End = Source.end(); Cur != End; ++Cur) {
+      if (*Cur == '\\') {
+        if (Cur + 1 != End) {
+          ++Cur;
+          if (Cur != End && *Cur == '\r' && (Cur + 1 != End) && Cur[1] == '\n')
+            ++Cur;
+        }
+      } else if (*Cur == '\n')
+        break;
+    }
+    // Tokenize line.
+    StringRef Line(Start, Cur - Start);
+    cl::TokenizeGNUCommandLine(Line, Saver, NewArgv, MarkEOLs);
+  }
+}
+
 bool cl::ParseCommandLineOptions(int argc, const char *const *argv,
                                  const char *Overview, bool IgnoreErrors) {
   return GlobalParser->ParseCommandLineOptions(argc, argv, Overview,
                                                IgnoreErrors);
 }
 
 void CommandLineParser::ResetAllOptionOccurrences() {
   // So that we can parse different command lines multiple times in succession

unittests/Support/CommandLineTest.cpp

 //===- llvm/unittest/Support/CommandLineTest.cpp - CommandLine tests ------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Config/config.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/StringSaver.h"
 #include "gtest/gtest.h"
+#include <fstream>
 #include <stdlib.h>
 #include <string>
 
 using namespace llvm;
 
 namespace {
 
 class TempEnvVar {
TEST(CommandLineTest, TokenizeWindowsCommandLine) {
   const char *Input = "a\\b c\\\\d e\\\\\"f g\" h\\\"i j\\\\\\\"k \"lmn\" o pqr "
                       "\"st \\\"u\" \\v";
   const char *const Output[] = { "a\\b", "c\\\\d", "e\\f g", "h\"i", "j\\\"k",
                                  "lmn", "o", "pqr", "st \"u", "\\v" };
   testCommandLineTokenizer(cl::TokenizeWindowsCommandLine, Input, Output,
                            array_lengthof(Output));
 }
 
+TEST(CommandLineTest, TokenizeConfigFile1) {
+  const char *Input = "\\";
+  const char *const Output[] = { "\\" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile2) {
+  const char *Input = "\\abc";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile3) {
+  const char *Input = "abc\\";
+  const char *const Output[] = { "abc\\" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile4) {
+  const char *Input = "abc\\\n123";
+  const char *const Output[] = { "abc123" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile5) {
+  const char *Input = "abc\\\r\n123";
+  const char *const Output[] = { "abc123" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile6) {
+  const char *Input = "abc\\\n";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile7) {
+  const char *Input = "abc\\\r\n";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile8) {
+  SmallVector<const char *, 0> Actual;
+  BumpPtrAllocator A;
+  StringSaver Saver(A);
+  cl::TokenizeConfigFile("\\\n", Saver, Actual, /*MarkEOLs=*/false);
+  EXPECT_TRUE(Actual.empty());
+}
+
+TEST(CommandLineTest, TokenizeConfigFile9) {
+  SmallVector<const char *, 0> Actual;
+  BumpPtrAllocator A;
+  StringSaver Saver(A);
+  cl::TokenizeConfigFile("\\\r\n", Saver, Actual, /*MarkEOLs=*/false);
+  EXPECT_TRUE(Actual.empty());
+}
+
+TEST(CommandLineTest, TokenizeConfigFile10) {
+  const char *Input = "\\\nabc";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
+TEST(CommandLineTest, TokenizeConfigFile11) {
+  const char *Input = "\\\r\nabc";
+  const char *const Output[] = { "abc" };
+  testCommandLineTokenizer(cl::TokenizeConfigFile, Input, Output,
+                           array_lengthof(Output));
+}
+
 TEST(CommandLineTest, AliasesWithArguments) {
   static const size_t ARGC = 3;
   const char *const Inputs[][ARGC] = {
     { "-tool", "-actual=x", "-extra" },
     { "-tool", "-actual", "x" },
     { "-tool", "-alias=x", "-extra" },
     { "-tool", "-alias", "x" }
   };
TEST(CommandLineTest, RemoveFromAllSubCommands) {
   cl::ResetAllOptionOccurrences();
   EXPECT_FALSE(cl::ParseCommandLineOptions(2, args0, nullptr, true));
   cl::ResetAllOptionOccurrences();
   EXPECT_FALSE(cl::ParseCommandLineOptions(3, args1, nullptr, true));
   cl::ResetAllOptionOccurrences();
   EXPECT_FALSE(cl::ParseCommandLineOptions(3, args2, nullptr, true));
 }
 
+TEST(CommandLineTest, ConfigFileErrors) {
+  llvm::SmallString<128> CfgFileName;
+  llvm::SmallVector<const char *, 1> Argv = { "test/test" };
+  ArrayRef<const char *> Dirs;
+  cl::SearchResult Result;
+
+  // Config in not specified.
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+  EXPECT_TRUE(Result == cl::SearchResult::NotSpecified);
+
+  // Missed argument of --config
+  Argv = { "--config" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+  EXPECT_TRUE(Result == cl::SearchResult::NoArgument);
+
+  // Multiple options --config
+  Argv = { "--config", "a", "--config", "b" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+  EXPECT_TRUE(Result == cl::SearchResult::Multiple);
+
+  // Inexistent file specified by full path.
+  Argv = { "--config", "/abcd.cfg" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundOpt);
+  EXPECT_TRUE(Argv.empty());
+  EXPECT_TRUE(CfgFileName.equals("/abcd.cfg"));
+
+  // Inexistent configuration.
+  Argv = { "--config", "abcd.cfg" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundCfg);
+  EXPECT_TRUE(Argv.empty());
+  EXPECT_TRUE(CfgFileName.equals("abcd.cfg"));
+
+  // Inexistent configuration, configuration without extension.
+  Argv = { "--config", "abcd" };
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+  EXPECT_TRUE(Result == cl::SearchResult::NotFoundCfg);
+  EXPECT_TRUE(Argv.empty());
+  EXPECT_TRUE(CfgFileName.equals("abcd.cfg"));
+}
+
+TEST(CommandLineTest, ReadConfigFile) {
+  llvm::SmallString<128> CfgFileName;
+  llvm::SmallVector<const char *, 1> Argv;
+  const char *ProgramFullPath = "/test/test";
+  ArrayRef<const char *> Dirs;
+  cl::SearchResult Result;
+
+  llvm::SmallString<128> TestCfg;
+  std::error_code EC =
+    llvm::sys::fs::createTemporaryFile("unittest", "cfg", TestCfg);
+  EXPECT_TRUE(!EC);
+  std::string Directory = llvm::sys::path::parent_path(TestCfg);
+  std::string FileName = llvm::sys::path::filename(TestCfg);
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  std::ofstream Cfg(TestCfg.c_str());
+  Cfg << "# Comment\n";
+  Cfg << "-option_1 -option_2\n";
+  Cfg << "-option_3=abcd\n";
+  Cfg.close();
+
+  Argv = { ProgramFullPath, "-flag_1", "--config", FileName.c_str() };
+  Dirs = ArrayRef<const char *>(Directory.c_str());
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_TRUE(CfgFileName.equals(TestCfg));
+  EXPECT_EQ(Argv.size(), 2);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-flag_1");
+
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv);
+
+  EXPECT_EQ(Argv.size(), 5);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-option_1");
+  EXPECT_STREQ(Argv[2], "-option_2");
+  EXPECT_STREQ(Argv[3], "-option_3=abcd");
+  EXPECT_STREQ(Argv[4], "-flag_1");
+
+  // Reading from file with lines concatenated by trailing \.
+  Cfg.open(TestCfg.c_str(), std::ofstream::trunc);
+  EXPECT_TRUE(Cfg.is_open());
+
+  Cfg << "\n\n# Comment\n";
+  Cfg << "-option_\\\n";
+  Cfg << "1 -option_3=abcd\n";
+  Cfg.close();
+
+  Argv = { ProgramFullPath, "-flag_1", "--config", FileName.c_str(),
+           "-flag_2" };
+  Dirs = ArrayRef<const char *>(Directory.c_str());
+  Result = cl::findConfigFileFromArgs(CfgFileName, Argv, Dirs);
+
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_TRUE(CfgFileName.equals(TestCfg));
+  EXPECT_EQ(Argv.size(), 3);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-flag_1");
+  EXPECT_STREQ(Argv[2], "-flag_2");
+
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv);
+
+  EXPECT_EQ(Argv.size(), 5);
+  EXPECT_STREQ(Argv[0], ProgramFullPath);
+  EXPECT_STREQ(Argv[1], "-option_1");
+  EXPECT_STREQ(Argv[2], "-option_3=abcd");
+  EXPECT_STREQ(Argv[3], "-flag_1");
+  EXPECT_STREQ(Argv[4], "-flag_2");
+
+  llvm::sys::fs::remove(TestCfg);
+}
+
+TEST(CommandLineTest, ReadDefaultConfigFile) {
+  llvm::SmallString<128> TestDir;
+  std::error_code EC =
+      llvm::sys::fs::createUniqueDirectory("unittest", TestDir);
+  EXPECT_TRUE(!EC);
+
+  llvm::SmallString<128> TestTool;
+  llvm::sys::path::append(TestTool, TestDir, "testtool");
+  std::ofstream Tool(TestTool.c_str());
+  EXPECT_TRUE(Tool.is_open());
+  Tool << std::endl;
+  Tool.close();
+
+  llvm::SmallString<128> TestCfg;
+  llvm::sys::path::append(TestCfg, TestDir, "test.cfg");
+  std::ofstream Cfg(TestCfg.c_str());
+  EXPECT_TRUE(Cfg.is_open());
+  Cfg << "-option_1 -option_2\n"
+         "-option_3=abcd\n";
+  Cfg.close();
+
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  llvm::SmallVector<const char *, 1> Argv = { TestTool.c_str(), "-flag_1" };
+  llvm::SmallString<128> CfgFileName;
+  cl::SearchResult Result = cl::findDefaultCfgFile(CfgFileName,
+                                 ArrayRef<const char *>(), Argv[0], "test.cfg");
+
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_STREQ(CfgFileName.c_str(), TestCfg.c_str());
+
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv);
+
+  EXPECT_EQ(Argv.size(), 5);
+  EXPECT_STREQ(Argv[0], TestTool.c_str());
+  EXPECT_STREQ(Argv[1], "-option_1");
+  EXPECT_STREQ(Argv[2], "-option_2");
+  EXPECT_STREQ(Argv[3], "-option_3=abcd");
+  EXPECT_STREQ(Argv[4], "-flag_1");
+
+  llvm::sys::fs::remove(TestCfg);
+  llvm::sys::fs::remove(TestTool);
+  llvm::sys::fs::remove(TestDir);
+}
+
+TEST(CommandLineTest, ReadDefaultConfigFileFromStdDir) {
+  llvm::SmallString<128> TestDir;
+  std::error_code EC =
+    llvm::sys::fs::createUniqueDirectory("unittest", TestDir);
+  EXPECT_TRUE(!EC);
+
+  llvm::SmallString<128> TestTool;
+  llvm::sys::path::append(TestTool, TestDir, "testtool");
+  std::ofstream Tool(TestTool.c_str());
+  EXPECT_TRUE(Tool.is_open());
+  Tool << std::endl;
+  Tool.close();
+
+  llvm::SmallString<128> TestCfgDir;
+  llvm::sys::path::append(TestCfgDir, TestDir, "stddir");
+  EC = llvm::sys::fs::create_directory(TestCfgDir);
+  EXPECT_TRUE(!EC);
+
+  llvm::SmallString<128> TestCfg;
+  llvm::sys::path::append(TestCfg, TestCfgDir, "test.cfg");
+  std::ofstream Cfg(TestCfg.c_str());
+  EXPECT_TRUE(Cfg.is_open());
+  Cfg << "-option_3=abcd\n";
+  Cfg.close();
+
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver(A);
+
+  llvm::SmallVector<const char *, 1> Argv = { TestTool.c_str(), "-flag_1" };
+  ArrayRef<const char *> Dirs = { TestCfgDir.c_str() };
+  llvm::SmallString<128> CfgFileName;
+  cl::SearchResult Result = cl::findDefaultCfgFile(CfgFileName, Dirs, Argv[0],
+                                                   "test.cfg");
+
+  EXPECT_TRUE(Result == cl::SearchResult::Successful);
+  EXPECT_STREQ(CfgFileName.c_str(), TestCfg.c_str());
+
+  llvm::cl::readConfigFile(CfgFileName, Saver, Argv);
+
+  EXPECT_EQ(Argv.size(), 3);
+  EXPECT_STREQ(Argv[0], TestTool.c_str());
+  EXPECT_STREQ(Argv[1], "-option_3=abcd");
+  EXPECT_STREQ(Argv[2], "-flag_1");
+
+  llvm::sys::fs::remove(TestCfg);
+  llvm::sys::fs::remove(TestCfgDir);
+  llvm::sys::fs::remove(TestTool);
+  llvm::sys::fs::remove(TestDir);
+}
+
 }  // anonymous namespace