[Support] Class for response file expansion

[sepavloff]

Summary

Functions that implement expansion of response and config files depend
on many options, which are passes as arguments. Extending the expansion
requires new options, it in turn causes changing calls in various places
making them even more bulky.

This change introduces a class ExpansionContext, which represents set of
options that control the expansion. Its methods implements expansion of
responce files including config files. It makes extending the expansion
easier.

No functional changes.

[thakis]

Can you say what you're trying to do?

[sepavloff]

In D132379#3740102, @thakis wrote:

Can you say what you're trying to do?

There is an intention to extend algorithm of included files search. In particular it requires passing down paths where config files may be searched for. It could solve the problem discussed in https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606, there are also other users that need such feature.

If config files are used to tune compiler for a platform which has variants, it is convenient to extract common platform settings into separate config file, which could be included by config files of particular variants. Now such solution does not work because included config files are searched differently than the file in --config option.

[sepavloff]

Move change for VFS support into separate patch

[sepavloff]

Rebased

[rnk]

This makes sense to me.

This probably affects nobody, but this reminds me of my first LLVM change: https://github.com/llvm/llvm-project/commit/fc8a2d5a8390952029e1c47a623e046b744f44d4

llvm/include/llvm/Support/CommandLine.h

2099

StringSaver is a stateless class that wraps a BumpPtrAllocator. To simplify the call site, I suggest making this parameter into a BumpPtrAllocator &, and construct a field StringSaver from the allocator.

2100

It feels like you are using the builder pattern to handle setting uncommon options. If that's what we're doing, should we commit completely and add a .setVFS() helper and remove the optional parameter?

[sepavloff]

Address reviewer's notes

• Pass BumpPtrAllocator instead of StringSaver,
• Use setter to specify file system object.

[sepavloff]

llvm/include/llvm/Support/CommandLine.h

2099

Indeed in many cases StringSaver in caller can be omitted if the parameter is BumpPtrAllocator.

2100

You are right. Changed.

[rnk]

lgtm

[sepavloff]

Thanks!

clang/lib/Driver/Driver.cpp

static void appendOneArg(InputArgList &Args, const Arg *Opt,
   Copy->setOwnsValues(Opt->getOwnsValues());
   Opt->setOwnsValues(false);
   Args.append(Copy);
 }
 
 bool Driver::readConfigFile(StringRef FileName) {
   // Try reading the given file.
   SmallVector<const char *, 32> NewCfgArgs;
-  if (!llvm::cl::readConfigFile(FileName, Saver, NewCfgArgs, getVFS())) {
+  llvm::cl::ExpansionContext ExpCtx(Alloc, llvm::cl::tokenizeConfigFile);
+  ExpCtx.setVFS(&getVFS());
+  if (!ExpCtx.readConfigFile(FileName, NewCfgArgs)) {
     Diag(diag::err_drv_cannot_read_config_file) << FileName;
     return true;
   }
 
   // Read options from config file.
   llvm::SmallString<128> CfgFileName(FileName);
   llvm::sys::path::native(CfgFileName);
   bool ContainErrors;

clang/lib/Tooling/ExpandResponseFilesCompilationDatabase.cpp

for (auto &Cmd : Cmds) {
       for (auto &Arg : Cmd.CommandLine) {
         Argv.push_back(Arg.c_str());
         if (!Arg.empty())
           SeenRSPFile |= Arg.front() == '@';
       }
       if (!SeenRSPFile)
         continue;
       llvm::BumpPtrAllocator Alloc;
-      llvm::StringSaver Saver(Alloc);
-      llvm::cl::ExpandResponseFiles(Saver, Tokenizer, Argv, false, false, false,
-                                    llvm::StringRef(Cmd.Directory), *FS);
+      llvm::cl::ExpansionContext ECtx(Alloc, Tokenizer);
+      ECtx.setVFS(FS.get())
+          .setCurrentDir(Cmd.Directory)
+          .expandResponseFiles(Argv);
       // Don't assign directly, Argv aliases CommandLine.
       std::vector<std::string> ExpandedArgv(Argv.begin(), Argv.end());
       Cmd.CommandLine = std::move(ExpandedArgv);
     }
     return Cmds;
   }
 
 private:

clang/tools/driver/driver.cpp

 static int ExecuteCC1Tool(SmallVectorImpl<const char *> &ArgV) {
   // If we call the cc1 tool from the clangDriver library (through
   // Driver::CC1Main), we need to clean up the options usage count. The options
   // are currently global, and they might have been used previously by the
   // driver.
   llvm::cl::ResetAllOptionOccurrences();
 
   llvm::BumpPtrAllocator A;
-  llvm::StringSaver Saver(A);
-  llvm::cl::ExpandResponseFiles(Saver, &llvm::cl::TokenizeGNUCommandLine, ArgV,
-                                /*MarkEOLs=*/false);
+  llvm::cl::ExpansionContext ECtx(A, llvm::cl::TokenizeGNUCommandLine);
+  ECtx.expandResponseFiles(ArgV);
   StringRef Tool = ArgV[1];
   void *GetExecutablePathVP = (void *)(intptr_t)GetExecutablePath;
   if (Tool == "-cc1")
     return cc1_main(makeArrayRef(ArgV).slice(1), ArgV[0], GetExecutablePathVP);
   if (Tool == "-cc1as")
     return cc1as_main(makeArrayRef(ArgV).slice(2), ArgV[0],
                       GetExecutablePathVP);
   if (Tool == "-cc1gen-reproducer")
int clang_main(int Argc, char **Argv) {
   llvm::cl::TokenizerCallback Tokenizer;
   if (RSPQuoting == Windows || (RSPQuoting == Default && ClangCLMode))
     Tokenizer = &llvm::cl::TokenizeWindowsCommandLine;
   else
     Tokenizer = &llvm::cl::TokenizeGNUCommandLine;
 
   if (MarkEOLs && Args.size() > 1 && StringRef(Args[1]).startswith("-cc1"))
     MarkEOLs = false;
-  llvm::cl::ExpandResponseFiles(Saver, Tokenizer, Args, MarkEOLs);
+  llvm::cl::ExpansionContext ECtx(A, Tokenizer);
+  ECtx.setMarkEOLs(MarkEOLs).expandResponseFiles(Args);
 
   // Handle -cc1 integrated tools, even if -cc1 was expanded from a response
   // file.
   auto FirstArg = llvm::find_if(llvm::drop_begin(Args),
                                 [](const char *A) { return A != nullptr; });
   if (FirstArg != Args.end() && StringRef(*FirstArg).startswith("-cc1")) {
     // If -cc1 came from a response file, remove the EOL sentinels.
     if (MarkEOLs) {

llvm/include/llvm/Support/CommandLine.h

 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/StringSaver.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cassert>
 #include <climits>
 #include <cstddef>
 #include <functional>
 #include <initializer_list>
 #include <string>
 #include <type_traits>

 /// \param [in] MarkEOLs Added for compatibility with TokenizerCallback.
 ///
 /// It works like TokenizeGNUCommandLine with ability to skip comment lines.
 ///
 void tokenizeConfigFile(StringRef Source, StringSaver &Saver,
                         SmallVectorImpl<const char *> &NewArgv,
                         bool MarkEOLs = false);
 
+/// Contains options that control response file expansion.
+class ExpansionContext {
+  /// Provides persistent storage for parsed strings.
+  StringSaver Saver;
+
+  /// Tokenization strategy. Typically Unix or Windows.
+  TokenizerCallback Tokenizer;
+
+  /// File system used for all file access when running the expansion.
+  vfs::FileSystem *FS;
+
+  /// Path used to resolve relative rsp files. If empty, the file system
+  /// current directory is used instead.
+  StringRef CurrentDir;
+
+  /// True if names of nested response files must be resolved relative to
+  /// including file.
+  bool RelativeNames = false;
+
+  /// If true, mark end of lines and the end of the response file with nullptrs
+  /// in the Argv vector.
+  bool MarkEOLs = false;
+
+  /// If true, body of config file is expanded.
+  bool InConfigFile = false;
+
+  llvm::Error expandResponseFile(StringRef FName,
+                                 SmallVectorImpl<const char *> &NewArgv);
+
+public:
+  ExpansionContext(BumpPtrAllocator &A, TokenizerCallback T);
+
+  ExpansionContext &setMarkEOLs(bool X) {
+    MarkEOLs = X;
+    return *this;
+  }
+
+  ExpansionContext &setRelativeNames(bool X) {
+    RelativeNames = X;
+    return *this;
+  }
+
+  ExpansionContext &setCurrentDir(StringRef X) {
+    CurrentDir = X;
+    return *this;
+  }
+
+  ExpansionContext &setVFS(vfs::FileSystem *X) {
+    FS = X;
+    return *this;
+  }
+
   /// Reads command line options from the given configuration file.
   ///
-/// \param [in] CfgFileName Path to configuration file.
-/// \param [in] Saver  Objects that saves allocated strings.
+  /// \param [in] CfgFile Path to configuration file.
   /// \param [out] Argv Array to which the read options are added.
   /// \return true if the file was successfully read.
   ///
   /// It reads content of the specified file, tokenizes it and expands "@file"
   /// commands resolving file names in them relative to the directory where
   /// CfgFilename resides. It also expands "<CFGDIR>" to the base path of the
   /// current config file.
-///
-bool readConfigFile(StringRef CfgFileName, StringSaver &Saver,
-                    SmallVectorImpl<const char *> &Argv,
-                    llvm::vfs::FileSystem &FS);
-
-/// Expand response files on a command line recursively using the given
-/// StringSaver and tokenization strategy.  Argv should contain the command line
-/// before expansion and will be modified in place. If requested, Argv will
-/// also be populated with nullptrs indicating where each response file line
-/// ends, which is useful for the "/link" argument that needs to consume all
-/// remaining arguments only until the next end of line, when in a response
-/// file.
-///
-/// \param [in] Saver Delegates back to the caller for saving parsed strings.
-/// \param [in] Tokenizer Tokenization strategy. Typically Unix or Windows.
-/// \param [in,out] Argv Command line into which to expand response files.
-/// \param [in] MarkEOLs Mark end of lines and the end of the response file
-/// with nullptrs in the Argv vector.
-/// \param [in] RelativeNames true if names of nested response files must be
-/// resolved relative to including file.
-/// \param [in] ExpandBasePath If true, "<CFGDIR>" expands to the base path of
-/// the current response file.
-/// \param [in] FS File system used for all file access when running the tool.
-/// \param [in] CurrentDir Path used to resolve relative rsp files. If set to
-/// None, the file system current directory is used instead.
+  bool readConfigFile(StringRef CfgFile, SmallVectorImpl<const char *> &Argv);
+
+  /// Expands constructs "@file" in the provided array of arguments recursively.
+  bool expandResponseFiles(SmallVectorImpl<const char *> &Argv);
+};
+
+/// A convenience helper which concatenates the options specified by the
+/// environment variable EnvVar and command line options, then expands
+/// response files recursively.
 /// \return true if all @files were expanded successfully or there were none.
-bool ExpandResponseFiles(StringSaver &Saver, TokenizerCallback Tokenizer,
-                         SmallVectorImpl<const char *> &Argv, bool MarkEOLs,
-                         bool RelativeNames, bool ExpandBasePath,
-                         llvm::Optional<llvm::StringRef> CurrentDir,
-                         llvm::vfs::FileSystem &FS);
-
-/// An overload of ExpandResponseFiles() that uses
-/// llvm::vfs::getRealFileSystem().
-bool ExpandResponseFiles(
-    StringSaver &Saver, TokenizerCallback Tokenizer,
-    SmallVectorImpl<const char *> &Argv, bool MarkEOLs = false,
-    bool RelativeNames = false, bool ExpandBasePath = false,
-    llvm::Optional<llvm::StringRef> CurrentDir = llvm::None);
+bool expandResponseFiles(int Argc, const char *const *Argv, const char *EnvVar,
+                         SmallVectorImpl<const char *> &NewArgv);
+
+/// A convenience helper which supports the typical use case of expansion
+/// function call.
+inline bool ExpandResponseFiles(StringSaver &Saver, TokenizerCallback Tokenizer,
+                                SmallVectorImpl<const char *> &Argv) {
+  ExpansionContext ECtx(Saver.getAllocator(), Tokenizer);
+  return ECtx.expandResponseFiles(Argv);
+}
 
 /// A convenience helper which concatenates the options specified by the
 /// environment variable EnvVar and command line options, then expands response
 /// files recursively. The tokenizer is a predefined GNU or Windows one.
 /// \return true if all @files were expanded successfully or there were none.
 bool expandResponseFiles(int Argc, const char *const *Argv, const char *EnvVar,
                          StringSaver &Saver,
                          SmallVectorImpl<const char *> &NewArgv);

llvm/include/llvm/Support/StringSaver.h

 /// Saves strings in the provided stable storage and returns a
 /// StringRef with a stable character pointer.
 class StringSaver final {
   BumpPtrAllocator &Alloc;
 
 public:
   StringSaver(BumpPtrAllocator &Alloc) : Alloc(Alloc) {}
 
+  BumpPtrAllocator &getAllocator() const { return Alloc; }
+
   // All returned strings are null-terminated: *save(S).end() == 0.
   StringRef save(const char *S) { return save(StringRef(S)); }
   StringRef save(StringRef S);
   StringRef save(const Twine &S) { return save(StringRef(S.str())); }
   StringRef save(const std::string &S) { return save(StringRef(S)); }
 };
 
 /// Saves strings in the provided stable storage and returns a StringRef with a

llvm/lib/Support/CommandLine.cpp

if (!ResponseFile.empty()) {
     const StringRef Remaining = ArgString.substr(StartPos);
     if (!Remaining.empty())
       llvm::sys::path::append(ResponseFile, Remaining);
     Arg = Saver.save(ResponseFile.str()).data();
   }
 }
 
 // FName must be an absolute path.
-static llvm::Error ExpandResponseFile(StringRef FName, StringSaver &Saver,
-                                      TokenizerCallback Tokenizer,
-                                      SmallVectorImpl<const char *> &NewArgv,
-                                      bool MarkEOLs, bool RelativeNames,
-                                      bool ExpandBasePath,
-                                      llvm::vfs::FileSystem &FS) {
+llvm::Error
+ExpansionContext::expandResponseFile(StringRef FName,
+                                     SmallVectorImpl<const char *> &NewArgv) {
   assert(sys::path::is_absolute(FName));
   llvm::ErrorOr<std::unique_ptr<MemoryBuffer>> MemBufOrErr =
-      FS.getBufferForFile(FName);
+      FS->getBufferForFile(FName);
   if (!MemBufOrErr)
     return llvm::errorCodeToError(MemBufOrErr.getError());
   MemoryBuffer &MemBuf = *MemBufOrErr.get();
   StringRef Str(MemBuf.getBufferStart(), MemBuf.getBufferSize());
 
   // If we have a UTF-16 byte order mark, convert to UTF-8 for parsing.
   ArrayRef<char> BufRef(MemBuf.getBufferStart(), MemBuf.getBufferEnd());
   std::string UTF8Buf;
ExpansionContext::expandResponseFile(StringRef FName,
   // If names of nested response files should be resolved relative to including
   // file, replace the included response file names with their full paths
   // obtained by required resolution.
   for (auto &Arg : NewArgv) {
     if (!Arg)
       continue;
 
     // Substitute <CFGDIR> with the file's base path.
-    if (ExpandBasePath)
+    if (InConfigFile)
       ExpandBasePaths(BasePath, Saver, Arg);
 
     // Skip non-rsp file arguments.
     if (Arg[0] != '@')
       continue;
 
     StringRef FileName(Arg + 1);
     // Skip if non-relative.
     if (!llvm::sys::path::is_relative(FileName))
       continue;
 
     SmallString<128> ResponseFile;
     ResponseFile.push_back('@');
     ResponseFile.append(BasePath);
     llvm::sys::path::append(ResponseFile, FileName);
     Arg = Saver.save(ResponseFile.str()).data();
   }
   return Error::success();
 }
 
 /// Expand response files on a command line recursively using the given
 /// StringSaver and tokenization strategy.
-bool cl::ExpandResponseFiles(StringSaver &Saver, TokenizerCallback Tokenizer,
-                             SmallVectorImpl<const char *> &Argv, bool MarkEOLs,
-                             bool RelativeNames, bool ExpandBasePath,
-                             llvm::Optional<llvm::StringRef> CurrentDir,
-                             llvm::vfs::FileSystem &FS) {
+bool ExpansionContext::expandResponseFiles(
+    SmallVectorImpl<const char *> &Argv) {
   bool AllExpanded = true;
   struct ResponseFileRecord {
     std::string File;
     size_t End;
   };
 
   // To detect recursive response files, we maintain a stack of files and the
   // position of the last argument in the file. This position is updated
if (Arg[0] != '@') {
       continue;
     }
 
     const char *FName = Arg + 1;
     // Note that CurrentDir is only used for top-level rsp files, the rest will
     // always have an absolute path deduced from the containing file.
     SmallString<128> CurrDir;
     if (llvm::sys::path::is_relative(FName)) {
-      if (!CurrentDir) {
-        if (auto CWD = FS.getCurrentWorkingDirectory()) {
+      if (CurrentDir.empty()) {
+        if (auto CWD = FS->getCurrentWorkingDirectory()) {
           CurrDir = *CWD;
         } else {
           // TODO: The error should be propagated up the stack.
           llvm::consumeError(llvm::errorCodeToError(CWD.getError()));
           return false;
         }
       } else {
-        CurrDir = *CurrentDir;
+        CurrDir = CurrentDir;
       }
       llvm::sys::path::append(CurrDir, FName);
       FName = CurrDir.c_str();
     }
-    auto IsEquivalent = [FName, &FS](const ResponseFileRecord &RFile) {
-      llvm::ErrorOr<llvm::vfs::Status> LHS = FS.status(FName);
+    auto IsEquivalent = [FName, this](const ResponseFileRecord &RFile) {
+      llvm::ErrorOr<llvm::vfs::Status> LHS = FS->status(FName);
       if (!LHS) {
         // TODO: The error should be propagated up the stack.
         llvm::consumeError(llvm::errorCodeToError(LHS.getError()));
         return false;
       }
-      llvm::ErrorOr<llvm::vfs::Status> RHS = FS.status(RFile.File);
+      llvm::ErrorOr<llvm::vfs::Status> RHS = FS->status(RFile.File);
       if (!RHS) {
         // TODO: The error should be propagated up the stack.
         llvm::consumeError(llvm::errorCodeToError(RHS.getError()));
         return false;
       }
       return LHS->equivalent(*RHS);
     };
 
     // Check for recursive response files.
     if (any_of(drop_begin(FileStack), IsEquivalent)) {
       // This file is recursive, so we leave it in the argument stream and
       // move on.
       AllExpanded = false;
       ++I;
       continue;
     }
 
     // Replace this response file argument with the tokenization of its
     // contents.  Nested response files are expanded in subsequent iterations.
     SmallVector<const char *, 0> ExpandedArgv;
-    if (llvm::Error Err =
-            ExpandResponseFile(FName, Saver, Tokenizer, ExpandedArgv, MarkEOLs,
-                               RelativeNames, ExpandBasePath, FS)) {
+    if (llvm::Error Err = expandResponseFile(FName, ExpandedArgv)) {
       // We couldn't read this file, so we leave it in the argument stream and
       // move on.
       // TODO: The error should be propagated up the stack.
       llvm::consumeError(std::move(Err));
       AllExpanded = false;
       ++I;
       continue;
     }
bool ExpansionContext::expandResponseFiles(
   // stream. A failure here indicates a bug in the stack popping logic above.
   // Note that FileStack may have more than one element at this point because we
   // don't have a chance to pop the stack when encountering recursive files at
   // the end of the stream, so seeing that doesn't indicate a bug.
   assert(FileStack.size() > 0 && Argv.size() == FileStack.back().End);
   return AllExpanded;
 }
 
-bool cl::ExpandResponseFiles(StringSaver &Saver, TokenizerCallback Tokenizer,
-                             SmallVectorImpl<const char *> &Argv, bool MarkEOLs,
-                             bool RelativeNames, bool ExpandBasePath,
-                             llvm::Optional<StringRef> CurrentDir) {
-  return ExpandResponseFiles(Saver, std::move(Tokenizer), Argv, MarkEOLs,
-                             RelativeNames, ExpandBasePath,
-                             std::move(CurrentDir), *vfs::getRealFileSystem());
-}
-
 bool cl::expandResponseFiles(int Argc, const char *const *Argv,
                              const char *EnvVar, StringSaver &Saver,
                              SmallVectorImpl<const char *> &NewArgv) {
   auto Tokenize = Triple(sys::getProcessTriple()).isOSWindows()
                       ? cl::TokenizeWindowsCommandLine
                       : cl::TokenizeGNUCommandLine;
   // The environment variable specifies initial options.
   if (EnvVar)
     if (llvm::Optional<std::string> EnvValue = sys::Process::GetEnv(EnvVar))
       Tokenize(*EnvValue, Saver, NewArgv, /*MarkEOLs=*/false);
 
   // Command line options can override the environment variable.
   NewArgv.append(Argv + 1, Argv + Argc);
-  return ExpandResponseFiles(Saver, Tokenize, NewArgv);
+  ExpansionContext ECtx(Saver.getAllocator(), Tokenize);
+  return ECtx.expandResponseFiles(NewArgv);
 }
 
-bool cl::readConfigFile(StringRef CfgFile, StringSaver &Saver,
-                        SmallVectorImpl<const char *> &Argv,
-                        llvm::vfs::FileSystem &FS) {
+ExpansionContext::ExpansionContext(BumpPtrAllocator &A, TokenizerCallback T)
+    : Saver(A), Tokenizer(T), FS(vfs::getRealFileSystem().get()) {}
+
+bool ExpansionContext::readConfigFile(StringRef CfgFile,
+                                      SmallVectorImpl<const char *> &Argv) {
   SmallString<128> AbsPath;
   if (sys::path::is_relative(CfgFile)) {
     AbsPath.assign(CfgFile);
-    if (std::error_code EC = FS.makeAbsolute(AbsPath))
+    if (std::error_code EC = FS->makeAbsolute(AbsPath))
       return false;
     CfgFile = AbsPath.str();
   }
-  if (llvm::Error Err =
-          ExpandResponseFile(CfgFile, Saver, cl::tokenizeConfigFile, Argv,
-                             /*MarkEOLs=*/false, /*RelativeNames=*/true,
-                             /*ExpandBasePath=*/true, FS)) {
+  InConfigFile = true;
+  RelativeNames = true;
+  if (llvm::Error Err = expandResponseFile(CfgFile, Argv)) {
     // TODO: The error should be propagated up the stack.
     llvm::consumeError(std::move(Err));
     return false;
   }
-  return ExpandResponseFiles(Saver, cl::tokenizeConfigFile, Argv,
-                             /*MarkEOLs=*/false, /*RelativeNames=*/true,
-                             /*ExpandBasePath=*/true, llvm::None, FS);
+  return expandResponseFiles(Argv);
 }
 
 static void initCommonOptions();
 bool cl::ParseCommandLineOptions(int argc, const char *const *argv,
                                  StringRef Overview, raw_ostream *Errs,
                                  const char *EnvVar,
                                  bool LongOptionsUseDoubleDash) {
   initCommonOptions();
bool CommandLineParser::ParseCommandLineOptions(int argc,
                                                 StringRef Overview,
                                                 raw_ostream *Errs,
                                                 bool LongOptionsUseDoubleDash) {
   assert(hasOptions() && "No options specified!");
 
   // Expand response files.
   SmallVector<const char *, 20> newArgv(argv, argv + argc);
   BumpPtrAllocator A;
-  StringSaver Saver(A);
-  ExpandResponseFiles(Saver,
-         Triple(sys::getProcessTriple()).isOSWindows() ?
-         cl::TokenizeWindowsCommandLine : cl::TokenizeGNUCommandLine,
-         newArgv);
+  ExpansionContext ECtx(A, Triple(sys::getProcessTriple()).isOSWindows()
+                               ? cl::TokenizeWindowsCommandLine
+                               : cl::TokenizeGNUCommandLine);
+  ECtx.expandResponseFiles(newArgv);
   argv = &newArgv[0];
   argc = static_cast<int>(newArgv.size());
 
   // Copy the program name into ProgName, making sure not to overflow it.
   ProgramName = std::string(sys::path::filename(StringRef(argv[0])));
 
   ProgramOverview = Overview;
   bool IgnoreErrors = Errs;

llvm/unittests/Support/CommandLineTest.cpp

#endif
   SmallString<128> IncRef;
   IncRef.append(1, '@');
   IncRef.append(IncludedFileName);
   llvm::SmallVector<const char *, 4> Argv = {"test/test", "-flag_1",
                                              IncRef.c_str(), "-flag_2"};
 
   // Expand response files.
   llvm::BumpPtrAllocator A;
-  llvm::StringSaver Saver(A);
-  ASSERT_TRUE(llvm::cl::ExpandResponseFiles(
-      Saver, llvm::cl::TokenizeGNUCommandLine, Argv, false, true, false,
-      /*CurrentDir=*/StringRef(TestRoot), FS));
+  llvm::cl::ExpansionContext ECtx(A, llvm::cl::TokenizeGNUCommandLine);
+  ECtx.setVFS(&FS).setCurrentDir(TestRoot).setRelativeNames(true);
+  ASSERT_TRUE(ECtx.expandResponseFiles(Argv));
   EXPECT_THAT(Argv, testing::Pointwise(
                         StringEquality(),
                         {"test/test", "-flag_1", "-option_1", "-option_2",
                          "-option_21", "-option_22", "-option_23=abcd",
                          "-option_3=abcd", "-option_31", "-option_32",
                          "-option_33=abcd", "-option_4=efjk", "-flag_2"}));
 }
 
#endif
 
   // Ensure:
   // Recursive expansion terminates
   // Recursive files never expand
   // Non-recursive repeats are allowed
   SmallVector<const char *, 4> Argv = {"test/test", SelfFileRef.c_str(),
                                        "-option_3"};
   BumpPtrAllocator A;
-  StringSaver Saver(A);
 #ifdef _WIN32
   cl::TokenizerCallback Tokenizer = cl::TokenizeWindowsCommandLine;
 #else
   cl::TokenizerCallback Tokenizer = cl::TokenizeGNUCommandLine;
 #endif
-  ASSERT_FALSE(
-      cl::ExpandResponseFiles(Saver, Tokenizer, Argv, false, false, false,
-                              /*CurrentDir=*/llvm::StringRef(TestRoot), FS));
+  llvm::cl::ExpansionContext ECtx(A, Tokenizer);
+  ECtx.setVFS(&FS).setCurrentDir(TestRoot);
+  ASSERT_FALSE(ECtx.expandResponseFiles(Argv));
 
   EXPECT_THAT(Argv,
               testing::Pointwise(StringEquality(),
                                  {"test/test", "-option_1", "-option_x",
                                   "-option_2", "-option_x", SelfFileRef.c_str(),
                                   NestedFileRef.c_str(), SelfFileRef.c_str(),
                                   "-option_3"}));
 }
#endif
   // Ensure we expand rsp files after lots of non-rsp arguments starting with @.
   constexpr size_t NON_RSP_AT_ARGS = 64;
   SmallVector<const char *, 4> Argv = {"test/test"};
   Argv.append(NON_RSP_AT_ARGS, "@non_rsp_at_arg");
   std::string ResponseFileRef = ("@" + ResponseFilePath).str();
   Argv.push_back(ResponseFileRef.c_str());
 
   BumpPtrAllocator A;
-  StringSaver Saver(A);
-  ASSERT_FALSE(cl::ExpandResponseFiles(Saver, cl::TokenizeGNUCommandLine, Argv,
-                                       false, false, false,
-                                       /*CurrentDir=*/StringRef(TestRoot), FS));
+  llvm::cl::ExpansionContext ECtx(A, cl::TokenizeGNUCommandLine);
+  ECtx.setVFS(&FS).setCurrentDir(TestRoot);
+  ASSERT_FALSE(ECtx.expandResponseFiles(Argv));
 
   // ASSERT instead of EXPECT to prevent potential out-of-bounds access.
   ASSERT_EQ(Argv.size(), 1 + NON_RSP_AT_ARGS + 2);
   size_t i = 0;
   EXPECT_STREQ(Argv[i++], "test/test");
   for (; i < 1 + NON_RSP_AT_ARGS; ++i)
     EXPECT_STREQ(Argv[i], "@non_rsp_at_arg");
   EXPECT_STREQ(Argv[i++], "-foo");
#endif
 
   StringRef InnerFile = "dir/inner.rsp";
   StringRef InnerFileContents = "-flag";
   FS.addFile(InnerFile, 0, MemoryBuffer::getMemBuffer(InnerFileContents));
 
   SmallVector<const char *, 2> Argv = {"test/test", "@dir/outer.rsp"};
 
   BumpPtrAllocator A;
-  StringSaver Saver(A);
-  ASSERT_TRUE(cl::ExpandResponseFiles(Saver, cl::TokenizeGNUCommandLine, Argv,
-                                      false, true, false,
-                                      /*CurrentDir=*/StringRef(TestRoot), FS));
+  llvm::cl::ExpansionContext ECtx(A, cl::TokenizeGNUCommandLine);
+  ECtx.setVFS(&FS).setCurrentDir(TestRoot).setRelativeNames(true);
+  ASSERT_TRUE(ECtx.expandResponseFiles(Argv));
   EXPECT_THAT(Argv,
               testing::Pointwise(StringEquality(), {"test/test", "-flag"}));
 }
 
 TEST(CommandLineTest, ResponseFileEOLs) {
   vfs::InMemoryFileSystem FS;
 #ifdef _WIN32
   const char *TestRoot = "C:\\";
 #else
   const char *TestRoot = "//net";
 #endif
   FS.setCurrentWorkingDirectory(TestRoot);
   FS.addFile("eols.rsp", 0,
              MemoryBuffer::getMemBuffer("-Xclang -Wno-whatever\n input.cpp"));
   SmallVector<const char *, 2> Argv = {"clang", "@eols.rsp"};
   BumpPtrAllocator A;
-  StringSaver Saver(A);
-  ASSERT_TRUE(cl::ExpandResponseFiles(Saver, cl::TokenizeWindowsCommandLine,
-                                      Argv, true, true, false,
-                                      /*CurrentDir=*/StringRef(TestRoot), FS));
+  llvm::cl::ExpansionContext ECtx(A, cl::TokenizeWindowsCommandLine);
+  ECtx.setVFS(&FS).setCurrentDir(TestRoot).setMarkEOLs(true).setRelativeNames(
+      true);
+  ASSERT_TRUE(ECtx.expandResponseFiles(Argv));
   const char *Expected[] = {"clang", "-Xclang", "-Wno-whatever", nullptr,
                             "input.cpp"};
   ASSERT_EQ(std::size(Expected), Argv.size());
   for (size_t I = 0, E = std::size(Expected); I < E; ++I) {
     if (Expected[I] == nullptr) {
       ASSERT_EQ(Argv[I], nullptr);
     } else {
       ASSERT_STREQ(Expected[I], Argv[I]);
TEST(CommandLineTest, ReadConfigFile) {
   // 'subconfig' unless it resolves nested inclusions relative to the including
   // file.
   llvm::SmallString<128> CurrDir;
   std::error_code EC = llvm::sys::fs::current_path(CurrDir);
   EXPECT_TRUE(!EC);
   EXPECT_NE(CurrDir.str(), TestDir.path());
 
   llvm::BumpPtrAllocator A;
-  llvm::StringSaver Saver(A);
-  bool Result = llvm::cl::readConfigFile(ConfigFile.path(), Saver, Argv,
-                             *llvm::vfs::getRealFileSystem());
+  llvm::cl::ExpansionContext ECtx(A, cl::tokenizeConfigFile);
+  bool Result = ECtx.readConfigFile(ConfigFile.path(), Argv);
 
   EXPECT_TRUE(Result);
   EXPECT_EQ(Argv.size(), 13U);
   EXPECT_STREQ(Argv[0], "-option_1");
   EXPECT_STREQ(Argv[1],
                ("-option_2=" + TestDir.path() + "/dir1").str().c_str());
   EXPECT_STREQ(Argv[2], ("-option_3=" + TestDir.path()).str().c_str());
   EXPECT_STREQ(Argv[3], "-option_4");