[lit] Support reading arguments from a file

[phosek]

Summary

This allows reading arguments from file using the response file syntax.
We would like to use this in the LLVM build to pass test suites from
subbuilds.

[jdenny]

So this is intended to help fix all the problems discussed at the reviews and issues below?

• https://github.com/llvm/llvm-project/issues/55082
• https://github.com/llvm/llvm-project/issues/54154
• https://reviews.llvm.org/D121276
• https://reviews.llvm.org/D121647

Please add documentation of the new feature to llvm/docs/CommandGuide/lit.rst.

[phosek]

In D132437#3742772, @jdenny wrote:

So this is intended to help fix all the problems discussed at the reviews and issues below?

• https://github.com/llvm/llvm-project/issues/55082
• https://github.com/llvm/llvm-project/issues/54154
• https://reviews.llvm.org/D121276
• https://reviews.llvm.org/D121647

Yes, that's correct, it's used in D132438.

Please add documentation of the new feature to llvm/docs/CommandGuide/lit.rst.

Done.

[thieta]

I am not Lit expert - but the python method seems to make sense reading the documentation and it doesn't seem to have any real downsides. I'll accept this - but if you want to wait for someone that understands lit better, I totally understand that.

[jdenny]

llvm/docs/CommandGuide/lit.rst

47

I'm not sure everyone reading this document will know what a response file is, and googling gave me inconsistent answers. Would you please add a brief explanation of the syntax? Is it one command-line option per line in the file? Can @ be used in a response file?

[phosek]

llvm/docs/CommandGuide/lit.rst

47

Updated in rG4f5cdbae887e9ab971a841dd123dec110ed12254.

[jdenny]

llvm/docs/CommandGuide/lit.rst

47

Thanks.

llvm/docs/CommandGuide/lit.rst

 subset of the options specified on the command line, see
 :ref:`selection-options` for more information.
 
 :program:`lit` parses options from the environment variable ``LIT_OPTS`` after
 parsing options from the command line.  ``LIT_OPTS`` is primarily useful for
 supplementing or overriding the command-line options supplied to :program:`lit`
 by ``check`` targets defined by a project's build system.
 
+:program:`lit` can also read options from response files which are specified as
+inputs using the ``@path/to/file.rsp`` syntax.
+
 Users interested in the :program:`lit` architecture or designing a
 :program:`lit` testing implementation should see :ref:`lit-infrastructure`.
 
 GENERAL OPTIONS
 ---------------
 
 .. option:: -h, --help
 

llvm/utils/lit/lit/cl_arguments.py

 @enum.unique
 class TestOrder(enum.Enum):
     LEXICAL = 'lexical'
     RANDOM = 'random'
     SMART = 'smart'
 
 
 def parse_args():
-    parser = argparse.ArgumentParser(prog='lit')
+    parser = argparse.ArgumentParser(prog='lit', fromfile_prefix_chars='@')
     parser.add_argument('test_paths',
             nargs='+',
             metavar="TEST_PATH",
             help='File or path to include in the test suite')
 
     parser.add_argument('--version',
             action='version',
             version='%(prog)s ' + lit.__version__)