[lit] Add --show-substitutions

[eush]

Summary

This adds a flag for llvm-lit to print all substitutions for each discovered
test suite as suggested in http://bugs.llvm.org/show_bug.cgi?id=31609.

This flag can be used to quickly look up all defined substitutions for a
given configuration without diving into Lit configs.

The implementation tries to minimize escaping but also avoid ambiguity, so in
the output quoted keys/values are always escaped, and unquoted keys/values are
always literal.

[delcypher]

utils/lit/lit/main.py

352

Why is showing substitutions coupled with listing the test suites?

433

Is all this regex really necessary? Couldn't we just always print repr(pattern), repr(replacement)?

[eush]

utils/lit/lit/main.py

352

Substitutions are defined per test suite, so we have to list test suites anyway. We could hide some lines though
(such as available features).

433

My only concern is that otherwise it would be harder to read regexes since they would be (doubly) escaped.

[delcypher]

utils/lit/lit/main.py

433

Then why even use repr(...)? Just print as a regular string always.

E.g.

>>> print(repr(r'\s\w'))
'\\s\\w'
>>> print(str(r'\s\w'))
\s\w

I honestly don't understand what your regex logic is trying to achieve so I really think you should drop it.

[delcypher]

@eush Also sorry for the delay. I just noticed that this patch was in my list of unreviewed patches. Please feel free to ping me on this review if I don't respond in a timely manner.

[atrick]

Thanks for doing this.

[serge-sans-paille]

Apart from the regexp pretty-printing, this LGTM.

utils/lit/lit/main.py

432

[nits] the parenthesis around the tuple are not needed.

433

I agree with @delcypher . Just printing the pattern « as is » should be fine.

[jdenny]

FYI: A similar patch already went in: D77818

[serge-sans-paille]

This bug should probably be closed then

utils/lit/lit/main.py

debug_group.add_argument("--debug",
                       help="Enable debugging (for 'lit' development)",
                       action="store_true", default=False)
     debug_group.add_argument("--show-suites", dest="showSuites",
                       help="Show discovered test suites",
                       action="store_true", default=False)
     debug_group.add_argument("--show-tests", dest="showTests",
                       help="Show all discovered tests",
                       action="store_true", default=False)
+    debug_group.add_argument("--show-substitutions", dest="showSubstitutions",
+                      help="Show all substitutions (implies --show-suites)",
+                      action="store_true", default=False)
     debug_group.add_argument("--single-process", dest="singleProcess",
                       help="Don't run tests in parallel.  Intended for debugging "
                       "single test failures",
                       action="store_true", default=False)
 
     opts = parser.parse_args()
     args = opts.test_paths
 
     if opts.show_version:
         print("lit %s" % (lit.__version__,))
         return
 
     if not args:
         parser.error('No inputs specified')
 
     if opts.numThreads is None:
         opts.numThreads = lit.util.detectCPUs()
 
     if opts.maxFailures == 0:
         parser.error("Setting --max-failures to 0 does not have any effect.")
 
     if opts.echoAllCommands:
         opts.showOutput = True
 
+    if opts.showSubstitutions:
+        opts.showSuites = True
+
     inputs = args
 
     # Create the user defined parameters.
     userParams = dict(builtinParameters)
     for entry in opts.userParameters:
         if '=' not in entry:
             name,val = entry,''
         else:
if opts.showSuites or opts.showTests:
             print('-- Test Suites --')
             for ts,ts_tests in suitesAndTests:
                 print('  %s - %d tests' %(ts.name, len(ts_tests)))
                 print('    Source Root: %s' % ts.source_root)
                 print('    Exec Root  : %s' % ts.exec_root)
                 if ts.config.available_features:
                     print('    Available Features : %s' % ' '.join(
                         sorted(ts.config.available_features)))
+                if opts.showSubstitutions and ts.config.substitutions:
+                    print('    Substitutions')
+                    for (pattern, replacement) in ts.config.substitutions:
+                        if re.match(r'[\'"]|.*\s', pattern):
+                            pattern = repr(pattern)
+                        if re.match(r'[\'"]|.*\s|$', replacement):
+                            replacement = repr(replacement)
+                        print('        %s = %s' %(pattern, replacement))
 
         # Show the tests, if requested.
         if opts.showTests:
             print('-- Available Tests --')
             for ts,ts_tests in suitesAndTests:
                 ts_tests.sort(key = lambda test: test.path_in_suite)
                 for test in ts_tests:
                     print('  %s' % (test.getFullName(),))

utils/lit/tests/Inputs/show-substitutions/lit.cfg

+import lit.formats
+
+config.name = 'shtest-shell'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+
+config.substitutions = [('%foo', 'foo'),
+                        ('%foo', ''),
+                        (r'\|%lli\b(?!(\.))', '/bin/lli'),
+                        (r'\| not', '/bin/not '),
+                        ("'foo'", '"foo"')]

utils/lit/tests/Inputs/show-substitutions/test.txt

+# RUN: true

utils/lit/tests/show-substitutions.py

+# RUN: %{lit} --show-substitutions %{inputs}/show-substitutions | FileCheck %s
+
+# CHECK: Substitutions
+# CHECK-NEXT: %foo = foo
+# CHECK-NEXT: %foo = ''
+# CHECK-NEXT: \|%lli\b(?!(\.)) = /bin/lli
+# CHECK-NEXT: '\\| not' = '/bin/not '
+# CHECK-NEXT: "'foo'" = '"foo"'