Robustify update_test_checks.py to non-autogened tests, and add a mode to skip non-autogenerated ones

[reames]

Summary

Intended use case is:
./utils/update_test_checks.py test/Transform/PassDir/* --update-only
(i.e. rapidly be able to see changes in autogened filed, before handing non-autogened tests individually)

[RKSimon]

Could this functionality be added to all the update*py scripts?

[MaskRay]

utils/update_test_checks.py

65

Prefer single quotes.

92

'autogenerated'

113

'...........{}'.format(l) is usually preferred over print('WARNING: Skipping unparseable RUN line: %s' % (l,), file=sys.stderr) (note comma after ll)

The comma makes it a one-element tuple. (l) is the same as l.

[reames]

In D65610#1611868, @RKSimon wrote:

Could this functionality be added to all the update*py scripts?

Could be reasonable done, but given the divergence already existing, would need to be hand implemented for each. I'm not volunteering.

utils/update_test_checks.py

113

I'd prefer to follow the style of all the surrounding code which uses this construct.

p.s. The common vs non-common form appear to evaluate exactly the same. (Based on the output of a test only)

[xbolva00]

Thanks, useful!

I'm not volunteering.

You already did all coding work, copy (move to common) and test it shouldn’t be so hard. Anyway, after you land this, I could port your work to other update scripts.

[xbolva00]

utils/update_test_checks.py

65

Can you add short option?

[MaskRay]

utils/update_test_checks.py

93

"Skipping test which isn't autogenerated: " + test

112

'|'

113

Alternatively, 'WARNING: Skipping unparseable RUN line: ' + l

%s can mean str, int, list ([1,2,3]) and many other strings that can be converted to a str via str().

utils/update_test_checks.py

def main():
   from argparse import RawTextHelpFormatter
   parser = argparse.ArgumentParser(description=__doc__, formatter_class=RawTextHelpFormatter)
   parser.add_argument('-v', '--verbose', action='store_true',
                       help='Show verbose output')
   parser.add_argument('--opt-binary', default='opt',
                       help='The opt binary used to generate the test case')
   parser.add_argument(
       '--function', help='The function in the test file to update')
+  parser.add_argument('--update-only', action="store_true",
+                      help='Only update test if it was already autogened')
   parser.add_argument('tests', nargs='+')
   args = parser.parse_args()
 
   autogenerated_note = (ADVERT + 'utils/' + os.path.basename(__file__))
 
   opt_basename = os.path.basename(args.opt_binary)
   if not re.match(r'^opt(-\d+)?$', opt_basename):
     print('ERROR: Unexpected opt name: ' + opt_basename, file=sys.stderr)
     sys.exit(1)
   opt_basename = 'opt'
 
   test_paths = []
   for test in args.tests:
     if not glob.glob(test):
       print('WARNING: Test file \'%s\' was not found. Ignoring it.' % (test,), file=sys.stderr)
       continue
     test_paths.append(test)
 
   for test in test_paths:
     if args.verbose:
       print('Scanning for RUN lines in test file: %s' % (test,), file=sys.stderr)
     with open(test) as f:
       input_lines = [l.rstrip() for l in f]
 
+    if args.update_only:
+      if len(input_lines) == 0 or "autogenerated" not in input_lines[0]:
+        print('Skipping test which isn\'t autogenerated: %s' % (test), file=sys.stderr)
+        continue;
+
     raw_lines = [m.group(1)
                  for m in [common.RUN_LINE_RE.match(l) for l in input_lines] if m]
     run_lines = [raw_lines[0]] if len(raw_lines) > 0 else []
     for l in raw_lines[1:]:
       if run_lines[-1].endswith("\\"):
         run_lines[-1] = run_lines[-1].rstrip("\\") + " " + l
       else:
         run_lines.append(l)
 
     if args.verbose:
       print('Found %d RUN lines:' % (len(run_lines),), file=sys.stderr)
       for l in run_lines:
         print('  RUN: ' + l, file=sys.stderr)
 
     prefix_list = []
     for l in run_lines:
+      if "|" not in l:
+        print('WARNING: Skipping unparseable RUN line: %s' % (l), file=sys.stderr)
+        continue
+        
       (tool_cmd, filecheck_cmd) = tuple([cmd.strip() for cmd in l.split('|', 1)])
       common.verify_filecheck_prefixes(filecheck_cmd)
       if not tool_cmd.startswith(opt_basename + ' '):
         print('WARNING: Skipping non-%s RUN line: %s' % (opt_basename, l), file=sys.stderr)
         continue
 
       if not filecheck_cmd.startswith('FileCheck '):
         print('WARNING: Skipping non-FileChecked RUN line: ' + l, file=sys.stderr)