[utils] Refactor utils/update_{,llc_}test_checks.py to share more code

[MaskRay]

Summary

This revision refactors 1. parser 2. CHECK line adder of utils/update_{,llc_}test_checks.py
so that thir functionality can be re-used by other utility scripts (e.g. D42712)

[MaskRay]

Rebase on D42805

[• dberlin]

These were actually deliberately split apart in the past year, so not sure what makes sense here.

[asb]

In D42805#996333, @dberlin wrote:

These were actually deliberately split apart in the past year, so not sure what makes sense here.

Are you referring to rL305208? It seems to me that attempts to make update_test_checks also handle llc checks were abandoned, but this patch takes a different strategy: maintaining separate 'frontends' and separate logic where it makes sense, but adding common utility functions where possible.

[spatel]

In D42805#996342, @asb wrote:

In D42805#996333, @dberlin wrote:

These were actually deliberately split apart in the past year, so not sure what makes sense here.

Are you referring to rL305208? It seems to me that attempts to make update_test_checks also handle llc checks were abandoned, but this patch takes a different strategy: maintaining separate 'frontends' and separate logic where it makes sense, but adding common utility functions where possible.

Yep - I killed the duplicate functionality because people couldn't tell which script to use when updating llc tests (and the scripts produced slightly different output, so it was causing bogus diffs).

It's entirely my fault that I didn't do this refactoring myself because I don't know python well enough. So I fully support making these more maintainable by people who actually know what they're doing. :)

One thing that might make the usage clearer - change the script name from "update_test_checks.py" to "update_opt_test_checks.py"? We would then want to update every regression test file that has the old name as part of the 'ADVERT' string, but that should be a mechanical/scriptable change? Or better - use the new script on every test file with the old string to prove that nothing is broken in this refactoring.

[MaskRay]

If you prefer different scripts for different purposes:

utils/update_opt_test_checks.py   input: LLVM IR  ; CHECK lines: LLVM IR
utils/update_llc_test_checks.py   input: LLVM IR  ; CHECK lines: assembly
utils/update_cc_test_checks.py    input: C/C++/ObjC/..  ; CHECK lines: assembly or LLVM IR

If you did not mind merging update_test_checks.py and update_llc_test_checks.py, we could rename and merge these scripts by their input format:

utils/update_ir_test_checks.py      input: LLVM IR  ; CHECK lines: assembly or LLVM IR
utils/update_cc_test_checks.py     input: LLVM IR  ; CHECK lines: assembly or LLVM IR

Instead of just script name on the ADVERT line, we could also put an extra option to indicate its usage e.g.:

; NOTE: Assertions have been autogenerated by utils/update_ir_test_checks.py --llc
; NOTE: Assertions have been autogenerated by utils/update_ir_test_checks.py --opt

--llc and --opt restrict the script to update ; RUN lines that use the two tools, respectively. By merging the two scripts, we can reduce more boilerplate.

I can do that in another revision (after this revision and D42712 are landed)

I know little about MIR but it seems utils/update_mir_test_checks.py is totally different.

[spatel]

In D42805#996490, @MaskRay wrote:

If you prefer different scripts for different purposes:

utils/update_opt_test_checks.py   input: LLVM IR  ; CHECK lines: LLVM IR
utils/update_llc_test_checks.py   input: LLVM IR  ; CHECK lines: assembly
utils/update_cc_test_checks.py    input: C/C++/ObjC/..  ; CHECK lines: assembly or LLVM IR

If you did not mind merging update_test_checks.py and update_llc_test_checks.py, we could rename and merge these scripts by their input format:

utils/update_ir_test_checks.py      input: LLVM IR  ; CHECK lines: assembly or LLVM IR
utils/update_cc_test_checks.py     input: LLVM IR  ; CHECK lines: assembly or LLVM IR

Instead of just script name on the ADVERT line, we could also put an extra option to indicate its usage e.g.:

; NOTE: Assertions have been autogenerated by utils/update_ir_test_checks.py --llc
; NOTE: Assertions have been autogenerated by utils/update_ir_test_checks.py --opt

--llc and --opt restrict the script to update ; RUN lines that use the two tools, respectively. By merging the two scripts, we can reduce more boilerplate.

I can do that in another revision (after this revision and D42712 are landed)

One script with an option to choose the output format sounds good to me.

Note that we do have some regression tests where we check both IR and asm in one file, so make sure that case still works. Recent example is in D42607 - test/Transforms/LoopStrengthReduce/X86/macro-fuse-cmp.ll

[MaskRay]

In D42805#996534, @spatel wrote:

In D42805#996490, @MaskRay wrote:

If you prefer different scripts for different purposes:

utils/update_opt_test_checks.py   input: LLVM IR  ; CHECK lines: LLVM IR
utils/update_llc_test_checks.py   input: LLVM IR  ; CHECK lines: assembly
utils/update_cc_test_checks.py    input: C/C++/ObjC/..  ; CHECK lines: assembly or LLVM IR

If you did not mind merging update_test_checks.py and update_llc_test_checks.py, we could rename and merge these scripts by their input format:

utils/update_ir_test_checks.py      input: LLVM IR  ; CHECK lines: assembly or LLVM IR
utils/update_cc_test_checks.py     input: LLVM IR  ; CHECK lines: assembly or LLVM IR

Instead of just script name on the ADVERT line, we could also put an extra option to indicate its usage e.g.:

; NOTE: Assertions have been autogenerated by utils/update_ir_test_checks.py --llc
; NOTE: Assertions have been autogenerated by utils/update_ir_test_checks.py --opt

--llc and --opt restrict the script to update ; RUN lines that use the two tools, respectively. By merging the two scripts, we can reduce more boilerplate.

I can do that in another revision (after this revision and D42712 are landed)

One script with an option to choose the output format sounds good to me.

Note that we do have some regression tests where we check both IR and asm in one file, so make sure that case still works. Recent example is in D42607 - test/Transforms/LoopStrengthReduce/X86/macro-fuse-cmp.ll

It still works

~/Dev/llvm/utils/update_llc_test_checks.py -v --llc-binary=~/Dev/llvm/release/bin/llc test/Transforms/LoopStrengthReduce/X86/macro-fuse-cmp.ll
# No change

~/Dev/llvm/utils/update_test_checks.py -v --opt-binary=~/Dev/llvm/release/bin/opt test/Transforms/LoopStrengthReduce/X86/macro-fuse-cmp.ll
# There are expected changes. After all the ADVERT line is update_llc_test_checks.py

[MaskRay]

Ping?

[MaskRay]

rebase

[MaskRay]

Friendly ping :) Is anyone against the idea to clean up these utils/update*check_tests.py scripts?

[RKSimon]

LGTM - thanks

[lebedev.ri]

llvm/trunk/utils/UpdateTestChecks/common.py

64

Hi @MaskRay.

I think there is something wrong with this particular line.

If the whitespace isn't leading/trailing, removing it will change the line,
and FileCheck will no longer match this line. So, how can we even do this?

This particular line breaks utils/update_analyze_test_checks.py
Try on test/Analysis/ScalarEvolution/*mask.ll

[MaskRay]

llvm/trunk/utils/UpdateTestChecks/common.py

64

Sorry but I don't have any idea what has been going on... When I spli clean up the script... utils/update_analyze_test_checks.py did not exist then..

[RKSimon]

llvm/trunk/utils/UpdateTestChecks/common.py

64

I wouldn't be surprised if I got something wrong on the analyze script - I wrote it for cost models only so other analysis passes weren't tested. If you can create a bugzilla with a simple repro I'll take a look (but I may need guidance from people who spend more time with python that I do).

[lebedev.ri]

llvm/trunk/utils/UpdateTestChecks/common.py

64

This is the only problem. I think this line is just wrong.
How can we change the inner context of the check line,
if we don't replace it with some [[:whitespace:]] globs or something?

llvm/trunk/utils/UpdateTestChecks/asm.py

 import re
-import string
+import sys
 
 from . import common
 
+if sys.version_info[0] > 2:
+  class string:
+    expandtabs = str.expandtabs
+else:
+  import string
+
 # RegEx: this is where the magic happens.
 
+##### Assembly parser
+
 ASM_FUNCTION_X86_RE = re.compile(
     r'^_?(?P<func>[^:]+):[ \t]*#+[ \t]*@(?P=func)\n[^:]*?'
     r'(?P<body>^##?[ \t]+[^:]+:.*?)\s*'
     r'^\s*(?:[^:\n]+?:\s*\n\s*\.size|\.cfi_endproc|\.globl|\.comm|\.(?:sub)?section|#+ -- End function)',
     flags=(re.M | re.S))
 
 ASM_FUNCTION_ARM_RE = re.compile(
         r'^(?P<func>[0-9a-zA-Z_]+):\n' # f: (name of function)
if triple.startswith(prefix):
       break
   else:
     raise KeyError('Triple %r is not supported' % (triple))
 
   scrubber, function_re = handlers
   common.build_function_body_dictionary(
           function_re, scrubber, [args], raw_tool_output, prefixes,
           func_dict, args.verbose)
+
+##### Generator of assembly CHECK lines
+
+def add_asm_checks(output_lines, comment_marker, run_list, func_dict, func_name):
+  printed_prefixes = []
+  for p in run_list:
+    checkprefixes = p[0]
+    for checkprefix in checkprefixes:
+      if checkprefix in printed_prefixes:
+        break
+      # TODO func_dict[checkprefix] may be None, '' or not exist.
+      # Fix the call sites.
+      if func_name not in func_dict[checkprefix] or not func_dict[checkprefix][func_name]:
+        continue
+      # Add some space between different check prefixes.
+      if len(printed_prefixes) != 0:
+        output_lines.append(comment_marker)
+      printed_prefixes.append(checkprefix)
+      output_lines.append('%s %s-LABEL: %s:' % (comment_marker, checkprefix, func_name))
+      func_body = func_dict[checkprefix][func_name].splitlines()
+      output_lines.append('%s %s:       %s' % (comment_marker, checkprefix, func_body[0]))
+      for func_line in func_body[1:]:
+        output_lines.append('%s %s-NEXT:  %s' % (comment_marker, checkprefix, func_line))
+      # Add space between different check prefixes and the first line of code.
+      # output_lines.append(';')
+      break

llvm/trunk/utils/UpdateTestChecks/common.py

 from __future__ import print_function
 import re
+import string
 import subprocess
 import sys
 
-RUN_LINE_RE = re.compile('^\s*;\s*RUN:\s*(.*)$')
-CHECK_PREFIX_RE = re.compile('--?check-prefix(?:es)?=(\S+)')
-CHECK_RE = re.compile(r'^\s*;\s*([^:]+?)(?:-NEXT|-NOT|-DAG|-LABEL)?:')
+if sys.version_info[0] > 2:
+  class string:
+    expandtabs = str.expandtabs
+else:
+  import string
 
-IR_FUNCTION_RE = re.compile('^\s*define\s+(?:internal\s+)?[^@]*@(\w+)\s*\(')
-TRIPLE_IR_RE = re.compile(r'^target\s+triple\s*=\s*"([^"]+)"$')
-TRIPLE_ARG_RE = re.compile(r'-mtriple=([^ ]+)')
-
-SCRUB_LEADING_WHITESPACE_RE = re.compile(r'^(\s+)')
-SCRUB_WHITESPACE_RE = re.compile(r'(?!^(|  \w))[ \t]+', flags=re.M)
-SCRUB_TRAILING_WHITESPACE_RE = re.compile(r'[ \t]+$', flags=re.M)
-SCRUB_KILL_COMMENT_RE = re.compile(r'^ *#+ +kill:.*\n')
-SCRUB_LOOP_COMMENT_RE = re.compile(
-    r'# =>This Inner Loop Header:.*|# in Loop:.*', flags=re.M)
+##### Common utilities for update_*test_checks.py
 
 def should_add_line_to_output(input_line, prefix_set):
   # Skip any blank comment lines in the IR.
   if input_line.strip() == ';':
     return False
   # Skip any blank lines in the IR.
   #if input_line.strip() == '':
   #  return False
def invoke_tool(exe, cmd_args, ir):
   with open(ir) as ir_file:
     stdout = subprocess.check_output(exe + ' ' + cmd_args,
                                      shell=True, stdin=ir_file)
     if sys.version_info[0] > 2:
       stdout = stdout.decode()
   # Fix line endings to unix CR style.
   return stdout.replace('\r\n', '\n')
 
+##### LLVM IR parser
+
+RUN_LINE_RE = re.compile('^\s*;\s*RUN:\s*(.*)$')
+CHECK_PREFIX_RE = re.compile('--?check-prefix(?:es)?=(\S+)')
+CHECK_RE = re.compile(r'^\s*;\s*([^:]+?)(?:-NEXT|-NOT|-DAG|-LABEL)?:')
+
+OPT_FUNCTION_RE = re.compile(
+    r'^\s*define\s+(?:internal\s+)?[^@]*@(?P<func>[\w-]+?)\s*\('
+    r'(\s+)?[^)]*[^{]*\{\n(?P<body>.*?)^\}$',
+    flags=(re.M | re.S))
+
+IR_FUNCTION_RE = re.compile('^\s*define\s+(?:internal\s+)?[^@]*@(\w+)\s*\(')
+TRIPLE_IR_RE = re.compile(r'^target\s+triple\s*=\s*"([^"]+)"$')
+TRIPLE_ARG_RE = re.compile(r'-mtriple=([^ ]+)')
+
+SCRUB_LEADING_WHITESPACE_RE = re.compile(r'^(\s+)')
+SCRUB_WHITESPACE_RE = re.compile(r'(?!^(|  \w))[ \t]+', flags=re.M)
+SCRUB_TRAILING_WHITESPACE_RE = re.compile(r'[ \t]+$', flags=re.M)
+SCRUB_KILL_COMMENT_RE = re.compile(r'^ *#+ +kill:.*\n')
+SCRUB_LOOP_COMMENT_RE = re.compile(
+    r'# =>This Inner Loop Header:.*|# in Loop:.*', flags=re.M)
+
+def scrub_body(body):
+  # Scrub runs of whitespace out of the assembly, but leave the leading
+  # whitespace in place.
+  body = SCRUB_WHITESPACE_RE.sub(r' ', body)
+  # Expand the tabs used for indentation.
+  body = string.expandtabs(body, 2)
+  # Strip trailing whitespace.
+  body = SCRUB_TRAILING_WHITESPACE_RE.sub(r'', body)
+  return body
+
 # Build up a dictionary of all the function bodies.
 def build_function_body_dictionary(function_re, scrubber, scrubber_args, raw_tool_output, prefixes, func_dict, verbose):
   for m in function_re.finditer(raw_tool_output):
     if not m:
       continue
     func = m.group('func')
     scrubbed_body = scrubber(m.group('body'), *scrubber_args)
     if func.startswith('stress'):
       # We only use the last line of the function body for stress tests.
       scrubbed_body = '\n'.join(scrubbed_body.splitlines()[-1:])
     if verbose:
       print('Processing function: ' + func, file=sys.stderr)
       for l in scrubbed_body.splitlines():
         print('  ' + l, file=sys.stderr)
     for prefix in prefixes:
       if func in func_dict[prefix] and func_dict[prefix][func] != scrubbed_body:
         if prefix == prefixes[-1]:
           print('WARNING: Found conflicting asm under the '
                                'same prefix: %r!' % (prefix,), file=sys.stderr)
         else:
           func_dict[prefix][func] = None
           continue
 
       func_dict[prefix][func] = scrubbed_body
+
+##### Generator of LLVM IR CHECK lines
+
+SCRUB_IR_COMMENT_RE = re.compile(r'\s*;.*')
+
+# Match things that look at identifiers, but only if they are followed by
+# spaces, commas, paren, or end of the string
+IR_VALUE_RE = re.compile(r'(\s+)%([\w\.]+?)([,\s\(\)]|\Z)')
+
+# Create a FileCheck variable name based on an IR name.
+def get_value_name(var):
+  if var.isdigit():
+    var = 'TMP' + var
+  var = var.replace('.', '_')
+  return var.upper()
+
+
+# Create a FileCheck variable from regex.
+def get_value_definition(var):
+  return '[[' + get_value_name(var) + ':%.*]]'
+
+
+# Use a FileCheck variable.
+def get_value_use(var):
+  return '[[' + get_value_name(var) + ']]'
+
+# Replace IR value defs and uses with FileCheck variables.
+def genericize_check_lines(lines):
+  # This gets called for each match that occurs in
+  # a line. We transform variables we haven't seen
+  # into defs, and variables we have seen into uses.
+  def transform_line_vars(match):
+    var = match.group(2)
+    if var in vars_seen:
+      rv = get_value_use(var)
+    else:
+      vars_seen.add(var)
+      rv = get_value_definition(var)
+    # re.sub replaces the entire regex match
+    # with whatever you return, so we have
+    # to make sure to hand it back everything
+    # including the commas and spaces.
+    return match.group(1) + rv + match.group(3)
+
+  vars_seen = set()
+  lines_with_def = []
+
+  for i, line in enumerate(lines):
+    # An IR variable named '%.' matches the FileCheck regex string.
+    line = line.replace('%.', '%dot')
+    # Ignore any comments, since the check lines will too.
+    scrubbed_line = SCRUB_IR_COMMENT_RE.sub(r'', line)
+    lines[i] =  IR_VALUE_RE.sub(transform_line_vars, scrubbed_line)
+  return lines
+
+
+def add_ir_checks(output_lines, prefix_list, func_dict, func_name, opt_basename):
+  # Label format is based on IR string.
+  check_label_format = "; %s-LABEL: @%s("
+
+  printed_prefixes = []
+  for checkprefixes, _ in prefix_list:
+    for checkprefix in checkprefixes:
+      if checkprefix in printed_prefixes:
+        break
+      if not func_dict[checkprefix][func_name]:
+        continue
+      # Add some space between different check prefixes, but not after the last
+      # check line (before the test code).
+      #if len(printed_prefixes) != 0:
+      #  output_lines.append(';')
+      printed_prefixes.append(checkprefix)
+      output_lines.append(check_label_format % (checkprefix, func_name))
+      func_body = func_dict[checkprefix][func_name].splitlines()
+
+      # For IR output, change all defs to FileCheck variables, so we're immune
+      # to variable naming fashions.
+      func_body = genericize_check_lines(func_body)
+
+      # This could be selectively enabled with an optional invocation argument.
+      # Disabled for now: better to check everything. Be safe rather than sorry.
+
+      # Handle the first line of the function body as a special case because
+      # it's often just noise (a useless asm comment or entry label).
+      #if func_body[0].startswith("#") or func_body[0].startswith("entry:"):
+      #  is_blank_line = True
+      #else:
+      #  output_lines.append('; %s:       %s' % (checkprefix, func_body[0]))
+      #  is_blank_line = False
+
+      is_blank_line = False
+
+      for func_line in func_body:
+        if func_line.strip() == '':
+          is_blank_line = True
+          continue
+        # Do not waste time checking IR comments.
+        func_line = SCRUB_IR_COMMENT_RE.sub(r'', func_line)
+
+        # Skip blank lines instead of checking them.
+        if is_blank_line == True:
+          output_lines.append('; %s:       %s' % (checkprefix, func_line))
+        else:
+          output_lines.append('; %s-NEXT:  %s' % (checkprefix, func_line))
+        is_blank_line = False
+
+      # Add space between different check prefixes and also before the first
+      # line of code in the test function.
+      output_lines.append(';')
+      break
+  return output_lines

llvm/trunk/utils/update_llc_test_checks.py

 import sys
 import re
 
 from UpdateTestChecks import asm, common
 
 ADVERT = '; NOTE: Assertions have been autogenerated by '
 
 
-def add_checks(output_lines, run_list, func_dict, func_name):
-  printed_prefixes = []
-  for p in run_list:
-    checkprefixes = p[0]
-    for checkprefix in checkprefixes:
-      if checkprefix in printed_prefixes:
-        break
-      if not func_dict[checkprefix][func_name]:
-        continue
-      # Add some space between different check prefixes.
-      if len(printed_prefixes) != 0:
-        output_lines.append(';')
-      printed_prefixes.append(checkprefix)
-      output_lines.append('; %s-LABEL: %s:' % (checkprefix, func_name))
-      func_body = func_dict[checkprefix][func_name].splitlines()
-      output_lines.append('; %s:       %s' % (checkprefix, func_body[0]))
-      for func_line in func_body[1:]:
-        output_lines.append('; %s-NEXT:  %s' % (checkprefix, func_line))
-      # Add space between different check prefixes and the first line of code.
-      # output_lines.append(';')
-      break
-  return output_lines
-
-
 def main():
   parser = argparse.ArgumentParser(description=__doc__)
   parser.add_argument('-v', '--verbose', action='store_true',
                       help='Show verbose output')
   parser.add_argument('--llc-binary', default='llc',
                       help='The "llc" binary to use to generate the test case')
   parser.add_argument(
       '--function', help='The function in the test file to update')
for input_line in input_lines:
           continue
         if input_line.lstrip().startswith(';'):
           m = common.CHECK_RE.match(input_line)
           if not m or m.group(1) not in prefix_set:
             output_lines.append(input_line)
             continue
 
         # Print out the various check lines here.
-        output_lines = add_checks(output_lines, run_list, func_dict, func_name)
+        asm.add_asm_checks(output_lines, ';', run_list, func_dict, func_name)
         is_in_function_start = False
 
       if is_in_function:
         if common.should_add_line_to_output(input_line, prefix_set):
           # This input line of the function body will go as-is into the output.
           output_lines.append(input_line)
         else:
           continue

llvm/trunk/utils/update_test_checks.py

 import re
 
 from UpdateTestChecks import common
 
 ADVERT = '; NOTE: Assertions have been autogenerated by '
 
 # RegEx: this is where the magic happens.
 
-SCRUB_IR_COMMENT_RE = re.compile(r'\s*;.*')
-
 IR_FUNCTION_RE = re.compile('^\s*define\s+(?:internal\s+)?[^@]*@([\w-]+)\s*\(')
-OPT_FUNCTION_RE = re.compile(
-    r'^\s*define\s+(?:internal\s+)?[^@]*@(?P<func>[\w-]+?)\s*\('
-    r'(\s+)?[^)]*[^{]*\{\n(?P<body>.*?)^\}$',
-    flags=(re.M | re.S))
-# Match things that look at identifiers, but only if they are followed by
-# spaces, commas, paren, or end of the string
-IR_VALUE_RE = re.compile(r'(\s+)%([\w\.]+?)([,\s\(\)]|\Z)')
-
-
-
-def scrub_body(body, opt_basename):
-  # Scrub runs of whitespace out of the assembly, but leave the leading
-  # whitespace in place.
-  body = common.SCRUB_WHITESPACE_RE.sub(r' ', body)
-  # Expand the tabs used for indentation.
-  body = string.expandtabs(body, 2)
-  # Strip trailing whitespace.
-  body = common.SCRUB_TRAILING_WHITESPACE_RE.sub(r'', body)
-  return body
-
 
 
-# Create a FileCheck variable name based on an IR name.
-def get_value_name(var):
-  if var.isdigit():
-    var = 'TMP' + var
-  var = var.replace('.', '_')
-  return var.upper()
-
-
-# Create a FileCheck variable from regex.
-def get_value_definition(var):
-  return '[[' + get_value_name(var) + ':%.*]]'
-
-
-# Use a FileCheck variable.
-def get_value_use(var):
-  return '[[' + get_value_name(var) + ']]'
-
-# Replace IR value defs and uses with FileCheck variables.
-def genericize_check_lines(lines):
-  # This gets called for each match that occurs in
-  # a line. We transform variables we haven't seen
-  # into defs, and variables we have seen into uses.
-  def transform_line_vars(match):
-    var = match.group(2)
-    if var in vars_seen:
-      rv = get_value_use(var)
-    else:
-      vars_seen.add(var)
-      rv = get_value_definition(var)
-    # re.sub replaces the entire regex match
-    # with whatever you return, so we have
-    # to make sure to hand it back everything
-    # including the commas and spaces.
-    return match.group(1) + rv + match.group(3)
-
-  vars_seen = set()
-  lines_with_def = []
-
-  for i, line in enumerate(lines):
-    # An IR variable named '%.' matches the FileCheck regex string.
-    line = line.replace('%.', '%dot')
-    # Ignore any comments, since the check lines will too.
-    scrubbed_line = SCRUB_IR_COMMENT_RE.sub(r'', line)
-    lines[i] =  IR_VALUE_RE.sub(transform_line_vars, scrubbed_line)
-  return lines
-
-
-def add_checks(output_lines, prefix_list, func_dict, func_name, opt_basename):
-  # Label format is based on IR string.
-  check_label_format = "; %s-LABEL: @%s("
-
-  printed_prefixes = []
-  for checkprefixes, _ in prefix_list:
-    for checkprefix in checkprefixes:
-      if checkprefix in printed_prefixes:
-        break
-      if not func_dict[checkprefix][func_name]:
-        continue
-      # Add some space between different check prefixes, but not after the last
-      # check line (before the test code).
-      #if len(printed_prefixes) != 0:
-      #  output_lines.append(';')
-      printed_prefixes.append(checkprefix)
-      output_lines.append(check_label_format % (checkprefix, func_name))
-      func_body = func_dict[checkprefix][func_name].splitlines()
-
-      # For IR output, change all defs to FileCheck variables, so we're immune
-      # to variable naming fashions.
-      func_body = genericize_check_lines(func_body)
-
-      # This could be selectively enabled with an optional invocation argument.
-      # Disabled for now: better to check everything. Be safe rather than sorry.
-
-      # Handle the first line of the function body as a special case because
-      # it's often just noise (a useless asm comment or entry label).
-      #if func_body[0].startswith("#") or func_body[0].startswith("entry:"):
-      #  is_blank_line = True
-      #else:
-      #  output_lines.append('; %s:       %s' % (checkprefix, func_body[0]))
-      #  is_blank_line = False
-
-      is_blank_line = False
-
-      for func_line in func_body:
-        if func_line.strip() == '':
-          is_blank_line = True
-          continue
-        # Do not waste time checking IR comments.
-        func_line = SCRUB_IR_COMMENT_RE.sub(r'', func_line)
-
-        # Skip blank lines instead of checking them.
-        if is_blank_line == True:
-          output_lines.append('; %s:       %s' % (checkprefix, func_line))
-        else:
-          output_lines.append('; %s-NEXT:  %s' % (checkprefix, func_line))
-        is_blank_line = False
 
-      # Add space between different check prefixes and also before the first
-      # line of code in the test function.
-      output_lines.append(';')
-      break
-  return output_lines
 
 
 def main():
   from argparse import RawTextHelpFormatter
   parser = argparse.ArgumentParser(description=__doc__, formatter_class=RawTextHelpFormatter)
   parser.add_argument('-v', '--verbose', action='store_true',
                       help='Show verbose output')
   parser.add_argument('--opt-binary', default='opt',
for prefixes, _ in prefix_list:
         func_dict.update({prefix: dict()})
     for prefixes, opt_args in prefix_list:
       if args.verbose:
         print >>sys.stderr, 'Extracted opt cmd: ' + opt_basename + ' ' + opt_args
         print >>sys.stderr, 'Extracted FileCheck prefixes: ' + str(prefixes)
 
       raw_tool_output = common.invoke_tool(args.opt_binary, opt_args, test)
       common.build_function_body_dictionary(
-              OPT_FUNCTION_RE, scrub_body, [opt_basename], raw_tool_output,
-              prefixes, func_dict, args.verbose)
+              common.OPT_FUNCTION_RE, common.scrub_body, [],
+              raw_tool_output, prefixes, func_dict, args.verbose)
 
     is_in_function = False
     is_in_function_start = False
     prefix_set = set([prefix for prefixes, _ in prefix_list for prefix in prefixes])
     if args.verbose:
       print >>sys.stderr, 'Rewriting FileCheck prefixes: %s' % (prefix_set,)
     output_lines = []
     output_lines.append(autogenerated_note)
 
     for input_line in input_lines:
       if is_in_function_start:
         if input_line == '':
           continue
         if input_line.lstrip().startswith(';'):
           m = common.CHECK_RE.match(input_line)
           if not m or m.group(1) not in prefix_set:
             output_lines.append(input_line)
             continue
 
         # Print out the various check lines here.
-        output_lines = add_checks(output_lines, prefix_list, func_dict, func_name, opt_basename)
+        output_lines = common.add_ir_checks(output_lines, prefix_list, func_dict, func_name, opt_basename)
         is_in_function_start = False
 
       if is_in_function:
         if common.should_add_line_to_output(input_line, prefix_set):
           # This input line of the function body will go as-is into the output.
           # Except make leading whitespace uniform: 2 spaces.
           input_line = common.SCRUB_LEADING_WHITESPACE_RE.sub(r'  ', input_line)
           output_lines.append(input_line)