utils/update_mir_test_checks.py: allow checking fixedStack in .mir files

[gbossu]

Summary

Generation of CHECK lines for fixedStack can be enabled with --print-fixed-stack.
This is particularly useful for tests which need to inspect how the
stack looks, e.g. for ABI tests.

See the other stacked revision building on top of this one which enables UTC_ARGS (in a similar fashion to other test updaters in utils/).

[gbossu]

Update email address in commit...

[nhaehnle]

The overall approach seems good to me.

llvm/test/tools/UpdateTestChecks/update_mir_test_checks/Inputs/print-stack-first.mir

4–6

This comment isn't really necessary.

llvm/utils/update_mir_test_checks.py

249–250

I don't think we really have a Python coding style guide, but this should probably be func_info: FunctionInfo

271

This assignment seems pointless?

[gbossu]

llvm/test/tools/UpdateTestChecks/update_mir_test_checks/Inputs/print-stack-first.mir

4–6

I do agree, but this was brought to my attention that such a comment would be useful to people not too familiar with llvm's test structure. If you feel it's not needed or too verbose, I do not mind removing it.

llvm/utils/update_mir_test_checks.py

249–250

Will change

271

Good catch, I think it's a remnant from some conflict resolution...

[nhaehnle]

llvm/test/tools/UpdateTestChecks/update_mir_test_checks/Inputs/print-stack-first.mir

4–6

All tests in these folders are structured the same way, so it feels strange to have the comment here. But I also don't really care, it's really a bikeshed :)

[gbossu]

Fix coding style & useless assignment

[arsenm]

I don't know this code but seems reasonable

[gbossu]

Thank you! Could someone commit the change on my behalf? I do not think I have write access, since it's my first patch here.

[gbossu]

Rebase on latest main, otherwise tests seem to timeout.

[gbossu]

@nhaehnle @arsenm Thank you for having a look at this patch. I unfortunately don't have commit "powers", would it be possible for one of you to commit on my behalf?
Here is the commit header I usually have: Gaëtan Bossu <[e-mail]>, if you need it. Thank you in advance :)

Note that there is also a stacked patch on top of this one which I would like to commit, but it's currently failing on something un-related. I'll restart the build and hope for the best I guess.

[foad]

This broke regenerating checks for tests like test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll which use -simplify-mir and do not print the fixedStack info. Can you fix update_mir_test_checks to cope, or should we remove -simplify-mir from files with generated checks?

[gbossu]

In D135579#4417921, @foad wrote:

This broke regenerating checks for tests like test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll which use -simplify-mir and do not print the fixedStack info. Can you fix update_mir_test_checks to cope, or should we remove -simplify-mir from files with generated checks?

Sorry about this, I'll have a look this week.

[foad]

In D135579#4420335, @gbossu wrote:

In D135579#4417921, @foad wrote:

This broke regenerating checks for tests like test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll which use -simplify-mir and do not print the fixedStack info. Can you fix update_mir_test_checks to cope, or should we remove -simplify-mir from files with generated checks?

Sorry about this, I'll have a look this week.

No worries. I actually have a simple fix already, I just need to write a test for it. I'll try to put something up for review today.

[arsenm]

In D135579#4417921, @foad wrote:

This broke regenerating checks for tests like test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll which use -simplify-mir and do not print the fixedStack info. Can you fix update_mir_test_checks to cope, or should we remove -simplify-mir from files with generated checks?

I think -simplify-mir should just be the global default and this should tolerate anything missing

[foad]

In D135579#4420338, @foad wrote:

In D135579#4420335, @gbossu wrote:

In D135579#4417921, @foad wrote:

This broke regenerating checks for tests like test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.s.buffer.load.ll which use -simplify-mir and do not print the fixedStack info. Can you fix update_mir_test_checks to cope, or should we remove -simplify-mir from files with generated checks?

Sorry about this, I'll have a look this week.

No worries. I actually have a simple fix already, I just need to write a test for it. I'll try to put something up for review today.

D152896

llvm/test/tools/UpdateTestChecks/update_mir_test_checks/Inputs/print-stack-first.mir

+# RUN: llc -mtriple=x86_64-linux-gnu -run-pass=none -o - %s | FileCheck %s
+
+# Note that this file isn't a test in itself (Inputs/ is excluded from lit's
+# test discovery). Instead, it is an input to the print-stack.test test that
+# verifies the --print-fixed-stack option of update_mir_test_checks.py.
+
+---
+name:            test_arg_stack
+fixedStack:
+  - { id: 0, type: default, offset: 0, size: 4, alignment: 4 }
+  - { id: 1, type: default, offset: 4, size: 4, alignment: 8 }
+stack:           []
+body:             |
+  bb.1:
+    RET 0, implicit $eax
+...
+
+---
+name:            test_arg_nostack
+body:             |
+  bb.1:
+    RET 0, implicit $eax
+...

llvm/test/tools/UpdateTestChecks/update_mir_test_checks/Inputs/print-stack.mir.expected

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=x86_64-linux-gnu -run-pass=none -o - %s | FileCheck %s
+
+# Note that this file isn't a test in itself (Inputs/ is excluded from lit's
+# test discovery). Instead, it is an input to the print-stack.test test that
+# verifies the --print-fixed-stack option of update_mir_test_checks.py.
+
+---
+name:            test_arg_stack
+fixedStack:
+  - { id: 0, type: default, offset: 0, size: 4, alignment: 4 }
+  - { id: 1, type: default, offset: 4, size: 4, alignment: 8 }
+stack:           []
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: test_arg_stack
+    ; CHECK: fixedStack:
+    ; CHECK-NEXT:   - { id: 0, type: default, offset: 4, size: 4, alignment: 8, stack-id: default, 
+    ; CHECK-NEXT:       isImmutable: false, isAliased: false, callee-saved-register: '', 
+    ; CHECK-NEXT:       callee-saved-restored: true, debug-info-variable: '', debug-info-expression: '', 
+    ; CHECK-NEXT:       debug-info-location: '' }
+    ; CHECK-NEXT:   - { id: 1, type: default, offset: 0, size: 4, alignment: 4, stack-id: default, 
+    ; CHECK-NEXT:       isImmutable: false, isAliased: false, callee-saved-register: '', 
+    ; CHECK-NEXT:       callee-saved-restored: true, debug-info-variable: '', debug-info-expression: '', 
+    ; CHECK-NEXT:       debug-info-location: '' }
+    ; CHECK: RET 0, implicit $eax
+    RET 0, implicit $eax
+...
+
+---
+name:            test_arg_nostack
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: test_arg_nostack
+    ; CHECK: fixedStack:
+    ; CHECK: RET 0, implicit $eax
+    RET 0, implicit $eax
+...

llvm/test/tools/UpdateTestChecks/update_mir_test_checks/print-stack.test

+# REQUIRES: x86-registered-target
+## Check that update_mir_test_checks handles --print-fixed-stack properly.
+
+## Verify with --print-fixed-stack, the proper CHECK lines for fixedStack are
+## generated.
+# RUN: cp -f %S/Inputs/print-stack-first.mir %t.mir && %update_mir_test_checks %t.mir --print-fixed-stack
+# RUN: diff -u %S/Inputs/print-stack.mir.expected %t.mir

llvm/utils/update_mir_test_checks.py

 #!/usr/bin/env python3
 
 """Updates FileCheck checks in MIR tests.
 
 This script is a utility to update MIR based tests with new FileCheck
 patterns.
 
 The checks added by this script will cover the entire body of each
 function it handles. Virtual registers used are given names via
 FileCheck patterns, so if you do want to check a subset of the body it
 should be straightforward to trim out the irrelevant parts. None of
-the YAML metadata will be checked, other than function names.
+the YAML metadata will be checked, other than function names, and fixedStack
+if the --print-fixed-stack option is used.
 
 If there are multiple llc commands in a test, the full set of checks
 will be repeated for each different check pattern. Checks for patterns
 that are common between different commands will be left as-is by
 default, or removed if the --remove-common-prefixes flag is provided.
 """
 
 from __future__ import print_function
IR_FUNC_NAME_RE = re.compile(
     r'^\s*define\s+(?:internal\s+)?[^@]*@(?P<func>[A-Za-z0-9_.]+)\s*\(')
 IR_PREFIX_DATA_RE = re.compile(r'^ *(;|$)')
 
 MIR_FUNC_RE = re.compile(
     r'^---$'
     r'\n'
     r'^ *name: *(?P<func>[A-Za-z0-9_.-]+)$'
     r'.*?'
+    r'^ *fixedStack: *(\[\])? *\n'
+    r'(?P<fixedStack>.*?)\n?'
+    r'^ *stack:'
+    r'.*?'
     r'^ *body: *\|\n'
     r'(?P<body>.*?)\n'
     r'^\.\.\.$',
     flags=(re.M | re.S))
 
 
 class LLC:
     def __init__(self, bin):
for line in lines:
         m = MIR_BASIC_BLOCK_RE.match(line)
         if m:
             bbs += 1
     if bbs == 1:
         result.append(cur_func)
     return result
 
 
-def build_function_body_dictionary(test, raw_tool_output, triple, prefixes,
+class FunctionInfo:
+    def __init__(self, body, fixedStack):
+        self.body = body
+        self.fixedStack = fixedStack
+
+    def __eq__(self, other):
+        if not isinstance(other, FunctionInfo):
+            return False
+        return self.body == other.body and self.fixedStack == other.fixedStack
+
+
+def build_function_info_dictionary(test, raw_tool_output, triple, prefixes,
                                    func_dict, verbose):
     for m in MIR_FUNC_RE.finditer(raw_tool_output):
         func = m.group('func')
+        fixedStack = m.group('fixedStack')
         body = m.group('body')
         if verbose:
             log('Processing function: {}'.format(func))
             for l in body.splitlines():
                 log('  {}'.format(l))
 
         # Vreg mangling
         mangled = []
         vreg_map = {}
         for func_line in body.splitlines(keepends=True):
             m = VREG_DEF_RE.match(func_line)
             if m:
                 for vreg in VREG_RE.finditer(m.group('vregs')):
                     name = mangle_vreg(m.group('opcode'), vreg_map.values())
                     vreg_map[vreg.group(1)] = name
                     func_line = func_line.replace(
                         vreg.group(1), '[[{}:%[0-9]+]]'.format(name), 1)
             for number, name in vreg_map.items():
                 func_line = re.sub(r'{}\b'.format(number), '[[{}]]'.format(name),
                                 func_line)
             mangled.append(func_line)
         body = ''.join(mangled)
 
         for prefix in prefixes:
+            info = FunctionInfo(body, fixedStack)
             if func in func_dict[prefix]:
-                if func_dict[prefix][func] != body:
+                if func_dict[prefix][func] != info:
                     func_dict[prefix][func] = None
             else:
-                func_dict[prefix][func] = body
+                func_dict[prefix][func] = info
 
 
 def add_checks_for_function(test, output_lines, run_list, func_dict, func_name,
-                            single_bb, verbose=False):
+                            single_bb, args):
     printed_prefixes = set()
     for run in run_list:
         for prefix in run.prefixes:
             if prefix in printed_prefixes:
                 break
             if not func_dict[prefix][func_name]:
                 continue
             # if printed_prefixes:
             #     # Add some space between different check prefixes.
             #     output_lines.append('')
             printed_prefixes.add(prefix)
-            log('Adding {} lines for {}'.format(prefix, func_name), verbose)
+            log('Adding {} lines for {}'.format(prefix, func_name), args.verbose)
             add_check_lines(test, output_lines, prefix, func_name, single_bb,
-                            func_dict[prefix][func_name].splitlines())
+                            func_dict[prefix][func_name], args)
             break
         else:
             common.warn(
                 'Found conflicting asm for function: {}'.format(func_name),
                 test_file=test)
     return output_lines
 
 
 def add_check_lines(test, output_lines, prefix, func_name, single_bb,
-                    func_body):
+                    func_info: FunctionInfo, args):
+    func_body = func_info.body.splitlines()
     if single_bb:
         # Don't bother checking the basic block label for a single BB
         func_body.pop(0)
 
     if not func_body:
         common.warn('Function has no instructions to check: {}'.format(func_name),
              test_file=test)
         return
 
     first_line = func_body[0]
     indent = len(first_line) - len(first_line.lstrip(' '))
     # A check comment, indented the appropriate amount
     check = '{:>{}}; {}'.format('', indent, prefix)
 
     output_lines.append('{}-LABEL: name: {}'.format(check, func_name))
-    first_check = True
 
+    if args.print_fixed_stack:
+        output_lines.append('{}: fixedStack:'.format(check))
+        for stack_line in func_info.fixedStack.splitlines():
+            filecheck_directive = check + '-NEXT'
+            output_lines.append('{}: {}'.format(filecheck_directive, stack_line))
+
+    first_check = True
     for func_line in func_body:
         if not func_line.strip():
             # The mir printer prints leading whitespace so we can't use CHECK-EMPTY:
             output_lines.append(check + '-NEXT: {{' + func_line + '$}}')
             continue
         filecheck_directive = check if first_check else check + '-NEXT'
         first_check = False
         check_line = '{}: {}'.format(filecheck_directive, func_line[indent:]).rstrip()
for prefixes, llc_args, triple_in_cmd in run_list:
         log('Extracted LLC cmd: llc {}'.format(llc_args), args.verbose)
         log('Extracted FileCheck prefixes: {}'.format(prefixes), args.verbose)
 
         raw_tool_output = args.llc(llc_args, test)
         if not triple_in_cmd and not triple_in_ir:
             common.warn('No triple found: skipping file', test_file=test)
             return
 
-        build_function_body_dictionary(test, raw_tool_output,
+        build_function_info_dictionary(test, raw_tool_output,
                                        triple_in_cmd or triple_in_ir,
                                        prefixes, func_dict, args.verbose)
 
     state = 'toplevel'
     func_name = None
     prefix_set = set([prefix for run in run_list for prefix in run.prefixes])
     log('Rewriting FileCheck prefixes: {}'.format(prefix_set), args.verbose)
 
for input_line in input_lines:
             if m:
                 if func_name in simple_functions:
                     # If there's only one block, put the checks inside it
                     state = 'mir function prefix'
                     continue
                 state = 'mir function body'
                 add_checks_for_function(test, output_lines, run_list,
                                         func_dict, func_name, single_bb=False,
-                                        verbose=args.verbose)
+                                        args=args)
         elif state == 'mir function prefix':
             m = MIR_PREFIX_DATA_RE.match(input_line)
             if not m:
                 state = 'mir function body'
                 add_checks_for_function(test, output_lines, run_list,
                                         func_dict, func_name, single_bb=True,
-                                        verbose=args.verbose)
+                                        args=args)
 
             if should_add_line_to_output(input_line, prefix_set):
                 output_lines.append(input_line)
         elif state == 'mir function body':
             if input_line.strip() == '...':
                 state = 'toplevel'
                 func_name = None
             if should_add_line_to_output(input_line, prefix_set):
                 output_lines.append(input_line)
         elif state == 'ir function prefix':
             m = IR_PREFIX_DATA_RE.match(input_line)
             if not m:
                 state = 'ir function body'
                 add_checks_for_function(test, output_lines, run_list,
                                         func_dict, func_name, single_bb=False,
-                                        verbose=args.verbose)
+                                        args=args)
 
             if should_add_line_to_output(input_line, prefix_set):
                 output_lines.append(input_line)
         elif state == 'ir function body':
             if input_line.strip() == '}':
                 state = 'toplevel'
                 func_name = None
             if should_add_line_to_output(input_line, prefix_set):
                 output_lines.append(input_line)
 
 
     log('Writing {} lines to {}...'.format(len(output_lines), test), args.verbose)
 
     with open(test, 'wb') as fd:
         fd.writelines(['{}\n'.format(l).encode('utf-8') for l in output_lines])
 
 
 def main():
     parser = argparse.ArgumentParser(
         description=__doc__, formatter_class=argparse.RawTextHelpFormatter)
     parser.add_argument('--llc-binary', dest='llc', default='llc', type=LLC,
                         help='The "llc" binary to generate the test case with')
+    parser.add_argument('--print-fixed-stack', action='store_true',
+                        help='Add check lines for fixedStack')
     parser.add_argument('tests', nargs='+')
     args = common.parse_commandline_args(parser)
 
     test_paths = [test for pattern in args.tests for test in glob.glob(pattern)]
     for test in test_paths:
         try:
             update_test_file(args, test)
         except Exception:
             common.warn('Error processing file', test_file=test)
             raise
 
 
 if __name__ == '__main__':
   main()