[UpdateTestChecks] Auto-generate stub bodies for unused prefixes

[mtrofin]

Summary

This is scoped to autogenerated tests.

The goal is to support having each RUN line specify a list of
check-prefixes where one can specify potentially redundant prefixes. For example,
for X86, if one specified prefixes for both AVX1 and AVX2, and the codegen happened to
match today, one of the prefixes would be used and the onther one not.
If the unused prefix were dropped, and later, codegen differences were
introduced, one would have to go figure out where to add what prefix
(paraphrasing
https://lists.llvm.org/pipermail/llvm-dev/2021-February/148326.html)

To avoid getting errors due to unused prefixes, whole directories can be
opted out (as discussed on that thread), but that means that tests that
aren't autogenerated in such directories could have undetected unused
prefix bugs.

This patch proposes an alternative that both avoids the above, dir-level
optout, and supports the main autogen scenario discussed first. The autogen
tool appends at the end of the test file the list of unused prefixes,
together with a note explaining that is the case. Each prefix is set up
to always pass.

This way, unexpected unused prefixes are easily discoverable, and
expected cases "just work".

[MaskRay]

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/amdgpu_no_merge_comments.ll.expected

33

NOTE is a regular comment, not a check prefix. I wonder whether using ;; for non-RUN non-CHECK lines will be clearer.

[mtrofin]

double-commenting the "NOTE", also fix in update_cc_test_checks

[mtrofin]

@RKSimon @lebedev.ri @probinson @nikic @xbolva00 - any pushback to this patch, is it helping/hindering in any way? Thanks!

[nikic]

I haven't checked the implementation, but conceptually this looks reasonable.

[UpdateTestChecks] Auto-generate stub bodies for conflicting outputs

Patch title seems unrelated though?

[RKSimon]

I'm not entirely happy with adding the "unused" noise at the end of the file, but I understand the need for such a workaround to appease lit.

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/amdgpu_no_merge_comments.ll.expected

34

Will the script strip these again if a later edit adds uses of a previously unused prefix?

[mtrofin]

make sure 1) re-running tool doesn't repeat the unused note; and 2) adding a use for a previously unused label removes that label from the note list.

[mtrofin]

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/amdgpu_no_merge_comments.ll.expected

34

Good catch - added a test for that. Also made the 'note' a bit more verbose.

[mtrofin]

In D124306#3534450, @nikic wrote:

I haven't checked the implementation, but conceptually this looks reasonable.

[UpdateTestChecks] Auto-generate stub bodies for conflicting outputs

Patch title seems unrelated though?

Ah, yes, fixed. It's actually kind of related - turns out one reason some prefixes may end up unused is if they produce conflicting outputs for functions, but that's too specific of a nuance.

[RKSimon]

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/common-label-different-bodies-1-next.ll

30

;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:

[mtrofin]

fix

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/common-label-different-bodies-1-next.ll

30

fixed, thanks!

[RKSimon]

LGTM

[MatzeB]

This change is breaking the build for me: BooleanOptionalAction is only available in python 3.9!

[MatzeB]

In D124306#3540768, @MatzeB wrote:

This change is breaking the build for me: BooleanOptionalAction is only available in python 3.9!

Well not failing the build but failing various tests for the scripts.

[MatzeB]

llvm/docs/GettingStarted.rst only asks for python >= 3.6 at the moment.

[mtrofin]

In D124306#3540769, @MatzeB wrote:

In D124306#3540768, @MatzeB wrote:

This change is breaking the build for me: BooleanOptionalAction is only available in python 3.9!

Well not failing the build but failing various tests for the scripts.

Sorry about that - fix coming in a moment.

[dyung]

Wrong change, sorry!

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/amdgpu_no_merge_comments.ll.expected

 ; GFX9-O3-NEXT:    v_mul_lo_u32 v0, v1, v0
 ; GFX9-O3-NEXT:    v_sub_u32_e32 v0, v0, v1
 ; GFX9-O3-NEXT:    s_setpc_b64 s[30:31]
   %add = add i32 %a, %a
   %mul = mul i32 %add, %a
   %sub = sub i32 %mul, %add
   ret i32 %sub
 }
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; GCN: {{.*}}

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/common-label-different-bodies-1-next.ll

+; RUN: llc < %s -mtriple=i686-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=A,B
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --allow-unused-prefixes=true --check-prefixes=C,A,UNUSED
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --allow-unused-prefixes=true --check-prefixe=A
+
+declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
+
+define <2 x i64> @fold_v2i64() {
+; B-LABEL: fold_v2i64:
+; B:       # %bb.0: # %entry
+; B-NEXT:    movaps {{.*#+}} xmm0 = [0,4278190080,4294967295,4294967295]
+; B-NEXT:    retl
+;
+; C-LABEL: fold_v2i64:
+; C:       # %bb.0: # %entry
+; C-NEXT:    movaps {{.*#+}} xmm0 = [18374686479671623680,18446744073709551615]
+; C-NEXT:    retq
+entry:
+  %r = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> <i64 255, i64 -1>)
+  ret <2 x i64> %r
+}
+
+declare <4 x i32> @llvm.bswap.v4i32(<4 x i32>)
+
+define <4 x i32> @test2(<4 x i32> %v) {
+  %r = call <4 x i32> @llvm.bswap.v4i32(<4 x i32> %v)
+  ret <4 x i32> %r
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; A: {{.*}}
+; UNUSED: {{.*}}

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/common-label-different-bodies-1.ll

 ; RUN: llc < %s -mtriple=i686-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=A,B
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --allow-unused-prefixes=true --check-prefixes=C,A,UNUSED
 
 declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
-; A: declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
 
 define <2 x i64> @fold_v2i64() {
 entry:
   %r = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> <i64 255, i64 -1>)
   ret <2 x i64> %r
 }

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/common-label-different-bodies-2.ll

 ; RUN: llc < %s -mtriple=i686-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=A,B
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefixes=C,A
 
 declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
-; A: declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
 
 define <2 x i64> @fold_v2i64() {
 entry:
   %r = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> <i64 255, i64 -1>)
   ret <2 x i64> %r
 }

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/common-label-different-bodies-3.ll

 ; RUN: llc < %s -mtriple=i686-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=A,B
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefixes=A,C
 
 declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
-; A: declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
 
 define <2 x i64> @fold_v2i64() {
 entry:
   %r = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> <i64 255, i64 -1>)
   ret <2 x i64> %r
 }

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/redundant-and-unmatching-prefixes.ll

+; RUN: llc < %s -mtriple=i686-unknown-linux-gnu -mattr=+sse2 | FileCheck %s --check-prefixes=A,B,REDUNDANT
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefixes=C,A,UNUSED
+
+; prefix 'A' has conflicting outputs, while the REDUNDANT and UNUSED ones are
+; unused
+declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>)
+
+define <2 x i64> @function_1() {
+entry:
+  %r = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> <i64 255, i64 -1>)
+  ret <2 x i64> %r
+}
+
+define <2 x i64> @function_2() {
+entry:
+  %r = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> <i64 16, i64 -1>)
+  ret <2 x i64> %r
+}

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/common-label-different-bodies.test

 # REQUIRES: x86-registered-target
 
 # RUN: cp -f %S/Inputs/common-label-different-bodies-1.ll %t-1.ll
 # RUN: cp -f %S/Inputs/common-label-different-bodies-2.ll %t-2.ll
 # RUN: cp -f %S/Inputs/common-label-different-bodies-3.ll %t-3.ll
 # RUN: %update_llc_test_checks %t-1.ll
+
+# re-running update_llc_test_checks leaves the file as-is.
+# RUN: cp -f %t-1.ll %t-1-copy.ll
+# RUN: %update_llc_test_checks %t-1.ll
+# RUN: diff %t-1.ll %t-1-copy.ll
 # RUN: %update_llc_test_checks %t-2.ll
 # RUN: %update_llc_test_checks %t-3.ll
-# RUN: FileCheck --input-file=%t-1.ll %s
+# RUN: FileCheck --input-file=%t-1.ll %s --check-prefixes=CHECK,CHECK-UNUSED
 # RUN: FileCheck --input-file=%t-2.ll %s
 # RUN: FileCheck --input-file=%t-3.ll %s
 
 # CHECK: B-LABEL: fold_v2i64
-# CHECK-NOT: A-LABEL: fold_v2i64
+# CHECK-NOT: A-LABEL: fold_v2i64
+# CHECK: NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+# CHECK-NEXT: A:
+# CHECK-UNUSED-NEXT: UNUSED:
+
+# adding a test removes that label from "unused" prefixes list
+# the input file is like "1" after the tool was run, and then we added a new test
+# RUN: cp -f %S/Inputs/common-label-different-bodies-1-next.ll %t-1-next.ll
+# RUN: %update_llc_test_checks %t-1-next.ll
+# RUN: FileCheck --input-file=%t-1-next.ll %s --check-prefixes=AFTER-CHANGE
+
+# AFTER-CHANGE: NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+# AFTER-CHANGE-NEXT: UNUSED:
+# AFTER-CHANGE-NOT: A:

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/prefix-never-matches.test

 # REQUIRES: x86-registered-target
 
 # RUN: cp -f %S/Inputs/prefix-never-matches.ll %t.ll
-# RUN: %update_llc_test_checks %t.ll 2>&1 | FileCheck %s
+# RUN: %update_llc_test_checks --no-generate-body-for-unused-prefixes %t.ll 2>&1 | FileCheck %s
 # RUN: FileCheck --input-file=%t.ll %s --check-prefix=OUTPUT
 
 # CHECK: WARNING: Prefix A had conflicting output
-# OUTPUT-NOT: A:
+# OUTPUT-NOT: A:
 

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/redundant-and-unmatched-prefixes.test

+# REQUIRES: x86-registered-target
+
+# RUN: cp -f %S/Inputs/redundant-and-unmatching-prefixes.ll %t-1.ll
+# RUN: %update_llc_test_checks %t-1.ll
+# RUN: FileCheck --input-file=%t-1.ll %s
+
+# CHECK: B-LABEL: function_1:
+# CHECK-NOT: A-LABEL: function_1
+# CHECK-NOT: REDUNDANT-LABEL: function_1:
+
+# CHECK: B-LABEL: function_2:
+# CHECK-NOT: A-LABEL: function_2:
+# CHECK-NOT: UNUSED-LABEL: function_2:
+
+# CHECK: NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+# CHECK-NEXT: A:
+# CHECK-NEXT: REDUNDANT:
+# CHECK-NEXT: UNUSED:

llvm/utils/UpdateTestChecks/asm.py

def get_run_handler(triple):
   return handler
 
 ##### Generator of assembly CHECK lines
 
 def add_checks(output_lines, comment_marker, prefix_list, func_dict,
                func_name, global_vars_seen_dict, is_filtered):
   # Label format is based on ASM string.
   check_label_format = '{} %s-LABEL: %s%s%s'.format(comment_marker)
-  common.add_checks(output_lines, comment_marker, prefix_list, func_dict,
+  return common.add_checks(output_lines, comment_marker, prefix_list, func_dict,
                            func_name, check_label_format, True, False,
                            global_vars_seen_dict, is_filtered=is_filtered)

llvm/utils/UpdateTestChecks/common.py

 from __future__ import print_function
 
 import argparse
 import copy
 import glob
 import itertools
 import os
 import re
 import subprocess
 import sys
 import shlex
 
+from typing import List
+
 ##### Common utilities for update_*test_checks.py
 
 
 _verbose = False
 _prefix_filecheck_ir_name = ''
 
 class Regex(object):
   """Wrap a compiled regular expression object to allow deep copy of a regexp.
def parse_commandline_args(parser):
   parser.add_argument('--replace-value-regex', nargs='+', default=[],
                       help='List of regular expressions to replace matching value names')
   parser.add_argument('--prefix-filecheck-ir-name', default='',
                       help='Add a prefix to FileCheck IR value names to avoid conflicts with scripted names')
   parser.add_argument('--global-value-regex', nargs='+', default=[],
                       help='List of regular expressions that a global value declaration must match to generate a check (has no effect if checking globals is not enabled)')
   parser.add_argument('--global-hex-value-regex', nargs='+', default=[],
                       help='List of regular expressions such that, for matching global value declarations, literal integer values should be encoded in hex in the associated FileCheck directives')
+  parser.add_argument('--generate-body-for-unused-prefixes',
+                      action=argparse.BooleanOptionalAction,
+                      dest='gen_unused_prefix_body',
+                      default=True,
+                      help='Generate a function body that always matches for unused prefixes. This is useful when unused prefixes are desired, and it avoids needing to annotate each FileCheck as allowing them.')
   args = parser.parse_args()
   global _verbose, _global_value_regex, _global_hex_value_regex
   _verbose = args.verbose
   _global_value_regex = args.global_value_regex
   _global_hex_value_regex = args.global_hex_value_regex
   return args
 
 
def __init__(self, test, parser, script_name, input_lines, args, argv,
     if self.comment_prefix is None:
       if self.path.endswith('.mir'):
         self.comment_prefix = '#'
       else:
         self.comment_prefix = ';'
     self.autogenerated_note_prefix = self.comment_prefix + ' ' + UTC_ADVERT
     self.test_autogenerated_note = self.autogenerated_note_prefix + script_name
     self.test_autogenerated_note += get_autogennote_suffix(parser, self.args)
+    self.test_unused_note = self.comment_prefix + self.comment_prefix + ' ' + UNUSED_NOTE
 
   def ro_iterlines(self):
     for line_num, input_line in enumerate(self.input_lines):
       args, argv = check_for_command(input_line, self.parser,
                                      self.args, self.argv, self.argparse_callback)
       yield InputLineInfo(input_line, line_num, args, argv)
 
   def iterlines(self, output_lines):
     output_lines.append(self.test_autogenerated_note)
     for line_info in self.ro_iterlines():
       input_line = line_info.line
       # Discard any previous script advertising.
       if input_line.startswith(self.autogenerated_note_prefix):
         continue
       self.args = line_info.args
       self.argv = line_info.argv
       if not self.args.enabled:
         output_lines.append(input_line)
         continue
       yield line_info
 
+  def get_checks_for_unused_prefixes(self, run_list, used_prefixes: List[str]) -> List[str]:
+    unused_prefixes = set(
+        [prefix for sublist in run_list for prefix in sublist[0]]).difference(set(used_prefixes))
+
+    ret = []
+    if not unused_prefixes:
+      return ret
+    ret.append(self.test_unused_note)
+    for unused in sorted(unused_prefixes):
+      ret.append('{comment} {prefix}: {match_everything}'.format(
+        comment=self.comment_prefix,
+        prefix=unused,
+        match_everything=r"""{{.*}}"""
+      ))
+    return ret
+
 def itertests(test_patterns, parser, script_name, comment_prefix=None, argparse_callback=None):
   for pattern in test_patterns:
     # On Windows we must expand the patterns ourselves.
     tests_list = glob.glob(pattern)
     if not tests_list:
       warn("Test file pattern '%s' was not found. Ignoring it." % (pattern,))
       continue
     for test in tests_list:
       with open(test) as f:
         input_lines = [l.rstrip() for l in f]
       args = parser.parse_args()
       if argparse_callback is not None:
         argparse_callback(args)
       argv = sys.argv[:]
       first_line = input_lines[0] if input_lines else ""
       if UTC_ADVERT in first_line:
         if script_name not in first_line and not args.force_update:
           warn("Skipping test which wasn't autogenerated by " + script_name, test)
           continue
         args, argv = check_for_command(first_line, parser, args, argv, argparse_callback)
       elif args.update_only:
         assert UTC_ADVERT not in first_line
         warn("Skipping test which isn't autogenerated: " + test)
         continue
-      yield TestInfo(test, parser, script_name, input_lines, args, argv,
+      final_input_lines = []
+      for l in input_lines:
+        if UNUSED_NOTE in l:
+          break
+        final_input_lines.append(l)
+      yield TestInfo(test, parser, script_name, final_input_lines, args, argv,
                      comment_prefix, argparse_callback)
 
 
 def should_add_line_to_output(input_line, prefix_set, skip_global_checks = False, comment_marker = ';'):
   # Skip any blank comment lines in the IR.
   if not skip_global_checks and input_line.strip() == comment_marker:
     return False
   # Skip a special double comment line we use as a separator.

 RUN_LINE_RE = re.compile(r'^\s*(?://|[;#])\s*RUN:\s*(.*)$')
 CHECK_PREFIX_RE = re.compile(r'--?check-prefix(?:es)?[= ](\S+)')
 PREFIX_RE = re.compile('^[a-zA-Z0-9_-]+$')
 CHECK_RE = re.compile(r'^\s*(?://|[;#])\s*([^:]+?)(?:-NEXT|-NOT|-DAG|-LABEL|-SAME|-EMPTY)?:')
 
 UTC_ARGS_KEY = 'UTC_ARGS:'
 UTC_ARGS_CMD = re.compile(r'.*' + UTC_ARGS_KEY + '\s*(?P<cmd>.*)\s*$')
 UTC_ADVERT = 'NOTE: Assertions have been autogenerated by '
+UNUSED_NOTE = 'NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:'
 
 OPT_FUNCTION_RE = re.compile(
     r'^(\s*;\s*Function\sAttrs:\s(?P<attrs>[\w\s]+?))?\s*define\s+(?:internal\s+)?[^@]*@(?P<func>[\w.$-]+?)\s*'
     r'(?P<args_and_sig>\((\)|(.*?[\w.-]+?)\))[^{]*\{)\n(?P<body>.*?)^\}$',
     flags=(re.M | re.S))
 
 ANALYZE_FUNCTION_RE = re.compile(
     r'^\s*\'(?P<analysis>[\w\s-]+?)\'\s+for\s+function\s+\'(?P<func>[\w.$-]+?)\':'
def __init__(self, run_list, flags, scrubber_args, path):
     self._global_var_dict = {}
     for tuple in run_list:
       for prefix in tuple[0]:
         self._func_dict.update({prefix:dict()})
         self._func_order.update({prefix: []})
         self._global_var_dict.update({prefix:dict()})
 
   def finish_and_get_func_dict(self):
-    for prefix in self._get_failed_prefixes():
+    for prefix in self.get_failed_prefixes():
       warn('Prefix %s had conflicting output from different RUN lines for all functions in test %s' % (prefix,self._path,))
     return self._func_dict
 
   def func_order(self):
     return self._func_order
 
   def global_var_dict(self):
     return self._global_var_dict
for m in function_re.finditer(raw_tool_output):
               # indicate that.
               self._func_dict[prefix][func] = None
               continue
 
         self._func_dict[prefix][func] = function_body(
             scrubbed_body, scrubbed_extra, args_and_sig, attrs, func_name_separator)
         self._func_order[prefix].append(func)
 
-  def _get_failed_prefixes(self):
+  def get_failed_prefixes(self):
     # This returns the list of those prefixes that failed to match any function,
     # because there were conflicting bodies produced by different RUN lines, in
-    # all instances of the prefix. Effectively, this prefix is unused and should
-    # be removed.
+    # all instances of the prefix.
     for prefix in self._func_dict:
       if (self._func_dict[prefix] and
           (not [fct for fct in self._func_dict[prefix]
                 if self._func_dict[prefix][fct] is not None])):
         yield prefix
 
 
 ##### Generator of LLVM IR CHECK lines
for checkprefix in checkprefixes:
       # line of code in the test function.
       output_lines.append(comment_marker)
 
       # Remember new global variables we have not seen before
       for key in global_vars_seen:
         if key not in global_vars_seen_before:
           global_vars_seen_dict[checkprefix][key] = global_vars_seen[key]
       break
+  return printed_prefixes
 
 def add_ir_checks(output_lines, comment_marker, prefix_list, func_dict,
                   func_name, preserve_names, function_sig,
                   global_vars_seen_dict, is_filtered):
   # Label format is based on IR string.
   function_def_regex = 'define {{[^@]+}}' if function_sig else ''
   check_label_format = '{} %s-LABEL: {}@%s%s%s'.format(comment_marker, function_def_regex)
-  add_checks(output_lines, comment_marker, prefix_list, func_dict, func_name,
+  return add_checks(output_lines, comment_marker, prefix_list, func_dict, func_name,
                     check_label_format, False, preserve_names, global_vars_seen_dict,
                     is_filtered)
 
 def add_analyze_checks(output_lines, comment_marker, prefix_list, func_dict, func_name, is_filtered):
   check_label_format = '{} %s-LABEL: \'%s%s%s\''.format(comment_marker)
   global_vars_seen_dict = {}
-  add_checks(output_lines, comment_marker, prefix_list, func_dict, func_name,
+  return add_checks(output_lines, comment_marker, prefix_list, func_dict, func_name,
                     check_label_format, False, True, global_vars_seen_dict,
                     is_filtered)
 
 def build_global_values_dictionary(glob_val_dict, raw_tool_output, prefixes):
   for nameless_value in itertools.chain(ir_nameless_values, asm_nameless_values):
     if nameless_value.global_ir_prefix is None:
       continue
 
     lhs_re_str = nameless_value.global_ir_prefix + nameless_value.global_ir_prefix_regexp
     rhs_re_str = nameless_value.global_ir_rhs_regexp
if line.lstrip().startswith(comment_string):
       m = CHECK_RE.match(line)
       if m and m.group(1) in prefix_set:
         continue
     output_lines.append(line.rstrip('\n'))
 
 def add_checks_at_end(output_lines, prefix_list, func_order,
                       comment_string, check_generator):
   added = set()
+  generated_prefixes = []
   for prefix in prefix_list:
     prefixes = prefix[0]
     tool_args = prefix[1]
     for prefix in prefixes:
       for func in func_order[prefix]:
         if added:
           output_lines.append(comment_string)
         added.add(func)
 
         # The add_*_checks routines expect a run list whose items are
         # tuples that have a list of prefixes as their first element and
         # tool command args string as their second element.  They output
         # checks for each prefix in the list of prefixes.  By doing so, it
         # implicitly assumes that for each function every run line will
         # generate something for that function.  That is not the case for
         # generated functions as some run lines might not generate them
         # (e.g. -fopenmp vs. no -fopenmp).
         #
         # Therefore, pass just the prefix we're interested in.  This has
         # the effect of generating all of the checks for functions of a
         # single prefix before moving on to the next prefix.  So checks
         # are ordered by prefix instead of by function as in "normal"
         # mode.
-        check_generator(output_lines,
+        generated_prefixes.extend(check_generator(output_lines,
                         [([prefix], tool_args)],
-                        func)
+                        func))
+  return generated_prefixes

llvm/utils/UpdateTestChecks/isel.py

def get_run_handler(triple):
   return handler
 
 ##### Generator of iSel CHECK lines
 
 def add_checks(output_lines, comment_marker, prefix_list, func_dict, func_name,
                global_vars_seen_dict, is_filtered):
   # Label format is based on iSel string.
   check_label_format = '{} %s-LABEL: %s%s%s'.format(comment_marker)
-  common.add_checks(output_lines, comment_marker, prefix_list, func_dict,
+  return common.add_checks(output_lines, comment_marker, prefix_list, func_dict,
                            func_name, check_label_format, True, False,
-                    global_vars_seen_dict, is_filtered = is_filtered)
+                           global_vars_seen_dict, is_filtered=is_filtered)

llvm/utils/update_cc_test_checks.py

if include_generated_funcs:
       # can't insert check lines before each source function and instead have to
       # put them at the end.  So the first thing to do is dump out the source
       # lines.
       common.dump_input_lines(output_lines, ti, prefix_set, '//')
 
       # Now generate all the checks.
       def check_generator(my_output_lines, prefixes, func):
         if '-emit-llvm' in clang_args:
-          common.add_ir_checks(my_output_lines, '//',
+          return common.add_ir_checks(my_output_lines, '//',
                                       prefixes,
                                       func_dict, func, False,
                                       ti.args.function_signature,
                                       global_vars_seen_dict,
                                       is_filtered=builder.is_filtered())
         else:
-          asm.add_checks(my_output_lines, '//',
+          return asm.add_checks(my_output_lines, '//',
                                 prefixes,
                                 func_dict, func, global_vars_seen_dict,
                                 is_filtered=builder.is_filtered())
 
       if ti.args.check_globals:
         common.add_global_checks(builder.global_var_dict(), '//', run_list,
                                  output_lines, global_vars_seen_dict, True,
                                  True)
       common.add_checks_at_end(output_lines, filecheck_run_list, builder.func_order(),
                                '//', lambda my_output_lines, prefixes, func:
                                check_generator(my_output_lines,

llvm/utils/update_llc_test_checks.py

for ti in common.itertests(initial_args.tests, parser,
     common.debug('Rewriting FileCheck prefixes:', str(prefix_set))
     output_lines = []
 
     include_generated_funcs = common.find_arg_in_test(ti,
                                                       lambda args: ti.args.include_generated_funcs,
                                                       '--include-generated-funcs',
                                                       True)
 
+    generated_prefixes = []
     if include_generated_funcs:
       # Generate the appropriate checks for each function.  We need to emit
       # these in the order according to the generated output so that CHECK-LABEL
       # works properly.  func_order provides that.
 
       # We can't predict where various passes might insert functions so we can't
       # be sure the input function order is maintained.  Therefore, first spit
       # out all the source lines.
       common.dump_input_lines(output_lines, ti, prefix_set, ';')
 
       # Now generate all the checks.
-      common.add_checks_at_end(output_lines, run_list, builder.func_order(),
-                               check_indent + ';',
-                               lambda my_output_lines, prefixes, func:
-                               output_type.add_checks(my_output_lines,
-                                                  check_indent + ';',
-                                                  prefixes, func_dict, func,
-                                                  global_vars_seen_dict,
-                                                  is_filtered=builder.is_filtered()))
+      generated_prefixes = common.add_checks_at_end(
+          output_lines, run_list, builder.func_order(),
     else:
       for input_info in ti.iterlines(output_lines):
         input_line = input_info.line
         args = input_info.args
         if is_in_function_start:
           if input_line == '':
             continue
           if input_line.lstrip().startswith(';'):
             m = common.CHECK_RE.match(input_line)
             if not m or m.group(1) not in prefix_set:
               output_lines.append(input_line)
               continue
 
           # Print out the various check lines here.
+          generated_prefixes.extend(
               output_type.add_checks(output_lines, check_indent + ';', run_list,
                                      func_dict, func_name, global_vars_seen_dict,
-                                 is_filtered=builder.is_filtered())
+                                     is_filtered=builder.is_filtered()))
           is_in_function_start = False
 
         if is_in_function:
           if common.should_add_line_to_output(input_line, prefix_set):
             # This input line of the function body will go as-is into the output.
             output_lines.append(input_line)
           else:
             continue
           if input_line.strip() == '}':
             is_in_function = False
           continue
 
         # If it's outside a function, it just gets copied to the output.
         output_lines.append(input_line)
 
         m = common.IR_FUNCTION_RE.match(input_line)
         if not m:
           continue
         func_name = m.group(1)
         if args.function is not None and func_name != args.function:
           # When filtering on a specific function, skip all others.
           continue
         is_in_function = is_in_function_start = True
 
-    common.debug('Writing %d lines to %s...' % (len(output_lines), ti.path))
+    if ti.args.gen_unused_prefix_body:
+      output_lines.extend(ti.get_checks_for_unused_prefixes(
+          run_list, generated_prefixes))
     
+    common.debug('Writing %d lines to %s...' % (len(output_lines), ti.path))
     with open(ti.path, 'wb') as f:
       f.writelines(['{}\n'.format(l).encode('utf-8') for l in output_lines])
 
 
 if __name__ == '__main__':
   main()