[Utils] Deal with occasionally deleted functions

[jdoerfert]

Summary

When functions exist for some but not all run lines we need to be
careful when selecting the prefix. So far, a common prefix was
potentially chosen as there was never a "conflict" that would have
caused otherwise. With this patch we avoid common prefixes if they
are used by run lines that do not emit the function.

Tested as part of D68766 and the follow up that adds the Attributor test
lines to all argument promotion tests.

[jdoerfert]

The difference is when we run update_test_checks.py on the following code

; RUN: opt -S < %s | FileCheck %s --check-prefixes=ALL,FIRST
; RUN: opt -S -globalopt < %s | FileCheck %s --check-prefixes=ALL,SECOND

define internal void @f() {
  ret void
}

we will see

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -S < %s | FileCheck %s --check-prefixes=ALL,FIRST
; RUN: opt -S -globalopt < %s | FileCheck %s --check-prefixes=ALL,SECOND

define internal void @f() {
; FIRST-LABEL: define {{[^@]+}}@f(
; FIRST-NEXT:    ret void
;
  ret void
}

and not

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -S < %s | FileCheck %s --check-prefixes=ALL,FIRST
; RUN: opt -S -globalopt < %s | FileCheck %s --check-prefixes=ALL,SECOND

define internal void @f() {
; ALL-LABEL: @f(
; ALL-NEXT:    ret void
;
  ret void
}

---

ping

[lebedev.ri]

Now that i believe there's some testing infra for these utils, add/update tests for this?

[jdoerfert]

Add a regression test

[merge_guards_bot]

Unit tests: unknown.

clang-tidy: unknown.

clang-format: unknown.

Build artifacts: diff.json, console-log.txt

[lebedev.ri]

llvm/utils/UpdateTestChecks/common.py

270

I'm having a hard time figuring out whether this blacklist is per-function or global.
I.e. what happens if sometimes_deleted_function.ll has a second function,
that *does* exist for all check prefixes. Will it also only be checked in FIRST check prefix?

[jdoerfert]

Make the test more expressive

[jdoerfert]

llvm/utils/UpdateTestChecks/common.py

270

It's a *per function* blacklist. I updated the test to show that ALL is used for a function that is always present.

[lebedev.ri]

Really convoluted, but seems to be okay.

llvm/utils/UpdateTestChecks/common.py

270

Name is too generic, should maybe be prefix_blacklist.

[merge_guards_bot]

Unit tests: fail. 61156 tests passed, 1 failed and 728 were skipped.

failed: libc++.std/thread/thread_mutex/thread_mutex_requirements/thread_sharedtimedmutex_requirements/thread_sharedtimedmutex_class/try_lock.pass.cpp

clang-tidy: pass.

clang-format: pass.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

llvm/test/tools/UpdateTestChecks/update_test_checks/Inputs/sometimes_deleted_function.ll

+; RUN: opt -S < %s | FileCheck %s --check-prefixes=ALL,FIRST
+; RUN: opt -S -globalopt < %s | FileCheck %s --check-prefixes=ALL,SECOND
+;
+; Make sure we use FIRST to check for @sometimes_here as ALL does not work.
+
+define internal void @sometimes_here() {
+  ret void
+}
+
+define void @always_here() {
+  ret void
+}

llvm/test/tools/UpdateTestChecks/update_test_checks/Inputs/sometimes_deleted_function.ll.expected

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -S < %s | FileCheck %s --check-prefixes=ALL,FIRST
+; RUN: opt -S -globalopt < %s | FileCheck %s --check-prefixes=ALL,SECOND
+;
+; Make sure we use FIRST to check for @sometimes_here as ALL does not work.
+
+define internal void @sometimes_here() {
+; FIRST-LABEL: @sometimes_here(
+; FIRST-NEXT:    ret void
+;
+  ret void
+}
+
+define void @always_here() {
+; ALL-LABEL: @always_here(
+; ALL-NEXT:    ret void
+;
+  ret void
+}

llvm/test/tools/UpdateTestChecks/update_test_checks/sometimes_deleted_function.test

+# RUN: cp -f %S/Inputs/sometimes_deleted_function.ll %t.ll && %update_test_checks %t.ll
+# RUN: diff -u %t.ll %S/Inputs/sometimes_deleted_function.ll.expected
+## Check that running the script again does not change the result:
+# RUN: %update_test_checks %t.ll
+# RUN: diff -u %t.ll %S/Inputs/sometimes_deleted_function.ll.expected

llvm/utils/UpdateTestChecks/common.py

for i, line in enumerate(lines):
     if is_analyze:
       lines[i] = scrubbed_line
     else:
       lines[i] = IR_VALUE_RE.sub(transform_line_vars, scrubbed_line)
   return lines
 
 
 def add_checks(output_lines, comment_marker, prefix_list, func_dict, func_name, check_label_format, is_asm, is_analyze):
+  # blacklist are prefixes we cannot use to print the function because it doesn't exist in run lines that use these prefixes as well.
+  blacklist = set()
   printed_prefixes = []
   for p in prefix_list:
     checkprefixes = p[0]
+    # If not all checkprefixes of this run line produced the function we cannot check for it as it does not
+    # exist for this run line. A subset of the check prefixes might know about the function but only because
+    # other run lines created it.
+    if any(map(lambda checkprefix: func_name not in func_dict[checkprefix], checkprefixes)):
+        blacklist |= set(checkprefixes)
+        continue
+
+  # blacklist is constructed, we can now emit the output
+  for p in prefix_list:
+    checkprefixes = p[0]
+    saved_output = None
     for checkprefix in checkprefixes:
       if checkprefix in printed_prefixes:
         break
-      # TODO func_dict[checkprefix] may be None, '' or not exist.
-      # Fix the call sites.
-      if func_name not in func_dict[checkprefix] or not func_dict[checkprefix][func_name]:
+
+      # prefix is blacklisted. We remember the output as we might need it later but we will not emit anything for the prefix.
+      if checkprefix in blacklist:
+          if not saved_output and func_name in func_dict[checkprefix]:
+              saved_output = func_dict[checkprefix][func_name]
+          continue
+
+      # If we do not have output for this prefix but there is one saved, we go ahead with this prefix and the saved output.
+      if not func_dict[checkprefix][func_name]:
+        if not saved_output:
             continue
+        func_dict[checkprefix][func_name] = saved_output
 
       # Add some space between different check prefixes, but not after the last
       # check line (before the test code).
       if is_asm:
         if len(printed_prefixes) != 0:
           output_lines.append(comment_marker)
 
       printed_prefixes.append(checkprefix)