[Utils] Allow update_test_checks to scrub attribute annotations

[jdoerfert]

Summary

Attribute annotations, e.g., #0, are not useful on their own. This
patch adds a flag to update_test_checks to scrub them.

[jdoerfert]

Make the decision per test (in preperation of D69701)

[lebedev.ri]

Now that i believe there's some testing infra for these utils, add/update tests for this?

[jdoerfert]

Add a regression test

[merge_guards_bot]

Unit tests: unknown.

clang-tidy: unknown.

clang-format: unknown.

Build artifacts: diff.json, console-log.txt

[lebedev.ri]

This is opt-in so should be ok.

[arichardson]

Could you also make this change to update_cc_test_checks.py?

This code could move to common and we add a generates_ir flag to common.parse_commandline_args(parser). If that flag is set, parse_commandline_args adds the --function-signature and --scrub-attributes command line option and performs the SCRUB_TRAILING_WHITESPACE_TEST_RE -> SCRUB_TRAILING_WHITESPACE_AND_ATTRIBUTES_RE subsitituion?
If it's in common there should be no need for a separate update_cc_test_checks test (however, it probably makes sense to wait until D71565 has landed to avoid regresssions).

[jdoerfert]

In D68851#1799748, @arichardson wrote:

Could you also make this change to update_cc_test_checks.py?

This code could move to common and we add a generates_ir flag to common.parse_commandline_args(parser). If that flag is set, parse_commandline_args adds the --function-signature and --scrub-attributes command line option and performs the SCRUB_TRAILING_WHITESPACE_TEST_RE -> SCRUB_TRAILING_WHITESPACE_AND_ATTRIBUTES_RE subsitituion?
If it's in common there should be no need for a separate update_cc_test_checks test (however, it probably makes sense to wait until D71565 has landed to avoid regresssions).

I'm in favor of this happening and I can put it on my list. Though, if you want me to do this it will take some time.

llvm/utils/UpdateTestChecks/common.py

 IR_FUNCTION_RE = re.compile(r'^\s*define\s+(?:internal\s+)?[^@]*@(\w+)\s*\(')
 TRIPLE_IR_RE = re.compile(r'^\s*target\s+triple\s*=\s*"([^"]+)"$')
 TRIPLE_ARG_RE = re.compile(r'-mtriple[= ]([^ ]+)')
 MARCH_ARG_RE = re.compile(r'-march[= ]([^ ]+)')
 
 SCRUB_LEADING_WHITESPACE_RE = re.compile(r'^(\s+)')
 SCRUB_WHITESPACE_RE = re.compile(r'(?!^(|  \w))[ \t]+', flags=re.M)
 SCRUB_TRAILING_WHITESPACE_RE = re.compile(r'[ \t]+$', flags=re.M)
+SCRUB_TRAILING_WHITESPACE_TEST_RE = SCRUB_TRAILING_WHITESPACE_RE
+SCRUB_TRAILING_WHITESPACE_AND_ATTRIBUTES_RE = re.compile(r'([ \t]|(#[0-9]+))+$', flags=re.M)
 SCRUB_KILL_COMMENT_RE = re.compile(r'^ *#+ +kill:.*\n')
 SCRUB_LOOP_COMMENT_RE = re.compile(
     r'# =>This Inner Loop Header:.*|# in Loop:.*', flags=re.M)
 
 
 def error(msg, test_file=None):
   if test_file:
     msg = '{}: {}'.format(msg, test_file)
   print('ERROR: {}'.format(msg), file=sys.stderr)
 
 def warn(msg, test_file=None):
   if test_file:
     msg = '{}: {}'.format(msg, test_file)
   print('WARNING: {}'.format(msg), file=sys.stderr)
 
 def scrub_body(body):
   # Scrub runs of whitespace out of the assembly, but leave the leading
   # whitespace in place.
   body = SCRUB_WHITESPACE_RE.sub(r' ', body)
   # Expand the tabs used for indentation.
   body = string.expandtabs(body, 2)
   # Strip trailing whitespace.
-  body = SCRUB_TRAILING_WHITESPACE_RE.sub(r'', body)
+  body = SCRUB_TRAILING_WHITESPACE_TEST_RE.sub(r'', body)
   return body
 
 def do_scrub(body, scrubber, scrubber_args, extra):
   if scrubber_args:
     local_args = copy.deepcopy(scrubber_args)
     local_args[0].extra_scrub = extra
     return scrubber(body, *local_args)
   return scrubber(body, *scrubber_args)

llvm/utils/update_test_checks.py

def main():
   parser.add_argument(
       '--function', help='The function in the test file to update')
   parser.add_argument('-u', '--update-only', action='store_true',
                       help='Only update test if it was already autogened')
   parser.add_argument('-p', '--preserve-names', action='store_true',
                       help='Do not scrub IR names')
   parser.add_argument('--function-signature', action='store_true',
                       help='Keep function signature information around for the check line')
+  parser.add_argument('--scrub-attributes', action='store_true',
+                      help='Remove attribute annotations (#0) from the end of check line')
   parser.add_argument('tests', nargs='+')
   args = parser.parse_args()
 
   script_name = os.path.basename(__file__)
   autogenerated_note = (ADVERT + 'utils/' + script_name)
 
   opt_basename = os.path.basename(args.opt_binary)
   if not re.match(r'^opt(-\d+)?$', opt_basename):
if 'autogenerated' in first_line and script_name not in first_line:
       common.warn("Skipping test which wasn't autogenerated by " + script_name, test)
       continue
 
     if args.update_only:
       if not first_line or 'autogenerated' not in first_line:
         common.warn("Skipping test which isn't autogenerated: " + test)
         continue
 
+    # If requested we scrub trailing attribute annotations, e.g., '#0', together with whitespaces
+    if args.scrub_attributes:
+      common.SCRUB_TRAILING_WHITESPACE_TEST_RE = common.SCRUB_TRAILING_WHITESPACE_AND_ATTRIBUTES_RE
+    else:
+      common.SCRUB_TRAILING_WHITESPACE_TEST_RE = common.SCRUB_TRAILING_WHITESPACE_RE
+
     raw_lines = [m.group(1)
                  for m in [common.RUN_LINE_RE.match(l) for l in input_lines] if m]
     run_lines = [raw_lines[0]] if len(raw_lines) > 0 else []
     for l in raw_lines[1:]:
       if run_lines[-1].endswith('\\'):
         run_lines[-1] = run_lines[-1].rstrip('\\') + ' ' + l
       else:
         run_lines.append(l)