There are a lot of common pattern that results in testcases that look like they
are checking something, but in fact they silently hide failures, leading to
accidental false negatives.
CHECK: legitimate test CHECK gotcha 1 CHECK : gotcha 2 CHECK-SAME-DAG: gotcha 3 CHECKNEXT: gotcha 4 CHECKDAG: gotcha 5
This patch attempts to catch 1 & 2, but it unfortunately still has a lot of
false positives, as it triggers on comments in testcases that refer to the
CHECKs themselves. The vast majority of false positives come from this
diagnostic triggering on RUN lines, which could be ameliorated (mostly) with a
precursor patch that quotes all the CHECK prefixes:
find -E llvm/test -regex ".*\.(c|cpp|m|ll|mir|yaml|s|td|test|txt)" -exec perl -p -i -e "s/check-prefix(es)?[= ]([^ \r\n]+)/check-prefix\1=\"\2\"/g" {} \;
But even with that, there are still ~455 "failures" this causes in llvm/test alone.
Is this worth moving forward on? If so, how else can we improve FileCheck to
prevent these kinds of issues?
clang-format: please reformat the code
13 diff lines are omitted. See full diff.