Page MenuHomePhabricator

[clangd] Errors in TestTU cause test failures unless suppressed with error-ok.

Authored by sammccall on Jan 22 2020, 7:38 AM.



The historic behavior of TestTU is to gather diagnostics and otherwise ignore
them. So if a test has a syntax error, and doesn't assert diagnostics, it
silently misbehaves.
This can be annoying when developing tests, as evidenced by various tests
gaining "assert no diagnostics" where that's not really the point of the test.

This patch aims to make that default behavior. For the first error
(not warning), TestTU will call ADD_FAILURE().

This can be suppressed with a comment containing "error-ok". For now that will
suppress any errors in the TU. We can make this stricter later -verify style.
(-verify itself is hard to reuse because of DiagnosticConsumer interfaces...)
A magic-comment was chosen over a TestTU option because of table-driven tests.

In addition to the behavior change, this patch:

  • adds //error-ok where we're knowingly testing invalid code (e.g. for diagnostics, crash-resilience, or token-level tests)
  • fixes a bunch of errors in the checked-in tests, mostly trivial (missing ;)
  • removes a bunch of now-redundant instances of "assert no diagnostics"

Diff Detail

Event Timeline

sammccall created this revision.Jan 22 2020, 7:38 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 22 2020, 7:38 AM

Unit tests: unknown.

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt

kadircet accepted this revision.Jan 23 2020, 12:12 AM

LGTM, thanks for the cleanup!


should we make this more obscure, something like // TESTTU: error-ok ?

Even though it sounds implausible, just in case someone spells error-ok as part of the test, and silently drops the error :D

This revision is now accepted and ready to land.Jan 23 2020, 12:12 AM
sammccall marked an inline comment as done.Jan 24 2020, 2:07 AM
sammccall added inline comments.

I don't think this is the worst or most-likely failure mode, compared to suppressing too many errors.
The extra-verbosity would make it annoying to put inline.

I'd rather switch to clang -verify or otherwise ensure it only suppresses errors on the same line.
But that adds complexity, and this is a strict improvement on what we have today.

This revision was automatically updated to reflect the committed changes.