HomePhabricator

[lit] Protect full test suite from FILECHECK_OPTS

Authored by jdenny on Jul 24 2019, 8:14 PM.

Description

[lit] Protect full test suite from FILECHECK_OPTS

lit's test suite calls lit multiple times for various sample test
suites. FILECHECK_OPTS is safe for FileCheck calls in lit's test
suite. It's not safe for FileCheck calls in the sample test suites,
whose output affects the results of lit's test suite.

Without this patch, only one such sample test suite is protected from
FILECHECK_OPTS, and currently shtest-shell.py breaks with
FILECHECK_OPTS=-vv. Moreover, it's hard to predict the future,
especially false passes. Thus, this patch protects all existing and
future sample test suites from FILECHECK_OPTS (and the deprecated
FILECHECK_DUMP_INPUT_ON_FAILURE).

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D65156