In D76178#1950193, @ldionne wrote:

You're right that if one sets up substitutions in the correct order (and assuming no circular dependencies), a single pass is sufficient to expand substitutions. However, the problem is precisely that one usually doesn't have control over the order in which substitutions are set up,

Sorry to be late to this review....

In D71408#1788582, @thopre wrote:

This is embarrassing.

This breaks the lit test suite. llvm/utils/lit/tests/shtest-format.py needs to be updated for the removed tests.

Thanks for the quick review. Will push hopefully tomorrow.

I have little experience with REQUIRES, etc., but a quick git grep suggests Paul's suggestion is a reasonable path forward.

@probinson : I followed your testing strategy, which I agree is better. Thanks for the quick review!

This patch handles libgcc_s.so, but what about libatomic.so? It's also not provided by x86_64-linux-gnu, at least on CentOS Linux release 7.7.1908.

Before proceeding, keep in mind that D54769 is closely related and was reviewed at the same time, so the discussion there should be considered too.

LGTM. Thanks!

In D70105#1745634, @probinson wrote:

In D70105#1745051, @jdenny wrote:

I think it's using utils\lit\tests\lit.site.cfg, which produces the error, when it would normally use utils/lit/tests/Inputs/nonexistent/lit.cfg. Creating even an empty version of that file would probably fix this.

Hmmm then the directory wouldn't be nonexistent...

What do you think of placing that empty lit.cfg directly within Inputs instead? I think this would generally isolate all Inputs/* test suites from lit/tests/lit.site.cfg and thus avoid such confusion for future tests.

I think it's using utils\lit\tests\lit.site.cfg, which produces the error, when it would normally use utils/lit/tests/Inputs/nonexistent/lit.cfg. Creating even an empty version of that file would probably fix this.

The number of shards is greater than the number of tests. Currently, this is never an error. Personally, I think we should consider making this an error by default; if this happens, you are doing something wrong. I added a warning but did not change the behavior, since this warrants more discussion.

Now that D65156 and its many dependencies seem to have survived the bots, I'm finally returning to this patch.

It appears the last update before today did not add documentation to llvm/docs/TestingGuide.rst as I intended, but today's update did, so that's another new change to look at.

• Rebased onto recent master.
• Adjusted tests not to use diff as an example in-process builtin because it's now an out-of-process builtin.
• Fixed discovery of out-of-process builtins behind not.

Can you please try adding an errors='ignore' argument to the failing decode call? Or I can push a patch if that's easier. Thanks.

While the test suite was already passing for me in Windows and Ubuntu using either Python 2 or 3, it raised an exception when passing -a to lit (or if the test suite failed and managed to print output containing special characters) in Ubuntu when using Python 2. This change fixes that.

Rebased onto updated D66574.

Rebased this onto D69207.

Rebased onto the most recent D66574.

Rebased onto D68839 (recently re-landed). As a result, the diff.py created here has to import lit.util for to_string. The only way I found to make that possible was to set PYTHONPATH in TestRunner.py when calling diff.py.

Rebased onto D68664, which I previously forgot this patch requires. Otherwise, when using Windows or when using Python 3, a lit test introduced here fails because it expects \r\n and \n to be different while, when using Windows or Python 3, \r\n is read as \n in text mode. D68664 reads in binary mode and thus prevents that problem.

Rebased onto master so it doesn't depend on D66574 anymore. Thus, it modifies TestRunner.py instead of diff.py.

• Added --strip-trailing-cr to another test that needs it.
• Removed some accidental FileCheck options.
• Rebased onto master so it doesn't depend on D66574 anymore. Thus, it modifies TestRunner.py instead of diff.py.