This is an archive of the discontinued LLVM Phabricator instance.

[lit] Make internal diff work in pipelines
ClosedPublic

Authored by jdenny on Aug 21 2019, 7:43 PM.

Details

Summary

When using lit's internal shell, RUN lines like the following
accidentally execute an external diff instead of lit's internal
diff:

# RUN: program | diff file -
# RUN: not diff file1 file2 | FileCheck %s

Such cases exist now, in clang/test/Analysis for example. We are
preparing patches to ensure lit's internal diff is called in such
cases, which will then fail because lit's internal diff cannot
currently be used in pipelines and doesn't recognize - as a
command-line option.

To enable pipelines, this patch moves lit's diff implementation into
an out-of-process script, similar to lit's cat implementation. A
follow-up patch will implement - to mean stdin.

Also, when lit's diff prints differences to stdout in Windows, this
patch ensures it always terminate lines with \n not \r\n. That
way, strict FileCheck directives checking the diff output succeed in
both Linux and Windows. This wasn't an issue when diff was internal
to lit because diff didn't then write to the true stdout, which is
where the \n -> \r\n conversion happened in Python.

Diff Detail

Event Timeline

jdenny created this revision.Aug 21 2019, 7:43 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 21 2019, 7:43 PM
Herald added a subscriber: delcypher. · View Herald Transcript

+Maggie who did the original diff implementation, AFAICT.

As before, someone with actual Python chops should review this. I've made some functionality remarks.

llvm/utils/lit/lit/builtin_commands/diff.py
197

I've also seen "a" and "U1" in my searches.
Does this correctly handle combined short options? e.g. "-aub" which I see used a fair amount.
I didn't notice --strip-trailing-cr anywhere but presumably the original author needed it, so probably should keep it.
I was going to say we don't need to bother with "r" but then I found a case that used it!

llvm/utils/lit/tests/shtest-shell.py
7

Is this for your temporary benefit or did you intend to make this change permanent? "-color" in particular seems odd here.

jdenny marked 2 inline comments as done.Aug 22 2019, 7:35 AM

+Maggie who did the original diff implementation, AFAICT.

Good idea.

Thanks for the review.

llvm/utils/lit/lit/builtin_commands/diff.py
197

I've also seen "a" and "U1" in my searches.

Looks like those will be easy to implement. difflib appears to provide functionality for the latter. I'll work on them.

Does this correctly handle combined short options? e.g. "-aub" which I see used a fair amount.

I believe so, but I'll add some tests to be sure.

Thanks for researching the current usage!

llvm/utils/lit/tests/shtest-shell.py
7

Thanks for catching that. Sorry, I meant to remove it.

jdenny updated this revision to Diff 216971.Aug 23 2019, 3:59 PM
jdenny marked an inline comment as done.
jdenny edited the summary of this revision. (Show Details)

I made several changes:

  • Added missing functools

I made several changes:

  • Added missing functools
  • ... import to diff.py.
  • Extracted - implementation. I'll put that in a separate patch to make it easier to review.
  • Fixed some missing newlines in diagnostics (bugs I introduced when converting from exceptions).
  • Cleaned up some tests.
jdenny marked 2 inline comments as done.Aug 23 2019, 4:15 PM
jdenny added inline comments.
llvm/utils/lit/lit/builtin_commands/diff.py
197

I'll address these in separate patches.

jdenny edited the summary of this revision. (Show Details)Aug 25 2019, 7:21 AM
jdenny set the repository for this revision to rG LLVM Github Monorepo.
stella.stamenova accepted this revision.Sep 11 2019, 5:35 AM

LGTM as long as all the tests still pass

This revision is now accepted and ready to land.Sep 11 2019, 5:35 AM

LGTM as long as all the tests still pass

Thanks. Yes, they pass for me. I'm busy now, but I'll try to find some time soon to push this and propose the next patch (support for - to mean stdin).

This revision was automatically updated to reflect the committed changes.

I had to revert this because it broke a Windows bot. I didn't consider that this patch not only enables internal diff to function in pipelines but also causes it to be used instead of external diffs when in pipelines. Thus, it cannot be pushed without additional patches that add features like -, which are sometimes used in the test suite when diff appears in pipelines.

jdenny updated this revision to Diff 225334.Oct 16 2019, 5:51 PM

Rebased onto D68839 (recently re-landed). As a result, the diff.py created here has to import lit.util for to_string. The only way I found to make that possible was to set PYTHONPATH in TestRunner.py when calling diff.py.

jdenny updated this revision to Diff 225715.Oct 18 2019, 4:37 PM
jdenny edited the summary of this revision. (Show Details)

Rebased this onto D69207.

Changed diff so that, when printing differences to stdout in Windows, it always terminate lines with \n not \r\n. That way, strict FileCheck directives checking the diff output succeed in both Linux and Windows. This somehow wasn't an issue when diff
was internal to lit.

jdenny edited the summary of this revision. (Show Details)Oct 18 2019, 4:52 PM
llvm/utils/lit/tests/Inputs/shtest-shell/diff-pipes.txt