This saves needing to specify -mtriple on nearly every use for MIR
reduction.
Details
- Reviewers
markus lebedev.ri aeubanks reames
Diff Detail
Event Timeline
Hi,
We are seeing test failures on Fuchsia Clang builder for windows-x64 after this change have landed.
Failed tests are:
LLVM :: tools/llvm-reduce/fail-file-open.test LLVM :: tools/llvm-reduce/mir/infer-triple-unknown-target.mir
Error messages:
Script: -- : 'RUN: at line 1'; not llvm-reduce --test=echo C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test -- Exit Code: 1 Command Output (stdout): -- $ ":" "RUN: at line 1" $ "not" "llvm-reduce" "--test=echo" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir" $ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test" # command stderr: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test:5:10: error: CHECK: expected string not found in input # CHECK: llvm-reduce: {{.*}}.NotAFileInTestingDir: error: ^ <stdin>:1:1: note: scanning from here llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory ^ <stdin>:1:81: note: possible intended match here llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory ^ Input file: <stdin> Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test -dump-input=help explains the following input dump. Input was: <<<<<< 1: llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory check:5'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found check:5'1 ? possible intended match >>>>>> error: command failed with exit status: 1 --
and
Script: -- : 'RUN: at line 1'; not llvm-reduce --test c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir -o /dev/null 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir -- Exit Code: 1 Command Output (stdout): -- $ ":" "RUN: at line 1" $ "not" "llvm-reduce" "--test" "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir" "-o" "/dev/null" $ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir" # command stderr: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir:3:10: error: CHECK: expected string not found in input # CHECK: llvm-reduce: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple. ^ <stdin>:1:1: note: scanning from here llvm-reduce.exe: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple. ^ Input file: <stdin> Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir -dump-input=help explains the following input dump. Input was: <<<<<< 1: llvm-reduce.exe: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple. check:3 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found >>>>>> error: command failed with exit status: 1 --
It is quite obvious that both tests did not consider that Windows has ".exe" suffix to its executables
Could you fix the breaking tests? If it takes a while to fix, could you revert your change please? Thanks.
No, it's still broken. Windows also prints the "N" of "No such file" in upper case.
See e.g. http://45.33.8.238/win/57129/step_11.txt for a failure log at HEAD.
Test LLVM :: tools/llvm-reduce/fail-file-open.test is still failing on our windows bots after the fix:
Script: -- : 'RUN: at line 1'; not llvm-reduce --test=echo C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test -- Exit Code: 1 Command Output (stdout): -- $ ":" "RUN: at line 1" $ "not" "llvm-reduce" "--test=echo" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir" $ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test" # command stderr: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test:5:10: error: CHECK: expected string not found in input # CHECK: llvm-reduce: {{.*}}.NotAFileInTestingDir: error: ^ <stdin>:1:1: note: scanning from here llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory ^ <stdin>:1:81: note: possible intended match here llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory ^ Input file: <stdin> Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test -dump-input=help explains the following input dump. Input was: <<<<<< 1: llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory check:5'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found check:5'1 ? possible intended match >>>>>> error: command failed with exit status: 1 --
This test still did not consider the ".exe" suffix on Windows
LLVM :: tools/llvm-reduce/mir/infer-triple-unknown-target.mir did passed on our builder but I think the capital "N" issue depends on the toolchain and SDK you use on Windows. Test like lld/test/COFF/driver.test use something like # RUN: FileCheck -DMSG=%errc_ENOENT -check-prefix=MISSING %s < %t.log to avoid this issue. And there is a dedicate CMake step in https://github.com/llvm/llvm-project/blob/c265170110b2e3b82ab8fca6658bfbbce79381c5/llvm/cmake/modules/GetErrcMessages.cmake#L31 to generate correct error string under different OS.
You probably want to use this approach in your tests.
I’m not sure I understand this approach. Maybe it’s just not worth testing the error message prefix
(ps: please include a line like "Differential Revision: D123998" in the commit message of https://github.com/llvm/llvm-project/commit/1747a93b28c3ee7de45c4591186b0972f799ba13 going forward, to make it easier to go from commit to review.)
Apologies, looks like the remaining failure (http://45.33.8.238/win/57162/step_11.txt) isn't due to this change but due to e39e9d339c94cb7f421403a2980998b4a4b6a1a1. (I failed to find the right review because the commit message didn't link to the review.) I tried fixing that failure in ddfffbeb319aa0cd0eb11c81e2cf989d38ab8a5f , let's see if it helps.
There's a windows bot on the public waterfall (https://lab.llvm.org/buildbot/#/console) at https://lab.llvm.org/buildbot/#/builders/123 that you can use to make sure your fixes work. Keeping a bot red for > 15h isn't ideal, of course.
Added an ugly downcast, like llc, to avoid the dependence on D123964