This is an archive of the discontinued LLVM Phabricator instance.

llvm-reduce: Try to parse triple/datalayout from module
ClosedPublic

Authored by arsenm on Apr 19 2022, 6:18 AM.

Details

Summary

This saves needing to specify -mtriple on nearly every use for MIR
reduction.

Diff Detail

Event Timeline

arsenm created this revision.Apr 19 2022, 6:18 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 19 2022, 6:18 AM
arsenm requested review of this revision.Apr 19 2022, 6:18 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 19 2022, 6:18 AM
Herald added a subscriber: wdng. · View Herald Transcript
markus added a subscriber: reames.

I think @reames requested this back in D110527

arsenm updated this revision to Diff 423635.Apr 19 2022, 8:33 AM

A bit more diagnostic consistency

markus accepted this revision.Apr 20 2022, 5:29 AM

LGTM

This revision is now accepted and ready to land.Apr 20 2022, 5:29 AM
arsenm added inline comments.Apr 27 2022, 2:37 PM
llvm/tools/llvm-reduce/ReducerWorkItem.cpp
292

Added an ugly downcast, like llc, to avoid the dependence on D123964

haowei added a subscriber: haowei.Apr 27 2022, 4:18 PM

Hi,

We are seeing test failures on Fuchsia Clang builder for windows-x64 after this change have landed.

Failed tests are:

LLVM :: tools/llvm-reduce/fail-file-open.test
LLVM :: tools/llvm-reduce/mir/infer-triple-unknown-target.mir

Error messages:

Script:
--
: 'RUN: at line 1';   not llvm-reduce --test=echo C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "not" "llvm-reduce" "--test=echo" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir"
$ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test"
# command stderr:
C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test:5:10: error: CHECK: expected string not found in input
# CHECK: llvm-reduce: {{.*}}.NotAFileInTestingDir: error:
         ^
<stdin>:1:1: note: scanning from here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
^
<stdin>:1:81: note: possible intended match here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
                                                                                ^

Input file: <stdin>
Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory 
check:5'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:5'1                                                                                     ?                                                                                                     possible intended match
>>>>>>

error: command failed with exit status: 1

--

and

Script:
--
: 'RUN: at line 1';   not llvm-reduce --test c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir -o /dev/null 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "not" "llvm-reduce" "--test" "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir" "-o" "/dev/null"
$ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir"
# command stderr:
C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir:3:10: error: CHECK: expected string not found in input
# CHECK: llvm-reduce: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple.
         ^
<stdin>:1:1: note: scanning from here
llvm-reduce.exe: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple.
^

Input file: <stdin>
Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir

-dump-input=help explains the following input dump.

Input was:
<<<<<<
         1: llvm-reduce.exe: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple. 
check:3     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
>>>>>>

error: command failed with exit status: 1

--

It is quite obvious that both tests did not consider that Windows has ".exe" suffix to its executables

Could you fix the breaking tests? If it takes a while to fix, could you revert your change please? Thanks.

Hi,

We are seeing test failures on Fuchsia Clang builder for windows-x64 after this change have landed.

Failed tests are:

LLVM :: tools/llvm-reduce/fail-file-open.test
LLVM :: tools/llvm-reduce/mir/infer-triple-unknown-target.mir

Error messages:

Script:
--
: 'RUN: at line 1';   not llvm-reduce --test=echo C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "not" "llvm-reduce" "--test=echo" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir"
$ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test"
# command stderr:
C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test:5:10: error: CHECK: expected string not found in input
# CHECK: llvm-reduce: {{.*}}.NotAFileInTestingDir: error:
         ^
<stdin>:1:1: note: scanning from here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
^
<stdin>:1:81: note: possible intended match here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
                                                                                ^

Input file: <stdin>
Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory 
check:5'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:5'1                                                                                     ?                                                                                                     possible intended match
>>>>>>

error: command failed with exit status: 1

--

and

Script:
--
: 'RUN: at line 1';   not llvm-reduce --test c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir -o /dev/null 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "not" "llvm-reduce" "--test" "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir" "-o" "/dev/null"
$ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir"
# command stderr:
C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir:3:10: error: CHECK: expected string not found in input
# CHECK: llvm-reduce: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple.
         ^
<stdin>:1:1: note: scanning from here
llvm-reduce.exe: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple.
^

Input file: <stdin>
Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\mir\infer-triple-unknown-target.mir

-dump-input=help explains the following input dump.

Input was:
<<<<<<
         1: llvm-reduce.exe: error: unable to get target for 'omgwtfbbqcpu64--', see --version and --triple. 
check:3     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
>>>>>>

error: command failed with exit status: 1

--

It is quite obvious that both tests did not consider that Windows has ".exe" suffix to its executables

Could you fix the breaking tests? If it takes a while to fix, could you revert your change please? Thanks.

This is already fixed

thakis added a subscriber: thakis.Apr 27 2022, 4:55 PM

No, it's still broken. Windows also prints the "N" of "No such file" in upper case.

See e.g. http://45.33.8.238/win/57129/step_11.txt for a failure log at HEAD.

Test LLVM :: tools/llvm-reduce/fail-file-open.test is still failing on our windows bots after the fix:

Script:
--
: 'RUN: at line 1';   not llvm-reduce --test=echo C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "not" "llvm-reduce" "--test=echo" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir"
$ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test"
# command stderr:
C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test:5:10: error: CHECK: expected string not found in input
# CHECK: llvm-reduce: {{.*}}.NotAFileInTestingDir: error:
         ^
<stdin>:1:1: note: scanning from here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
^
<stdin>:1:81: note: possible intended match here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
                                                                                ^

Input file: <stdin>
Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory 
check:5'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:5'1                                                                                     ?                                                                                                     possible intended match
>>>>>>

error: command failed with exit status: 1

--

This test still did not consider the ".exe" suffix on Windows

LLVM :: tools/llvm-reduce/mir/infer-triple-unknown-target.mir did passed on our builder but I think the capital "N" issue depends on the toolchain and SDK you use on Windows. Test like lld/test/COFF/driver.test use something like # RUN: FileCheck -DMSG=%errc_ENOENT -check-prefix=MISSING %s < %t.log to avoid this issue. And there is a dedicate CMake step in https://github.com/llvm/llvm-project/blob/c265170110b2e3b82ab8fca6658bfbbce79381c5/llvm/cmake/modules/GetErrcMessages.cmake#L31 to generate correct error string under different OS.

You probably want to use this approach in your tests.

Test LLVM :: tools/llvm-reduce/fail-file-open.test is still failing on our windows bots after the fix:

Script:
--
: 'RUN: at line 1';   not llvm-reduce --test=echo C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir 2>&1 | c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "not" "llvm-reduce" "--test=echo" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir"
$ "c:\b\s\w\ir\x\w\staging\llvm_build\bin\filecheck.exe" "C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test"
# command stderr:
C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test:5:10: error: CHECK: expected string not found in input
# CHECK: llvm-reduce: {{.*}}.NotAFileInTestingDir: error:
         ^
<stdin>:1:1: note: scanning from here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
^
<stdin>:1:81: note: possible intended match here
llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory
                                                                                ^

Input file: <stdin>
Check file: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: llvm-reduce.exe: C:\b\s\w\ir\x\w\llvm-llvm-project\llvm\test\tools\llvm-reduce\fail-file-open.test.NotAFileInTestingDir: error: Could not open input file: no such file or directory 
check:5'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:5'1                                                                                     ?                                                                                                     possible intended match
>>>>>>

error: command failed with exit status: 1

--

This test still did not consider the ".exe" suffix on Windows

LLVM :: tools/llvm-reduce/mir/infer-triple-unknown-target.mir did passed on our builder but I think the capital "N" issue depends on the toolchain and SDK you use on Windows. Test like lld/test/COFF/driver.test use something like # RUN: FileCheck -DMSG=%errc_ENOENT -check-prefix=MISSING %s < %t.log to avoid this issue. And there is a dedicate CMake step in https://github.com/llvm/llvm-project/blob/c265170110b2e3b82ab8fca6658bfbbce79381c5/llvm/cmake/modules/GetErrcMessages.cmake#L31 to generate correct error string under different OS.

You probably want to use this approach in your tests.

I’m not sure I understand this approach. Maybe it’s just not worth testing the error message prefix

(ps: please include a line like "Differential Revision: D123998" in the commit message of https://github.com/llvm/llvm-project/commit/1747a93b28c3ee7de45c4591186b0972f799ba13 going forward, to make it easier to go from commit to review.)

Test is still broken on Windows. Time to revert?

Test is still broken on Windows. Time to revert?

I just weakened the check in cf68b31f14a9de549a2f6052cc68ca1cc8198302

Apologies, looks like the remaining failure (http://45.33.8.238/win/57162/step_11.txt) isn't due to this change but due to e39e9d339c94cb7f421403a2980998b4a4b6a1a1. (I failed to find the right review because the commit message didn't link to the review.) I tried fixing that failure in ddfffbeb319aa0cd0eb11c81e2cf989d38ab8a5f , let's see if it helps.

There's a windows bot on the public waterfall (https://lab.llvm.org/buildbot/#/console) at https://lab.llvm.org/buildbot/#/builders/123 that you can use to make sure your fixes work. Keeping a bot red for > 15h isn't ideal, of course.