[test] Use host platform specific error message substitution in lit tests

[abhina.sreeskantharajan]

Summary

On z/OS, the following error message is not matched correctly in lit tests.

EDC5129I No such file or directory.

This patch uses a lit config substitution to check for platform specific error messages.

[muiez]

LGTM

[jhenderson]

Sorry, could you revert this please. I don't think this is a good fix, as you've reduced coverage within the test, and some changes are completly redundant/add extra noise. I've commented inline with examples. Skimming D94239 suggests that has the same issue.

Could you also please explain the messages your system is actually producing so we can provide a better solution than this fix.

I'm also concerned that you are doing this to fix this test for a system, yet there are no build bots that report this error. A future contributor is likely to break them in the same way/add new tests with the same issue. If your system is a system that is supposed to be supported by LLVM, there needs to be a build bot. If it isn't supported, you should bring this up on llvm-dev (if you haven't already) to get buy-in for support.

clang/test/Driver/clang-offload-bundler.c

73–75

Here and everywhere you've added the {{.*}}, the test will no longer pick up changes in the message that occur between the end of the literal match and the start of the "no such file..." part. For example, this will no longer fail if the message reported became "error: 'afile': fsdufbsdufbno such file or directory"

Even were adding .* desirable, it should be part of the following regex, i.e. {{.*N|n}} to reduce the noise of the {{ and }}.

llvm/test/Object/archive-extract-dir.test

11 ↗

(On Diff #318581)

{{.*}} at the end of a FileCheck line has literally no benefit - it is implicit unless --match-full-lines is specified to FileCheck.

[grimar]

As far I understand, looking on the description of D94239, the message on z/OS looks like "EDC5129I No such file or directory.".
I guess the EDC5129I is a stable error code? So why not to check for a possible EDC5129I prefix word instead of .*?
(The same applies for other possible errors)

[abhina.sreeskantharajan]

In D95246#2518989, @jhenderson wrote:

Sorry, could you revert this please. I don't think this is a good fix, as you've reduced coverage within the test, and some changes are completly redundant/add extra noise. I've commented inline with examples. Skimming D94239 suggests that has the same issue.

Could you also please explain the messages your system is actually producing so we can provide a better solution than this fix.

I'm also concerned that you are doing this to fix this test for a system, yet there are no build bots that report this error. A future contributor is likely to break them in the same way/add new tests with the same issue. If your system is a system that is supposed to be supported by LLVM, there needs to be a build bot. If it isn't supported, you should bring this up on llvm-dev (if you haven't already) to get buy-in for support.

Thanks for the feedback. I've reverted my changes from these two patches. We have indicated that we wish to add support for the z/OS platform and we are working on getting a buildbot set up.

In D95246#2519086, @grimar wrote:

As far I understand, looking on the description of D94239, the message on z/OS looks like "EDC5129I No such file or directory.".
I guess the EDC5129I is a stable error code? So why not to check for a possible EDC5129I prefix word instead of .*?
(The same applies for other possible errors)

As grimar noted, this is indeed the correct error message. "EDC5129I No such file or directory." (Note the extra period at the end)
Based on your feedback, these are the better alternatives that were suggested:

'{{.*N|n}}o such file or directory'

{{EDC5129I N|N|n}}o such file or directory'

Some testcases fail because of the extra period at the end. For those testcases, this is a possible alternative.

{{.*N|n}}o such file or directory{{\.?}}

Please let me know if there are better alternatives I could look into.

[jhenderson]

In D95246#2519642, @abhina.sreeskantharajan wrote:

In D95246#2518989, @jhenderson wrote:

Sorry, could you revert this please. I don't think this is a good fix, as you've reduced coverage within the test, and some changes are completly redundant/add extra noise. I've commented inline with examples. Skimming D94239 suggests that has the same issue.

Could you also please explain the messages your system is actually producing so we can provide a better solution than this fix.

I'm also concerned that you are doing this to fix this test for a system, yet there are no build bots that report this error. A future contributor is likely to break them in the same way/add new tests with the same issue. If your system is a system that is supposed to be supported by LLVM, there needs to be a build bot. If it isn't supported, you should bring this up on llvm-dev (if you haven't already) to get buy-in for support.

Thanks for the feedback. I've reverted my changes from these two patches. We have indicated that we wish to add support for the z/OS platform but we have not set up a buildbot yet.

In D95246#2519086, @grimar wrote:

As far I understand, looking on the description of D94239, the message on z/OS looks like "EDC5129I No such file or directory.".
I guess the EDC5129I is a stable error code? So why not to check for a possible EDC5129I prefix word instead of .*?
(The same applies for other possible errors)

As grimar noted, this is indeed the correct error message. "EDC5129I No such file or directory." (Note the extra period at the end)
Based on your feedback, these are the better alternatives that were suggested:

Slightly off-the-wall idea: I'm assuming you don't control your system in such a way that you can change the error message to omit the error code?

'{{.*N|n}}o such file or directory'

{{EDC5129I N|N|n}}o such file or directory'

Some testcases fail because of the extra period at the end. For those testcases, this is a possible alternative.

{{.*N|n}}o such file or directory{{\.?}}

Please let me know if there are better alternatives I could look into.

I think you can just omit the trailing full stop in those cases. If the test isn't using --match-full-lines, it should be fine. If it is, adding {{.?}} seems reasonable.

Having the error code explicitly in the pattern looks like the right solution for now, but a thought on that - it seems like tests will still have the fragility problem for when someone else writes a new test that checks the message due to the error code not being present on most systems. Is the error code different for each system error message (I'm guessing it is)? I wonder if we would be better off adding some sort of lit substitution or similar that expands to the right string for the given host OS, which could in turn be fed to FileCheck. It might look a bit like this in practice:

# RUN: not do-a-thing %t 2>&1 | FileCheck %s -DMSG=%enoent -DFILE=%t

# CHECK: error: '[[FILE]]': [[MSG]]

What do you think?

[abhina.sreeskantharajan]

In D95246#2519729, @jhenderson wrote:

In D95246#2519642, @abhina.sreeskantharajan wrote:

In D95246#2518989, @jhenderson wrote:

Sorry, could you revert this please. I don't think this is a good fix, as you've reduced coverage within the test, and some changes are completly redundant/add extra noise. I've commented inline with examples. Skimming D94239 suggests that has the same issue.

Could you also please explain the messages your system is actually producing so we can provide a better solution than this fix.

I'm also concerned that you are doing this to fix this test for a system, yet there are no build bots that report this error. A future contributor is likely to break them in the same way/add new tests with the same issue. If your system is a system that is supposed to be supported by LLVM, there needs to be a build bot. If it isn't supported, you should bring this up on llvm-dev (if you haven't already) to get buy-in for support.

Thanks for the feedback. I've reverted my changes from these two patches. We have indicated that we wish to add support for the z/OS platform but we have not set up a buildbot yet.

In D95246#2519086, @grimar wrote:

As far I understand, looking on the description of D94239, the message on z/OS looks like "EDC5129I No such file or directory.".
I guess the EDC5129I is a stable error code? So why not to check for a possible EDC5129I prefix word instead of .*?
(The same applies for other possible errors)

As grimar noted, this is indeed the correct error message. "EDC5129I No such file or directory." (Note the extra period at the end)
Based on your feedback, these are the better alternatives that were suggested:

Slightly off-the-wall idea: I'm assuming you don't control your system in such a way that you can change the error message to omit the error code?

Right, I'm not able to change the error message.

'{{.*N|n}}o such file or directory'

{{EDC5129I N|N|n}}o such file or directory'

Some testcases fail because of the extra period at the end. For those testcases, this is a possible alternative.

{{.*N|n}}o such file or directory{{\.?}}

Please let me know if there are better alternatives I could look into.

I think you can just omit the trailing full stop in those cases. If the test isn't using --match-full-lines, it should be fine. If it is, adding {{.?}} seems reasonable.

Having the error code explicitly in the pattern looks like the right solution for now, but a thought on that - it seems like tests will still have the fragility problem for when someone else writes a new test that checks the message due to the error code not being present on most systems. Is the error code different for each system error message (I'm guessing it is)? I wonder if we would be better off adding some sort of lit substitution or similar that expands to the right string for the given host OS, which could in turn be fed to FileCheck. It might look a bit like this in practice:

# RUN: not do-a-thing %t 2>&1 | FileCheck %s -DMSG=%enoent -DFILE=%t

# CHECK: error: '[[FILE]]': [[MSG]]

What do you think?

I like the lit substitution solution, it will be a lot cleaner compared to a complicated regex. I've noticed there are already different regex for the same error message so this will help the error messages be uniform as well. I can look into implementing this.

[abhina.sreeskantharajan]

I've implemented the initial solution suggested by James. I haven't updated all the testcases yet because I wanted to get feedback on the implementation first.

[abhina.sreeskantharajan]

FIx CI, check if host_triple exists.

[grimar]

I like this approach.

[jhenderson]

I did briefly consider one alternative, which was to build these directly into FileCheck, but it doesn't quite feel like the right approach for FileCheck to need to know system level details. In practice, I think the lit substitution may be the best way forward overall. In the future, it should be easy to expand with more error messages.

You should add the new substitution to the lit command guide, so that it's documented. It might also be worth a heads up on llvm-dev to get feedback from those not directly on this review, to see if there are other ideas.

llvm/utils/lit/lit/llvm/config.py

349–354

These lines are quite long, so probably want reflowing.

I wonder if %errc_... might be a better name? That way, it ties to the std::errc values these match up with.

369–370

Under what conditions can there not be a host_triple? In those cases, what happens to the tests that use the new substitution?

372

Nit: too many blank lines (compared to what was there before)

[abhina.sreeskantharajan]

This patch makes the following changes:

• Define LLVM_HOST_TRIPLE for lld tests. (This was the project that didn't have host defined.)
• Change %err_no_such_file_or_directory to %errc_ENOENT as suggested by jhenderson

I've also created a post on llvm-dev about this patch: https://lists.llvm.org/pipermail/llvm-dev/2021-January/148089.html

[abhina.sreeskantharajan]

llvm/utils/lit/lit/llvm/config.py

349–354

Thanks, I've changed the error messages to your suggestion.

369–370

This was not defined for lld. I added changes to define this for lld and removed the check. I think this is defined in all the other projects.

[abhina.sreeskantharajan]

Fix CI: Other projects like flang also do not specify LLVM_HOST_TRIPLE. If this is not specified, set the triple to an empty string.

I also added one more error code %errc_EISDIR and updated the TestingGuide.
Please let me know if there are other guides I will need to update that I'm not aware of.

[jhenderson]

In D95246#2522913, @abhina.sreeskantharajan wrote:

Please let me know if there are other guides I will need to update that I'm not aware of.

There's also the lit CommandGuide located at llvm/docs/CommandGuide/lit.rst.

clang/test/Driver/clang-offload-bundler.c

73–76

Nit: These are probably good candidates to split up over multiple lines, as in the inline edit.

lld/CMakeLists.txt

115

The target and the host may be two completely different things. This variable setting doesn't look right to me. In a situation where someone is building LLD to run on Windows but targeting Linux, the host triple value will end up being set to a Linux triple, and the tests would fail.

The variable setting can also be moved inside the earlier if about 5 lines up, because that block is hit when LLD_BUILD_STANDALONE is set.

llvm/docs/TestingGuide.rst

545–547 ↗

(On Diff #319309)

This is a slightly different format to the other examples above. I think it should be like the inline edit. this also helps distinguish the difference between the two.

Note: I can't remember which way around the two versions are, so you might need to swap them!

llvm/utils/lit/lit/llvm/config.py

349–351

I'm concerned that someone might start using these substitutions in a project for the first time, and get confused why they don't work on non-windows platforms. Maybe the solution is simply to require LLVM_HOST_TRIPLE to be set in all projects, i.e. go back to what you were doing before, and letting python fail if it isn't set.

Happy to hear other ideas too.

[abhina.sreeskantharajan]

I've changed the host check to use python's sys.platform to determine the host as an alternative to using LLVM_HOST_TRIPLE. It will save us the effort of making sure it's defined in all projects as well.

I noticed that llvm/docs/CommandGuide/lit.rst only lists substitutions that are done in TestRunner.py and refers to llvm/docs/TestingGuide.rst for information on the remaining substitutions. I think if we move the error substitution code into TestRunner.py, then it would be appropriate to update the lit.rst. Where do you think is the appropriate place to define the error substitutions, TestRunner.py or config.py?

[abhina.sreeskantharajan]

llvm/utils/lit/lit/llvm/config.py

349–351

I think using sys.platform or platform.system() could be a better alternative. What do you think?

[abhina.sreeskantharajan]

Fix CI: Rebase

[jhenderson]

clang/test/Driver/clang-offload-bundler.c

74

Add one more space - it helps the indentation stand out more and therefore make it clearer this line is a continuation.

76

llvm/docs/TestingGuide.rst

545–547 ↗

(On Diff #319309)

We probably need to list what error codes are currently supported somewhere.

llvm/utils/lit/lit/llvm/config.py

349–351

Makes sense to me. Slight issue: cygwin on Windows uses cygwin. What error message does it produce though?

[abhina.sreeskantharajan]

llvm/utils/lit/lit/llvm/config.py

349–351

I tested this out. On a cygwin terminal on Windows it emits the same error message as Linux so it should take the else path.

[abhina.sreeskantharajan]

Address syntax comments, and add a list of supported error code substitutions in the guide.

If these changes are ok, I would like to start making changes to all affected testcases.

[jhenderson]

One nit, but otherwise looks good to me, thanks! Please go ahead with the other test updates. Do you plan on doing them in this patch?

llvm/docs/TestingGuide.rst

545 ↗

(On Diff #319839)

Adding "currently" implies to me that someone could add to the list easily.

[abhina.sreeskantharajan]

In D95246#2527961, @jhenderson wrote:

One nit, but otherwise looks good to me, thanks! Please go ahead with the other test updates. Do you plan on doing them in this patch?

This was my initial thought. But if it's preferred to create a second patch to make the remaining changes to affected testcases, I have no issue with doing that.

[abhina.sreeskantharajan]

Add currently in TestingGuide.

[jhenderson]

In D95246#2527999, @abhina.sreeskantharajan wrote:

In D95246#2527961, @jhenderson wrote:

One nit, but otherwise looks good to me, thanks! Please go ahead with the other test updates. Do you plan on doing them in this patch?

This was my initial thought. But if it's preferred to create a second patch to make the remaining changes to affected testcases, I have no issue with doing that.

I'm happy either way, whichever you prefer.

[abhina.sreeskantharajan]

Update all testcases to use -DMSG.
I've also updated the summary to match the changes in the patch.

[jhenderson]

Great work! LGTM.

[ASDenysPetrov]

This patch causes fails in a bunch of test files. When I roll it back, it passes.
Below you can see fail output of ubsan-blacklist-vfs.c and basic-block-sections.c.

fail_output.txt17 KBDownload

Please, pay attention to this.
I'm using build on Win10 + GCC config.

P.S. The similar trouble is in D95808. I've commented there as well.

[ASDenysPetrov]

One more mention. There are a lot of popups like this which stops the process untill press OK.

That's very annoying. Please revert or fix this patch.

[abhina.sreeskantharajan]

In D95246#2564137, @ASDenysPetrov wrote:

This patch causes fails in a bunch of test files. When I roll it back, it passes.
Below you can see fail output of ubsan-blacklist-vfs.c and basic-block-sections.c.

fail_output.txt17 KBDownload

Please, pay attention to this.
I'm using build on Win10 + GCC config.

P.S. The similar trouble is in D95808. I've commented there as well.

Hi Denys, thanks for bringing this to my attention, I'll look into this failure. I do know that the CI test for this patch and the other patch ran on Windows and did not report any errors. https://buildkite.com/llvm-project/premerge-checks/builds/23727#63570ad0-0e74-4825-b1dc-8f53a4d4dad0
Do you know what is different between your environments which is causing the spelling to be capitalized? This might help me determine how to fix the current host platform check in llvm/utils/lit/lit/llvm/config.py.

[jhenderson]

In D95246#2565351, @abhina.sreeskantharajan wrote:

Do you know what is different between your environments which is causing the spelling to be capitalized? This might help me determine how to fix the current host platform check in llvm/utils/lit/lit/llvm/config.py.

Could it be something to do with a different standard library being used?

[ASDenysPetrov]

In D95246#2565351, @abhina.sreeskantharajan wrote:

Do you know what is different between your environments which is causing the spelling to be capitalized? This might help me determine how to fix the current host platform check in llvm/utils/lit/lit/llvm/config.py.

I'd love to help you. I'm using ordinary Win10. Here is my python output:

>>> os.name
'nt'
>>> sys.platform
'win32'
>>> platform.system()
'Windows'
>>> platform.release()
'10'
>>>

If you would say what exact info you need I would execute and send you.
BTW my python3 uses Sentence case capitalization in errors as you can see:

>>> open('some')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'some'
>>>

[abhina.sreeskantharajan]

In D95246#2566029, @ASDenysPetrov wrote:

In D95246#2565351, @abhina.sreeskantharajan wrote:

Do you know what is different between your environments which is causing the spelling to be capitalized? This might help me determine how to fix the current host platform check in llvm/utils/lit/lit/llvm/config.py.

I'd love to help you. I'm using ordinary Win10. Here is my python output:

>>> os.name
'nt'
>>> sys.platform
'win32'
>>> platform.system()
'Windows'
>>> platform.release()
'10'
>>>

If you would say what exact info you need I would execute and send you.
BTW my python3 uses Sentence case capitalization in errors as you can see:

>>> open('some')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'some'
>>>

Thanks! The following Windows bot (clang-x64-windows-msvc) is also passing http://lab.llvm.org:8011/#/builders/123, I expect it would have the same output in python as your machine. James mentioned that this issue might be caused by a difference in standard library.

I was curious if all of these modified testcases are now failing or only the three you mentioned. I also don't know how this change can cause a pop-up to occur, is there more information you can provide?
And lastly, if you have the time, does this testcase pass for you?

llvm/test/tools/llvm-elfabi/fail-file-write-windows.test

I did not modify this but it requires system-windows and has a lower-case error message. I expect if my change causes failures for you, then this testcase should also fail.

[ASDenysPetrov]

In D95246#2566417, @abhina.sreeskantharajan wrote:

I was curious if all of these modified testcases are now failing or only the three you mentioned.

Here is the list of the tests which fail after the patch (I haven't build LLD so don't have tests as well):

Clang :: CodeGen/basic-block-sections.c
Clang :: CodeGen/ubsan-blacklist-vfs.c
Clang :: Frontend/output-paths.c
Clang :: Frontend/stats-file.c  
LLVM :: DebugInfo/symbolize-missing-file.test
LLVM :: Object/archive-extract-dir.test
LLVM :: Object/directory.ll
LLVM :: tools/llvm-ar/error-opening-directory.test
LLVM :: tools/llvm-ar/missing-thin-archive-member.test
LLVM :: tools/llvm-ar/move.test
LLVM :: tools/llvm-ar/print.test
LLVM :: tools/llvm-ar/quick-append.test
LLVM :: tools/llvm-ar/replace.test
LLVM :: tools/llvm-ar/response.test
LLVM :: tools/llvm-cxxdump/trivial.test
LLVM :: tools/llvm-libtool-darwin/filelist.test
LLVM :: tools/llvm-libtool-darwin/invalid-input-output-args.test
LLVM :: tools/llvm-lipo/create-arch.test
LLVM :: tools/llvm-lipo/replace-invalid-input.test
LLVM :: tools/llvm-lto/error.ll
LLVM :: tools/llvm-lto2/X86/stats-file-option.ll
LLVM :: tools/llvm-mc/basic.test
LLVM :: tools/llvm-mca/invalid_input_file_name.test
LLVM :: tools/llvm-ml/basic.test
LLVM :: tools/llvm-objcopy/COFF/add-section.test
LLVM :: tools/llvm-objcopy/ELF/add-section.test
LLVM :: tools/llvm-objcopy/ELF/error-format.test
LLVM :: tools/llvm-objcopy/MachO/add-section-error.test
LLVM :: tools/llvm-objcopy/redefine-symbols.test
LLVM :: tools/llvm-objcopy/wasm/dump-section.test
LLVM :: tools/llvm-profdata/weight-instr.test
LLVM :: tools/llvm-profdata/weight-sample.test
LLVM :: tools/llvm-readobj/ELF/thin-archive-paths.test
LLVM :: tools/llvm-readobj/basic.test
LLVM :: tools/llvm-readobj/thin-archive.test
LLVM :: tools/llvm-size/no-input.test
LLVM :: tools/llvm-xray/X86/no-such-file.txt
LLVM :: tools/obj2yaml/invalid_input_file.test
LLVM :: tools/yaml2obj/output-file.yaml

I also don't know how this change can cause a pop-up to occur, is there more information you can provide?

I got this. The popups is not the fault of this patch. They happened earlier. I'll investigate it later.

llvm/test/tools/llvm-elfabi/fail-file-write-windows.test

I did not modify this but it requires system-windows and has a lower-case error message. I expect if my change causes failures for you, then this testcase should also fail.

Yes, this fails for me too. Didn't check it earlier.

[jhenderson]

@ASDenysPetrov, could you provide your link command-line for one of the tools that produces the failing message, please, e.g. llvm-ar? That may give us a hint about whether different libraries are being used. It's possible we need some additional cmake-time configuration steps to detect this difference. Also are you using cygwin at all?

Not that I'm saying we shouldn't try to fix your use-case, but if you aren't using Cygwin, it's worth noting that https://llvm.org/docs/GettingStarted.html#hardware states that only Visual Studio builds (whether via the IDE or some other tool like ninja) are known to work for Windows.

[ASDenysPetrov]

@jhenderson

@ASDenysPetrov, could you provide your link command-line for one of the tools that produces the failing message, please, e.g. llvm-ar?

Here is output of running tests in llvm-ar folder:

fail_output_llvm_ar.txt93 KBDownload

Not that I'm saying we shouldn't try to fix your use-case, but if you aren't using Cygwin, it's worth noting that https://llvm.org/docs/GettingStarted.html#hardware states that only Visual Studio builds (whether via the IDE or some other tool like ninja) are known to work for Windows.

I'm using MSYS2 instead of Cygwin. I found it as a better alternative. I'm using GCC+MSYS2+Ninja.

[jhenderson]

In D95246#2568084, @ASDenysPetrov wrote:

@jhenderson

@ASDenysPetrov, could you provide your link command-line for one of the tools that produces the failing message, please, e.g. llvm-ar?

Here is output of running tests in llvm-ar folder:

fail_output_llvm_ar.txt93 KBDownload

Sorry, what I meant was, what is the command used to build the llvm-ar executable, not what does it print when used. The command used to build llvm-ar should tell us what libraries are being used, which could impact this behaviour.

[ASDenysPetrov]

@jhenderson
I'm using
cmake -GNinja ../llvm -DLLVM_LIT_ARGS=-sv -DLLVM_ENABLE_PROJECTS=clang -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
for setup and ninja for build. Here is my CMakeCache.txt config file.

CMakeCache.txt76 KBDownload

[abhina.sreeskantharajan]

In D95246#2568084, @ASDenysPetrov wrote:

@jhenderson

@ASDenysPetrov, could you provide your link command-line for one of the tools that produces the failing message, please, e.g. llvm-ar?

Here is output of running tests in llvm-ar folder:

fail_output_llvm_ar.txt93 KBDownload

Not that I'm saying we shouldn't try to fix your use-case, but if you aren't using Cygwin, it's worth noting that https://llvm.org/docs/GettingStarted.html#hardware states that only Visual Studio builds (whether via the IDE or some other tool like ninja) are known to work for Windows.

I'm using MSYS2 instead of Cygwin. I found it as a better alternative. I'm using GCC+MSYS2+Ninja.

MSYS2 is based on a modified version of Cygwin according to its description, maybe if we can detect that in config.py this will solve your issue.
Can you share the output of

sysconfig.get_platform()

[ASDenysPetrov]

@abhina.sreeskantharajan

Can you share the output of

sysconfig.get_platform()

Sure,

>>> sysconfig.get_platform()
'win32'
>>>

[abhina.sreeskantharajan]

@ASDenysPetrov are you running python inside MSYS bash? that is installed by

pacman -S --needed base-devel mingw-w64-x86_64-toolchain

I learned that the python should give different output

>>> sys.platform
msys
>>> platform.system()
MSYS_NT-10.0-19041
>>> sysconfig.get_platform()
msys-3.1.7-x86_64

After taking a look at the CMakeCache.txt, I think a potential workaround would be to check if config.host_cxx contains MSYS2 which is set to CMAKE_CXX_COMPILER:FILEPATH=D:/WORK/Utilities/MSYS2/mingw64/bin/c++.exe if we can't determine MSYS through python.

[ASDenysPetrov]

@abhina.sreeskantharajan wrote:

@ASDenysPetrov are you running python inside MSYS bash? that is installed by

No, I installed Python as usual via Windows msi got from official site.

Here is what I've got from Python:

>>> sys.platform
'win32'
>>> platform.system()
'Windows'
>>> sysconfig.get_platform()
'win32'
>>>

[jhenderson]

@ASDenysPetrov, if you run "ninja -v llvm-ar", what is the output of the line that actually builds the llvm-ar executable? That'll tell us which libraries are in use, which should hopefully point to the difference to our own environments. For example, the last line of output for this on my Linux machine is:

[811/811] : && /usr/bin/c++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -Wl,-rpath-link,/home/binutils/llvm/llvm-project/build/./lib  -Wl,-O3 -Wl,--gc-sections tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.o -o bin/llvm-ar  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMARMAsmParser.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMHexagonAsmParser.a  lib/libLLVMLanaiAsmParser.a  lib/libLLVMMipsAsmParser.a  lib/libLLVMMSP430AsmParser.a  lib/libLLVMPowerPCAsmParser.a  lib/libLLVMRISCVAsmParser.a  lib/libLLVMSparcAsmParser.a  lib/libLLVMSystemZAsmParser.a  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMX86AsmParser.a  lib/libLLVMAArch64Desc.a  lib/libLLVMAMDGPUDesc.a  lib/libLLVMARMDesc.a  lib/libLLVMAVRDesc.a  lib/libLLVMBPFDesc.a  lib/libLLVMHexagonDesc.a  lib/libLLVMLanaiDesc.a  lib/libLLVMMipsDesc.a  lib/libLLVMMSP430Desc.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMPowerPCDesc.a  lib/libLLVMRISCVDesc.a  lib/libLLVMSparcDesc.a  lib/libLLVMSystemZDesc.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMX86Desc.a  lib/libLLVMXCoreDesc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAMDGPUInfo.a  lib/libLLVMARMInfo.a  lib/libLLVMAVRInfo.a  lib/libLLVMBPFInfo.a  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiInfo.a  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVInfo.a  lib/libLLVMSparcInfo.a  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMX86Info.a  lib/libLLVMXCoreInfo.a  lib/libLLVMBinaryFormat.a  lib/libLLVMCore.a  lib/libLLVMDlltoolDriver.a  lib/libLLVMLibDriver.a  lib/libLLVMObject.a  lib/libLLVMSupport.a  -lpthread  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMUtils.a  lib/libLLVMMCDisassembler.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMTextAPI.a  lib/libLLVMOption.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lrt  -ldl  -lpthread  -lm  /usr/lib/x86_64-linux-gnu/libz.so  /usr/lib/x86_64-linux-gnu/libtinfo.so  lib/libLLVMDemangle.a && :

This shows, for example, that the libz.so my build uses is located in /usr/lib/x86_64-linux-gnu. By having your equivalent path, we can hopefully confirm that the issue is caused by a different set of system libraries being used.

[ASDenysPetrov]

This shows, for example, that the libz.so my build uses is located in /usr/lib/x86_64-linux-gnu. By having your equivalent path, we can hopefully confirm that the issue is caused by a different set of system libraries being used.

Hi, @jhenderson
I changed llvm-ar.cpp then ran ninja -v llvm-ar and got this:

[1/2] D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -DGTEST_HAS_RTTI=0 -D_DEBUG -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/llvm-ar -ID:/WORK/LLVM/llvm-project/llvm/tools/llvm-ar -Iinclude -ID:/WORK/LLVM/llvm-project/llvm/include -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment  -O2   -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -MF tools\llvm-ar\CMakeFiles\llvm-ar.dir\llvm-ar.cpp.obj.d -o tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -c D:/WORK/LLVM/llvm-project/llvm/tools/llvm-ar/llvm-ar.cpp
[2/2] cmd.exe /C "cd . && D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment  -O2 -Wl,--stack,16777216 tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -o bin\llvm-ar.exe -Wl,--out-implib,lib\libllvm-ar.dll.a -Wl,--major-image-version,0,--minor-image-version,0  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a  lib/libLLVMX86Info.a  lib/libLLVMBinaryFormat.a  lib/libLLVMCore.a  lib/libLLVMDlltoolDriver.a  lib/libLLVMLibDriver.a  lib/libLLVMObject.a  lib/libLLVMSupport.a  lib/libLLVMMCDisassembler.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMTextAPI.a  lib/libLLVMOption.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lpsapi  -lshell32  -lole32  -luuid  -ladvapi32  D:/WORK/Utilities/MSYS2/mingw64/lib/libz.dll.a  lib/libLLVMDemangle.a  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 && cd ."

As you can libz is here D:/WORK/Utilities/MSYS2/mingw64/lib/libz.dll.a

[jhenderson]

In D95246#2587123, @ASDenysPetrov wrote:

This shows, for example, that the libz.so my build uses is located in /usr/lib/x86_64-linux-gnu. By having your equivalent path, we can hopefully confirm that the issue is caused by a different set of system libraries being used.

Hi, @jhenderson
I changed llvm-ar.cpp then ran ninja -v llvm-ar and got this:

[1/2] D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -DGTEST_HAS_RTTI=0 -D_DEBUG -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/llvm-ar -ID:/WORK/LLVM/llvm-project/llvm/tools/llvm-ar -Iinclude -ID:/WORK/LLVM/llvm-project/llvm/include -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment  -O2   -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -MF tools\llvm-ar\CMakeFiles\llvm-ar.dir\llvm-ar.cpp.obj.d -o tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -c D:/WORK/LLVM/llvm-project/llvm/tools/llvm-ar/llvm-ar.cpp
[2/2] cmd.exe /C "cd . && D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment  -O2 -Wl,--stack,16777216 tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -o bin\llvm-ar.exe -Wl,--out-implib,lib\libllvm-ar.dll.a -Wl,--major-image-version,0,--minor-image-version,0  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a  lib/libLLVMX86Info.a  lib/libLLVMBinaryFormat.a  lib/libLLVMCore.a  lib/libLLVMDlltoolDriver.a  lib/libLLVMLibDriver.a  lib/libLLVMObject.a  lib/libLLVMSupport.a  lib/libLLVMMCDisassembler.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMTextAPI.a  lib/libLLVMOption.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lpsapi  -lshell32  -lole32  -luuid  -ladvapi32  D:/WORK/Utilities/MSYS2/mingw64/lib/libz.dll.a  lib/libLLVMDemangle.a  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 && cd ."

As you can libz is here D:/WORK/Utilities/MSYS2/mingw64/lib/libz.dll.a

Thanks (for clarity, libz isn't what I'm looking for, it was just an example of how this information might be useful). As it is, I think we might need more information still - that commandline hides which files are actually used slightly. Could you try explicitly running the c++ command that is being requested, but with the -Wl,--trace option added too. That will tell the linker to print what files it opens when it runs the link, and should give us an hint as to which system library is different for your usage than ours.

[ASDenysPetrov]

@jhenderson

Thanks (for clarity, libz isn't what I'm looking for, it was just an example of how this information might be useful). As it is, I think we might need more information still - that commandline hides which files are actually used slightly. Could you try explicitly running the c++ command that is being requested, but with the -Wl,--trace option added too. That will tell the linker to print what files it opens when it runs the link, and should give us an hint as to which system library is different for your usage than ours.

I added -W --trace to the end of cmdline. The output is in the file:

trace_output.txt56 KBDownload

[jhenderson]

In D95246#2587167, @ASDenysPetrov wrote:

@jhenderson

Thanks (for clarity, libz isn't what I'm looking for, it was just an example of how this information might be useful). As it is, I think we might need more information still - that commandline hides which files are actually used slightly. Could you try explicitly running the c++ command that is being requested, but with the -Wl,--trace option added too. That will tell the linker to print what files it opens when it runs the link, and should give us an hint as to which system library is different for your usage than ours.

I added -W --trace to the end of cmdline. The output is in the file:

trace_output.txt56 KBDownload

Sorry, but -W --trace is not the same as -Wl,--trace. The former is a pair of options used by the compiler (one of which describes the files used by the compiler). The latter is an option passed to the linker, and is what we need. Please try again!

[ASDenysPetrov]

@jhenderson

Sorry, but -W --trace is not the same as -Wl,--trace. The former is a pair of options used by the compiler (one of which describes the files used by the compiler). The latter is an option passed to the linker, and is what we need. Please try again!

I used -W, because my c++ doesn't know -Wl option.

C:\Users\Denis>c++ -Wl
c++: error: unrecognized command-line option '-Wl'; did you mean '-W'?
c++: fatal error: no input files
compilation terminated.

Could you provide exact cmd line me to use if I'm doing smth wrong?

[jhenderson]

In D95246#2587277, @ASDenysPetrov wrote:

@jhenderson

Sorry, but -W --trace is not the same as -Wl,--trace. The former is a pair of options used by the compiler (one of which describes the files used by the compiler). The latter is an option passed to the linker, and is what we need. Please try again!

I used -W, because my c++ doesn't know -Wl option.

C:\Users\Denis>c++ -Wl
c++: error: unrecognized command-line option '-Wl'; did you mean '-W'?
c++: fatal error: no input files
compilation terminated.

Could you provide exact cmd line me to use if I'm doing smth wrong?

-Wl on its own isn't an option. It is used as a prefix to pass an option to the linker, so the exact string you pass to the compiler is -Wl,--trace (no spaces, comma required). The compiler sees the -Wl, prefix, removes it and then passes the remainder straight to the linker. You can see examples of this in your existing commandline with things like -Wl,--stack,16777216.

The cmdline should be:

D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -DGTEST_HAS_RTTI=0 -D_DEBUG -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/llvm-ar -ID:/WORK/LLVM/llvm-project/llvm/tools/llvm-ar -Iinclude -ID:/WORK/LLVM/llvm-project/llvm/include -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment  -O2   -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -MF tools\llvm-ar\CMakeFiles\llvm-ar.dir\llvm-ar.cpp.obj.d -o tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -c D:/WORK/LLVM/llvm-project/llvm/tools/llvm-ar/llvm-ar.cpp
[2/2] cmd.exe /C "cd . && D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment  -O2 -Wl,--stack,16777216 tools/llvm-ar/CMakeFiles/llvm-ar.dir/llvm-ar.cpp.obj -o bin\llvm-ar.exe -Wl,--out-implib,lib\libllvm-ar.dll.a -Wl,--major-image-version,0,--minor-image-version,0  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a  lib/libLLVMX86Info.a  lib/libLLVMBinaryFormat.a  lib/libLLVMCore.a  lib/libLLVMDlltoolDriver.a  lib/libLLVMLibDriver.a  lib/libLLVMObject.a  lib/libLLVMSupport.a  lib/libLLVMMCDisassembler.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMTextAPI.a  lib/libLLVMOption.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lpsapi  -lshell32  -lole32  -luuid  -ladvapi32  D:/WORK/Utilities/MSYS2/mingw64/lib/libz.dll.a  lib/libLLVMDemangle.a  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--trace

[ASDenysPetrov]

@jhenderson
I think I'm done.
Here is output:

linker_trace_output.txt7 KBDownload

Is it OK now?

[jhenderson]

In D95246#2587410, @ASDenysPetrov wrote:

@jhenderson
I think I'm done.
Here is output:

linker_trace_output.txt7 KBDownload

Is it OK now?

Thanks, yes that does the trick. As I expected, you can see that all the libraries that are used are contained withing the msys2/mingw subdirectory, and therefore are not the standard Windows SDK files that you would typically use for a Windows build using Visual Studio. One of those will contain the definitions we are talking about.

@abhina.sreeskantharajan - I think you could this issue in one of two ways. One would be to detect when the tools/libraries used are within the msys2/mingw installation. Another way might be to actually build and run a miniature program at CMake time that prints out the messages, in such a way that CMake can inspect the output and use it to populate the lit configuration. Note: I am not a CMake expert, so don't know if this is strictly possible. The advantage of this approach is that you can remove the reliance on the system.platform call in python, and just always use the output of this simple program.

[abhina.sreeskantharajan]

In D95246#2587452, @jhenderson wrote:

In D95246#2587410, @ASDenysPetrov wrote:

@jhenderson
I think I'm done.
Here is output:

linker_trace_output.txt7 KBDownload

Is it OK now?

Thanks, yes that does the trick. As I expected, you can see that all the libraries that are used are contained withing the msys2/mingw subdirectory, and therefore are not the standard Windows SDK files that you would typically use for a Windows build using Visual Studio. One of those will contain the definitions we are talking about.

@abhina.sreeskantharajan - I think you could this issue in one of two ways. One would be to detect when the tools/libraries used are within the msys2/mingw installation. Another way might be to actually build and run a miniature program at CMake time that prints out the messages, in such a way that CMake can inspect the output and use it to populate the lit configuration. Note: I am not a CMake expert, so don't know if this is strictly possible. The advantage of this approach is that you can remove the reliance on the system.platform call in python, and just always use the output of this simple program.

Thanks, I'm also not sure if the second way is possible. I had another idea but I will need @ASDenysPetrov to test it out.
Does the following python program print out the correct error string for you?

import errno
import os
print ( os.strerror(errno.ENOENT))

If so, we can use this as the error message instead of hardcoding it per platform.

[abhina.sreeskantharajan]

I created the following review https://reviews.llvm.org/D97472 which I think should fix the error on all platforms. Let me know if this fixes the issue on MSYS2.

[abhina.sreeskantharajan]

This issue has been fixed in https://reviews.llvm.org/D97472.
@ASDenysPetrov I'm unable to close this revision and the other one https://reviews.llvm.org/D95808. Do you know what the appropriate action is here? To abandon this, or accept it again and close it?

[ASDenysPetrov]

In D95246#2606343, @abhina.sreeskantharajan wrote:

This issue has been fixed in https://reviews.llvm.org/D97472.
@ASDenysPetrov I'm unable to close this revision and the other one https://reviews.llvm.org/D95808. Do you know what the appropriate action is here? To abandon this, or accept it again and close it?

Done. Sorry for the delay.

clang/test/Driver/clang-offload-bundler.c

 
 // RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -outputs=%t.i,%t.tgt1,%t.tgt2 -inputs=%t.bundle.i 2>&1 | FileCheck %s --check-prefix CK-ERR3
 // CK-ERR3: error: only one output file supported in bundling mode
 
 // RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu -outputs=%t.i,%t.tgt1,%t.tgt2 -inputs=%t.bundle.i -unbundle 2>&1 | FileCheck %s --check-prefix CK-ERR4
 // RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -outputs=%t.i,%t.tgt1 -inputs=%t.bundle.i -unbundle 2>&1 | FileCheck %s --check-prefix CK-ERR4
 // CK-ERR4: error: number of output files and targets should match in unbundling mode
 
-// RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.i,%t.tgt1,%t.tgt2.notexist -outputs=%t.bundle.i 2>&1 | FileCheck %s -DFILE=%t.tgt2.notexist --check-prefix CK-ERR5
-// RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -outputs=%t.i,%t.tgt1,%t.tgt2 -inputs=%t.bundle.i.notexist -unbundle 2>&1 | FileCheck %s -DFILE=%t.bundle.i.notexist --check-prefix CK-ERR5
-// CK-ERR5: error: '[[FILE]]': {{N|n}}o such file or directory
+// RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.i,%t.tgt1,%t.tgt2.notexist -outputs=%t.bundle.i 2>&1 | FileCheck %s -DFILE=%t.tgt2.notexist -DMSG=%errc_ENOENT --check-prefix CK-ERR5
+// RUN: not clang-offload-bundler -type=i -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -outputs=%t.i,%t.tgt1,%t.tgt2 -inputs=%t.bundle.i.notexist -unbundle 2>&1 | FileCheck %s -DFILE=%t.bundle.i.notexist -DMSG=%errc_ENOENT --check-prefix CK-ERR5
+// CK-ERR5: error: '[[FILE]]': [[MSG]]
 
 // RUN: not clang-offload-bundler -type=invalid -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.i,%t.tgt1,%t.tgt2 -outputs=%t.bundle.i 2>&1 | FileCheck %s -DTYPE=invalid --check-prefix CK-ERR6
 // CK-ERR6: error: '[[TYPE]]': invalid file type specified
 
 // RUN: not clang-offload-bundler 2>&1 | FileCheck %s --check-prefix CK-ERR7
 // CK-ERR7: clang-offload-bundler: for the --type option: must be specified at least once!
 
 // RUN: not clang-offload-bundler -type=i -inputs=%t.i,%t.tgt1,%t.tgt2 2>&1 | FileCheck %s -check-prefix=CK-ERR7A
 // CK-ERR7A: error: for the --outputs option: must be specified at least once!

clang/test/Frontend/output-failures.c

 // RUN: not %clang_cc1 -emit-llvm -o %t.doesnotexist/somename %s 2> %t
-// RUN: FileCheck -check-prefix=OUTPUTFAIL -input-file=%t %s
+// RUN: FileCheck -check-prefix=OUTPUTFAIL -DMSG=%errc_ENOENT -input-file=%t %s
 
-// OUTPUTFAIL: error: unable to open output file '{{.*}}doesnotexist{{.}}somename': '{{[nN]}}o such file or directory'
+// OUTPUTFAIL: error: unable to open output file '{{.*}}doesnotexist{{.}}somename': '[[MSG]]'

clang/test/Frontend/stats-file.c

 // RUN: %clang_cc1 -emit-llvm -o /dev/null -stats-file=%t %s
 // RUN: FileCheck -input-file=%t %s
 // CHECK: {
 //  ... here come some json values ...
 // CHECK: }
 
-// RUN: %clang_cc1 -emit-llvm -o %t -stats-file=%t.doesnotexist/bla %s 2>&1 | FileCheck -check-prefix=OUTPUTFAIL %s
-// OUTPUTFAIL: warning: unable to open statistics output file '{{.*}}doesnotexist{{.}}bla': '{{[Nn]}}o such file or directory'
+// RUN: %clang_cc1 -emit-llvm -o %t -stats-file=%t.doesnotexist/bla %s 2>&1 | FileCheck -DMSG=%errc_ENOENT -check-prefix=OUTPUTFAIL %s
+// OUTPUTFAIL: warning: unable to open statistics output file '{{.*}}doesnotexist{{.}}bla': '[[MSG]]'

lld/CMakeLists.txt

if(LLVM_INCLUDE_TESTS)
     endif()
   endif()
 endif()
 
 set(LLD_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 set(LLD_INCLUDE_DIR ${LLD_SOURCE_DIR}/include )
 set(LLD_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
 
+if (LLD_BUILT_STANDALONE)
+  set(LLVM_HOST_TRIPLE ${TARGET_TRIPLE})
+endif()
+
 set(LLD_VENDOR ${PACKAGE_VENDOR} CACHE STRING
   "Vendor-specific text for showing with version information.")
 
 if(LLD_VENDOR)
   add_definitions(-DLLD_VENDOR="${LLD_VENDOR}")
 endif()
 
 # Compute the LLD version from the LLVM version.

lld/test/ELF/basic.s

 # WINRSP: cannot open c:\blah\foo
 
 # Test for the response file (invalid quoting style)
 # RUN: not ld.lld --rsp-quoting=patatino %t 2>&1 | FileCheck %s \
 # RUN:   --check-prefix=INVRSP
 # INVRSP: invalid response file quoting: patatino
 
 # RUN: not ld.lld %t.foo -o /dev/null 2>&1 | \
-# RUN:  FileCheck --check-prefix=MISSING %s
-# MISSING: cannot open {{.*}}.foo: {{[Nn]}}o such file or directory
+# RUN:  FileCheck -DMSG=%errc_ENOENT --check-prefix=MISSING %s
+# MISSING: cannot open {{.*}}.foo: [[MSG]]
 
 # RUN: not ld.lld -o /dev/null 2>&1 | \
 # RUN:  FileCheck --check-prefix=NO_INPUT %s
 # NO_INPUT: ld.lld{{.*}}: no input files
 
 # RUN: not ld.lld %t.no.such.file -o /dev/null 2>&1 | \
 # RUN:  FileCheck --check-prefix=CANNOT_OPEN %s
 # CANNOT_OPEN: cannot open {{.*}}.no.such.file: {{[Nn]}}o such file or directory
 
 # RUN: not ld.lld --foo 2>&1 | FileCheck --check-prefix=UNKNOWN %s
 # UNKNOWN: unknown argument '--foo'
 
 # RUN: not ld.lld %t -o /dev/null -m wrong_emul_fbsd 2>&1 | FileCheck --check-prefix=UNKNOWN_EMUL %s
 # UNKNOWN_EMUL: unknown emulation: wrong_emul_fbsd

lld/test/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import lit.util
 
 config.have_dia_sdk = lit.util.pythonize_bool("@LLVM_ENABLE_DIA_SDK@")
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
 config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
 config.lld_obj_root = "@LLD_BINARY_DIR@"
 config.lld_libs_dir = "@LLVM_LIBRARY_OUTPUT_INTDIR@"
 config.lld_tools_dir = "@LLVM_RUNTIME_OUTPUT_INTDIR@"
+config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@Python3_EXECUTABLE@"
 config.have_zlib = @LLVM_ENABLE_ZLIB@
 config.have_libxml2 = @LLVM_ENABLE_LIBXML2@
 config.sizeof_void_p = @CMAKE_SIZEOF_VOID_P@
 config.ld_lld_default_mingw = @LLD_DEFAULT_LD_LLD_IS_MINGW@
 
 # Support substitution of the tools and libs dirs with user parameters. This is

llvm/test/Object/archive-extract.test

 NOTFOUND: error: 'foo.o' was not found
 
 RUN: not llvm-ar x %p/Inputs/thin.a foo.o 2>&1 | FileCheck %s --check-prefix=THINEXTRACT
 THINEXTRACT: extracting from a thin archive is not supported
 
 RUN: llvm-ar p %p/Inputs/thin.a evenlen | FileCheck %s --check-prefix=EVENLEN
 EVENLEN: evenlen
 
-RUN: not llvm-ar p %p/Inputs/thin-path.a t/test2.o 2>&1 | FileCheck %s --check-prefix=MISSING
-MISSING: error: {{N|n}}o such file or directory
+RUN: not llvm-ar p %p/Inputs/thin-path.a t/test2.o 2>&1 | FileCheck %s --DMSG=%errc_ENOENT --check-prefix=MISSING
+MISSING: error: [[MSG]]

llvm/test/tools/llvm-ar/move.test

 # RUN: not llvm-ar m 2>&1 \
 # RUN:   | FileCheck %s --check-prefix=NO-ARCHIVE
 
 # NO-ARCHIVE: error: an archive name must be specified
 
 ## Member does not exist:
 # RUN: llvm-ar rc %t/missing.a %t/1.o %t/2.o %t/3.o
 # RUN: not llvm-ar m %t/missing.a %t/missing.txt 2>&1 \
-# RUN:   | FileCheck %s --check-prefix=MISSING-FILE -DFILE=%t/missing.txt
+# RUN:   | FileCheck %s --check-prefix=MISSING-FILE -DFILE=%t/missing.txt -DMSG=%errc_ENOENT
 
-# MISSING-FILE: error: [[FILE]]: {{[nN]}}o such file or directory
+# MISSING-FILE: error: [[FILE]]: [[MSG]]
 
 --- !ELF
 FileHeader:
   Class:   ELFCLASS64
   Data:    ELFDATA2LSB
   Type:    ET_REL
   Machine: EM_X86_64
 Sections:

llvm/utils/lit/lit/llvm/config.py

def add_tool_substitutions(self, tools, search_dirs=None):
             if not command:
                 return False
 
             substitutions.append((subst_key, tool_pipe + command))
 
         self.config.substitutions.extend(substitutions)
         return True
 
+    def add_err_msg_substitutions(self, triple):
+        if (re.match(r's390x-.*-zos', triple)):
+            self.config.substitutions.append(('%errc_ENOENT', '\'EDC5129I No such file or directory.\''))
+        elif (re.match(r'.*windows.*', triple)):
+            self.config.substitutions.append(('%errc_ENOENT', '\'no such file or directory\''))
+        else:
+            self.config.substitutions.append(('%errc_ENOENT', '\'No such file or directory\''))
+
     def use_default_substitutions(self):
         tool_patterns = [
             ToolSubst('FileCheck', unresolved='fatal'),
             # Handle these specially as they are strings searched for during testing.
             ToolSubst(r'\| \bcount\b', command=FindTool(
                 'count'), verbatim=True, unresolved='fatal'),
             ToolSubst(r'\| \bnot\b', command=FindTool('not'), verbatim=True, unresolved='fatal')]
 
         self.config.substitutions.append(('%python', '"%s"' % (sys.executable)))
 
         self.add_tool_substitutions(
             tool_patterns, [self.config.llvm_tools_dir])
 
+        self.add_err_msg_substitutions(self.config.host_triple)
+
     def use_llvm_tool(self, name, search_env=None, required=False, quiet=False):
         """Find the executable program 'name', optionally using the specified
         environment variable as an override before searching the
         configuration's PATH."""
         # If the override is specified in the environment, use it without
         # validation.
         if search_env:
             tool = self.config.environment.get(search_env)
             if tool:
                 return tool