This is an archive of the discontinued LLVM Phabricator instance.

[llvm-ar][test] Add tests failing on Darwin
ClosedPublic

Authored by gbreynoo on Jul 16 2019, 8:58 AM.

Details

Summary

The following reviews contained tests that failed on Darwin but passed on other machines. This is due to the default archive format differing on a Darwin machine, and what looks to be bugs in the output of this format. I can not investigate these issue further so the tests are considered expected failures on Darwin.

Old diffs:
D63197
D63935
D64330

From my limited investigation I believe I found 1 bug:
https://bugs.llvm.org/show_bug.cgi?id=42562

Diff Detail

Repository
rL LLVM

Event Timeline

gbreynoo created this revision.Jul 16 2019, 8:58 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 16 2019, 8:58 AM
MaskRay accepted this revision.Jul 17 2019, 6:47 AM
MaskRay added inline comments.
test/tools/llvm-ar/extract.test
1 ↗(On Diff #210108)

Can you check if the file uses the \r\n ending? Some other tests you added did that. I just fixed them in r366324.

9 ↗(On Diff #210108)

| count 0

Sorry I suggested this pattern before.. I recall a better one now.

This revision is now accepted and ready to land.Jul 17 2019, 6:47 AM
gbreynoo marked an inline comment as done.Jul 17 2019, 8:03 AM
gbreynoo added inline comments.
test/tools/llvm-ar/extract.test
1 ↗(On Diff #210108)

Thanks, I'll have a look before I commit.

This revision was automatically updated to reflect the committed changes.

@gbreynoo

Our team maintains a downstream embedded ARM cross compiler, and these tests are failing on our Mac validations.
The key problem here I believe is the 'XFAIL: darwin' doesn't seem to work how you expect it to, and I'm not aware of a good alternative that I can suggest. Hopefully you or someone else has one.

XFAIL: darwin triggers if either the 'available_features' has darwin in it, or if the target triple contains 'darwin'.
Available features is not populated with the current host platform, and the target triple in our case is for our embedded ARM. Thus, the XFAIL is not active, and the failures are treated as such.

Do you know of any good solution for this? In summary:

  1. Our tools (both clang and llvm-ar) are built on and run on Mac
  2. The compiler (just clang) generates embedded ARM code
  3. This test fails for llvm-ar built on and run on on mac
  4. These lit tests seem to assume 2 implies 3, when in actuality, 1 implies 3

Hi James,

From my understanding these has been resolving correctly as XFAIL's on upstream Darwin test machines and XFAIL: darwin is used in a few other lit tests. I do not know what cross compiling is tested on the build bots however.

Firstly, does use of XFAIL: system-darwin instead of XFAIL: darwin fix this issue?

Second can you run any of the llvm tests below on your machine and get the expected result? They all use XFAIL: darwin:

  • test\DebugInfo\Generic\empty.ll
  • test\DebugInfo\Generic\gmlt.test
  • test\ExecutionEngine\MCJIT\test-global-ctors.ll
  • test\ExecutionEngine\OrcMCJIT\test-global-ctors.ll

Finally I believe lit just uses the standard python commands to determine the platform, do the commands below give unexpected output?

import sys
import platform
print(platform.system())
print(sys.platform)

Hopefully this will make finding the cause of the issue easier. If anyone else has suggestions I would be happy to hear them.

Thanks

Hi James,

From my understanding these has been resolving correctly as XFAIL's on upstream Darwin test machines and XFAIL: darwin is used in a few other lit tests. I do not know what cross compiling is tested on the build bots however.

Firstly, does use of XFAIL: system-darwin instead of XFAIL: darwin fix this issue?

Second can you run any of the llvm tests below on your machine and get the expected result? They all use XFAIL: darwin:

  • test\DebugInfo\Generic\empty.ll
  • test\DebugInfo\Generic\gmlt.test
  • test\ExecutionEngine\MCJIT\test-global-ctors.ll
  • test\ExecutionEngine\OrcMCJIT\test-global-ctors.ll

Finally I believe lit just uses the standard python commands to determine the platform, do the commands below give unexpected output?

import sys
import platform
print(platform.system())
print(sys.platform)

Hopefully this will make finding the cause of the issue easier. If anyone else has suggestions I would be happy to hear them.

Thanks

Hi James,

From my understanding these has been resolving correctly as XFAIL's on upstream Darwin test machines and XFAIL: darwin is used in a few other lit tests. I do not know what cross compiling is tested on the build bots however.

Firstly, does use of XFAIL: system-darwin instead of XFAIL: darwin fix this issue?

Second can you run any of the llvm tests below on your machine and get the expected result? They all use XFAIL: darwin:

  • test\DebugInfo\Generic\empty.ll
  • test\DebugInfo\Generic\gmlt.test
  • test\ExecutionEngine\MCJIT\test-global-ctors.ll
  • test\ExecutionEngine\OrcMCJIT\test-global-ctors.ll

Finally I believe lit just uses the standard python commands to determine the platform, do the commands below give unexpected output?

import sys
import platform
print(platform.system())
print(sys.platform)

Hopefully this will make finding the cause of the issue easier. If anyone else has suggestions I would be happy to hear them.

Thanks

I had not considered system-darwin, and hadn't found it in my cursory look through the tests, though I now see where it gets added by the main LLVM lit config.py file.

XFAIL: system-darwin does fix the issue and marks the unexpected failure.

I think the main question here is if compiling llvm-ar on darwin causes the errors, or if compiling for darwin causes the errors.
I'm not sure of the functional differences between llvm-ar between x86_64-apple-darwin* and our arm-*-none-eabi. I'm running under the assumption that the archive format is agnostic of the target triple. If that's the case, then I believe the problem lies when llvm-ar is built and executed as a darwin-hosted tool, and the XFAIL should be system-darwin. Otherwise, it would be the case that our Arm implementation is exhibiting similar bugs to the darwin implementation, and our downstream copy should have "|| arm-*-none-eabi" in addition to "darwin".

Thanks for the response. I think we'll continue forward with the system-darwin approach unless more work is done on this front.

JB