This is an archive of the discontinued LLVM Phabricator instance.

[Builtin] Implement lit-test support (part 1 of 2: test cases update)
ClosedPublic

Authored by weimingz on Mar 9 2017, 6:16 PM.

Details

Summary

This patch implements adds RUN command for test cases.
Unit/arm/call_apsr.S is updated to support thumb1.
It also fixes a bug in arm/aeabi_uldivmod_test.c
gcc_personality_test is XFAILED as the framework cannot handle it so far.
cpu_model_test is also XFAILED for now as it is expected to return non-zero.

TODO: A few tests are XFAILed for armhf and aarch64. We need to investigate [1,2]

[1] https://bugs.llvm.org//show_bug.cgi?id=32260
[2] https://bugs.llvm.org//show_bug.cgi?id=32261

Diff Detail

Event Timeline

weimingz created this revision.Mar 9 2017, 6:16 PM

I tested it on ARM via QEMU with the CMAKE parames below:

cmake -G Ninja  ... -DCOMPILER_RT_DEFAULT_TARGET_ARCH=arm  -DCOMPILER_RT_EMULATOR=<path_to_qemu> -DCOMPILER_RT_TEST_COMPILER=<install_prefix>/bin/clang -DCOMPILER_RT_INCLUDE_TESTS=ON

ninja check-builtins

[1/1] Running the Builtins tests
Testing Time: 11.31s

Expected Passes    : 358
Unsupported Tests  : 24

It runs for both arm and armv6m. Unsupported tests are from ppc/ and gcc_personality_test (not sure how to run), clear_cache_test, enable_exec_stack_test (looks not supported by qemu), and bswap (not supported by armv6m ) and cpu_model_test (x86 only)

Unit/arm/call_apsr.S is updated to support thumb1.
It also fixes a bug in arm/aeabi_uldivmod_test.c

Mind fixing these in separate patches?

jroelofs added inline comments.Mar 9 2017, 6:34 PM
test/builtins/Unit/lit.cfg
67 ↗(On Diff #91248)

Where does %run get expanded?

83 ↗(On Diff #91248)

Please don't assume that $host == $target... This prevents remote cross testing.

Unit/arm/call_apsr.S is updated to support thumb1.
It also fixes a bug in arm/aeabi_uldivmod_test.c

Mind fixing these in separate patches?

Sure. Will post a separate patch.

weimingz added inline comments.Mar 10 2017, 1:43 AM
test/builtins/Unit/lit.cfg
67 ↗(On Diff #91248)

%run will be run natively by default. If config.emulator is set, then it will be used. (test/lit.common.cfg)

83 ↗(On Diff #91248)

I can remove that. I copied it from asan's lit test.

weimingz updated this revision to Diff 91290.Mar 10 2017, 1:50 AM
weimingz added reviewers: jroelofs, erik.pilkington.

Address Jonathan's comments.
Just rebased and find that https://reviews.llvm.org/D30136 is adding TestCase support.
I'm not sure what's the TestCase is used for. Looks only for Darwin and the directory is empty.

First of all, thank you for doing this! This was long overdue ;-).

However, after applying your patch I get 2 failures per arch:

Failing Tests (4):
    Builtins-i386-linux :: cpu_model_test.c
    Builtins-i386-linux :: gcc_personality_test.c
    Builtins-x86_64-linux :: cpu_model_test.c
    Builtins-x86_64-linux :: gcc_personality_test.c

FWICS, cpu_model_test is not strictly a test since it either returns 4 or 3 depending on whether avx2 is supported. So you should probably remove/disable it.

If you don't plan to fix the personality test in this comment, then it would probably be wise to disable it as well.

Just rebased and find that https://reviews.llvm.org/D30136 is adding TestCase support.
I'm not sure what's the TestCase is used for. Looks only for Darwin and the directory is empty.

Thanks for working on this!
I added TestCases as a workaround, once this lands I'll be more than happy to remove it and port the os_version_check_test I added to this infrastructure.

weimingz updated this revision to Diff 91420.Mar 10 2017, 2:50 PM
weimingz edited the summary of this revision. (Show Details)
weimingz added a reviewer: arphaman.

Thanks for trying it on x86, Michal.
I XFAILed cpu_model_test.c and gcc_personality_test.c for now and I leave "FIXME" around the %RUN line. We can fix them later on.

jroelofs accepted this revision.Mar 10 2017, 2:52 PM

LGTM

test/builtins/Unit/lit.cfg
67 ↗(On Diff #91248)

Ah, thanks.

This revision is now accepted and ready to land.Mar 10 2017, 2:52 PM
This revision was automatically updated to reflect the committed changes.
weimingz updated this revision to Diff 91821.Mar 14 2017, 10:07 PM

XFAIL some of the tests for armhf and aarch64.
See https://bugs.llvm.org//show_bug.cgi?id=32260 and https://bugs.llvm.org//show_bug.cgi?id=32261

Anyone knows how to address the powpc64 fails?
http://lab.llvm.org:8011/builders/sanitizer-ppc64le-linux/builds/1703
Seems the libclang.builtin was not built.

weimingz reopened this revision.Mar 14 2017, 10:07 PM
This revision is now accepted and ready to land.Mar 14 2017, 10:07 PM
weimingz updated this revision to Diff 92428.Mar 20 2017, 10:22 PM
weimingz retitled this revision from [Builtin] Implement lit-test support to [Builtin] Implement lit-test support (part 1 of 2: test cases update).

I split the original patch into 2 parts: This one only contains the test case updates: adding "RUN" or "UNSUPPORT". A couple of test cases are XFAILed for armhf/aarch64 target (bugs were opened [1,2]). This part is also safe to commit; The second part will includes the CMake, lit.cfg files for easy reviewing.

[1] https://bugs.llvm.org//show_bug.cgi?id=32260 and
[2] https://bugs.llvm.org//show_bug.cgi?id=32261

weimingz edited the summary of this revision. (Show Details)Mar 20 2017, 10:35 PM
This revision was automatically updated to reflect the committed changes.