Page MenuHomePhabricator

[DEXTER] Add support for windows in regression test suite.
ClosedPublic

Authored by TWeaver on Mar 23 2020, 6:36 AM.

Details

Summary

Hello fellow Dexter Contributors,

This patch addresses the issue of the regression suite not running on windows hardware. It changes the following things:

  • add new dexter regression suite command to lit.cfg.py that makes use of the clang-cl_vs2015 and dbgend builder and debuggers.
  • sprinkle the new regression suite command through the feature and tool tests that require them.
  • mark certain problem tests on windows
    • There's a couple of tests that fail (or pass) in unexpected ways on Windows.

Problem tests are both the penalty and perfect expect_watch_type.cpp tests. Type information reporting parity is not possible a this time in dexter due to the nature of how different debuggers report type information back to their users.

Happy Reviewing

Diff Detail

Event Timeline

TWeaver created this revision.Mar 23 2020, 6:36 AM
TWeaver added a reviewer: rnk.Mar 23 2020, 7:36 AM
Orlando accepted this revision.Mar 27 2020, 7:26 AM

LGTM

This revision is now accepted and ready to land.Mar 27 2020, 7:26 AM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptMar 27 2020, 11:30 AM
davide added a subscriber: davide.Mar 27 2020, 12:04 PM

This broke the lldb bot (which also runs the debuginfo tests).

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/13349/

Can you fix it, or tell me how to fix it, or revert?

Thanks,

D

Reverted in:

commit 470e82cd06062a3fc3c142782b4d46d711b7c19c (HEAD -> master, origin/master, origin/HEAD)
Author: Davide Italiano <ditaliano@apple.com>
Date:   Fri Mar 27 13:12:00 2020 -0700

    Revert "[Dexter] Add support for Windows to regression test suite."
    
    This reverts commit 89025da9f676aebff7daf055824d6fd102a70c34 as
    it breaks the lldb macOS bot.
rnk added a comment.Mar 28 2020, 9:18 AM

I synced to a revision where this was landed, ran the tests on Windows, and got these results:

-- Testing: 54 tests, 8 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..

Testing Time: 182.54s
  Expected Passes    : 28
  Expected Failures  : 3
  Unsupported Tests  : 23

That's pretty good! Thanks for polishing this up.

I noticed there were a few tests that seemed to hang. The process driving the debugger consumed spun consuming CPU with no memory growth. I think this was one of the XFAILed expect_watch_something.cpp tests, from checking it in process hacker. As a next step, I think it would be good to look into these slow tests. Previously the cdb tests ran in a nominal amount of time, < 20s.

I've actually mailed these via the mailing list by accident, here's a copy of what I mailed:

Hi Davide,

sorry about the failures, I'l get right on it, thanks for sharing.

Sorry I missed your comments too.
Tom W


Hi Reid,

We have an internal ticket for improving the performance of the regression tests, it's not clear what's causing the slow down at this point in time but we intend to fix it in the near future.

Thanks again,
Tom W

TWeaver added a comment.EditedMar 31 2020, 2:27 AM

Hi Davide,

I've marked tests that were 'REQUIRES: system-linux' before this change to 'UNSUPPORTED: system-darwin',

The failing tests should no longer cause any hangs up on you CI. If you run into any further issues please let me know,

https://reviews.llvm.org/rGb6d2212f5276bcf0e5ef6b5dc0c985b72664ae34

Thanks
Tom W

I've also added an internal ticket to add darwin support to the dexter regression suite... I don't think there'll be much appetite in-house for it but if there's a clear need to add it in the future then of course we'll consider putting in the work.

@rnk ,

The cdb tests taking longer may be a result of the dexter regression suite running along side them. SFAIK that's not intended behaviour.

The regression suite doesn't run as quick as we'd like right now but it seems as though those tests are running as well as the CDB dexter tests.

To avoid running the dexter regression suite it may be best to point lit at the test directory that only contains the CDB regression suite you're interested in.
At the time of writing I believe this is the dexter-tests directory.