Page MenuHomePhabricator

[debuginfo-tests][dexter] Add a test tool --calculate-average option
ClosedPublic

Authored by Pierre-vh on Feb 27 2020, 3:00 AM.

Details

Summary

During our experiments with DExTer, we found that it lacked a way to calculate the average score of every test ran inside a directory. For instance, if I have a directory full of DExTer tests and I run them all using something like
python3 dexter.py /foo/mytests --builder clang --debugger lldb --cflags="-g -O2", I have no way to know the average score of all the tests that I just ran. So, we implemented this feature under`--calculate-average`, and we now propose to upstream it:

When using that option, DExTer will calculate the average score of every test run and print it after all the tests have run,.

Example output:

$ python3 dexter/dexter.py test ~/dexter/tests/nostdlib/llvm_passes/GVN --builder clang --debugger lldb --calculate-average

NumGVNEqProp: (0.1429)
NumGVNLoad: (0.1429)
NumGVNPRE: (0.1613)
NumPRELoad: (0.1549)

@avg: (0.1505)

Diff Detail

Event Timeline

Pierre-vh created this revision.Feb 27 2020, 3:00 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 27 2020, 3:00 AM
Orlando accepted this revision.Feb 28 2020, 1:29 AM
Orlando added a subscriber: Orlando.

LGTM. Thanks for the patch, It's great to see someone using and contributing to dexter!

FYI It will help visibility if you add the debug-info tag to these patches.

This revision is now accepted and ready to land.Feb 28 2020, 1:29 AM
This revision was automatically updated to reflect the committed changes.