This is an archive of the discontinued LLVM Phabricator instance.

[test-suite] Update litsupport/module/microbenchmark.py to report individual timing results from 1 test.
ClosedPublic

Authored by homerdin on Feb 14 2018, 2:23 PM.

Details

Summary

These changes are dependent on the changes purposed in https://reviews.llvm.org/D43314.

With the purposed changes to lit we can report the individual timing results for each microbenchmark with one test. This updates the litsupport plugin to make further use of the benchmark library.

I have also changed the current microbenchmarks that use this plugin by removing the individual tests from the XRay microbenchmarks as they should no longer be needed. Please let me know if this causes any problems with the XRay tests uses @eizan .

With the purposed changes example output is:

Terminal:

PASS: test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench.test (1 of 1)
********** TEST 'test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench.test' RESULTS **********
MicroBenchmarks: 6 
compile_time: 0.6487 
hash: "daec0d37414da26fdae56f0d9bfa2cc0" 
link_time: 0.0637 
**********
*** MICRO-TEST 'BM_RDTSCP_Cost RESULTS ***
    exec_time:  12.7087 
*** MICRO-TEST 'BM_ReturnInstrumentedPatched RESULTS ***
    exec_time:  9.0223 
*** MICRO-TEST 'BM_ReturnInstrumentedPatchedThenUnpatched RESULTS ***
    exec_time:  2.6744 
*** MICRO-TEST 'BM_ReturnInstrumentedPatchedWithLogHandler RESULTS ***
    exec_time:  36.5216 
*** MICRO-TEST 'BM_ReturnInstrumentedUnPatched RESULTS ***
    exec_time:  2.6745 
*** MICRO-TEST 'BM_ReturnNeverInstrumented RESULTS ***
    exec_time:  2.0074 
**********

JSON output:

"tests": [
  {
    "code": "PASS",
    "elapsed": null,
    "metrics": {
      "exec_time": 2.67445
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench/BM_ReturnInstrumentedUnPatched.test",
    "output": ""
  },
  {
    "code": "PASS",
    "elapsed": null,
    "metrics": {
      "exec_time": 2.67435
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench/BM_ReturnInstrumentedPatchedThenUnpatched.test",
    "output": ""
  },
  {
    "code": "PASS",
    "elapsed": null,
    "metrics": {
      "exec_time": 36.5216
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench/BM_ReturnInstrumentedPatchedWithLogHandler.test",
    "output": ""
  },
  {
    "code": "PASS",
    "elapsed": null,
    "metrics": {
      "exec_time": 12.7087
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench/BM_RDTSCP_Cost.test",
    "output": ""
  },
  {
    "code": "PASS",
    "elapsed": null,
    "metrics": {
      "exec_time": 2.00737
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench/BM_ReturnNeverInstrumented.test",
    "output": ""
  },
  {
    "code": "PASS",
    "elapsed": null,
    "metrics": {
      "exec_time": 9.02227
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench/BM_ReturnInstrumentedPatched.test",
    "output": ""
  },
  {
    "code": "PASS",
    "elapsed": 5.6960108280181885,
    "metrics": {
      "MicroBenchmarks": 6,
      "compile_time": 0.6487,
      "hash": "daec0d37414da26fdae56f0d9bfa2cc0",
      "link_time": 0.0637
    },
    "name": "test-suite :: MicroBenchmarks/XRay/ReturnReference/retref-bench.test",
    "output": "\nMicroBenchmarks/XRay/ReturnReference/retref-bench --benchmark_format=csv > MicroBenchmarks/XRay/ReturnReference/Output/retref-bench.test.bench.csv"
  }

Diff Detail

Event Timeline

homerdin created this revision.Feb 14 2018, 2:23 PM
homerdin updated this revision to Diff 134436.Feb 15 2018, 8:32 AM

Including full context

homerdin updated this revision to Diff 135690.Feb 23 2018, 1:37 PM

Raising the lit.local.cfg into /Microbenchmark from comment in https://reviews.llvm.org/D43319

rengolin accepted this revision.Mar 9 2018, 5:26 AM

LGTM after D43314 goes in. Thanks!

This revision is now accepted and ready to land.Mar 9 2018, 5:26 AM
This revision was automatically updated to reflect the committed changes.
MicroBenchmarks/XRay/ReturnReference/retref-bench_BM_ReturnInstrumentedPatched.test