[NFC] Add utility to sum/merge stats files

[Tyker]

Summary

Add a small script to sum *.stats file given as input and output the totals
usage example:

merge-stats.py $(find ./builddir/ -name "*.stats") > total.stats

[lebedev.ri]

Is there really no such existing script? (i don't know of one, i've always hacked around with bash)
It's a non-blocker, but i just want to point out that aggregating via sum is not always correct for all stats - there's updateMax() setter.

[Tyker]

In D83505#2142556, @lebedev.ri wrote:

Is there really no such existing script? (i don't know of one, i've always hacked around with bash)

i tried to find one the first time i needed one and didn't find any. maybe there is one i didn't find.

It's a non-blocker, but i just want to point out that aggregating via sum is not always correct for all stats - there's updateMax() setter.

ok
i wasn't aware of it, i don't think it is really possible to differentiate between them. but we could count both the sum and the max for all stats.

[lebedev.ri]

In D83505#2142581, @Tyker wrote:

In D83505#2142556, @lebedev.ri wrote:

Is there really no such existing script? (i don't know of one, i've always hacked around with bash)

i tried to find one the first time i needed one and didn't find any. maybe there is one i didn't find.

It's a non-blocker, but i just want to point out that aggregating via sum is not always correct for all stats - there's updateMax() setter.

ok
i wasn't aware of it, i don't think it is really possible to differentiate between them.

Yep. Just feel like pointing out.

but we could count both the sum and the max for all stats.

[lebedev.ri]

This looks good and reasonable to me.
I think it would be useful to have that in-tree.

llvm/utils/merge-stats.py

1

I think it would be okay to hardcode python3 in a new script :)

[lebedev.ri]

llvm/utils/merge-stats.py

24

.. because python2 doesn't like this

llvm/utils/merge-stats.py

+#!/usr/bin/env python
+'''
+Merge .stats files generated by llvm tools
+
+merge-stats.py takes as argument a list of stats files to merge
+and output the result on stdout
+
+Usage:
+  merge-stats.py $(find ./builddir/ -name "*.stats") > total.stats
+'''
+
+import json
+import sys
+
+result = {}
+
+for arg in range(1, len(sys.argv)):
+  with open(sys.argv[arg], "r", encoding='utf-8',
+            errors='ignore') as f:
+    text = f.read()
+    try:
+      data = json.loads(text)
+    except:
+      print('ignored %s: failed to parse' % sys.argv[arg], file= sys.stderr)
+      continue
+    for key in data:
+      if key in result:
+        result[key] += data[key]
+      else:
+        result[key] = data[key]
+
+out = json.dumps(result, indent=2)
+print(out)