[Util] Add a helper script for converting -print-before-all output into a file based equivelent

[reames]

Summary

Simple little utility which takes a opt logfile generated with -print-before-all -print-module-scope, and splits into a series of individual "chunk-X.ll" files. The intended purpose is to help automate one step in failure reduction.

The imagined workflow is:

1. New crasher bug reported against clang or other frontend
2. Frontend run with -emit-llvm equivalent and manually confirmed that opt -O2 <emit.ll> crashes
3. Run this splitter script
4. Manually map pass name to invocation command (next on the to automate list)
5. Run bugpoint on last chunk file + manual command

I chose to dump every chunk rather than only the last since miscompile debugging frequently requires either manual step by step reduction, or cross feeding IR into different compiler versions. Not an immediate target, but there may be applications.

[skatkov]

I remember there were an idea to use git where each commit corresponds to one module output.
Have you considered this option?
It should save a lot of disk space and should not prevent you from the task you described.

utils/chunk-print-before-all.py

22 ↗

(On Diff #205197)

is_dump to have uniform variable naming.

[reames]

In D63461#1549688, @skatkov wrote:

I remember there were an idea to use git where each commit corresponds to one module output.
Have you considered this option?
It should save a lot of disk space and should not prevent you from the task you described.

I specifically wanted something which worked as a post process of existing output. (I believe that was the result of that proposal as well.) Taking the output of this, and creating the version history in a git repo would be straight-forward if you wanted to.

llvm/trunk/utils/chunk-print-before-all.py

+#!/usr/bin/env python
+# Given a -print-before-all -print-module-scope log from an opt invocation,
+# chunk it into a series of individual IR files, one for each pass invocation.
+# If the log ends with an obvious stack trace, try to split off a separate
+# "crashinfo.txt" file leaving only the valid input IR in the last chunk.
+# Files are written to current working directory.
+
+import sys
+
+basename = "chunk-"
+chunk_id = 0
+
+def print_chunk(lines):
+    global chunk_id
+    global basename
+    fname = basename + str(chunk_id) + ".ll"
+    chunk_id = chunk_id + 1
+    print "writing chunk " + fname + " (" + str(len(lines)) + " lines)"
+    with open(fname, "w") as f:
+        f.writelines(lines)
+
+is_dump = False
+cur = []
+for line in sys.stdin:
+    if line.startswith("*** IR Dump Before ") and len(cur) != 0:
+        print_chunk(cur);
+        cur = []
+        cur.append("; " + line)
+    elif line.startswith("Stack dump:"):
+        print_chunk(cur);
+        cur = []
+        cur.append(line)
+        is_dump = True
+    else:
+        cur.append(line)
+
+if is_dump:
+    print "writing crashinfo.txt (" + str(len(cur)) + " lines)"
+    with open("crashinfo.txt", "w") as f:
+        f.writelines(cur)
+else:
+    print_chunk(cur);