[opt-viewer] Python 3 support in opt-viewer.py

[modocache]

Summary

Minor changes that allow opt-stats.py to support both Python 2 and 3.
In addition to the same dictionary iterator changes that were necessary
in https://reviews.llvm.org/D34564, this diff also:

• Explcitly converts strings to bytes when reading from and writing to stdin and stdout.
• No longer uses dictionaries as a sort key for optimization remarks. Dictionary sort order in Python 2 is pretty esoteric anyway, so it's not clear that the additional sorting had a benefit for end users (for details, https://stackoverflow.com/a/3484456/679254 is a good resource on Python 2 dictionary sort order).

[anemet]

utils/opt-viewer/opt-viewer.py

193–195

The idea for sticking r.dict at the end was to have a deterministic index page. This allows diffing the before and after html directories to make sure a change is in fact NFC.

Is dict no longer available in Python3?

[modocache]

utils/opt-viewer/opt-viewer.py

193–195

dict is still available in Python 3, but it doesn't have a built-in sort order (that is, it no longer implements __cmp__). For details, see: https://stackoverflow.com/a/22334235/679254.

In any case, sorry about that, I'll take another look. Maybe instead of comparing the dict representation of the remark, I could compare more of its comparable attributes?

[modocache]

Keying on the pass name and demangled function when necessary prevents any non-determinism in the index ordering.

[anemet]

LGTM, one suggestion below.

utils/opt-viewer/opt-viewer.py

193–195

Can we use r.Function to avoid calling the demangler?

[modocache]

Yes, good catch. Use function name instead of demangled function name.

utils/opt-viewer/opt-viewer.py

def _render_file(source_dir, output_dir, ctx, entry):
     context = ctx
     filename, remarks = entry
     SourceFileRenderer(source_dir, output_dir, filename).render(remarks)
 
 
 def map_remarks(all_remarks):
     # Set up a map between function names and their source location for
     # function where inlining happened
-    for remark in all_remarks.itervalues():
+    for remark in optrecord.itervalues(all_remarks):
         if isinstance(remark, optrecord.Passed) and remark.Pass == "inline" and remark.Name == "Inlined":
             for arg in remark.Args:
                 caller = arg.get('Caller')
                 if caller:
                     context.caller_loc[caller] = arg['DebugLoc']
 
 
 def generate_report(pmap, all_remarks, file_remarks, source_dir, output_dir, should_display_hotness):
     try:
         os.makedirs(output_dir)
     except OSError as e:
         if e.errno == errno.EEXIST and os.path.isdir(output_dir):
             pass
         else:
             raise
 
     _render_file_bound = functools.partial(_render_file, source_dir, output_dir, context)
     pmap(_render_file_bound, file_remarks.items())
 
     if should_display_hotness:
-        sorted_remarks = sorted(all_remarks.itervalues(), key=lambda r: (r.Hotness, r.File, r.Line, r.Column, r.__dict__), reverse=True)
+        sorted_remarks = sorted(optrecord.itervalues(all_remarks), key=lambda r: (r.Hotness, r.File, r.Line, r.Column, r.PassWithDiffPrefix, r.yaml_tag, r.Function), reverse=True)
     else:
-        sorted_remarks = sorted(all_remarks.itervalues(), key=lambda r: (r.File, r.Line, r.Column, r.__dict__))
+        sorted_remarks = sorted(optrecord.itervalues(all_remarks), key=lambda r: (r.File, r.Line, r.Column, r.PassWithDiffPrefix, r.yaml_tag, r.Function))
     IndexRenderer(args.output_dir).render(sorted_remarks)
 
     shutil.copy(os.path.join(os.path.dirname(os.path.realpath(__file__)),
             "style.css"), output_dir)
 
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(description=desc)

utils/opt-viewer/optrecord.py

else:
     def itervalues(d):
         return d.itervalues()
     def iteritems(d):
         return d.iteritems()
 
 
 def demangle(name):
     with p_lock:
-        p.stdin.write(name + '\n')
-        return p.stdout.readline().rstrip()
+        p.stdin.write((name + '\n').encode('utf-8'))
+        p.stdin.flush()
+        return p.stdout.readline().rstrip().decode('utf-8')
 
 
 def html_file_name(filename):
     return filename.replace('/', '_').replace('#', '_') + ".html"
 
 
 def make_link(File, Line):
     return "\"{}#L{}\"".format(html_file_name(File), Line)