[optview] added C++ version of opt-viewer

[inglorion]

Summary

Adds a C++ implementation of opt-viewer. It works similarly to the Python implementation, with the following differences:

• Speed. The C++ implementation is faster, allowing it to be used on larger projects.
• The output directory is specified using the -o flag, before the inputs. This makes it easier to provide input filenames using xargs.
• When given a directory as an input path, this implementation crawls the directory and processes any *.opt.yaml files it finds.
• Percentages are rounded rather than truncated, and '<', '>', and '&' are replaced by HTML entities in the generated output.
• Uses LLVM to demangle C++ names, which sometimes generates output different from c++filt's.
• Only depends on LLVM - no dependencies outside the code base.
  
  This implementation does not currently copy the style.css file to the output directory - I can add that in a follow-up if the rest of the code here looks good.

[ruiu]

Function names should start with lowercase letters (I think that's the most recent policy).

utils/opt-viewer/HTMLWriter.cpp

65

You want to run clang-format.

utils/opt-viewer/YAMLProcessor.cpp

37

Can you use unique_ptr instead of free?

39

[inglorion]

Removed an extra level of HTML encoding that I had accidentally introduced

[inglorion]

utils/opt-viewer/HTMLWriter.cpp

65

Hmm, odd. It came out of clang-format this way. Let me reconfigure things. Thanks for pointing that out!

[inglorion]

@ruiu made a good point about the use of malloc and free: This isn't how we normally want to do things. In this case, we need to, because itaniumDemangle calls realloc on the buffer. I've added a comment to the code to explain this.

[inglorion]

fixed the indentation of the switch statements

[zturner]

utils/opt-viewer/YAMLProcessor.cpp

127–128

else if?

[anemet]

It does not compile for me:

../utils/opt-viewer/YAMLProcessor.cpp:130:19: error: conversion function from 'llvm::ErrorSuccess' to 'Expected<llvm::opt_viewer::Remark>' invokes a deleted function
  if (!MN) return Error::success();
                  ^~~~~~~~~~~~~~~~
../include/llvm/Support/Error.h:658:3: note: 'Expected' has been explicitly marked deleted here
  Expected(ErrorSuccess) = delete;
  ^
1 error generated.

[inglorion]

Fixed compile error (thanks, @anemet !) and inserted missing elses (thanks, @zturner !)

[inglorion]

changed function names to start in lowercase letters

[ruiu]

LGTM, but I'm not sure if I can sign off. Who is the owner of the tool?

utils/opt-viewer/HTMLWriter.cpp

105–108

It looks weird to mix << and literal string concatenation. Make it consistent.

[amccarth]

There is a lot fiddly HTML crafting going on. Have you tried running some samples through tidy to make sure everything is properly matched up and escaped? The browsers will accept a lot of badly-formed HTML.

utils/opt-viewer/HTMLWriter.cpp

36

"descending" seems wrong. A normal comparison function returns true if the left value comes strictly "before" the right value. When such a function is used with something like std::sort, that gives you an ascending ordering.

44

"ascending" seems wrong here for the same reason as the previous comment.

77

The implicit conversion from unsigned long long to double can lose precision, which is probably not important here, but it could trigger a compiler warning.

141

This assumes two's complement, right?

0 is a (signed) int, likely 32 bits. ~0 sets all 32 bits to 1. Implicitly converting that to a std::size_t (which is unsigned and likely 64 bits) seems difficult to predict. I believe this works because the signed int representation is two's complement.

The common advice is to use -1 which should be safe even if the representation is signed magnitude.
http://stackoverflow.com/questions/809227/is-it-safe-to-use-1-to-set-all-bits-to-true

221

How about case '"': Stream << "&quot"; break;?

You're using writeHTML to output href links (see writeLocationLinkPrologue). It's not clear if any quotes in those URIs are already escaped.

utils/opt-viewer/opt-viewer.cpp

56

This is portable nowadays, right? The original ANSI C standard made return EXIT_SUCCESS from main do exactly the wrong thing on VMS.

[anemet]

For the record, there is an llvm-dev discussion that wasn't referenced here. In the thread it was raised whether it makes sense to rewrite this in C++ without proper profiling of the existing Python implementation. The idea of this review was so that people interested could give this a try not to go in as is . (For example, the patch has obvious issues duplicating the code from llvm-opt-report.)

[inglorion]

utils/opt-viewer/HTMLWriter.cpp

36

It's a bit confusing. priority_queue orders elements so that the "greatest" one is popped first. The operator() returns true if R->Hotness is greater than L->Hotness, causing the queue to sort R before L. As you note, this is the opposite of what std::sort does.

77

I'm converting to double on purpose, because it avoids the problem of running out of bits in an unsigned long long before performing the division (or losing all the bits by performing the division first). The conversion to unsigned long long should be fine, too, because surely it has enough bits to represent values >= 0 and <= 100. Good point about the compiler warning - what's a good way to avoid that?

105–108

Yep, there was supposed to be a << there. Thanks!

141

I actually want whatever the maximum value of the type is, so maybe I should just write std::numeric_limits<size_t>::max().

utils/opt-viewer/opt-viewer.cpp

56

I'd be happy to change it if you want me to. On the other hand, we're using it in utils/unittest/googletest in a couple of places, so I imagine it's ok.

[inglorion]

@amccarth's comments

[inglorion]

There is a lot fiddly HTML crafting going on. Have you tried running some samples through tidy to make sure everything is properly matched up and escaped? The browsers will accept a lot of badly-formed HTML.

The HTML language lawyer in me doesn't immediately want to pop the champagne. I'm certain it won't meet all the requirements of the validator. That wasn't the goal - I've intentionally kept the output mostly the same as @anemet's original so that it's easier to compare what the tools generate, instead of going for full HTML-some-version-compliance. I expect that it will render just fine in any browser that supports tables. If we wanted it to pass the W3 validator, I'd say we should do that in a follow-up.

[amccarth]

utils/opt-viewer/HTMLWriter.cpp

77

Making the conversion explicit (e.g., static_cast<double>()) will suppress any "loss of precision" warning from the compiler. As a bonus, it also makes it more obvious to the reader what's going on.

I totally understand why you need double, and I'm not opposed to that.

141

Cool.

utils/opt-viewer/opt-viewer.cpp

56

No change necessary. The standards folks have fixed the glitch.

[inglorion]

added static casts and comment to relativeHotness()

CMakeLists.txt

 add_subdirectory(lib)
 
 if( LLVM_INCLUDE_UTILS )
   add_subdirectory(utils/FileCheck)
   add_subdirectory(utils/PerfectShuffle)
   add_subdirectory(utils/count)
   add_subdirectory(utils/not)
   add_subdirectory(utils/llvm-lit)
+  add_subdirectory(utils/opt-viewer)
   add_subdirectory(utils/yaml-bench)
   add_subdirectory(utils/unittest)
 else()
   if ( LLVM_INCLUDE_TESTS )
     message(FATAL_ERROR "Including tests when not building utils will not work.
     Either set LLVM_INCLUDE_UTILS to On, or set LLVM_INCLDE_TESTS to Off.")
   endif()
 endif()

utils/opt-viewer/CMakeLists.txt

+add_llvm_utility(opt-viewer
+  opt-viewer.cpp
+  HTMLWriter.cpp
+  YAMLProcessor.cpp
+  )
+
+target_link_libraries(opt-viewer LLVMSupport)

utils/opt-viewer/HTMLWriter.h

+//===- HTMLWriter.h - Write HTML reports from optimizer remarks -----------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_UTILS_OPT_VIEWER_HTML_H
+#define LLVM_UTILS_OPT_VIEWER_HTML_H
+
+#include "Remark.h"
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/Twine.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <string>
+
+namespace llvm {
+namespace opt_viewer {
+
+/// Creates a URI for a path.
+std::string pathToURI(const Twine &Path);
+
+/// Writes Text, properly encoded as HTML, to Stream.
+void writeHTML(StringRef Text, raw_ostream &Stream);
+
+/// Generates an HTML report from the given remark in the given output
+/// directory.
+Error writeReport(const Twine &OutputDirectory,
+                  SmallVectorImpl<Remark> &Remarks,
+                  unsigned long long MaxHotness);
+
+/// Writes an HTML link for the given location, using the location as the text.
+void writeLocationLink(const Location &L, raw_ostream &Stream);
+
+/// Writes an HTML link for the given location, using the given text.
+void writeLocationLink(const Location &L, StringRef Text, raw_ostream &Stream);
+
+}  // namespace opt_viewer
+}  // namespace llvm
+
+#endif

utils/opt-viewer/HTMLWriter.cpp

+//===- HTMLWriter.cpp - Write HTML reports from optimizer remarks ---------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "HTMLWriter.h"
+#include "Location.h"
+#include "Remark.h"
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/Twine.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/MD5.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <fstream>
+#include <limits>
+#include <queue>
+
+namespace llvm {
+namespace opt_viewer {
+
+static const char uri_suffix[] = ".html";
+static const size_t max_uri_length = 64;
+static const size_t uri_hash_length = 16;
+static const size_t uri_suffix_length = sizeof(uri_suffix);
+static const size_t uri_filename_length =
+    max_uri_length - uri_hash_length - uri_suffix_length;
+
+/// Function-like class that sorts remarks by hotness, descending.
+class SortRemarksByHotness {
+ public:
+  bool operator()(const Remark *L, const Remark *R) {
+    return L->Hotness < R->Hotness;
+  }
+};
+
+/// Function-like class that sorts remarks by line and column, ascending.
+class SortRemarksByPosition {
+ public:
+  bool operator()(const Remark *L, const Remark *R) {
+    auto &LL = L->Location;
+    auto &RL = R->Location;
+    return LL.Line > RL.Line || (LL.Line == RL.Line && LL.Column > RL.Column);
+  }
+};
+
+typedef std::priority_queue<const Remark *, std::vector<const Remark *>,
+                            SortRemarksByHotness>
+    RemarksByHotnessQueue;
+typedef std::priority_queue<const Remark *, std::vector<const Remark *>,
+                            SortRemarksByPosition>
+    RemarksByPositionQueue;
+typedef DenseMap<StringRef, RemarksByPositionQueue> PathRemarksMap;
+
+/// Maps Remark::Kind values to CSS classes used by the stylesheet.
+static const char *colorClass(enum Remark::Kind Kind) {
+  switch (Kind) {
+  case Remark::Kind::PASSED:
+    return "column-entry-green";
+  case Remark::Kind::MISSED:
+    return "column-entry-red";
+  default:
+    return "column-entry-white";
+  }
+}
+
+/// Returns Hotness as a percentage of MaxHotness.
+static unsigned long long relativeHotness(unsigned long long Hotness,
+                                          unsigned long long MaxHotness) {
+  // Convert the inputs to floating point numbers to avoid overflowing the
+  // fixed-precision integer type. A double has fewer mantissa bits than an
+  // unsigned long long, but since we are only going to return an integer in the
+  // range [0,100], there should be plenty of precision there. Similarly, we
+  // cast back to an unsigned long long, which cannot represent all values of a
+  // double, but we only want the integer value in the range [0,100], so it's
+  // fine. The static_casts are there to prevent the compiler from warning us
+  // about throwing away bits we don't need.
+  return static_cast<unsigned long long>(
+      100.0 * static_cast<double>(Hotness) / static_cast<double>(MaxHotness)
+      + 0.5);
+}
+
+static void writeLocationLinkEpilogue(raw_ostream &Stream) { Stream << "</a>"; }
+
+static void writeLocationLinkPrologue(const Location &L, raw_ostream &Stream) {
+  auto URI = pathToURI(L.File);
+  Stream << "<a href=\"";
+  writeHTML(URI, Stream);
+  Stream << "#L" << L.Line << "\">";
+}
+
+static Error writeRemark(const Remark &R, unsigned long long MaxHotness,
+                         raw_ostream &Stream) {
+  Stream << "<tr>\n<td>";
+  writeLocationLink(R.Location, Stream);
+  Stream << "</td>\n<td>" << relativeHotness(R.Hotness, MaxHotness)
+         << "%</td>\n<td>";
+  writeHTML(R.Function, Stream);
+  Stream << "</td>\n<td class=\"" << colorClass(R.Kind) << "\">" << R.Pass
+         << "</td>\n</tr>\n";
+  return Error::success();
+}
+
+static Error writeIndex(RemarksByHotnessQueue &HQ,
+                        unsigned long long MaxHotness, raw_ostream &Stream) {
+  Stream << "<html>\n<head>\n"
+         << "<link rel=\"stylesheet\" type=\"text/css\" href=\"style.css\">\n"
+         << "</head>\n<body>\n<div class=\"centered\">\n<table>\n"
+         << "<tr>\n<td>Source Location</td>\n<td>Hotness</td>\n"
+         << "<td>Function</td>\n<td>Pass</td>\n</tr>\n";
+  while (!HQ.empty()) {
+    if (auto E = writeRemark(*HQ.top(), MaxHotness, Stream)) return E;
+    HQ.pop();
+  }
+  Stream << "</table>\n</div>\n</body>\n</html>\n";
+  Stream.flush();
+  return Error::success();
+}
+
+static Error WriteSourceFile(const Twine &SourcePath,
+                             const Twine &OutputDirectory,
+                             RemarksByPositionQueue &Remarks,
+                             unsigned long long MaxHotness) {
+  size_t Lineno = 0;
+  SmallString<255> OutputPath;
+  OutputDirectory.toVector(OutputPath);
+  sys::path::append(OutputPath, pathToURI(SourcePath));
+  std::ifstream Source(SourcePath.str());
+  std::error_code EC;
+  raw_fd_ostream Output(OutputPath, EC, sys::fs::OpenFlags::F_None);
+  if (EC) return errorCodeToError(EC);
+  Output << "<html>\n<head>\n<link rel=\"stylesheet\" type=\"text/css\" "
+            "href=\"style.css\">\n"
+         << "<title>";
+  writeHTML(SourcePath.str(), Output);
+  Output << "</title>\n"
+         << "</head>\n<body>\n<div class=\"centered\">\n<table>\n<tr>\n"
+         << "<td>Line</td><td>Hotness</td><td>Optimization</td><td>Source</"
+            "td></tr>\n";
+  std::string Line;
+  const Remark *NextRemark;
+  size_t NextRemarkLine = std::numeric_limits<size_t>::max();
+  if (!Remarks.empty()) {
+    NextRemark = Remarks.top();
+    Remarks.pop();
+    NextRemarkLine = NextRemark->Location.Line;
+  }
+  while (std::getline(Source, Line)) {
+    ++Lineno;
+    Output << "<tr>\n<td><a name=\"L" << Lineno << "\">" << Lineno
+           << "</a></td>\n"
+           << "<td></td><td></td><td><pre>";
+    writeHTML(Line, Output);
+    Output << "</pre></td>\n</tr>\n";
+    while (Lineno >= NextRemarkLine) {
+      Output << "<tr>\n<td></td>\n<td>"
+             << relativeHotness(NextRemark->Hotness, MaxHotness) << "%</td>\n"
+             << "<td class=\"" << colorClass(NextRemark->Kind) << "\">";
+      writeHTML(NextRemark->Pass, Output);
+      Output << "</td>\n<td class=\"column-entry-yellow\">"
+             << NextRemark->Message << "</td>\n</tr>\n";
+      if (!Remarks.empty()) {
+        NextRemark = Remarks.top();
+        Remarks.pop();
+        NextRemarkLine = NextRemark->Location.Line;
+      } else {
+        NextRemarkLine = std::numeric_limits<size_t>::max();
+      }
+    }
+  }
+  Output << "</table>\n</div>\n</body>\n</html>\n";
+  Output.close();
+  return Error::success();
+}
+
+std::string pathToURI(const Twine &Path) {
+  SmallString<max_uri_length> PathStr;
+  SmallVectorImpl<char>::iterator PI;
+  std::string URI;
+  std::string::iterator UI;
+  Path.toVector(PathStr);
+  if (PathStr.size() <= max_uri_length - uri_suffix_length) {
+    URI.assign(PathStr.size(), 0);
+    PI = PathStr.begin();
+    UI = URI.begin();
+  } else {
+    MD5 Digest;
+    MD5::MD5Result Result;
+    Digest.update(PathStr);
+    Digest.final(Result);
+    SmallString<32> Hexed;
+    MD5::stringifyResult(Result, Hexed);
+    URI.assign(Hexed.begin(), Hexed.begin() + uri_hash_length);
+    URI.append(uri_filename_length, 0);
+    PI = PathStr.end() - uri_filename_length;
+    UI = URI.end() - uri_filename_length;
+  }
+  std::string::iterator UE = URI.end();
+  while (UI != UE) {
+    char c = *PI;
+    if (c == '/' || c == '\\') c = '_';
+    *UI = c;
+    ++UI;
+    ++PI;
+  }
+  URI.append(uri_suffix);
+  return URI;
+}
+
+void writeHTML(StringRef Text, raw_ostream &Stream) {
+  for (char C : Text) {
+    switch (C) {
+    case '<':
+      Stream << "&lt;";
+      break;
+    case '>':
+      Stream << "&gt;";
+      break;
+    case '&':
+      Stream << "&amp;";
+      break;
+    default:
+      Stream << C;
+    }
+  }
+}
+
+Error writeReport(const Twine &OutputDirectory,
+                  SmallVectorImpl<Remark> &Remarks,
+                  unsigned long long MaxHotness) {
+  // Sort remarks by hotness for the index, and by file and location for the
+  // source files.
+  RemarksByHotnessQueue HQ;
+  PathRemarksMap RemarksByPath;
+  for (const auto &R : Remarks) {
+    HQ.push(&R);
+    RemarksByPositionQueue &RQ =
+        RemarksByPath.FindAndConstruct(R.Location.File).getSecond();
+    RQ.push(&R);
+  }
+
+  // Create the output directory if it does not already exist.
+  std::error_code EC;
+  sys::fs::create_directory(OutputDirectory);
+
+  // Write the index.
+  SmallString<255> IndexPath;
+  OutputDirectory.toVector(IndexPath);
+  sys::path::append(IndexPath, "index.html");
+  raw_fd_ostream Index(IndexPath, EC, sys::fs::OpenFlags::F_None);
+  if (EC) return errorCodeToError(EC);
+  Error E = writeIndex(HQ, MaxHotness, Index);
+  Index.close();
+  if (E) return E;
+
+  // Write the source files.
+  for (auto &Entry : RemarksByPath) {
+    E = WriteSourceFile(Entry.getFirst(), OutputDirectory, Entry.getSecond(),
+                        MaxHotness);
+    if (E) return E;
+  }
+
+  return E;
+}
+
+void writeLocationLink(const Location &L, raw_ostream &Stream) {
+  writeLocationLinkPrologue(L, Stream);
+  writeHTML(L.File, Stream);
+  Stream << ":" << L.Line << ":" << L.Column;
+  writeLocationLinkEpilogue(Stream);
+}
+
+void writeLocationLink(const Location &L, StringRef Text, raw_ostream &Stream) {
+  writeLocationLinkPrologue(L, Stream);
+  writeHTML(Text, Stream);
+  writeLocationLinkEpilogue(Stream);
+}
+
+}  // namespace opt_viewer
+}  // namespace llvm

utils/opt-viewer/Location.h

+//===- Location.h - Source location ---------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_UTILS_OPT_VIEWER_LOCATION_H
+#define LLVM_UTILS_OPT_VIEWER_LOCATION_H
+
+#include <string>
+
+namespace llvm {
+namespace opt_viewer {
+
+struct Location {
+  std::string File;
+  size_t Line;
+  size_t Column;
+};
+
+}  // namespace opt_viewer
+}  // namespace llvm
+
+#endif

utils/opt-viewer/Remark.h

+//===- Remark.h - Optimizer remark ----------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_UTILS_OPT_VIEWER_REMARK_H
+#define LLVM_UTILS_OPT_VIEWER_REMARK_H
+
+#include "Location.h"
+
+#include <string>
+
+namespace llvm {
+namespace opt_viewer {
+
+struct Remark {
+  std::string Pass;
+  std::string Name;
+  Location Location;
+  std::string Function;
+  unsigned long long Hotness;
+  enum Kind {
+    NOTE,
+    MISSED,
+    PASSED,
+  } Kind;
+  std::string Message;
+};
+
+}  // namespace opt_viewer
+}  // namespace llvm
+
+#endif

utils/opt-viewer/YAMLProcessor.h

+//===- YAMLProcessor.h - Parses YAML files into Remarks -------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_UTILS_OPT_VIEWER_YAMLPROCESSOR_H
+#define LLVM_UTILS_OPT_VIEWER_YAMLPROCESSOR_H
+
+#include "Location.h"
+#include "Remark.h"
+
+#include "llvm/ADT/Twine.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+#include <string>
+
+namespace llvm {
+namespace yaml {
+class Node;
+}
+
+namespace opt_viewer {
+
+/// Parse YAML files into Remarks. The Process* methods are used to process
+/// inputs,
+/// after which the get* methods can be used to get the results.
+class YAMLProcessor {
+ public:
+  YAMLProcessor() : MaxHotness(1) {}
+
+  /// Finds files matching *.opt.yaml in the given directory and its
+  /// subdirectories,
+  /// then processes the remarks in them.
+  Error processDirectory(const Twine &Path);
+
+  /// Processes remarks from the file at Path.
+  Error processFile(const Twine &Path);
+
+  /// Processes remarks from the file or directory at Path.
+  Error processPath(const Twine &Path);
+
+  /// Returns a reference to the parsed Remarks.
+  SmallVectorImpl<Remark> &getRemarks() { return Remarks; }
+
+  /// Returns the highest hotness value seen during processing.
+  unsigned long long getMaxHotness() const { return MaxHotness; }
+
+ private:
+  void argToHTML(yaml::Node *Node, raw_ostream &stream);
+  std::string argsToHTML(yaml::Node *Node);
+  Expected<Location> parseDebugLoc(yaml::Node *Node);
+  Expected<Remark> parseRemark(yaml::Node *Node);
+  Error processMemoryBuffer(std::unique_ptr<MemoryBuffer> Buffer);
+  Error processNode(yaml::Node *Node);
+
+  unsigned long long MaxHotness;
+  SmallVector<Remark, 256> Remarks;
+};
+
+}  // namespace opt_viewer
+}  // namespace llvm
+
+#endif

utils/opt-viewer/YAMLProcessor.cpp

+//===- YAMLProcessor.cpp - Parses YAML files into Remarks -----------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "YAMLProcessor.h"
+#include "HTMLWriter.h"
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Demangle/Demangle.h"
+#include "llvm/Support/Casting.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Support/YAMLParser.h"
+#include "llvm/Support/raw_os_ostream.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <cstdlib>
+#include <limits>
+#include <sstream>
+#include <string>
+
+namespace llvm {
+namespace opt_viewer {
+
+static const unsigned long long InvalidHotness =
+    std::numeric_limits<unsigned long long>::max();
+
+/// Attempts to demangle a function name.
+/// If successful, returns the demangled name.
+/// Else, returns a copy of the original name.
+static std::string demangle(const std::string &Name) {
+  // itaniumDemangle has an API that is quite unlike how LLVM normally works.
+  // It will call realloc on the buffer if the buffer is too small.
+  // Because of this, the buffer needs to be something that realloc can be
+  // called on. To ensure this, we allocate it using malloc (and call
+  // free before leaving this function).
+  size_t BufferSize = 64;
+  char *Buffer = reinterpret_cast<char *>(std::malloc(BufferSize));
+  int Status;
+  Buffer = itaniumDemangle(Name.c_str(), Buffer, &BufferSize, &Status);
+  std::string Result((Status == 0) ? Buffer : Name);
+  std::free(Buffer);
+  return Result;
+}
+
+/// Given a yaml::ScalarNode, gets the string it contains.
+static StringRef YAMLToString(yaml::Node *Node,
+                              SmallVectorImpl<char> &Storage) {
+  return cast<yaml::ScalarNode>(Node)->getValue(Storage);
+}
+
+/// Given a yaml::ScalarNode, gets its value as an unsigned long long.
+static unsigned long long YAMLToUnsigned(yaml::Node *Node) {
+  SmallString<20> Storage;
+  auto Str = cast<yaml::ScalarNode>(Node)->getValue(Storage);
+  return std::stoull(Str);
+}
+
+/// Converts an Arg from a YAML optimizatin report to HTML.
+void YAMLProcessor::argToHTML(yaml::Node *Node, raw_ostream &Stream) {
+  auto *MN = cast<yaml::MappingNode>(Node);
+  std::string Demangled, Key;
+  Location Location;
+  SmallString<255> Storage;
+  yaml::Node *Value;
+  for (auto &It : *MN) {
+    Key = YAMLToString(It.getKey(), Storage);
+    Value = It.getValue();
+
+    if (Key == "DebugLoc") {
+      // If we have a demangled name, emit it, with a link to the debug
+      // location.
+      // Else, skip the debug location.
+      if (Demangled.empty()) continue;
+      auto DL = parseDebugLoc(Value);
+      if (DL) {
+        writeLocationLink(*DL, Demangled, Stream);
+        Demangled.clear();
+      }
+    }
+
+    // If we have a demangled name at this point, just emit it - we'll do
+    // without the link to the source.
+    if (!Demangled.empty()) {
+      Stream << Demangled;
+      Demangled.clear();
+    }
+
+    if (Key == "Callee") {
+      // Save the function name for later, so that we can emit it
+      // together with a link to the source.
+      Demangled = demangle(YAMLToString(Value, Storage).str());
+    } else if (Key == "Caller") {
+      writeHTML(demangle(YAMLToString(Value, Storage).str()), Stream);
+    } else {
+      if (Value->getType() == yaml::Node::NodeKind::NK_Scalar)
+        writeHTML(YAMLToString(Value, Storage), Stream);
+    }
+  }
+}
+
+/// Converts the Args from a YAML optimization report to HTML.
+std::string YAMLProcessor::argsToHTML(yaml::Node *Node) {
+  auto *SN = cast<yaml::SequenceNode>(Node);
+  std::ostringstream Result;
+  raw_os_ostream Stream(Result);
+  for (auto &N : *SN) {
+    argToHTML(&N, Stream);
+  }
+  Stream.flush();
+  return Result.str();
+}
+
+/// Parses a DebugLoc from a YAML optimization report.
+Expected<Location> YAMLProcessor::parseDebugLoc(yaml::Node *Node) {
+  auto *MN = cast<yaml::MappingNode>(Node);
+  SmallString<255> Storage;
+  Location D;
+  for (auto &It : *MN) {
+    auto Key = YAMLToString(It.getKey(), Storage);
+    if (Key == "File") D.File = YAMLToString(It.getValue(), Storage);
+    else if (Key == "Line") D.Line = YAMLToUnsigned(It.getValue());
+    else if (Key == "Column") D.Column = YAMLToUnsigned(It.getValue());
+  }
+  return D;
+}
+
+/// Parses a remark from a YAML optimization report.
+Expected<Remark> YAMLProcessor::parseRemark(yaml::Node *Node) {
+  // Ignore Nodes that aren't KeyValue.
+  auto *MN = dyn_cast<yaml::MappingNode>(Node);
+  Remark R;
+  R.Hotness = InvalidHotness;
+  if (!MN) return R;
+  if (MN->getVerbatimTag() == "!Passed")
+    R.Kind = Remark::Kind::PASSED;
+  else if (MN->getVerbatimTag() == "!Missed")
+    R.Kind = Remark::Kind::MISSED;
+  SmallString<255> Storage;
+  for (auto &It : *MN) {
+    auto *Key = cast<yaml::ScalarNode>(It.getKey());
+    auto *Value = It.getValue();
+    auto KeyStr = Key->getValue(Storage);
+    if (KeyStr == "DebugLoc") {
+      auto D = parseDebugLoc(Value);
+      if (!D) return D.takeError();
+      R.Location = *D;
+    } else if (KeyStr == "Name") {
+      R.Name = YAMLToString(Value, Storage);
+    } else if (KeyStr == "Function") {
+      R.Function = demangle(YAMLToString(Value, Storage));
+    } else if (KeyStr == "Hotness") {
+      R.Hotness = YAMLToUnsigned(Value);
+      if (R.Hotness == InvalidHotness) --R.Hotness;
+      if (R.Hotness > MaxHotness) MaxHotness = R.Hotness;
+    } else if (KeyStr == "Pass") {
+      R.Pass = YAMLToString(Value, Storage);
+    } else if (KeyStr == "Args") {
+      R.Message = argsToHTML(Value);
+    }
+  }
+  return R;
+}
+
+Error YAMLProcessor::processNode(yaml::Node *Node) {
+  auto R = parseRemark(Node);
+  if (!R) return R.takeError();
+  if (R->Hotness != InvalidHotness) Remarks.push_back(*R);
+  return Error::success();
+}
+
+Error YAMLProcessor::processMemoryBuffer(std::unique_ptr<MemoryBuffer> Buffer) {
+  SourceMgr SM;
+  yaml::Stream Stream(MemoryBufferRef(*Buffer), SM);
+  for (auto &Document : Stream) {
+    yaml::Node *Node = Document.getRoot();
+    if (Node)
+      if (auto E = processNode(Node)) return E;
+  }
+  return Error::success();
+}
+
+Error YAMLProcessor::processFile(const Twine &path) {
+  auto Buffer = MemoryBuffer::getFile(path);
+  if (auto EC = Buffer.getError()) return errorCodeToError(EC);
+  return processMemoryBuffer(std::move(*Buffer));
+}
+
+Error YAMLProcessor::processDirectory(const Twine &path) {
+  std::error_code EC;
+  for (sys::fs::recursive_directory_iterator I(path, EC), EI;
+       I != EI && !EC;
+       I.increment(EC)) {
+    const StringRef Name = I->path();
+    if (Name.endswith(".opt.yaml"))
+      if (auto E = processFile(Name)) return E;
+  }
+  return errorCodeToError(EC);
+}
+
+Error YAMLProcessor::processPath(const Twine &path) {
+  if (sys::fs::is_directory(path))
+    return processDirectory(path);
+  else
+    return processFile(path);
+}
+
+}  // namespace opt_viewer
+}  // namespace llvm

utils/opt-viewer/opt-viewer.cpp

+//===- opt-viewer.cpp - Generates HTML optimization reports ---------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Generate HTML output to visualize optimization records from the YAML files
+// generated with -fsave-optimization-record and -fdiagnostics-show-hotness.
+//
+// Usage: opt-viewer -o <output directory> <input files>
+//
+//===----------------------------------------------------------------------===//
+
+#include "HTMLWriter.h"
+#include "YAMLProcessor.h"
+
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Error.h"
+
+#include <cstdlib>
+#include <string>
+
+using namespace llvm;
+using namespace llvm::opt_viewer;
+
+static cl::opt<std::string> OutputDirectory(
+    "output-directory", cl::desc("Directory to generate output in."),
+    cl::Required);
+static cl::alias OutputDirectoryA("o", cl::desc("Short for -output-directory"),
+                                  cl::aliasopt(OutputDirectory));
+static cl::list<std::string> Inputs(cl::Positional, cl::desc("<input files>"),
+                                    cl::OneOrMore);
+
+static Error do_main(int argc, char **argv) {
+  cl::ParseCommandLineOptions(argc, argv);
+  YAMLProcessor P;
+  for (auto &Input : Inputs)
+    if (auto E = P.processPath(Input)) return E;
+
+  if (auto E = writeReport(OutputDirectory, P.getRemarks(), P.getMaxHotness()))
+    return E;
+
+  return Error::success();
+}
+
+int main(int argc, char *argv[]) {
+  Error E = do_main(argc, argv);
+  if (E) {
+    logAllUnhandledErrors(std::move(E), errs(), "opt-viewer: ");
+    return EXIT_FAILURE;
+  }
+  return EXIT_SUCCESS;
+}