[llvm-profdata] Support JSON as as an output-only format

[kazu]

Summary

This patch teaches llvm-profdata to output the sample profile in the
JSON format. The new option is intended to be used for research and
development purposes. For example, one can write a Python script to
take a JSON file and analyze how similar different inline instances of
a given function are to each other.

I've chosen JSON because Python can parse it reasonably fast, and it
just takes a couple of lines to read the whole data:

import json
with open ('profile.json') as f:
  profile = json.load(f)

[hoy]

Perhaps make it explicit in the title that this is json an output-only format so far?

llvm/lib/ProfileData/SampleProfWriter.cpp

539 ↗

(On Diff #449128)

Wondering if this is necessary since S.getBodySamples() returns a BodySampleMap object which is already ordered.

Similarly with getCallsiteSamples below.

llvm/test/tools/llvm-profdata/sample-profile-json.test

4

nit: use JSON-next for this line and below

[wenlei]

Do you absolutely need this to be a supported format and generated natively in llvm tools?

For a profile format, we'd expect full reader/writer support, in both llvm-profdata and llvm-profgen. Having a partially supported output only format seems less than ideal.

For research purpose, would it be possible to generate json from a text profile through offline scripts?

Another option is to leverage llvm-profdata show to have an extra flag to output in json form. show output doesn't need to be a valid profile format and it doesn't need to be loadable by profile reader.

[davidxl]

Re-reply in Phab: +1 on the show command direction. If we have a reader in the future, consider promote this into a supported format.

[kazu]

Updated to move the JSON format support to "llvm-profdata show".

[kazu]

In D130944#3695666, @wenlei wrote:

Do you absolutely need this to be a supported format and generated natively in llvm tools?

For a profile format, we'd expect full reader/writer support, in both llvm-profdata and llvm-profgen. Having a partially supported output only format seems less than ideal.

For research purpose, would it be possible to generate json from a text profile through offline scripts?

Another option is to leverage llvm-profdata show to have an extra flag to output in json form. show output doesn't need to be a valid profile format and it doesn't need to be loadable by profile reader.

The main reason for adding the JSON support in an llvm tool is that I can leverage the parser.

llvm-profdata show makes sense.

I wrote another "dumper" just for the JSON format in SampleProfileReader. It's harder than I thought to wire SampleProfileWriter* to llvm-profdata show. Specifically, I had trouble casting raw_fd_ostream to raw_ostream for json::OStream purposes.

[kazu]

I've manually updated the patch description as that's not part of the diff in Phabricator.

[davidxl]

llvm/lib/ProfileData/SampleProfReader.cpp

87

too many nesting levels. Perhaps split out the lamda?

115

split out the lamda?

[kazu]

Broke up the big lambda into smaller pieces.

[kazu]

I've broken up the big lambda into smaller pieces. It should be easier to see that the output has two major parts, namely`BodySamples` and CallsiteSamples, with the implementation details factored out elsewhere.

PTAL. Thanks!

[davidxl]

lgtm

llvm/include/llvm/ProfileData/SampleProfReader.h

public:
   /// support loading function profiles on demand, return true when the
   /// reader has been given a module. Always return false for reader
   /// which doesn't support loading function profiles on demand.
   virtual bool collectFuncsFromModule() { return false; }
 
   /// Print all the profiles on stream \p OS.
   void dump(raw_ostream &OS = dbgs());
 
+  /// Print all the profiles on stream \p OS in the JSON format.
+  void dumpJson(raw_ostream &OS = dbgs());
+
   /// Return the samples collected for function \p F.
   FunctionSamples *getSamplesFor(const Function &F) {
     // The function name may have been updated by adding suffix. Call
     // a helper to (optionally) strip off suffixes so that we can
     // match against the original function name in the profile.
     StringRef CanonName = FunctionSamples::getCanonicalFnName(F);
     return getSamplesFor(CanonName);
   }

llvm/lib/ProfileData/SampleProfReader.cpp

 #include "llvm/ADT/StringRef.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ProfileSummary.h"
 #include "llvm/ProfileData/ProfileCommon.h"
 #include "llvm/ProfileData/SampleProf.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compression.h"
 #include "llvm/Support/ErrorOr.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/LineIterator.h"
 #include "llvm/Support/MD5.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>

 /// Dump all the function profiles found on stream \p OS.
 void SampleProfileReader::dump(raw_ostream &OS) {
   std::vector<NameFunctionSamples> V;
   sortFuncProfiles(Profiles, V);
   for (const auto &I : V)
     dumpFunctionProfile(I.first, OS);
 }
 
+static void dumpFunctionProfileJson(const FunctionSamples &S,
+                                    json::OStream &JOS, bool TopLevel = false) {
+  auto DumpBody = [&](const BodySampleMap &BodySamples) {
+    for (const auto &I : BodySamples) {
+      const LineLocation &Loc = I.first;
+      const SampleRecord &Sample = I.second;
+      JOS.object([&] {
+        JOS.attribute("line", Loc.LineOffset);
+        if (Loc.Discriminator)
+          JOS.attribute("discriminator", Loc.Discriminator);
+        JOS.attribute("samples", Sample.getSamples());
+
+        auto CallTargets = Sample.getSortedCallTargets();
+        if (!CallTargets.empty()) {
+          JOS.attributeArray("calls", [&] {
+            for (const auto &J : CallTargets) {
+              JOS.object([&] {
+                JOS.attribute("function", J.first);
+                JOS.attribute("samples", J.second);
+              });
+            }
+          });
+        }
+      });
+    }
+  };
+
+  auto DumpCallsiteSamples = [&](const CallsiteSampleMap &CallsiteSamples) {
+    for (const auto &I : CallsiteSamples)
+      for (const auto &FS : I.second) {
+        const LineLocation &Loc = I.first;
+        const FunctionSamples &CalleeSamples = FS.second;
+        JOS.object([&] {
+          JOS.attribute("line", Loc.LineOffset);
+          if (Loc.Discriminator)
+            JOS.attribute("discriminator", Loc.Discriminator);
+          JOS.attributeArray(
+              "samples", [&] { dumpFunctionProfileJson(CalleeSamples, JOS); });
+        });
+      }
+  };
+
+  JOS.object([&] {
+    JOS.attribute("name", S.getName());
+    JOS.attribute("total", S.getTotalSamples());
+    if (TopLevel)
+      JOS.attribute("head", S.getHeadSamples());
+
+    const auto &BodySamples = S.getBodySamples();
+    if (!BodySamples.empty())
+      JOS.attributeArray("body", [&] { DumpBody(BodySamples); });
+
+    const auto &CallsiteSamples = S.getCallsiteSamples();
+    if (!CallsiteSamples.empty())
+      JOS.attributeArray("callsites",
+                         [&] { DumpCallsiteSamples(CallsiteSamples); });
+  });
+}
+
+/// Dump all the function profiles found on stream \p OS in the JSON format.
+void SampleProfileReader::dumpJson(raw_ostream &OS) {
+  std::vector<NameFunctionSamples> V;
+  sortFuncProfiles(Profiles, V);
+  json::OStream JOS(OS, 2);
+  JOS.arrayBegin();
+  for (const auto &[FC, FS] : V)
+    dumpFunctionProfileJson(*FS, JOS, true);
+  JOS.arrayEnd();
+
+  // Emit a newline character at the end as json::OStream doesn't emit one.
+  OS << "\n";
+}
+
 /// Parse \p Input as function head.
 ///
 /// Parse one line of \p Input, and update function name in \p FName,
 /// function's total sample count in \p NumSamples, function's entry
 /// count in \p NumHeadSamples.
 ///
 /// \returns true if parsing is successful.
 static bool ParseHead(const StringRef &Input, StringRef &FName,

llvm/test/tools/llvm-profdata/sample-profile-json.test

+RUN: llvm-profdata show --sample --json %p/Inputs/sample-profile.proftext | FileCheck %s --check-prefix=JSON
+JSON:      [
+JSON-NEXT:   {
+JSON-NEXT:     "name": "main",
+JSON-NEXT:     "total": 184019,
+JSON-NEXT:     "head": 0,
+JSON-NEXT:     "body": [
+JSON-NEXT:       {
+JSON-NEXT:         "line": 4,
+JSON-NEXT:         "samples": 534
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 4,
+JSON-NEXT:         "discriminator": 2,
+JSON-NEXT:         "samples": 534
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 5,
+JSON-NEXT:         "samples": 1075
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 5,
+JSON-NEXT:         "discriminator": 1,
+JSON-NEXT:         "samples": 1075
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 6,
+JSON-NEXT:         "samples": 2080
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 7,
+JSON-NEXT:         "samples": 534
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 9,
+JSON-NEXT:         "samples": 2064,
+JSON-NEXT:         "calls": [
+JSON-NEXT:           {
+JSON-NEXT:             "function": "_Z3bari",
+JSON-NEXT:             "samples": 1471
+JSON-NEXT:           },
+JSON-NEXT:           {
+JSON-NEXT:             "function": "_Z3fooi",
+JSON-NEXT:             "samples": 631
+JSON-NEXT:           }
+JSON-NEXT:         ]
+JSON-NEXT:       }
+JSON-NEXT:     ],
+JSON-NEXT:     "callsites": [
+JSON-NEXT:       {
+JSON-NEXT:         "line": 10,
+JSON-NEXT:         "samples": [
+JSON-NEXT:           {
+JSON-NEXT:             "name": "inline1",
+JSON-NEXT:             "total": 1000,
+JSON-NEXT:             "body": [
+JSON-NEXT:               {
+JSON-NEXT:                 "line": 1,
+JSON-NEXT:                 "samples": 1000
+JSON-NEXT:               }
+JSON-NEXT:             ]
+JSON-NEXT:           }
+JSON-NEXT:         ]
+JSON-NEXT:       },
+JSON-NEXT:       {
+JSON-NEXT:         "line": 10,
+JSON-NEXT:         "samples": [
+JSON-NEXT:           {
+JSON-NEXT:             "name": "inline2",
+JSON-NEXT:             "total": 2000,
+JSON-NEXT:             "body": [
+JSON-NEXT:               {
+JSON-NEXT:                 "line": 1,
+JSON-NEXT:                 "samples": 2000
+JSON-NEXT:               }
+JSON-NEXT:             ]
+JSON-NEXT:           }
+JSON-NEXT:         ]
+JSON-NEXT:       }
+JSON-NEXT:     ]
+JSON-NEXT:   },
+JSON-NEXT:   {
+JSON-NEXT:     "name": "_Z3bari",
+JSON-NEXT:     "total": 20301,
+JSON-NEXT:     "head": 1437,
+JSON-NEXT:     "body": [
+JSON-NEXT:       {
+JSON-NEXT:         "line": 1,
+JSON-NEXT:         "samples": 1437
+JSON-NEXT:       }
+JSON-NEXT:     ]
+JSON-NEXT:   },
+JSON-NEXT:   {
+JSON-NEXT:     "name": "_Z3fooi",
+JSON-NEXT:     "total": 7711,
+JSON-NEXT:     "head": 610,
+JSON-NEXT:     "body": [
+JSON-NEXT:       {
+JSON-NEXT:         "line": 1,
+JSON-NEXT:         "samples": 610
+JSON-NEXT:       }
+JSON-NEXT:     ]
+JSON-NEXT:   }
+JSON-NEXT: ]

llvm/tools/llvm-profdata/llvm-profdata.cpp

 }
 
 static int showSampleProfile(const std::string &Filename, bool ShowCounts,
                              uint32_t TopN, bool ShowAllFunctions,
                              bool ShowDetailedSummary,
                              const std::string &ShowFunction,
                              bool ShowProfileSymbolList,
                              bool ShowSectionInfoOnly, bool ShowHotFuncList,
-                             raw_fd_ostream &OS) {
+                             bool JsonFormat, raw_fd_ostream &OS) {
   using namespace sampleprof;
   LLVMContext Context;
   auto ReaderOrErr =
       SampleProfileReader::create(Filename, Context, FSDiscriminatorPassOption);
   if (std::error_code EC = ReaderOrErr.getError())
     exitWithErrorCode(EC, Filename);
 
   auto Reader = std::move(ReaderOrErr.get());
   if (ShowSectionInfoOnly) {
     showSectionInfo(Reader.get(), OS);
     return 0;
   }
 
   if (std::error_code EC = Reader->read())
     exitWithErrorCode(EC, Filename);
 
-  if (ShowAllFunctions || ShowFunction.empty())
-    Reader->dump(OS);
+  if (ShowAllFunctions || ShowFunction.empty()) {
+    if (JsonFormat)
+      Reader->dumpJson(OS);
     else
+      Reader->dump(OS);
+  } else {
+    if (JsonFormat)
+      exitWithError(
+          "the JSON format is supported only when all functions are to "
+          "be printed");
+
     // TODO: parse context string to support filtering by contexts.
     Reader->dumpFunctionProfile(StringRef(ShowFunction), OS);
+  }
 
   if (ShowProfileSymbolList) {
     std::unique_ptr<sampleprof::ProfileSymbolList> ReaderList =
         Reader->getProfileSymbolList();
     ReaderList->dump(OS);
   }
 
   if (ShowDetailedSummary) {

 static int show_main(int argc, const char *argv[]) {
   cl::opt<std::string> Filename(cl::Positional, cl::desc("<profdata-file>"));
 
   cl::opt<bool> ShowCounts("counts", cl::init(false),
                            cl::desc("Show counter values for shown functions"));
   cl::opt<bool> TextFormat(
       "text", cl::init(false),
       cl::desc("Show instr profile data in text dump format"));
+  cl::opt<bool> JsonFormat(
+      "json", cl::init(false),
+      cl::desc("Show sample profile data in the JSON format"));
   cl::opt<bool> ShowIndirectCallTargets(
       "ic-targets", cl::init(false),
       cl::desc("Show indirect call site target values for shown functions"));
   cl::opt<bool> ShowMemOPSizes(
       "memop-sizes", cl::init(false),
       cl::desc("Show the profiled sizes of the memory intrinsic calls "
                "for shown functions"));
   cl::opt<bool> ShowDetailedSummary("detailed-summary", cl::init(false),
static int show_main(int argc, const char *argv[]) {
 
   if (ProfileKind == instr)
     return showInstrProfile(
         Filename, ShowCounts, TopNFunctions, ShowIndirectCallTargets,
         ShowMemOPSizes, ShowDetailedSummary, DetailedSummaryCutoffs,
         ShowAllFunctions, ShowCS, ValueCutoff, OnlyListBelow, ShowFunction,
         TextFormat, ShowBinaryIds, ShowCovered, OS);
   if (ProfileKind == sample)
-    return showSampleProfile(Filename, ShowCounts, TopNFunctions,
-                             ShowAllFunctions, ShowDetailedSummary,
-                             ShowFunction, ShowProfileSymbolList,
-                             ShowSectionInfoOnly, ShowHotFuncList, OS);
+    return showSampleProfile(
+        Filename, ShowCounts, TopNFunctions, ShowAllFunctions,
+        ShowDetailedSummary, ShowFunction, ShowProfileSymbolList,
+        ShowSectionInfoOnly, ShowHotFuncList, JsonFormat, OS);
   return showMemProfProfile(Filename, ProfiledBinary, OS);
 }
 
 int main(int argc, const char *argv[]) {
   InitLLVM X(argc, argv);
 
   StringRef ProgName(sys::path::filename(argv[0]));
   if (argc > 1) {