[RemarkUtil] Add an option to collect remark count information given a list of keys.

[zjaffal]

Summary

There are remarks that have a key,value pair where it would be
benificial to collect inforation for a given key and report it on file
or function basis.

llvm-remarkutil remark-count provides the utility to collect counts
for a set of comma seperated keys for a provided remarkname.

By default the tool will collect and report numbers for the whole remark
file provided
e.g

llvm-remarkutil remark-count --remark-name=name --keys=key1,key2 ...

Output:
key1,key2
123,321

if collect-per-source or collect-per-function are passed then the

count is broken down by source file path or function name
respectively.

[thegameg]

I think having the key as a regex is a bit more flexible, wdyt?

Something like --keys=count.*` that also prints the different keys it matched:

count1,count2,count3,count4,counter
0,0,0,0,0

[zjaffal]

D156869 should cover all the changes that this patch is doing.

llvm/docs/CommandGuide/llvm-remarkutil.rst

 
 Subcommands
 -----------
 
   * :ref:`bitstream2yaml_subcommand` - Reserialize bitstream remarks to YAML.
   * :ref:`yaml2bitstream_subcommand` - Reserialize YAML remarks to bitstream.
   * :ref:`instruction-count_subcommand` - Output function instruction counts.
   * :ref:`annotation-count_subcommand` - Output remark type count from annotation remarks.
+  * :ref:`remark-count_subcommand` - Output total count for a provided keys in a remark.
 
 .. _bitstream2yaml_subcommand:
 
 bitstream2yaml
 ~~~~~~~~~~~~~~
 
 .. program:: llvm-remarkutil bitstream2yaml
 
::
   Function,Count
   foo,123
 
 if `--use-debug-loc` is passed then the CSV will include the source path, line number and column. 
 
 ::
   Source,Function,Count
   path:line:column,foo,3
+
+.. _remark-count_subcommand:
+
+remark-count
+~~~~~~~~~~~~~
+.. program:: llvm-remarkutil remark-count 
+
+USAGE: :program:`llvm-remarkutil` remark-count <input file> --parser=<bitstream|yaml> --remark-name=<string> --keys=<string>  [--collect-per-source] [--collect-per-function] -o <output file>
+
+Summary
+^^^^^^^
+
+Output the count for a specified set of `keys` passed as a comma-seperated string for a given `remark-name`. By default the count will represent the total count for each key for the remark file.
+
+CSV format is as follows:
+
+::
+  Key1,Key2
+  123,321 
+
+If `--collect-per-source` is passed then the count will be broken down by source file. This requires the remark to have `DebugLoc` information present. 
+
+::
+  Source,Key1,Key2
+  path/to/foo,123,321
+
+if `--collect-per-function` is passed then the count will be broken down by function name.
+
+::
+  Function,Key1,Key2
+  func0,123,321

llvm/test/tools/llvm-remarkutil/Inputs/remark-count-with-dbg-loc.yaml

+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - String:          'CountsSummary '
+  - count1:           '1'
+  - count2:           '2'
+  - count3:           '1'
+  - count4:           '2'
+...
+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts
+DebugLoc:        { File: path/to/anno.c, Line: 3, Column: 2 }
+Function:        func1
+Args:
+  - String:          'CountsSummary '
+  - count1:           '1'
+  - count2:           '2'
+...
+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts
+DebugLoc:        { File: path/to/anno2.c, Line: 3, Column: 2 }
+Function:        func1
+Args:
+  - String:          'CountsSummary '
+  - count1:           '3'
+  - count2:           '4'
+...
+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts
+DebugLoc:        { File: path/to/anno2.c, Line: 3, Column: 2 }
+Function:        func1
+Args:
+  - String:          'CountsSummary '
+  - count1:           '4'

llvm/test/tools/llvm-remarkutil/Inputs/remark-count-with-incorrect-keys.yaml

+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - String:          'CountsSummary '
+  - count:           'one'

llvm/test/tools/llvm-remarkutil/Inputs/remark-count.yaml

+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts 
+Function:        func1 
+Args:
+  - String:          'CountsSummary '
+  - count1:           '3'
+  - count2:           '4'
+...
+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts 
+Function:        func1 
+Args:
+  - String:          'CountsSummary '
+  - count:           '1'
+  - count2:           '4'
+...
+--- !Analysis
+Pass:            AnyRemarkPass 
+Name:            RemarkCounts 
+Function:        func1 
+Args:
+  - String:          'CountsSummary '
+  - count3:           '1'
+  - count4:           '4'

llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test

 RUN: not llvm-remarkutil bitstream2yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil instruction-count --parser=bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil annotation-count --parser=bitstream --annotation-type=remark %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+RUN: not llvm-remarkutil remark-count --remark-name=remark --keys=count,count1 --parser=bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+
 CHECK: error: Unknown magic number: expecting RMRK, got --- .

llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test

 RUN: not llvm-remarkutil yaml2bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil instruction-count --parser=yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil annotation-count --parser=yaml --annotation-type=remark %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+RUN: not llvm-remarkutil remark-count --parser=yaml --remark-name=remark --keys=count1,count2 %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+
 CHECK: error: Type, Pass, Name or Function missing

llvm/test/tools/llvm-remarkutil/empty-file.test

 RUN: not llvm-remarkutil yaml2bitstream %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --check-prefix=YAMLPARSER
 RUN: not llvm-remarkutil instruction-count --parser=yaml %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --check-prefix=YAMLPARSER
 RUN: not llvm-remarkutil annotation-count --parser=yaml --annotation-type=remark %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --check-prefix=YAMLPARSER
+RUN: not llvm-remarkutil remark-count --parser=yaml --remark-name=remark --keys=count,count1 %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --check-prefix=YAMLPARSER
 RUN: llvm-remarkutil bitstream2yaml %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --allow-empty --check-prefix=BITSTREAM2YAML
 RUN: llvm-remarkutil instruction-count --parser=bitstream %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --allow-empty --check-prefix=SIZEBITSTREAM
 RUN: llvm-remarkutil annotation-count --parser=bitstream --annotation-type=remark %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --allow-empty --check-prefix=ANNOTATIONBITSTREAM
+RUN: llvm-remarkutil remark-count --parser=bitstream --remark-name=remark --keys=count,count1 %p/Inputs/empty-file -o - 2>&1 | FileCheck %s --allow-empty --check-prefix=REMARKBITSTREAM
 
 ; YAMLPARSER: error: document root is not of mapping type.
 
 ; An empty bitstream file is valid.
 ; BITSTREAM2YAML-NOT: error
 
 ; SIZEBITSTREAM-LABEL: Function,InstructionCount
 ; SIZEBITSTREAM-EMPTY:
 
 ; ANNOTATIONBITSTREAM-LABEL: Function,Count
 ; ANNOTATIONBITSTREAM-EMPTY: 
+
+; REMARKBITSTREAM-LABEL: count,count1 
+; REMARKBITSTREAM: 0,0 

llvm/test/tools/llvm-remarkutil/instruction-count.test

 RUN: llvm-remarkutil instruction-count --parser=yaml %p/Inputs/instruction-count.yaml | FileCheck %s
 RUN: llvm-remarkutil yaml2bitstream %p/Inputs/instruction-count.yaml | llvm-remarkutil instruction-count --parser=bitstream | FileCheck %s
+RUN: llvm-remarkutil remark-count --parser=yaml --collect-per-function --keys=NumInstructions --remark-name=InstructionCount %p/Inputs/instruction-count.yaml  | FileCheck --check-prefix=REMARKCOUNT %s
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/instruction-count.yaml | llvm-remarkutil remark-count --parser=bitstream --collect-per-function --keys=NumInstructions --remark-name=InstructionCount  | FileCheck --check-prefix=REMARKCOUNT %s
 
 ; CHECK-LABEL: Function,InstructionCount
 ; CHECK: func1,1
 ; CHECK: func2,2
 ; CHECK: func3,3
+
+; REMARKCOUNT-LABEL: Function,NumInstructions
+; REMARKCOUNT: func1,1
+; REMARKCOUNT: func2,2
+; REMARKCOUNT: func3,3

llvm/test/tools/llvm-remarkutil/no-debug-log.test

 RUN: llvm-remarkutil instruction-count --use-debug-loc --parser=yaml %p/Inputs/instruction-count.yaml | FileCheck --check-prefix=INSTRUCTIONCOUNT %s
 RUN: llvm-remarkutil yaml2bitstream %p/Inputs/instruction-count.yaml | llvm-remarkutil instruction-count --use-debug-loc --parser=bitstream | FileCheck %s --check-prefix=INSTRUCTIONCOUNT
 RUN: llvm-remarkutil annotation-count --use-debug-loc --parser=yaml --annotation-type=remark %p/Inputs/annotation-count.yaml | FileCheck --check-prefix=ANNOTATIONCOUNT %s
 RUN: llvm-remarkutil yaml2bitstream %p/Inputs/instruction-count.yaml | llvm-remarkutil annotation-count --use-debug-loc --parser=bitstream --annotation-type=remark | FileCheck %s --check-prefix=ANNOTATIONCOUNT
+RUN: llvm-remarkutil remark-count --collect-per-source --keys=count1,count2 --parser=yaml --remark-name=RemarkCounts %p/Inputs/remark-count.yaml | FileCheck %s --check-prefix=REMARKCOUNT
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/remark-count.yaml | llvm-remarkutil remark-count --collect-per-source --keys=count1,count2 --parser=bitstream --remark-name=RemarkCounts | FileCheck %s --check-prefix=REMARKCOUNT
 
 ; ANNOTATIONCOUNT-LABEL: Source,Function,Count
 ; INSTRUCTIONCOUNT-LABEL: Source,Function,InstructionCount
+; REMARKCOUNT-LABEL: Source,count1,count2 

llvm/test/tools/llvm-remarkutil/remark-count-incorrect-key.test

+RUN: llvm-remarkutil remark-count --remark-name=RemarkCount --keys=count --parser=yaml %p/Inputs/remark-count-with-incorrect-keys.yaml | FileCheck %s
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/remark-count-with-incorrect-keys.yaml | llvm-remarkutil remark-count --remark-name=RemarkCount --keys=count --parser=yaml %p/Inputs/remark-count-with-incorrect-keys.yaml | FileCheck %s
+
+; CHECK-LABEL: count
+; CHECK-NEXT: 0 

llvm/test/tools/llvm-remarkutil/remark-count-key-not-found.test

+RUN: llvm-remarkutil remark-count --collect-per-source --remark-name=RemarkCounts --keys=nocount --parser=yaml %p/Inputs/remark-count.yaml | FileCheck %s
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/remark-count.yaml | llvm-remarkutil remark-count --collect-per-source --remark-name=RemarkCounts --keys=nocount --parser=bitstream | FileCheck %s
+
+; CHECK-LABEL: Source,nocount
+; CHECK-EMPTY:

llvm/test/tools/llvm-remarkutil/remark-count-remark-name-not-found.test

+RUN: llvm-remarkutil remark-count --remark-name=Remark --keys=count1,count2,count3,count4 --parser=yaml %p/Inputs/remark-count.yaml | FileCheck %s
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/remark-count.yaml | llvm-remarkutil remark-count --remark-name=Remark --keys=count1,count2,count3,count4 --parser=bitstream | FileCheck %s
+
+; CHECK-LABEL: count1,count2,count3,count4
+; CHECK-NEXT: 0,0,0,0

llvm/test/tools/llvm-remarkutil/remark-count-with-dbg-loc.test

+RUN: llvm-remarkutil remark-count --collect-per-source --remark-name=RemarkCounts --keys=count1,count2,count3,count4 --parser=yaml %p/Inputs/remark-count-with-dbg-loc.yaml | FileCheck %s
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/remark-count-with-dbg-loc.yaml | llvm-remarkutil remark-count --collect-per-source --remark-name=RemarkCounts --keys=count1,count2,count3,count4 --parser=bitstream | FileCheck %s
+
+; CHECK-LABEL: Source,count1,count2,count3,count4
+; CHECK-NEXT: path/to/anno.c,2,4,1,2
+; CHECK-NEXT: path/to/anno2.c,7,4,0,0

llvm/test/tools/llvm-remarkutil/remark-count.test

+RUN: llvm-remarkutil remark-count --remark-name=RemarkCounts --keys=count1,count2,count3,count4 --parser=yaml %p/Inputs/remark-count.yaml | FileCheck %s
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/remark-count.yaml | llvm-remarkutil remark-count --remark-name=RemarkCounts --keys=count1,count2,count3,count4 --parser=bitstream | FileCheck %s
+
+; CHECK-LABEL: count1,count2,count3,count4
+; CHECK-NEXT: 3,8,1,4

llvm/tools/llvm-remarkutil/RemarkUtil.cpp

 //===--------- llvm-remarkutil/RemarkUtil.cpp -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 /// Utility for remark files.
 //===----------------------------------------------------------------------===//
 
 #include "llvm-c/Remarks.h"
+#include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Remarks/Remark.h"
 #include "llvm/Remarks/RemarkFormat.h"
 #include "llvm/Remarks/RemarkParser.h"
 #include "llvm/Remarks/YAMLRemarkSerializer.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Error.h"
static cl::SubCommand
     YAML2Bitstream("yaml2bitstream",
                    "Convert YAML remarks to bitstream remarks");
 static cl::SubCommand
     Bitstream2YAML("bitstream2yaml",
                    "Convert bitstream remarks to YAML remarks");
 static cl::SubCommand InstructionCount(
     "instruction-count",
     "Function instruction count information (requires asm-printer remarks)");
+static cl::SubCommand RemarkCount(
+    "remark-count",
+    "Collect count information for a remark based on used provided keys.");
 static cl::SubCommand
     AnnotationCount("annotation-count",
                     "Collect count information from annotation remarks (uses "
                     "AnnotationRemarksPass)");
 } // namespace subopts
 
 // Keep input + output help + names consistent across the various modes via a
 // hideous macro.

 } // namespace bitstream2yaml
 
 namespace instructioncount {
 INPUT_FORMAT_COMMAND_LINE_OPTIONS(subopts::InstructionCount)
 INPUT_OUTPUT_COMMAND_LINE_OPTIONS(subopts::InstructionCount)
 DEBUG_LOC_INFO_COMMAND_LINE_OPTIONS(subopts::InstructionCount)
 } // namespace instructioncount
 
+namespace remarkcount {
+INPUT_FORMAT_COMMAND_LINE_OPTIONS(subopts::RemarkCount)
+static cl::opt<std::string>
+    RemarkNameToCollectFor("remark-name",
+                           cl::desc("Specific remark name to collect info for"),
+                           cl::sub(subopts::RemarkCount));
+static cl::list<std::string> Keys("keys",
+                                  cl::desc("Key names to collect count from"),
+                                  cl::CommaSeparated,
+                                  cl::sub(subopts::RemarkCount));
+static cl::opt<bool>
+    CollectPerSource("collect-per-source",
+                     cl::desc("collect remark count per source file"),
+                     cl::init(false), cl::sub(subopts::RemarkCount));
+static cl::opt<bool>
+    CollectPerFunction("collect-per-function",
+                       cl::desc("collect remark count per function"),
+                       cl::init(false), cl::sub(subopts::RemarkCount));
+bool CollectRemarksPerFeature;
+INPUT_OUTPUT_COMMAND_LINE_OPTIONS(subopts::RemarkCount)
+} // namespace remarkcount
+
 namespace annotationcount {
 INPUT_FORMAT_COMMAND_LINE_OPTIONS(subopts::AnnotationCount)
 static cl::opt<std::string> AnnotationTypeToCollect(
     "annotation-type", cl::desc("annotation-type remark to collect count for"),
     cl::sub(subopts::AnnotationCount));
 INPUT_OUTPUT_COMMAND_LINE_OPTIONS(subopts::AnnotationCount)
 DEBUG_LOC_INFO_COMMAND_LINE_OPTIONS(subopts::AnnotationCount)
 } // namespace annotationcount
static Error tryInstructionCount() {
   if (!E.isA<EndOfFileError>())
     return E;
   consumeError(std::move(E));
   OF->keep();
   return Error::success();
 }
 } // namespace instructioncount
 
+static std::optional<unsigned> getValForKey(StringRef Key,
+                                            const Remark &Remark) {
+
+  APInt KeyVal;
+  auto *RemarkArg = find_if(Remark.Args, [&Key, &KeyVal](const Argument &Arg) {
+    return Arg.Key == Key && !Arg.Val.getAsInteger(10, KeyVal);
+  });
+  if (RemarkArg == Remark.Args.end())
+    return std::nullopt;
+  return KeyVal.getSExtValue();
+}
+
+namespace remarkcount {
+static Error tryRemarkCount() {
+  CollectRemarksPerFeature = CollectPerFunction || CollectPerSource;
+  // Create the output buffer.
+  auto MaybeOF = getOutputFileWithFlags(OutputFileName,
+                                        /*Flags = */ sys::fs::OF_TextWithCRLF);
+  if (!MaybeOF)
+    return MaybeOF.takeError();
+  auto OF = std::move(*MaybeOF);
+  // Create a parser for the user-specified input format.
+  auto MaybeBuf = getInputMemoryBuffer(InputFileName);
+  if (!MaybeBuf)
+    return MaybeBuf.takeError();
+  auto MaybeParser = createRemarkParser(InputFormat, (*MaybeBuf)->getBuffer());
+  if (!MaybeParser)
+    return MaybeParser.takeError();
+
+  SmallVector<StringRef> KeyRefVals{Keys.begin(), Keys.end()};
+  // Maping from key value to index.
+  DenseMap<StringRef, unsigned> KeyIdxPairing;
+  for (unsigned KeyIdx = 0; KeyIdx < Keys.size(); KeyIdx++)
+    KeyIdxPairing.insert({KeyRefVals[KeyIdx], KeyIdx});
+  // collect key count for the file.
+  SmallVector<unsigned> KeyCount(KeyRefVals.size());
+  // collect key count for each source file in the remark file.
+  MapVector<StringRef, SmallVector<unsigned, 4>> SourceRowInfo;
+  auto &Parser = **MaybeParser;
+  auto MaybeRemark = Parser.next();
+  for (; MaybeRemark; MaybeRemark = Parser.next()) {
+    auto &Remark = **MaybeRemark;
+    if (Remark.RemarkName != RemarkNameToCollectFor)
+      continue;
+    if (shouldSkipRemark(CollectPerSource, Remark))
+      continue;
+    SmallVector<unsigned, 4> DataRow(KeyRefVals.size());
+    for (auto Key : KeyRefVals)
+      if (auto Count = getValForKey(Key, Remark))
+        DataRow[KeyIdxPairing[Key]] = Count.value();
+    if (CollectRemarksPerFeature) {
+      StringRef Feature =
+          CollectPerSource ? Remark.Loc->SourceFilePath : Remark.FunctionName;
+      auto Iter = SourceRowInfo.insert(
+          {Feature, SmallVector<unsigned, 4>(KeyRefVals.size())});
+      for (unsigned Idx = 0; Idx < Iter.first->second.size(); Idx++)
+        Iter.first->second[Idx] += DataRow[Idx];
+    }
+    for (unsigned Idx = 0; Idx < KeyCount.size(); Idx++)
+      KeyCount[Idx] += DataRow[Idx];
+  }
+  if (!CollectRemarksPerFeature) {
+    OF->os() << join(KeyRefVals, ",") << "\n";
+    for (unsigned Idx = 0; Idx < KeyCount.size(); Idx++) {
+      OF->os() << KeyCount[Idx];
+      if (Idx < KeyCount.size() - 1)
+        OF->os() << ",";
+    }
+    OF->os() << "\n";
+  } else {
+    std::string Feature = CollectPerSource ? "Source" : "Function";
+    OF->os() << Feature << "," << join(KeyRefVals, ",") << "\n";
+    for (auto &[Source, SourceRowCount] : SourceRowInfo) {
+      OF->os() << Source << ",";
+      for (unsigned Idx = 0; Idx < SourceRowCount.size(); Idx++) {
+        OF->os() << SourceRowCount[Idx];
+        if (Idx < SourceRowCount.size() - 1)
+          OF->os() << ",";
+      }
+      OF->os() << "\n";
+    }
+  }
+  auto E = MaybeRemark.takeError();
+  if (!E.isA<EndOfFileError>())
+    return E;
+  consumeError(std::move(E));
+  OF->keep();
+  return Error::success();
+}
+} // namespace remarkcount
+
 namespace annotationcount {
 static Error tryAnnotationCount() {
   // Create the output buffer.
   auto MaybeOF = getOutputFileWithFlags(OutputFileName,
                                         /*Flags = */ sys::fs::OF_TextWithCRLF);
   if (!MaybeOF)
     return MaybeOF.takeError();
   auto OF = std::move(*MaybeOF);

 /// specified. Otherwise, Error::success().
 static Error handleSuboptions() {
   if (subopts::Bitstream2YAML)
     return bitstream2yaml::tryBitstream2YAML();
   if (subopts::YAML2Bitstream)
     return yaml2bitstream::tryYAML2Bitstream();
   if (subopts::InstructionCount)
     return instructioncount::tryInstructionCount();
+  if (subopts::RemarkCount)
+    return remarkcount::tryRemarkCount();
   if (subopts::AnnotationCount)
     return annotationcount::tryAnnotationCount();
-
   return make_error<StringError>(
       "Please specify a subcommand. (See -help for options)",
       inconvertibleErrorCode());
 }
 
 int main(int argc, const char **argv) {
   InitLLVM X(argc, argv);
   cl::HideUnrelatedOptions(RemarkUtilCategory);
   cl::ParseCommandLineOptions(argc, argv, "Remark file utilities\n");
   ExitOnErr.setBanner(std::string(argv[0]) + ": error: ");
   ExitOnErr(handleSuboptions());
 }