[rfc] target stats

[wallace]

Summary

D45547 added a while ago the skeleton for some target stats, along with a command to enable, disable, and dump them. However, it seems it wasn't further developed.

My team is in need of a bunch of other target stats so that we can fix some bottlenecks. Some of these stats are related to expressions (e.g. # of expr eval that trigger global lookups, time per expr eval), and some others are are related to modules (e.g. amount of debug info parsed, time spent indexing dwarf, etc.).

In order to collect this metrics, I'm proposing improving the existing code and create a TargetStats class, that can keep track of them.

Some things to consider:

• I think it's useful to have colletion enabled by default. You might encounter some perf issues and you might want to dump the stats right away, instead of rerunning the debug session but this time with collection enabled.
• The information that is very cheap to collect, should be collected on the fly, e.g. # of failed frame vars.
• The information that is expensive to collect, should be collected at retrieval time (e.g. when invoking ToJSON or Dump). That way the collection can be enabled by default without paying any noticeable penalty.

As an interesting case, I added a metric for the amount of time spent indexing dwarf per module, which is not as trivial as the other existing metrics because the Target is not available there. However, it was easy to implement and can be extended to all symbol files. This metric is collected at retrieval time. This doesn't account for cases in which a dynamic library is unloaded at runtime, but I'm just making the current changes just for demonstration purposes.

Btw, the code is not really polished, but it's fully functional.

Example:

(lldb) statistics dump                                                             
Number of expr evaluation successes: 0
Number of expr evaluation failures: 0
Number of frame var successes: 0
Number of frame var failures: 0
Modules: 
  /home/wallace/a.out:
    Symbol loading time in seconds: 2.500064272
  /lib64/libc.so.6:
    Symbol loading time in seconds: not loaded
  /lib64/libgcc_s.so.1:
    Symbol loading time in seconds: not loaded
  /lib64/libm.so.6:
    Symbol loading time in seconds: not loaded
  /lib64/libstdc++.so.6:
    Symbol loading time in seconds: not loaded
  /usr/lib64/ld-2.28.so:
    Symbol loading time in seconds: not loaded
  [vdso]:
    Symbol loading time in seconds: not loaded
(lldb) statistics dump --json
{
  "exprEval": {
    "failures": 0,
    "successes": 0
  },
  "frameVal": {
    "failures": 0,
    "successes": 0
  },
  "modules": {
    "/home/wallace/a.out": {
      "symbolLoadingTimeInSec:": "2.500064272"
    },
    "/lib64/libc.so.6": {
      "symbolLoadingTimeInSec:": null
    },
    "/lib64/libgcc_s.so.1": {
      "symbolLoadingTimeInSec:": null
    },
    "/lib64/libm.so.6": {
      "symbolLoadingTimeInSec:": null
    },
    "/lib64/libstdc++.so.6": {
      "symbolLoadingTimeInSec:": null
    },
    "/usr/lib64/ld-2.28.so": {
      "symbolLoadingTimeInSec:": null
    },
    "[vdso]": {
      "symbolLoadingTimeInSec:": null
    }
  }
}

[clayborg]

lldb/include/lldb/Symbol/SymbolFile.h

302–305

I would add an instance variable to this base class and then return it here. Then subclasses can just access the ivar, and clients can use this accessor to access it and no need to be virtual

lldb/include/lldb/Target/TargetStats.h

43

Does this belong here? It seems like this is something we should gather at the time the stats are asked to be returned. If it does belong here, what is this saying? Total symbol parsing time?

lldb/source/API/SBTarget.cpp

213–226

We can't replace this if this was doing something before. Was this doing nothing previously? Or are we doing everything this used to do and more?

228–230

We can't change this if it did something valid prior to this diff? Or are we doing everything this used to do and more?

234

We can't change this if it did something valid prior to this diff? Or are we doing everything this used to do and more?

lldb/source/Commands/CommandObjectStats.cpp

16

We should leave this command in place. Not a good idea to remote a command, if we do, it might make some lldb command files fail in the middle or parsing them and change people's script behavior.

40

We should leave this command in place. Not a good idea to remote a command, if we do, it might make some lldb command files fail in the middle or parsing them and change people's script behavior.

lldb/source/Plugins/SymbolFile/DWARF/ManualDWARFIndex.cpp

26–38

Maybe use a scope C++ class to help get the time of the index here and avoid making the Index() and DoIndex() calls? The "ScopedTimer" class would be something like:

class ScopedTimer {
  llvm::Optional<std::chrono::duration<double>> &m_opt_time;
  std::chrono::duration<double>m_start;
public:
  ScopedTimer (llvm::Optional<std::chrono::duration<double>> &opt_time) : m_opt_time(opt_time) {
    m_start = std::chrono::steady_clock::now().time_since_epoch();
  }
  ~ScopedTimer() {
    m_opt_time = std::chrono::steady_clock::now().time_since_epoch() - m_start;
  }
}

And we should probably put this in a header file so it can be reused elsewhere.

lldb/source/Plugins/SymbolFile/DWARF/ManualDWARFIndex.h

58

Remove, see comment in DWARFIndex.h

72

Remove and see "ScopedTimer" comment

86

This should be in base class.

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

517–522

Remove this and just set the SymbolFile::m_index_time directly after indexing

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

323–324

Remove, see comment in SymbolFile.h

lldb/include/lldb/Symbol/SymbolFile.h

 //===-- SymbolFile.h --------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SYMBOL_SYMBOLFILE_H
 #define LLDB_SYMBOL_SYMBOLFILE_H
 
 #include "lldb/Core/PluginInterface.h"
 #include "lldb/Core/SourceLocationSpec.h"
 #include "lldb/Symbol/CompilerDecl.h"
 #include "lldb/Symbol/CompilerDeclContext.h"
 #include "lldb/Symbol/CompilerType.h"
 #include "lldb/Symbol/Function.h"
 #include "lldb/Symbol/SourceModule.h"
 #include "lldb/Symbol/Type.h"
 #include "lldb/Symbol/TypeList.h"
public:
   /// function.
   virtual llvm::Expected<lldb::addr_t> GetParameterStackSize(Symbol &symbol) {
     return llvm::createStringError(make_error_code(llvm::errc::not_supported),
                                    "Operation not supported.");
   }
 
   virtual void Dump(Stream &s);
 
+  virtual llvm::Optional<std::chrono::duration<double>> GetSymbolLoadingTime() {
+    return llvm::None;
+  }
+
 protected:
   void AssertModuleLock();
   virtual uint32_t CalculateNumCompileUnits() = 0;
   virtual lldb::CompUnitSP ParseCompileUnitAtIndex(uint32_t idx) = 0;
   virtual TypeList &GetTypeList() { return m_type_list; }
 
   void SetCompileUnitAtIndex(uint32_t idx, const lldb::CompUnitSP &cu_sp);
 

lldb/include/lldb/Target/Target.h

 #define LLDB_TARGET_TARGET_H
 
 #include <list>
 #include <map>
 #include <memory>
 #include <string>
 #include <vector>
 
 #include "lldb/Breakpoint/BreakpointList.h"
 #include "lldb/Breakpoint/BreakpointName.h"
 #include "lldb/Breakpoint/WatchpointList.h"
 #include "lldb/Core/Architecture.h"
 #include "lldb/Core/Disassembler.h"
 #include "lldb/Core/ModuleList.h"
 #include "lldb/Core/UserSettingsController.h"
 #include "lldb/Expression/Expression.h"
 #include "lldb/Host/ProcessLaunchInfo.h"
 #include "lldb/Symbol/TypeSystem.h"
 #include "lldb/Target/ExecutionContextScope.h"
 #include "lldb/Target/PathMappingList.h"
 #include "lldb/Target/SectionLoadHistory.h"
+#include "lldb/Target/TargetStats.h"
 #include "lldb/Target/ThreadSpec.h"
 #include "lldb/Utility/ArchSpec.h"
 #include "lldb/Utility/Broadcaster.h"
 #include "lldb/Utility/LLDBAssert.h"
 #include "lldb/Utility/Timeout.h"
 #include "lldb/lldb-public.h"
 
 namespace lldb_private {
protected:
   /// information of this target.
   lldb::TraceSP m_trace_sp;
   /// Stores the frame recognizers of this target.
   lldb::StackFrameRecognizerManagerUP m_frame_recognizer_manager_up;
 
   static void ImageSearchPathsChanged(const PathMappingList &path_list,
                                       void *baton);
 
-  // Utilities for `statistics` command.
+  /// \{
+  /// Utilities for `statistics` command.
 private:
-  std::vector<uint32_t> m_stats_storage;
-  bool m_collecting_stats = false;
+  TargetStats m_stats;
 
 public:
-  void SetCollectingStats(bool v) { m_collecting_stats = v; }
-
-  bool GetCollectingStats() { return m_collecting_stats; }
-
-  void IncrementStats(lldb_private::StatisticKind key) {
-    if (!GetCollectingStats())
-      return;
-    lldbassert(key < lldb_private::StatisticKind::StatisticMax &&
-               "invalid statistics!");
-    m_stats_storage[key] += 1;
-  }
-
-  std::vector<uint32_t> GetStatistics() { return m_stats_storage; }
+  TargetStats &GetStatistics() { return m_stats; }
+  /// \}
 
 private:
   /// Construct with optional file and arch.
   ///
   /// This member is private. Clients must use
   /// TargetList::CreateTarget(const FileSpec*, const ArchSpec*)
   /// so all targets can be tracked from the central target list.
   ///

lldb/include/lldb/Target/TargetStats.h

+//===-- TargetStats.h -------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_TARGET_TARGETSTATS_H
+#define LLDB_TARGET_TARGETSTATS_H
+
+#include <chrono>
+#include <vector>
+
+#include "llvm/Support/JSON.h"
+
+#include "lldb/Utility/Stream.h"
+#include "lldb/lldb-forward.h"
+
+namespace lldb_private {
+
+class TargetStats {
+public:
+  struct Operation {
+    void Notify(bool success);
+
+    llvm::json::Value ToJSON() const;
+
+    uint32_t successes = 0;
+    uint32_t failures = 0;
+  };
+
+  struct ModuleStats {
+    llvm::json::Value ToJSON() const;
+
+    llvm::Optional<std::chrono::duration<double>> symbol_loading_time;
+  };
+
+  void NotifyExprEval(bool success);
+
+  void NotifyFrameVar(bool success);
+
+  void NotifyModuleSymbolParsing(std::chrono::duration<double> duration);
+
+  llvm::json::Value ToJSON(Target &target);
+
+  void Dump(Stream &s, Target &target);
+
+private:
+  void CollectModuleStats(Target &target);
+
+  Operation m_expr_eval;
+  Operation m_frame_var;
+  std::map<std::string, ModuleStats> m_module_stats;
+};
+
+} // namespace lldb_private
+
+#endif // LLDB_TARGET_TARGETSTATS_H

lldb/include/lldb/lldb-forward.h

 class SymbolVendor;
 class Symtab;
 class SyntheticChildren;
 class SyntheticChildrenFrontEnd;
 class SystemRuntime;
 class Target;
 class TargetList;
 class TargetProperties;
+class TargetStats;
 class Thread;
 class ThreadCollection;
 class ThreadList;
 class ThreadPlan;
 class ThreadPlanBase;
 class ThreadPlanRunToAddress;
 class ThreadPlanStepInstruction;
 class ThreadPlanStepOut;

lldb/source/API/SBTarget.cpp

 //===-- SBTarget.cpp ------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/API/SBTarget.h"
 #include "SBReproducerPrivate.h"
 
 #include "lldb/lldb-public.h"
 
 #include "lldb/API/SBBreakpoint.h"
 #include "lldb/API/SBDebugger.h"
 #include "lldb/API/SBEnvironment.h"
 #include "lldb/API/SBEvent.h"
SBDebugger SBTarget::GetDebugger() const {
 
   SBDebugger debugger;
   TargetSP target_sp(GetSP());
   if (target_sp)
     debugger.reset(target_sp->GetDebugger().shared_from_this());
   return LLDB_RECORD_RESULT(debugger);
 }
 
 SBStructuredData SBTarget::GetStatistics() {
   LLDB_RECORD_METHOD_NO_ARGS(lldb::SBStructuredData, SBTarget, GetStatistics);
 
   SBStructuredData data;
   TargetSP target_sp(GetSP());
   if (!target_sp)
     return LLDB_RECORD_RESULT(data);
 
-  auto stats_up = std::make_unique<StructuredData::Dictionary>();
-  int i = 0;
-  for (auto &Entry : target_sp->GetStatistics()) {
-    std::string Desc = lldb_private::GetStatDescription(
-        static_cast<lldb_private::StatisticKind>(i));
-    stats_up->AddIntegerItem(Desc, Entry);
-    i += 1;
-  }
+  std::string json_str =
+      llvm::formatv("{0}", target_sp->GetStatistics().ToJSON(*target_sp)).str();
+  data.m_impl_up->SetObjectSP(StructuredData::ParseJSON(json_str));
 
-  data.m_impl_up->SetObjectSP(std::move(stats_up));
   return LLDB_RECORD_RESULT(data);
 }
 
 void SBTarget::SetCollectingStats(bool v) {
   LLDB_RECORD_METHOD(void, SBTarget, SetCollectingStats, (bool), v);
-
-  TargetSP target_sp(GetSP());
-  if (!target_sp)
-    return;
-  return target_sp->SetCollectingStats(v);
 }
 
 bool SBTarget::GetCollectingStats() {
   LLDB_RECORD_METHOD_NO_ARGS(bool, SBTarget, GetCollectingStats);
-
-  TargetSP target_sp(GetSP());
-  if (!target_sp)
-    return false;
-  return target_sp->GetCollectingStats();
+  return true;
 }
 
 SBProcess SBTarget::LoadCore(const char *core_file) {
   LLDB_RECORD_METHOD(lldb::SBProcess, SBTarget, LoadCore, (const char *),
                      core_file);
 
   lldb::SBError error; // Ignored
   return LLDB_RECORD_RESULT(LoadCore(core_file, error));

lldb/source/Commands/CommandObjectExpression.cpp

if (!m_fixed_expression.empty() && target.GetEnableNotifyAboutFixIts()) {
         // Add in any options that might have been in the original command:
         fixed_command.append(std::string(args.GetArgStringWithDelimiter()));
         fixed_command.append(m_fixed_expression);
       } else
         fixed_command.append(m_fixed_expression);
       history.AppendString(fixed_command);
     }
     // Increment statistics to record this expression evaluation success.
-    target.IncrementStats(StatisticKind::ExpressionSuccessful);
+    target.GetStatistics().NotifyExprEval(true);
     return true;
   }
 
   // Increment statistics to record this expression evaluation failure.
-  target.IncrementStats(StatisticKind::ExpressionFailure);
+  target.GetStatistics().NotifyExprEval(false);
   result.SetStatus(eReturnStatusFailed);
   return false;
 }

lldb/source/Commands/CommandObjectFrame.cpp

bool DoExecute(Args &command, CommandReturnObject &result) override {
     if (m_interpreter.TruncationWarningNecessary()) {
       result.GetOutputStream().Printf(m_interpreter.TruncationWarningText(),
                                       m_cmd_name.c_str());
       m_interpreter.TruncationWarningGiven();
     }
 
     // Increment statistics.
     bool res = result.Succeeded();
-    Target &target = GetSelectedOrDummyTarget();
-    if (res)
-      target.IncrementStats(StatisticKind::FrameVarSuccess);
-    else
-      target.IncrementStats(StatisticKind::FrameVarFailure);
+    GetSelectedOrDummyTarget().GetStatistics().NotifyFrameVar(res);
     return res;
   }
 
   OptionGroupOptions m_option_group;
   OptionGroupVariable m_option_variable;
   OptionGroupFormat m_option_format;
   OptionGroupValueObjectDisplay m_varobj_options;
 };

lldb/source/Commands/CommandObjectStats.cpp

 //===-- CommandObjectStats.cpp --------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "CommandObjectStats.h"
+
+#include "lldb/Host/OptionParser.h"
 #include "lldb/Interpreter/CommandReturnObject.h"
 #include "lldb/Target/Target.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
-class CommandObjectStatsEnable : public CommandObjectParsed {
-public:
-  CommandObjectStatsEnable(CommandInterpreter &interpreter)
-      : CommandObjectParsed(interpreter, "enable",
-                            "Enable statistics collection", nullptr,
-                            eCommandProcessMustBePaused) {}
-
-  ~CommandObjectStatsEnable() override = default;
-
-protected:
-  bool DoExecute(Args &command, CommandReturnObject &result) override {
-    Target &target = GetSelectedOrDummyTarget();
-
-    if (target.GetCollectingStats()) {
-      result.AppendError("statistics already enabled");
-      return false;
-    }
+#pragma mark CommandObjectStats
 
-    target.SetCollectingStats(true);
-    result.SetStatus(eReturnStatusSuccessFinishResult);
-    return true;
-  }
-};
+#define LLDB_OPTIONS_statistics_dump
+#include "CommandOptions.inc"
 
-class CommandObjectStatsDisable : public CommandObjectParsed {
+class CommandObjectStatsDump : public CommandObjectParsed {
+public:
+  class CommandOptions : public Options {
   public:
-  CommandObjectStatsDisable(CommandInterpreter &interpreter)
-      : CommandObjectParsed(interpreter, "disable",
-                            "Disable statistics collection", nullptr,
-                            eCommandProcessMustBePaused) {}
+    CommandOptions() : Options() { OptionParsingStarting(nullptr); }
 
-  ~CommandObjectStatsDisable() override = default;
+    Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_arg,
+                          ExecutionContext *execution_context) override {
+      Status error;
+      const int short_option = m_getopt_table[option_idx].val;
 
-protected:
-  bool DoExecute(Args &command, CommandReturnObject &result) override {
-    Target &target = GetSelectedOrDummyTarget();
+      switch (short_option) {
+      case 'j': {
+        m_json = true;
+        break;
+      }
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+      return error;
+    }
 
-    if (!target.GetCollectingStats()) {
-      result.AppendError("need to enable statistics before disabling them");
-      return false;
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      m_json = false;
     }
 
-    target.SetCollectingStats(false);
-    result.SetStatus(eReturnStatusSuccessFinishResult);
-    return true;
+    llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
+      return llvm::makeArrayRef(g_statistics_dump_options);
     }
+
+    bool m_json;
   };
 
-class CommandObjectStatsDump : public CommandObjectParsed {
-public:
   CommandObjectStatsDump(CommandInterpreter &interpreter)
       : CommandObjectParsed(interpreter, "dump", "Dump statistics results",
-                            nullptr, eCommandProcessMustBePaused) {}
+                            nullptr, eCommandProcessMustBePaused),
+        m_options() {}
 
   ~CommandObjectStatsDump() override = default;
 
+  Options *GetOptions() override { return &m_options; }
+
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
-    Target &target = GetSelectedOrDummyTarget();
+    Target &target = m_exe_ctx.GetTargetRef();
+    TargetStats &stats = target.GetStatistics();
+
+    if (m_options.m_json)
+      result.AppendMessageWithFormatv("{0:2}", stats.ToJSON(target));
+    else
+      stats.Dump(result.GetOutputStream(), target);
 
-    uint32_t i = 0;
-    for (auto &stat : target.GetStatistics()) {
-      result.AppendMessageWithFormat(
-          "%s : %u\n",
-          lldb_private::GetStatDescription(
-              static_cast<lldb_private::StatisticKind>(i))
-              .c_str(),
-          stat);
-      i += 1;
-    }
     result.SetStatus(eReturnStatusSuccessFinishResult);
     return true;
   }
+
+  CommandOptions m_options;
 };
 
 CommandObjectStats::CommandObjectStats(CommandInterpreter &interpreter)
     : CommandObjectMultiword(interpreter, "statistics",
                              "Print statistics about a debugging session",
                              "statistics <subcommand> [<subcommand-options>]") {
-  LoadSubCommand("enable",
-                 CommandObjectSP(new CommandObjectStatsEnable(interpreter)));
-  LoadSubCommand("disable",
-                 CommandObjectSP(new CommandObjectStatsDisable(interpreter)));
   LoadSubCommand("dump",
                  CommandObjectSP(new CommandObjectStatsDump(interpreter)));
 }
 
 CommandObjectStats::~CommandObjectStats() = default;

lldb/source/Commands/Options.td

let Command = "trace dump" in {
   def trace_dump_verbose : Option<"verbose", "v">, Group<1>,
     Desc<"Show verbose trace information.">;
 }
 
 let Command = "trace schema" in {
   def trace_schema_verbose : Option<"verbose", "v">, Group<1>,
     Desc<"Show verbose trace schema logging for debugging the plug-in.">;
 }
+
+let Command = "statistics dump" in {
+  def statistics_dump_json: Option<"json", "j">, Group<1>,
+    Desc<"Dump the statistics in JSON format.">;
+}

lldb/source/Plugins/SymbolFile/DWARF/DWARFIndex.h

 //===-- DWARFIndex.h -------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_DWARFINDEX_H
 #define LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_DWARFINDEX_H
 
 #include "Plugins/SymbolFile/DWARF/DIERef.h"
 #include "Plugins/SymbolFile/DWARF/DWARFDIE.h"
 #include "Plugins/SymbolFile/DWARF/DWARFFormValue.h"
 
 class DWARFDeclContext;
 class DWARFDIE;
 
 namespace lldb_private {
 class DWARFIndex {
GetFunctions(ConstString name, SymbolFileDWARF &dwarf,
                uint32_t name_type_mask,
                llvm::function_ref<bool(DWARFDIE die)> callback) = 0;
   virtual void
   GetFunctions(const RegularExpression &regex,
                llvm::function_ref<bool(DWARFDIE die)> callback) = 0;
 
   virtual void Dump(Stream &s) = 0;
 
+  virtual llvm::Optional<std::chrono::duration<double>> GetSymbolLoadingTime() {
+    return llvm::None;
+  }
+
 protected:
   Module &m_module;
 
   /// Helper function implementing common logic for processing function dies. If
   /// the function given by "ref" matches search criteria given by
   /// "parent_decl_ctx" and "name_type_mask", it is inserted into the "dies"
   /// vector.
   bool ProcessFunctionDIE(llvm::StringRef name, DIERef ref,

lldb/source/Plugins/SymbolFile/DWARF/ManualDWARFIndex.h

 //===-- ManualDWARFIndex.h --------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_MANUALDWARFINDEX_H
 #define LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_MANUALDWARFINDEX_H
 
 #include "Plugins/SymbolFile/DWARF/DWARFIndex.h"
 #include "Plugins/SymbolFile/DWARF/NameToDIE.h"
 #include "llvm/ADT/DenseSet.h"
 
 class DWARFDebugInfo;
 class SymbolFileDWARFDwo;
 
 namespace lldb_private {
 class ManualDWARFIndex : public DWARFIndex {
void GetFunctions(ConstString name, SymbolFileDWARF &dwarf,
                     const CompilerDeclContext &parent_decl_ctx,
                     uint32_t name_type_mask,
                     llvm::function_ref<bool(DWARFDIE die)> callback) override;
   void GetFunctions(const RegularExpression &regex,
                     llvm::function_ref<bool(DWARFDIE die)> callback) override;
 
   void Dump(Stream &s) override;
 
+  llvm::Optional<std::chrono::duration<double>> GetSymbolLoadingTime() override;
+
 private:
   struct IndexSet {
     NameToDIE function_basenames;
     NameToDIE function_fullnames;
     NameToDIE function_methods;
     NameToDIE function_selectors;
     NameToDIE objc_class_selectors;
     NameToDIE globals;
     NameToDIE types;
     NameToDIE namespaces;
   };
   void Index();
+  void DoIndex();
   void IndexUnit(DWARFUnit &unit, SymbolFileDWARFDwo *dwp, IndexSet &set);
 
   static void IndexUnitImpl(DWARFUnit &unit,
                             const lldb::LanguageType cu_language,
                             IndexSet &set);
 
   /// The DWARF file which we are indexing. Set to nullptr after the index is
   /// built.
   SymbolFileDWARF *m_dwarf;
   /// Which dwarf units should we skip while building the index.
   llvm::DenseSet<dw_offset_t> m_units_to_avoid;
 
   IndexSet m_set;
+  llvm::Optional<std::chrono::duration<double>> m_loading_time;
 };
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_MANUALDWARFINDEX_H

lldb/source/Plugins/SymbolFile/DWARF/ManualDWARFIndex.cpp

 //===-- ManualDWARFIndex.cpp ----------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Plugins/SymbolFile/DWARF/ManualDWARFIndex.h"
 #include "Plugins/Language/ObjC/ObjCLanguage.h"
 #include "Plugins/SymbolFile/DWARF/DWARFDebugInfo.h"
 #include "Plugins/SymbolFile/DWARF/DWARFDeclContext.h"
 #include "Plugins/SymbolFile/DWARF/LogChannelDWARF.h"
 #include "Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/Progress.h"
 #include "lldb/Symbol/ObjectFile.h"
 #include "lldb/Utility/Stream.h"
 #include "lldb/Utility/Timer.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/ThreadPool.h"
 
 using namespace lldb_private;
 using namespace lldb;
 
 void ManualDWARFIndex::Index() {
+  std::chrono::duration<double> t1 =
+      std::chrono::steady_clock::now().time_since_epoch();
+  DoIndex();
+  m_loading_time = std::chrono::steady_clock::now().time_since_epoch() - t1;
+}
+
+llvm::Optional<std::chrono::duration<double>>
+ManualDWARFIndex::GetSymbolLoadingTime() {
+  return m_loading_time;
+}
+
+void ManualDWARFIndex::DoIndex() {
   if (!m_dwarf)
     return;
 
   SymbolFileDWARF &main_dwarf = *m_dwarf;
   m_dwarf = nullptr;
 
   LLDB_SCOPED_TIMERF("%p", static_cast<void *>(&main_dwarf));
 

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

 #include <mutex>
 #include <unordered_map>
 #include <vector>
 
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SetVector.h"
 #include "llvm/Support/Threading.h"
 
 #include "lldb/Core/UniqueCStringMap.h"
 #include "lldb/Core/dwarf.h"
 #include "lldb/Symbol/DebugMacros.h"
 #include "lldb/Symbol/SymbolContext.h"
 #include "lldb/Symbol/SymbolFile.h"
 #include "lldb/Utility/ConstString.h"
 #include "lldb/Utility/Flags.h"
 #include "lldb/Utility/RangeMap.h"
 #include "lldb/lldb-private.h"
public:
   static DWARFDeclContext GetDWARFDeclContext(const DWARFDIE &die);
 
   static lldb::LanguageType LanguageTypeFromDWARF(uint64_t val);
 
   static lldb::LanguageType GetLanguage(DWARFUnit &unit);
   /// Same as GetLanguage() but reports all C++ versions as C++ (no version).
   static lldb::LanguageType GetLanguageFamily(DWARFUnit &unit);
 
+  llvm::Optional<std::chrono::duration<double>> GetSymbolLoadingTime() override;
+
 protected:
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, lldb_private::Type *>
       DIEToTypePtr;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, lldb::VariableSP>
       DIEToVariableSP;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *,
                          lldb::opaque_compiler_type_t>
       DIEToClangType;

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

if (debug_names.GetByteSize() > 0) {
                      "Unable to read .debug_names data: {0}");
     }
   }
 
   m_index =
       std::make_unique<ManualDWARFIndex>(*GetObjectFile()->GetModule(), *this);
 }
 
+llvm::Optional<std::chrono::duration<double>>
+SymbolFileDWARF::GetSymbolLoadingTime() {
+  if (!m_index)
+    return llvm::None;
+  return m_index->GetSymbolLoadingTime();
+}
+
 bool SymbolFileDWARF::SupportedVersion(uint16_t version) {
   return version >= 2 && version <= 5;
 }
 
 uint32_t SymbolFileDWARF::CalculateAbilities() {
   uint32_t abilities = 0;
   if (m_objfile_sp != nullptr) {
     const Section *section = nullptr;

lldb/source/Target/CMakeLists.txt

add_lldb_library(lldbTarget
   StackFrameList.cpp
   StackFrameRecognizer.cpp
   StackID.cpp
   StopInfo.cpp
   StructuredDataPlugin.cpp
   SystemRuntime.cpp
   Target.cpp
   TargetList.cpp
+  TargetStats.cpp
   Thread.cpp
   ThreadCollection.cpp
   ThreadList.cpp
   ThreadPlan.cpp
   ThreadPlanBase.cpp
   ThreadPlanCallFunction.cpp
   ThreadPlanCallFunctionUsingABI.cpp
   ThreadPlanCallOnFunctionExit.cpp

lldb/source/Target/Target.cpp

 //===-- Target.cpp --------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Target/Target.h"
 #include "lldb/Breakpoint/BreakpointIDList.h"
 #include "lldb/Breakpoint/BreakpointPrecondition.h"
 #include "lldb/Breakpoint/BreakpointResolver.h"
 #include "lldb/Breakpoint/BreakpointResolverAddress.h"
 #include "lldb/Breakpoint/BreakpointResolverFileLine.h"
 #include "lldb/Breakpoint/BreakpointResolverFileRegex.h"
 #include "lldb/Breakpoint/BreakpointResolverName.h"
 #include "lldb/Breakpoint/BreakpointResolverScripted.h"
: TargetProperties(this),
       Broadcaster(debugger.GetBroadcasterManager(),
                   Target::GetStaticBroadcasterClass().AsCString()),
       ExecutionContextScope(), m_debugger(debugger), m_platform_sp(platform_sp),
       m_mutex(), m_arch(target_arch), m_images(this), m_section_load_history(),
       m_breakpoint_list(false), m_internal_breakpoint_list(true),
       m_watchpoint_list(), m_process_sp(), m_search_filter_sp(),
       m_image_search_paths(ImageSearchPathsChanged, this),
       m_source_manager_up(), m_stop_hooks(), m_stop_hook_next_id(0),
       m_latest_stop_hook_id(0),
       m_valid(true), m_suppress_stop_hooks(false),
       m_is_dummy_target(is_dummy_target),
       m_frame_recognizer_manager_up(
-          std::make_unique<StackFrameRecognizerManager>()),
-      m_stats_storage(static_cast<int>(StatisticKind::StatisticMax))
+          std::make_unique<StackFrameRecognizerManager>())
 
 {
   SetEventName(eBroadcastBitBreakpointChanged, "breakpoint-changed");
   SetEventName(eBroadcastBitModulesLoaded, "modules-loaded");
   SetEventName(eBroadcastBitModulesUnloaded, "modules-unloaded");
   SetEventName(eBroadcastBitWatchpointChanged, "watchpoint-changed");
   SetEventName(eBroadcastBitSymbolsLoaded, "symbols-loaded");
 

lldb/source/Target/TargetStats.cpp

+//===-- Target.cpp --------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Target/TargetStats.h"
+
+#include "lldb/Core/Module.h"
+#include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Target/Target.h"
+
+using namespace lldb;
+using namespace lldb_private;
+using namespace llvm;
+
+void TargetStats::Operation::Notify(bool success) {
+  if (success)
+    successes++;
+  else
+    failures++;
+}
+
+void TargetStats::NotifyExprEval(bool success) { m_expr_eval.Notify(success); }
+
+void TargetStats::NotifyFrameVar(bool success) { m_frame_var.Notify(success); }
+
+void TargetStats::CollectModuleStats(Target &target) {
+  for (ModuleSP module_sp : target.GetImages().Modules()) {
+    m_module_stats[module_sp->GetFileSpec().GetPath()] = {
+        module_sp->GetSymbolFile()->GetSymbolLoadingTime()};
+  }
+}
+
+void TargetStats::Dump(Stream &s, Target &target) {
+  CollectModuleStats(target);
+  s << llvm::formatv("Number of expr evaluation successes: {0}\n",
+                     m_expr_eval.successes)
+           .str();
+  s << llvm::formatv("Number of expr evaluation failures: {0}\n",
+                     m_expr_eval.failures)
+           .str();
+  s << llvm::formatv("Number of frame var successes: {0}\n",
+                     m_frame_var.successes)
+           .str();
+  s << llvm::formatv("Number of frame var failures: {0}\n",
+                     m_frame_var.failures)
+           .str();
+
+  s << "Modules: \n";
+
+  std::for_each(m_module_stats.begin(), m_module_stats.end(),
+                [&](const std::pair<std::string, ModuleStats> &module_entry) {
+                  s << "  " << module_entry.first << ":\n";
+                  s << "    Symbol loading time in seconds: ";
+                  if (module_entry.second.symbol_loading_time)
+                    s << llvm::formatv(
+                             "{0:f9}",
+                             module_entry.second.symbol_loading_time->count())
+                             .str();
+                  else
+                    s << "not loaded";
+                  s << "\n";
+                });
+}
+
+json::Value TargetStats::Operation::ToJSON() const {
+  return json::Value(json::Object{
+      {"successes", successes},
+      {"failures", failures},
+  });
+}
+
+json::Value TargetStats::ModuleStats::ToJSON() const {
+  if (symbol_loading_time) {
+    return json::Value(
+        json::Object{{"symbolLoadingTimeInSec:",
+                      formatv("{0:f9}", symbol_loading_time->count()).str()}});
+  } else {
+    return json::Value(json::Object{{"symbolLoadingTimeInSec:", nullptr}});
+  }
+}
+
+json::Value TargetStats::ToJSON(Target &target) {
+  CollectModuleStats(target);
+
+  json::Object modules;
+  std::for_each(
+      m_module_stats.begin(), m_module_stats.end(),
+      [&](const std::pair<std::string, ModuleStats> &module_entry) {
+        modules.insert({module_entry.first, module_entry.second.ToJSON()});
+      });
+
+  return json::Value(json::Object{
+      {"exprEval", m_expr_eval.ToJSON()},
+      {"frameVal", m_frame_var.ToJSON()},
+      {"modules", std::move(modules)},
+  });
+}