Modify "statistics dump" to dump JSON.

[clayborg]

Summary

This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804.

This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like:

(lldb) statistics dump
{

"expressionEvaluation": {
  "failures": 0,
  "successes": 0
},
"firstStopTime": 0.34164492800000001,
"frameVariable": {
  "failures": 0,
  "successes": 0
},
"launchOrAttachTime": 0.31969605400000001,
"targetCreateTime": 0.0040863039999999998

}

The top level keys are:

"expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts.
"frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts.
"targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled.
"launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs.
"firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test.

This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class.

[JDevlieghere]

I have a few comments but I think this is going in a direction that will work for both of us. Once this lands I can sign up for the work to untie the statistics form the target.

lldb/include/lldb/Target/Statistics.h

24

I'd use an enum class with Success and Failure here instead of a bool that requires you to know what the argument is named to know if true means success of failure.

29

I would add a member for the name so that ToJSON can generate the whole JSON object. I imagine something like

SuccessFailStats s("frameVariable");
s.Notify(SuccessFailStats::Success);
s.ToJSON()

which generates

"frameVariable": {
  "failures": 0,
  "successes": 1
},

32

The direction I outlined in the other patch is to move away from tying the statistics to the target. I think the class itself is fine, as long as we agree that the target stats would be held by an object spanning all debuggers.

36

Why don't we have these return an ElapsedTime. Then you can do something like

ElapsedTime elapsed_time = m_stats.GetCreateTime();

RVO should ensure there are no copies, but you can always delete the copy ctor to make sure. Maybe we should call it ScopedTime or something to make it more obvious that this is a RAII object?

52

In line with my earlier comment about not tying the stats tot he target, this would be controlled by the global (singleton) statistic class.

54

I would abstract this behind a TimeStats or TimeStats, similar to SuccessFailStats, so that every type of statistic has its own type.

lldb/include/lldb/Utility/ElapsedTime.h

34 ↗

(On Diff #379233)

This seems simple enough and specific enough to the statistics that it can be part of that header?

[wallace]

lldb/include/lldb/Target/Process.h

241–242

add a comment saying that this returns the stop id with the value before the bump, otherwise someone might think that this returns the modified value

lldb/include/lldb/Target/Statistics.h

24

Or maybe just NotifySuccess(bool) to keep callers simple

lldb/include/lldb/Target/Target.h

1466–1468

this param is not present

lldb/include/lldb/Utility/ElapsedTime.h

22 ↗

(On Diff #379233)

29 ↗

(On Diff #379233)

41 ↗

(On Diff #379233)

lldb/include/lldb/lldb-forward.h

121 ↗

(On Diff #379233)

move this to line ~277

lldb/packages/Python/lldbsuite/test/lldbtest.py

753

is this actually used somewhere?

lldb/source/API/SBTarget.cpp

216–219

a simpler version

lldb/source/Commands/CommandObjectExpression.cpp

661–664

actually this is not sufficient, we need to log as well in the SBAPI side, which is what lldb-vscode uses to evaluate expressions. So either move it to deeper point in the call chain or report this event from other locations

lldb/source/Commands/CommandObjectFrame.cpp

715

same here

[clayborg]

lldb/include/lldb/Target/Statistics.h

24

I like the idea of adding:

void NotifySuccess();
void NotifyFailure();

JDevlieghere you ok with that?

29

Good idea!

32

Yeah, my main patch did a little of both. The main idea behind the classes in this file are they don't need to be the storage for the stats, but they can be if that makes things easier. The main classes (Target, Module, Breakpoint, etc) can just have member variables in the main classes themselves, but anything inside of these classes can/should be used in case we want to aggregate stats from any contained Stats classes. For example, I want to have a ModuleStats class (see https://reviews.llvm.org/D110804), this class doesn't not live inside of Module itself, but can be constructed from a Module. Then the TargetStats class can have a std::vector<ModuleStats> classes that it gathers when TargetStats is asked to collect statistics. This same methodology can then be applied to the debugger where we have DebuggerStats.

What do you think of this approach:

• Any class can define a XXXStats class in this file which is used to collect statistics at any given point
• Classes can use the XXXStats class as a member variable if it makes sense, or just add member variables to store any objects needed to calculate the stats later when the object is asked to collect stats (target.GatherStatistics(target_stats);).

This would mean we could make a DebuggerStats class that can have a std::vector<TargetStats>, and when the debugger.GatherStatistics(...) is called. Then either the DebuggerStats or TargetStats object can be asked to be converted to JSON when presenting to the user in the "statistics dump" command?

36

These are setting the time at which things start. If we follow suggestions above, SetLaunchOrAttachTime(), SetFirstPrivateStopTime(), and SetFirstPublicStopTime() can be remove and all of the llvm::Optional<std::chrono::steady_clock::time_point> member variables can be moved to Target class, and we can store only "llvm::Optional<ElapsedTime::Duration> launch_or_attach_time;" member variables in the TargetStats class, which would get set when statistics are gathered?

52

I put this here because of the existing SB API:

bool SBTarget::GetCollectingStats();
void SBTarget::SetCollectingStats(bool);

So you want debugger to have this? This m_collecting_stats can be moved to Debugger, or DebuggerStats? But that makes the above API useless or it becomes deprecated. We really don't need to set if we are collecting stats IMHO, since collection should be cheap. You ok with deprecating the "statistics enable"/"statistics disable" and the above APIs?

54

I will make a ElapsedTime::TimePoint

lldb/include/lldb/Target/Target.h

1466–1468

I will remove it!

lldb/include/lldb/Utility/ElapsedTime.h

34 ↗

(On Diff #379233)

Yes, good idea.

lldb/include/lldb/lldb-forward.h

121 ↗

(On Diff #379233)

I actually need to remove this for now as we don't have this class yet!

lldb/packages/Python/lldbsuite/test/lldbtest.py

753

Not yet, I will remove it

lldb/source/API/SBTarget.cpp

216–219

Will switch!

lldb/source/Commands/CommandObjectExpression.cpp

661–664

Gotcha, will move this.

lldb/source/Commands/CommandObjectFrame.cpp

715

"frame variable" takes optional args. If there are none, it dumps everything which will never fail, if it has args it can be one or more variable names or variable expressions. I can hook into a few more places for this.

[clayborg]

Fixed issues identified in first review round:

• typos in comments
• create GlobalStats class to contain the global statistics
• Make sure all code is using the new StatsClock, StatsDuration, and StatsTimepoint definitions
• added a "--all-targets" option to dump options to dump stats for all targets and open further discussion

[clayborg]

Ok, take a look now. I added a GlobalStats class that contains the "enable/disable" state of the statistics gathering. This shows how things can be organized and dumped.

A few things about the current approach: Stats classes in Statistics.h are designed to be able to be fetched, like TargetStats, and then any class that owns a collection of these, like how GlobalStats could own a std::vector<TargetStats> could then aggregate any sub stats into key/value pairs at this higher level. The original patch will end up gathering ModuleStats for each module in TargetStats, and then collecting total stats for debug info size, debug info parse and index time, symbol table parse and index time, and these total stats can be aggregated and displayed. So hopefully this shows the direction I was thinking. Let me know what you think and what changes you would like to see here.

[clayborg]

Right now if you run:
(lldb) statistics dump
It will dump stats for the current target only as one target dictionary. We can change this if desired.
(lldb) statistics dump --all-targets
will dump an dictionary that contains a top level "targets" key/value pair where the value is an array of the all target dictionaries statistics. Of course new key/value pairs can be added to this top level that total up stats from and of the TargetStats statistics.

[clayborg]

Friendly ping

[JDevlieghere]

lldb/include/lldb/Target/Statistics.h

97

Do we expect there to be something like DebuggerStats? I think it would be nice from a hierarchy perspective that Global Stats have a map of debugger -> to debuggerstat who then in turn hold on to a map of target -> target stats. That hierarchy would work really well for JSON (except they would be lists instead of maps).

98–115

For things like how many lldb_asserts we've hit, this actually will contain its own stats, not just things per debugger and target. I think it makes more sense to make this a Singleton as I did in D110895.

[clayborg]

lldb/include/lldb/Target/Statistics.h

97

I can change this to DebuggerStats as this is essentially what this was.

115

This can be expensive if you start locking a mutex just to increment a stat that is a counter and will make statistics slow down the debugger. I would rather rely on std::atomic or the locks already built into Target or Debugger if possible. If lldb_asserts are firing off at a high rate, we can't be spending thousands of instructions locking and unlocking mutexes.

[clayborg]

Rename GlobalStats to DebuggerStats.

[JDevlieghere]

I'm not super happy with the debugger stats as they are right here. Given that it only affects a subset of this patch and I don't want to block other work that depends on this, it's probably easier for me to create a patch on top of this rather than iterating on this one.

LGTM

lldb/include/lldb/Target/Statistics.h

97

I do need something that sits about the debugger for the lldb_assert case.

115

Yeah, and atomic variable for a particular metric might do the trick.

[clayborg]

So there is a buildbot failure due to an expression which succeeds, but because the "expression" command options don't apply cleanly to the expression result the "expression" command fails. Question: do we want "expressionEvaluation" to truly track expression evaluation failures, or just track the "expression" command success and fails? I updated this patch to track expression failures only by modifying the Target::EvaluateExpression(), so if this succeeds and "expr" command fails, we consider the expression a success right now.

[teemperor]

In D111686#3079248, @clayborg wrote:

So there is a buildbot failure due to an expression which succeeds, but because the "expression" command options don't apply cleanly to the expression result the "expression" command fails. Question: do we want "expressionEvaluation" to truly track expression evaluation failures, or just track the "expression" command success and fails? I updated this patch to track expression failures only by modifying the Target::EvaluateExpression(), so if this succeeds and "expr" command fails, we consider the expression a success right now.

I have to look into this but could we skip that test in the meantime?

There is also another failure that looks like it's caused by some accidental change (see inline)

lldb/packages/Python/lldbsuite/test/lldbtest.py

753

I think you also removed some existing stuff here:

PASS: LLDB (C:\buildbot\lldb-x64-windows-ninja\build\bin\clang.exe-x86_64) :: test_zero_uuid_modules (TestMiniDumpUUID.MiniDumpUUIDTestCase)
======================================================================
ERROR: test_relative_module_name (TestMiniDumpUUID.MiniDumpUUIDTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot\lldb-x64-windows-ninja\llvm-project\lldb\test\API\functionalities\postmortem\minidump-new\TestMiniDumpUUID.py", line 317, in test_relative_module_name
    self.getSourcePath("relative_module_name.yaml"))
AttributeError: 'MiniDumpUUIDTestCase' object has no attribute 'getSourcePath'
Config=x86_64-C:\buildbot\lldb-x64-windows-ninja\build\bin\clang.exe

[clayborg]

In D111686#3079352, @teemperor wrote:

In D111686#3079248, @clayborg wrote:

So there is a buildbot failure due to an expression which succeeds, but because the "expression" command options don't apply cleanly to the expression result the "expression" command fails. Question: do we want "expressionEvaluation" to truly track expression evaluation failures, or just track the "expression" command success and fails? I updated this patch to track expression failures only by modifying the Target::EvaluateExpression(), so if this succeeds and "expr" command fails, we consider the expression a success right now.

I have to look into this but could we skip that test in the meantime?

There is also another failure that looks like it's caused by some accidental change (see inline)''

I will check in a fix in a few minutes for my buildbot issues.

[clayborg]

Fixed buildbots with:

commit 910838f07da7872d2b7cca5b07d64ea9915b6767 (HEAD -> main, origin/main, origin/HEAD)
Author: Greg Clayton <[e-mail]>
Date: Thu Oct 21 14:21:36 2021 -0700

Fix buildbots after https://reviews.llvm.org/D111686

lldb/include/lldb/Target/Process.h

if (this != &rhs) {
       m_stop_id = rhs.m_stop_id;
       m_memory_id = rhs.m_memory_id;
     }
     return *this;
   }
 
   ~ProcessModID() = default;
 
-  void BumpStopID() {
-    m_stop_id++;
+  uint32_t BumpStopID() {
+    const uint32_t prev_stop_id = m_stop_id++;
     if (!IsLastResumeForUserExpression())
       m_last_natural_stop_id++;
+    return prev_stop_id;
   }
 
   void BumpMemoryID() { m_memory_id++; }
 
   void BumpResumeID() {
     m_resume_id++;
     if (m_running_user_expression > 0)
       m_last_user_expression_resume = m_resume_id;

lldb/include/lldb/Target/Statistics.h

+//===-- Statistics.h --------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_TARGET_STATISTICS_H
+#define LLDB_TARGET_STATISTICS_H
+
+#include <chrono>
+#include <string>
+#include <vector>
+
+#include "lldb/Utility/Stream.h"
+#include "lldb/lldb-forward.h"
+#include "llvm/Support/JSON.h"
+
+namespace lldb_private {
+
+using StatsClock = std::chrono::high_resolution_clock;
+using StatsDuration = std::chrono::duration<double>;
+using StatsTimepoint = std::chrono::time_point<StatsClock>;
+
+/// A class that measures elapsed time in an exception safe way.
+///
+/// This is a RAII class is designed to help gather timing statistics within
+/// LLDB where objects have optional Duration variables that get updated with
+/// elapsed times. This helps LLDB measure statistics for many things that are
+/// then reported in LLDB commands.
+///
+/// Objects that need to measure elapsed times should have a variable of type
+/// "StatsDuration m_time_xxx;" which can then be used in the constructor of
+/// this class inside a scope that wants to measure something:
+///
+///   ElapsedTime elapsed(m_time_xxx);
+///   // Do some work
+///
+/// This class will increment the m_time_xxx variable with the elapsed time
+/// when the object goes out of scope. The "m_time_xxx" variable will be
+/// incremented when the class goes out of scope. This allows a variable to
+/// measure something that might happen in stages at different times, like
+/// resolving a breakpoint each time a new shared library is loaded.
+class ElapsedTime {
+public:
+  /// Set to the start time when the object is created.
+  StatsTimepoint m_start_time;
+  /// Elapsed time in seconds to increment when this object goes out of scope.
+  StatsDuration &m_elapsed_time;
+
+public:
+  ElapsedTime(StatsDuration &opt_time) : m_elapsed_time(opt_time) {
+    m_start_time = StatsClock::now();
+  }
+  ~ElapsedTime() {
+    StatsDuration elapsed = StatsClock::now() - m_start_time;
+    m_elapsed_time += elapsed;
+  }
+};
+
+/// A class to count success/fail statistics.
+struct StatsSuccessFail {
+  StatsSuccessFail(llvm::StringRef n) : name(n.str()) {}
+
+  void NotifySuccess() { ++successes; }
+  void NotifyFailure() { ++failures; }
+
+  llvm::json::Value ToJSON() const;
+  std::string name;
+  uint32_t successes = 0;
+  uint32_t failures = 0;
+};
+
+/// A class that represents statistics for a since lldb_private::Target.
+class TargetStats {
+public:
+  llvm::json::Value ToJSON();
+
+  void SetLaunchOrAttachTime();
+  void SetFirstPrivateStopTime();
+  void SetFirstPublicStopTime();
+
+  StatsDuration &GetCreateTime() { return m_create_time; }
+  StatsSuccessFail &GetExpressionStats() { return m_expr_eval; }
+  StatsSuccessFail &GetFrameVariableStats() { return m_frame_var; }
+
+protected:
+  StatsDuration m_create_time{0.0};
+  llvm::Optional<StatsTimepoint> m_launch_or_attach_time;
+  llvm::Optional<StatsTimepoint> m_first_private_stop_time;
+  llvm::Optional<StatsTimepoint> m_first_public_stop_time;
+  StatsSuccessFail m_expr_eval{"expressionEvaluation"};
+  StatsSuccessFail m_frame_var{"frameVariable"};
+};
+
+class DebuggerStats {
+public:
+  static void SetCollectingStats(bool enable) { g_collecting_stats = enable; }
+  static bool GetCollectingStats() { return g_collecting_stats; }
+
+  /// Get metrics associated with all targets in a debugger in JSON format.
+  ///
+  /// \return
+  ///     Returns a JSON value that contains all target metrics.
+  static llvm::json::Value ReportStatistics(Debugger &debugger);
+
+protected:
+  // Collecting stats can be set to true to collect stats that are expensive
+  // to collect. By default all stats that are cheap to collect are enabled.
+  // This settings is here to maintain compatibility with "statistics enable"
+  // and "statistics disable".
+  static bool g_collecting_stats;
+};
+
+} // namespace lldb_private
+
+#endif // LLDB_TARGET_STATISTICS_H

lldb/include/lldb/Target/Target.h

 #include "lldb/Core/ModuleList.h"
 #include "lldb/Core/UserSettingsController.h"
 #include "lldb/Expression/Expression.h"
 #include "lldb/Host/ProcessLaunchInfo.h"
 #include "lldb/Symbol/TypeSystem.h"
 #include "lldb/Target/ExecutionContextScope.h"
 #include "lldb/Target/PathMappingList.h"
 #include "lldb/Target/SectionLoadHistory.h"
+#include "lldb/Target/Statistics.h"
 #include "lldb/Target/ThreadSpec.h"
 #include "lldb/Utility/ArchSpec.h"
 #include "lldb/Utility/Broadcaster.h"
 #include "lldb/Utility/LLDBAssert.h"
 #include "lldb/Utility/Timeout.h"
 #include "lldb/lldb-public.h"
 
 namespace lldb_private {
protected:
   /// Stores the frame recognizers of this target.
   lldb::StackFrameRecognizerManagerUP m_frame_recognizer_manager_up;
 
   static void ImageSearchPathsChanged(const PathMappingList &path_list,
                                       void *baton);
 
   // Utilities for `statistics` command.
 private:
-  std::vector<uint32_t> m_stats_storage;
-  bool m_collecting_stats = false;
+  // Target metrics storage.
+  TargetStats m_stats;
 
 public:
-  void SetCollectingStats(bool v) { m_collecting_stats = v; }
-
-  bool GetCollectingStats() { return m_collecting_stats; }
-
-  void IncrementStats(lldb_private::StatisticKind key) {
-    if (!GetCollectingStats())
-      return;
-    lldbassert(key < lldb_private::StatisticKind::StatisticMax &&
-               "invalid statistics!");
-    m_stats_storage[key] += 1;
-  }
+  /// Get metrics associated with this target in JSON format.
+  ///
+  /// Target metrics help measure timings and information that is contained in
+  /// a target. These are designed to help measure performance of a debug
+  /// session as well as represent the current state of the target, like
+  /// information on the currently modules, currently set breakpoints and more.
+  ///
+  /// \return
+  ///     Returns a JSON value that contains all target metrics.
+  llvm::json::Value ReportStatistics();
 
-  std::vector<uint32_t> GetStatistics() { return m_stats_storage; }
+  TargetStats &GetStatistics() { return m_stats; }
 
 private:
   /// Construct with optional file and arch.
   ///
   /// This member is private. Clients must use
   /// TargetList::CreateTarget(const FileSpec*, const ArchSpec*)
   /// so all targets can be tracked from the central target list.
   ///

lldb/packages/Python/lldbsuite/test/lldbtest.py

def makeBuildDir(self):
         if os.path.isdir(bdir):
             shutil.rmtree(bdir)
         lldbutil.mkdir_p(bdir)
 
     def getBuildArtifact(self, name="a.out"):
         """Return absolute path to an artifact in the test's build directory."""
         return os.path.join(self.getBuildDir(), name)
 
-    def getSourcePath(self, name):
-        """Return absolute path to a file in the test's source directory."""
-        return os.path.join(self.getSourceDir(), name)
-
     @classmethod
     def setUpCommands(cls):
         commands = [
             # First of all, clear all settings to have clean state of global properties.
             "settings clear -all",
 
             # Disable Spotlight lookup. The testsuite creates
             # different binaries with the same UUID, because they only
             # differ in the debug info, which is not being hashed.

lldb/source/API/SBTarget.cpp

 
 SBStructuredData SBTarget::GetStatistics() {
   LLDB_RECORD_METHOD_NO_ARGS(lldb::SBStructuredData, SBTarget, GetStatistics);
 
   SBStructuredData data;
   TargetSP target_sp(GetSP());
   if (!target_sp)
     return LLDB_RECORD_RESULT(data);
-
-  auto stats_up = std::make_unique<StructuredData::Dictionary>();
-  int i = 0;
-  for (auto &Entry : target_sp->GetStatistics()) {
-    std::string Desc = lldb_private::GetStatDescription(
-        static_cast<lldb_private::StatisticKind>(i));
-    stats_up->AddIntegerItem(Desc, Entry);
-    i += 1;
-  }
-
-  data.m_impl_up->SetObjectSP(std::move(stats_up));
+  std::string json_str =
+      llvm::formatv("{0:2}", target_sp->ReportStatistics()).str();
+  data.m_impl_up->SetObjectSP(StructuredData::ParseJSON(json_str));
   return LLDB_RECORD_RESULT(data);
 }
 
 void SBTarget::SetCollectingStats(bool v) {
   LLDB_RECORD_METHOD(void, SBTarget, SetCollectingStats, (bool), v);
 
   TargetSP target_sp(GetSP());
   if (!target_sp)
     return;
-  return target_sp->SetCollectingStats(v);
+  return DebuggerStats::SetCollectingStats(v);
 }
 
 bool SBTarget::GetCollectingStats() {
   LLDB_RECORD_METHOD_NO_ARGS(bool, SBTarget, GetCollectingStats);
 
   TargetSP target_sp(GetSP());
   if (!target_sp)
     return false;
-  return target_sp->GetCollectingStats();
+  return DebuggerStats::GetCollectingStats();
 }
 
 SBProcess SBTarget::LoadCore(const char *core_file) {
   LLDB_RECORD_METHOD(lldb::SBProcess, SBTarget, LoadCore, (const char *),
                      core_file);
 
   lldb::SBError error; // Ignored
   return LLDB_RECORD_RESULT(LoadCore(core_file, error));

lldb/source/Commands/CommandObjectExpression.cpp

if (!m_fixed_expression.empty() && target.GetEnableNotifyAboutFixIts()) {
       std::string fixed_command("expression ");
       if (args.HasArgs()) {
         // Add in any options that might have been in the original command:
         fixed_command.append(std::string(args.GetArgStringWithDelimiter()));
         fixed_command.append(m_fixed_expression);
       } else
         fixed_command.append(m_fixed_expression);
       history.AppendString(fixed_command);
     }
-    // Increment statistics to record this expression evaluation success.
-    target.IncrementStats(StatisticKind::ExpressionSuccessful);
     return true;
   }
-
-  // Increment statistics to record this expression evaluation failure.
-  target.IncrementStats(StatisticKind::ExpressionFailure);
   result.SetStatus(eReturnStatusFailed);
   return false;
 }

lldb/source/Commands/CommandObjectFrame.cpp

bool DoExecute(Args &command, CommandReturnObject &result) override {
     if (m_interpreter.TruncationWarningNecessary()) {
       result.GetOutputStream().Printf(m_interpreter.TruncationWarningText(),
                                       m_cmd_name.c_str());
       m_interpreter.TruncationWarningGiven();
     }
 
     // Increment statistics.
     bool res = result.Succeeded();
-    Target &target = GetSelectedOrDummyTarget();
+    TargetStats &target_stats = GetSelectedOrDummyTarget().GetStatistics();
     if (res)
-      target.IncrementStats(StatisticKind::FrameVarSuccess);
+      target_stats.GetFrameVariableStats().NotifySuccess();
     else
-      target.IncrementStats(StatisticKind::FrameVarFailure);
+      target_stats.GetFrameVariableStats().NotifyFailure();
     return res;
   }
 
   OptionGroupOptions m_option_group;
   OptionGroupVariable m_option_variable;
   OptionGroupFormat m_option_format;
   OptionGroupValueObjectDisplay m_varobj_options;
 };

lldb/source/Commands/CommandObjectStats.cpp

 //===-- CommandObjectStats.cpp --------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "CommandObjectStats.h"
+#include "lldb/Core/Debugger.h"
+#include "lldb/Host/OptionParser.h"
 #include "lldb/Interpreter/CommandReturnObject.h"
 #include "lldb/Target/Target.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 class CommandObjectStatsEnable : public CommandObjectParsed {
 public:
   CommandObjectStatsEnable(CommandInterpreter &interpreter)
       : CommandObjectParsed(interpreter, "enable",
                             "Enable statistics collection", nullptr,
                             eCommandProcessMustBePaused) {}
 
   ~CommandObjectStatsEnable() override = default;
 
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
-    Target &target = GetSelectedOrDummyTarget();
-
-    if (target.GetCollectingStats()) {
+    if (DebuggerStats::GetCollectingStats()) {
       result.AppendError("statistics already enabled");
       return false;
     }
 
-    target.SetCollectingStats(true);
+    DebuggerStats::SetCollectingStats(true);
     result.SetStatus(eReturnStatusSuccessFinishResult);
     return true;
   }
 };
 
 class CommandObjectStatsDisable : public CommandObjectParsed {
 public:
   CommandObjectStatsDisable(CommandInterpreter &interpreter)
       : CommandObjectParsed(interpreter, "disable",
                             "Disable statistics collection", nullptr,
                             eCommandProcessMustBePaused) {}
 
   ~CommandObjectStatsDisable() override = default;
 
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
-    Target &target = GetSelectedOrDummyTarget();
-
-    if (!target.GetCollectingStats()) {
+    if (!DebuggerStats::GetCollectingStats()) {
       result.AppendError("need to enable statistics before disabling them");
       return false;
     }
 
-    target.SetCollectingStats(false);
+    DebuggerStats::SetCollectingStats(false);
     result.SetStatus(eReturnStatusSuccessFinishResult);
     return true;
   }
 };
 
+#define LLDB_OPTIONS_statistics_dump
+#include "CommandOptions.inc"
+
 class CommandObjectStatsDump : public CommandObjectParsed {
+  class CommandOptions : public Options {
+  public:
+    CommandOptions() : Options() { OptionParsingStarting(nullptr); }
+
+    Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_arg,
+                          ExecutionContext *execution_context) override {
+      Status error;
+      const int short_option = m_getopt_table[option_idx].val;
+
+      switch (short_option) {
+      case 'a':
+        m_all_targets = true;
+        break;
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+      return error;
+    }
+
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      m_all_targets = false;
+    }
+
+    llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
+      return llvm::makeArrayRef(g_statistics_dump_options);
+    }
+
+    bool m_all_targets = false;
+  };
+
 public:
   CommandObjectStatsDump(CommandInterpreter &interpreter)
-      : CommandObjectParsed(interpreter, "dump", "Dump statistics results",
-                            nullptr, eCommandProcessMustBePaused) {}
+      : CommandObjectParsed(
+            interpreter, "statistics dump", "Dump metrics in JSON format",
+            "statistics dump [<options>]", eCommandRequiresTarget) {}
 
   ~CommandObjectStatsDump() override = default;
 
+  Options *GetOptions() override { return &m_options; }
+
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
-    Target &target = GetSelectedOrDummyTarget();
-
-    uint32_t i = 0;
-    for (auto &stat : target.GetStatistics()) {
-      result.AppendMessageWithFormat(
-          "%s : %u\n",
-          lldb_private::GetStatDescription(
-              static_cast<lldb_private::StatisticKind>(i))
-              .c_str(),
-          stat);
-      i += 1;
+    if (m_options.m_all_targets) {
+      result.AppendMessageWithFormatv(
+          "{0:2}", DebuggerStats::ReportStatistics(GetDebugger()));
+    } else {
+      Target &target = m_exe_ctx.GetTargetRef();
+      result.AppendMessageWithFormatv("{0:2}", target.ReportStatistics());
     }
     result.SetStatus(eReturnStatusSuccessFinishResult);
     return true;
   }
+
+  CommandOptions m_options;
 };
 
 CommandObjectStats::CommandObjectStats(CommandInterpreter &interpreter)
     : CommandObjectMultiword(interpreter, "statistics",
                              "Print statistics about a debugging session",
                              "statistics <subcommand> [<subcommand-options>]") {
   LoadSubCommand("enable",
                  CommandObjectSP(new CommandObjectStatsEnable(interpreter)));
   LoadSubCommand("disable",
                  CommandObjectSP(new CommandObjectStatsDisable(interpreter)));
   LoadSubCommand("dump",
                  CommandObjectSP(new CommandObjectStatsDump(interpreter)));
 }
 
 CommandObjectStats::~CommandObjectStats() = default;

lldb/source/Commands/Options.td

let Command = "trace dump" in {
   def trace_dump_verbose : Option<"verbose", "v">, Group<1>,
     Desc<"Show verbose trace information.">;
 }
 
 let Command = "trace schema" in {
   def trace_schema_verbose : Option<"verbose", "v">, Group<1>,
     Desc<"Show verbose trace schema logging for debugging the plug-in.">;
 }
+
+let Command = "statistics dump" in {
+  def statistics_dump_all: Option<"all-targets", "a">, Group<1>,
+    Desc<"Include statistics for all targets.">;
+}

lldb/source/Target/CMakeLists.txt

add_lldb_library(lldbTarget
   RegisterNumber.cpp
   RemoteAwarePlatform.cpp
   SectionLoadHistory.cpp
   SectionLoadList.cpp
   StackFrame.cpp
   StackFrameList.cpp
   StackFrameRecognizer.cpp
   StackID.cpp
+  Statistics.cpp
   StopInfo.cpp
   StructuredDataPlugin.cpp
   SystemRuntime.cpp
   Target.cpp
   TargetList.cpp
   Thread.cpp
   ThreadCollection.cpp
   ThreadList.cpp

lldb/source/Target/Process.cpp

uint32_t Process::AssignIndexIDToThread(uint64_t thread_id) {
   return result;
 }
 
 StateType Process::GetState() {
   return m_public_state.GetValue();
 }
 
 void Process::SetPublicState(StateType new_state, bool restarted) {
+  const bool new_state_is_stopped = StateIsStoppedState(new_state, false);
+  if (new_state_is_stopped) {
+    // This will only set the time if the public stop time has no value, so
+    // it is ok to call this multiple times. With a public stop we can't look
+    // at the stop ID because many private stops might have happened, so we
+    // can't check for a stop ID of zero. This allows the "statistics" command
+    // to dump the time it takes to reach somewhere in your code, like a
+    // breakpoint you set.
+    GetTarget().GetStatistics().SetFirstPublicStopTime();
+  }
+
   Log *log(lldb_private::GetLogIfAnyCategoriesSet(LIBLLDB_LOG_STATE |
                                                   LIBLLDB_LOG_PROCESS));
   LLDB_LOGF(log, "Process::SetPublicState (state = %s, restarted = %i)",
             StateAsCString(new_state), restarted);
   const StateType old_state = m_public_state.GetValue();
   m_public_state.SetValue(new_state);
 
   // On the transition from Run to Stopped, we unlock the writer end of the run
   // lock.  The lock gets locked in Resume, which is the public API to tell the
   // program to run.
   if (!StateChangedIsExternallyHijacked()) {
     if (new_state == eStateDetached) {
       LLDB_LOGF(log,
                 "Process::SetPublicState (%s) -- unlocking run lock for detach",
                 StateAsCString(new_state));
       m_public_run_lock.SetStopped();
     } else {
       const bool old_state_is_stopped = StateIsStoppedState(old_state, false);
-      const bool new_state_is_stopped = StateIsStoppedState(new_state, false);
       if ((old_state_is_stopped != new_state_is_stopped)) {
         if (new_state_is_stopped && !restarted) {
           LLDB_LOGF(log, "Process::SetPublicState (%s) -- unlocking run lock",
                     StateAsCString(new_state));
           m_public_run_lock.SetStopped();
         }
       }
     }
if (StateIsStoppedState(new_state, false)) {
       // identify which threads are stopping or create a special thread list
       // containing only threads which actually stopped.
       //
       // The process plugin is responsible for managing the actual behavior of
       // the threads and should have stopped any threads that are going to stop
       // before we get here.
       m_thread_list.DidStop();
 
-      m_mod_id.BumpStopID();
+      if (m_mod_id.BumpStopID() == 0)
+        GetTarget().GetStatistics().SetFirstPrivateStopTime();
+
       if (!m_mod_id.IsLastResumeForUserExpression())
         m_mod_id.SetStopEventForLastNaturalStopID(event_sp);
       m_memory_cache.Clear();
       LLDB_LOGF(log, "Process::SetPrivateState (%s) stop_id = %u",
                 StateAsCString(new_state), m_mod_id.GetStopID());
     }
 
     m_private_state_broadcaster.BroadcastEvent(event_sp);

lldb/source/Target/Statistics.cpp

+//===-- Statistics.cpp ----------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Target/Statistics.h"
+
+#include "lldb/Core/Debugger.h"
+#include "lldb/Core/Module.h"
+#include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Target/Target.h"
+
+using namespace lldb;
+using namespace lldb_private;
+using namespace llvm;
+
+json::Value StatsSuccessFail::ToJSON() const {
+  return json::Object{{"successes", successes}, {"failures", failures}};
+}
+
+static double elapsed(const StatsTimepoint &start, const StatsTimepoint &end) {
+  StatsDuration elapsed = end.time_since_epoch() - start.time_since_epoch();
+  return elapsed.count();
+}
+
+json::Value TargetStats::ToJSON() {
+  json::Object target_metrics_json{{m_expr_eval.name, m_expr_eval.ToJSON()},
+                                   {m_frame_var.name, m_frame_var.ToJSON()}};
+  if (m_launch_or_attach_time && m_first_private_stop_time) {
+    double elapsed_time =
+        elapsed(*m_launch_or_attach_time, *m_first_private_stop_time);
+    target_metrics_json.try_emplace("launchOrAttachTime", elapsed_time);
+  }
+  if (m_launch_or_attach_time && m_first_public_stop_time) {
+    double elapsed_time =
+        elapsed(*m_launch_or_attach_time, *m_first_public_stop_time);
+    target_metrics_json.try_emplace("firstStopTime", elapsed_time);
+  }
+  target_metrics_json.try_emplace("targetCreateTime", m_create_time.count());
+
+  return target_metrics_json;
+}
+
+void TargetStats::SetLaunchOrAttachTime() {
+  m_launch_or_attach_time = StatsClock::now();
+  m_first_private_stop_time = llvm::None;
+}
+
+void TargetStats::SetFirstPrivateStopTime() {
+  // Launching and attaching has many paths depending on if synchronous mode
+  // was used or if we are stopping at the entry point or not. Only set the
+  // first stop time if it hasn't already been set.
+  if (!m_first_private_stop_time)
+    m_first_private_stop_time = StatsClock::now();
+}
+
+void TargetStats::SetFirstPublicStopTime() {
+  // Launching and attaching has many paths depending on if synchronous mode
+  // was used or if we are stopping at the entry point or not. Only set the
+  // first stop time if it hasn't already been set.
+  if (!m_first_public_stop_time)
+    m_first_public_stop_time = StatsClock::now();
+}
+
+bool DebuggerStats::g_collecting_stats = false;
+
+llvm::json::Value DebuggerStats::ReportStatistics(Debugger &debugger) {
+  json::Array targets;
+  for (const auto &target : debugger.GetTargetList().Targets()) {
+    targets.emplace_back(target->ReportStatistics());
+  }
+  json::Object global_stats{
+      {"targets", std::move(targets)},
+  };
+  return std::move(global_stats);
+}

lldb/source/Target/Target.cpp

: TargetProperties(this),
       Broadcaster(debugger.GetBroadcasterManager(),
                   Target::GetStaticBroadcasterClass().AsCString()),
       ExecutionContextScope(), m_debugger(debugger), m_platform_sp(platform_sp),
       m_mutex(), m_arch(target_arch), m_images(this), m_section_load_history(),
       m_breakpoint_list(false), m_internal_breakpoint_list(true),
       m_watchpoint_list(), m_process_sp(), m_search_filter_sp(),
       m_image_search_paths(ImageSearchPathsChanged, this),
       m_source_manager_up(), m_stop_hooks(), m_stop_hook_next_id(0),
-      m_latest_stop_hook_id(0),
-      m_valid(true), m_suppress_stop_hooks(false),
+      m_latest_stop_hook_id(0), m_valid(true), m_suppress_stop_hooks(false),
       m_is_dummy_target(is_dummy_target),
       m_frame_recognizer_manager_up(
-          std::make_unique<StackFrameRecognizerManager>()),
-      m_stats_storage(static_cast<int>(StatisticKind::StatisticMax))
-
-{
+          std::make_unique<StackFrameRecognizerManager>()) {
   SetEventName(eBroadcastBitBreakpointChanged, "breakpoint-changed");
   SetEventName(eBroadcastBitModulesLoaded, "modules-loaded");
   SetEventName(eBroadcastBitModulesUnloaded, "modules-unloaded");
   SetEventName(eBroadcastBitWatchpointChanged, "watchpoint-changed");
   SetEventName(eBroadcastBitSymbolsLoaded, "symbols-loaded");
 
   CheckInWithManager();
 

 }
 
 void Target::SetExecutableModule(ModuleSP &executable_sp,
                                  LoadDependentFiles load_dependent_files) {
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_TARGET));
   ClearModules(false);
 
   if (executable_sp) {
+    ElapsedTime elapsed(m_stats.GetCreateTime());
     LLDB_SCOPED_TIMERF("Target::SetExecutableModule (executable = '%s')",
                        executable_sp->GetFileSpec().GetPath().c_str());
 
     const bool notify = true;
     m_images.Append(executable_sp,
                     notify); // The first image is our executable file
 
     // If we haven't set an architecture yet, reset our architecture based on
ExpressionResults Target::EvaluateExpression(
     llvm::StringRef expr, ExecutionContextScope *exe_scope,
     lldb::ValueObjectSP &result_valobj_sp,
     const EvaluateExpressionOptions &options, std::string *fixed_expression,
     ValueObject *ctx_obj) {
   result_valobj_sp.reset();
 
   ExpressionResults execution_results = eExpressionSetupError;
 
-  if (expr.empty())
+  if (expr.empty()) {
+    m_stats.GetExpressionStats().NotifyFailure();
     return execution_results;
+  }
 
   // We shouldn't run stop hooks in expressions.
   bool old_suppress_value = m_suppress_stop_hooks;
   m_suppress_stop_hooks = true;
   auto on_exit = llvm::make_scope_exit([this, old_suppress_value]() {
     m_suppress_stop_hooks = old_suppress_value;
   });
 
ExpressionResults Target::EvaluateExpression(
   } else {
     llvm::StringRef prefix = GetExpressionPrefixContents();
     Status error;
     execution_results = UserExpression::Evaluate(exe_ctx, options, expr, prefix,
                                                  result_valobj_sp, error,
                                                  fixed_expression, ctx_obj);
   }
 
+  if (execution_results == eExpressionCompleted)
+    m_stats.GetExpressionStats().NotifySuccess();
+  else
+    m_stats.GetExpressionStats().NotifyFailure();
   return execution_results;
 }
 
 lldb::ExpressionVariableSP Target::GetPersistentVariable(ConstString name) {
   lldb::ExpressionVariableSP variable_sp;
   m_scratch_type_system_map.ForEach(
       [name, &variable_sp](TypeSystem *type_system) -> bool {
         if (PersistentExpressionState *persistent_state =
else
     stop_id = m_section_load_history.GetLastStopID();
   return m_section_load_history.SetSectionUnloaded(stop_id, section_sp,
                                                    load_addr);
 }
 
 void Target::ClearAllLoadedSections() { m_section_load_history.Clear(); }
 
 Status Target::Launch(ProcessLaunchInfo &launch_info, Stream *stream) {
+  m_stats.SetLaunchOrAttachTime();
   Status error;
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_TARGET));
 
   LLDB_LOGF(log, "Target::%s() called for %s", __FUNCTION__,
             launch_info.GetExecutableFile().GetPath().c_str());
 
   StateType state = eStateInvalid;
 

 
 llvm::Expected<TraceSP> Target::GetTraceOrCreate() {
   if (m_trace_sp)
     return m_trace_sp;
   return CreateTrace();
 }
 
 Status Target::Attach(ProcessAttachInfo &attach_info, Stream *stream) {
+  m_stats.SetLaunchOrAttachTime();
   auto state = eStateInvalid;
   auto process_sp = GetProcessSP();
   if (process_sp) {
     state = process_sp->GetState();
     if (process_sp->IsAlive() && state != eStateConnected) {
       if (state == eStateAttaching)
         return Status("process attach is in progress");
       return Status("a process is already being debugged");

 }
 
 std::recursive_mutex &Target::GetAPIMutex() {
   if (GetProcessSP() && GetProcessSP()->CurrentThreadIsPrivateStateThread())
     return m_private_mutex;
   else
     return m_mutex;
 }
+
+/// Get metrics associated with this target in JSON format.
+llvm::json::Value Target::ReportStatistics() { return m_stats.ToJSON(); }

lldb/test/API/commands/statistics/basic/TestStats.py

 import lldb
+import json
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 class TestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
-    def test(self):
+    def setUp(self):
+        TestBase.setUp(self)
         self.build()
-        lldbutil.run_to_source_breakpoint(self, "// break here", lldb.SBFileSpec("main.c"))
 
-        self.expect("statistics disable", substrs=['need to enable statistics before disabling'], error=True)
+    NO_DEBUG_INFO_TESTCASE = True
+
+    def test_enable_disable(self):
+        """
+        Test "statistics disable" and "statistics enable". These don't do
+        anything anymore for cheap to gather statistics. In the future if
+        statistics are expensive to gather, we can enable the feature inside
+        of LLDB and test that enabling and disabling stops expesive information
+        from being gathered.
+        """
+        target = self.createTestTarget()
 
-        # 'expression' should change the statistics.
+        self.expect("statistics disable", substrs=['need to enable statistics before disabling'], error=True)
         self.expect("statistics enable")
         self.expect("statistics enable", substrs=['already enabled'], error=True)
-        self.expect("expr patatino", substrs=['27'])
         self.expect("statistics disable")
-        self.expect("statistics dump", substrs=['expr evaluation successes : 1\n',
-                                                'expr evaluation failures : 0\n'])
+        self.expect("statistics disable", substrs=['need to enable statistics before disabling'], error=True)
 
-        self.expect("statistics enable")
-        # Doesn't parse.
+    def verify_key_in_dict(self, key, d, description):
+        self.assertEqual(key in d, True,
+            'make sure key "%s" is in dictionary %s' % (key, description))
+
+    def verify_key_not_in_dict(self, key, d, description):
+        self.assertEqual(key in d, False,
+            'make sure key "%s" is in dictionary %s' % (key, description))
+
+    def verify_keys(self, dict, description, keys_exist, keys_missing=None):
+        """
+            Verify that all keys in "keys_exist" list are top level items in
+            "dict", and that all keys in "keys_missing" do not exist as top
+            level items in "dict".
+        """
+        if keys_exist:
+            for key in keys_exist:
+                self.verify_key_in_dict(key, dict, description)
+        if keys_missing:
+            for key in keys_missing:
+                self.verify_key_not_in_dict(key, dict, description)
+
+    def verify_success_fail_count(self, stats, key, num_successes, num_fails):
+        self.verify_key_in_dict(key, stats, 'stats["%s"]' % (key))
+        success_fail_dict = stats[key]
+        self.assertEqual(success_fail_dict['successes'], num_successes,
+                         'make sure success count')
+        self.assertEqual(success_fail_dict['failures'], num_fails,
+                         'make sure success count')
+
+    def get_stats(self, options=None, log_path=None):
+        """
+            Get the output of the "statistics dump" with optional extra options
+            and return the JSON as a python dictionary.
+        """
+        # If log_path is set, open the path and emit the output of the command
+        # for debugging purposes.
+        if log_path is not None:
+            f = open(log_path, 'w')
+        else:
+            f = None
+        return_obj = lldb.SBCommandReturnObject()
+        command = "statistics dump "
+        if options is not None:
+            command += options
+        if f:
+            f.write('(lldb) %s\n' % (command))
+        self.ci.HandleCommand(command, return_obj, False)
+        metrics_json = return_obj.GetOutput()
+        if f:
+            f.write(metrics_json)
+        return json.loads(metrics_json)
+
+    def test_expressions_frame_var_counts(self):
+        lldbutil.run_to_source_breakpoint(self, "// break here",
+                                          lldb.SBFileSpec("main.c"))
+
+        self.expect("expr patatino", substrs=['27'])
+        stats = self.get_stats()
+        self.verify_success_fail_count(stats, 'expressionEvaluation', 1, 0)
         self.expect("expr doesnt_exist", error=True,
                     substrs=["undeclared identifier 'doesnt_exist'"])
         # Doesn't successfully execute.
         self.expect("expr int *i = nullptr; *i", error=True)
         # Interpret an integer as an array with 3 elements is also a failure.
         self.expect("expr -Z 3 -- 1", error=True,
                     substrs=["expression cannot be used with --element-count"])
-        self.expect("statistics disable")
         # We should have gotten 3 new failures and the previous success.
-        self.expect("statistics dump", substrs=['expr evaluation successes : 1\n',
-                                                'expr evaluation failures : 3\n'])
-
-        # 'frame var' with disabled statistics shouldn't change stats.
-        self.expect("frame var", substrs=['27'])
+        stats = self.get_stats()
+        self.verify_success_fail_count(stats, 'expressionEvaluation', 1, 3)
 
         self.expect("statistics enable")
         # 'frame var' with enabled statistics will change stats.
         self.expect("frame var", substrs=['27'])
-        self.expect("statistics disable")
-        self.expect("statistics dump", substrs=['frame var successes : 1\n',
-                                                'frame var failures : 0\n'])
+        stats = self.get_stats()
+        self.verify_success_fail_count(stats, 'frameVariable', 1, 0)
+
+    def test_default_no_run(self):
+        """Test "statistics dump" without running the target.
+
+        When we don't run the target, we expect to not see any 'firstStopTime'
+        or 'launchOrAttachTime' top level keys that measure the launch or
+        attach of the target.
+
+        Output expected to be something like:
+
+        (lldb) statistics dump
+        {
+          "targetCreateTime": 0.26566899599999999,
+          "expressionEvaluation": {
+            "failures": 0,
+            "successes": 0
+          },
+          "frameVariable": {
+            "failures": 0,
+            "successes": 0
+          },
+        }
+        """
+        target = self.createTestTarget()
+        stats = self.get_stats()
+        keys_exist = [
+            'expressionEvaluation',
+            'frameVariable',
+            'targetCreateTime',
+        ]
+        keys_missing = [
+            'firstStopTime',
+            'launchOrAttachTime'
+        ]
+        self.verify_keys(stats, '"stats"', keys_exist, keys_missing)
+        self.assertGreater(stats['targetCreateTime'], 0.0)
+
+    def test_default_with_run(self):
+        """Test "statistics dump" when running the target to a breakpoint.
+
+        When we run the target, we expect to see 'launchOrAttachTime' and
+        'firstStopTime' top level keys.
+
+        Output expected to be something like:
+
+        (lldb) statistics dump
+        {
+          "firstStopTime": 0.34164492800000001,
+          "launchOrAttachTime": 0.31969605400000001,
+          "targetCreateTime": 0.0040863039999999998
+          "expressionEvaluation": {
+            "failures": 0,
+            "successes": 0
+          },
+          "frameVariable": {
+            "failures": 0,
+            "successes": 0
+          },
+        }
+
+        """
+        target = self.createTestTarget()
+        lldbutil.run_to_source_breakpoint(self, "// break here",
+                                          lldb.SBFileSpec("main.c"))
+        stats = self.get_stats()
+        keys_exist = [
+            'expressionEvaluation',
+            'firstStopTime',
+            'frameVariable',
+            'launchOrAttachTime',
+            'targetCreateTime',
+        ]
+        self.verify_keys(stats, '"stats"', keys_exist, None)
+        self.assertGreater(stats['firstStopTime'], 0.0)
+        self.assertGreater(stats['launchOrAttachTime'], 0.0)
+        self.assertGreater(stats['targetCreateTime'], 0.0)

lldb/test/API/functionalities/stats_api/TestStatisticsAPI.py

 # Test the SBAPI for GetStatistics()
 
 import json
 import lldb
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class TestStatsAPI(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
+    NO_DEBUG_INFO_TESTCASE = True
+
     def test_stats_api(self):
         self.build()
         exe = self.getBuildArtifact("a.out")
         target = self.dbg.CreateTarget(exe)
 
         # Test enabling/disabling stats
         self.assertFalse(target.GetCollectingStats())
         target.SetCollectingStats(True)
         self.assertTrue(target.GetCollectingStats())
         target.SetCollectingStats(False)
         self.assertFalse(target.GetCollectingStats())
 
         # Test the function to get the statistics in JSON'ish.
         stats = target.GetStatistics()
         stream = lldb.SBStream()
         res = stats.GetAsJSON(stream)
-        stats_json = sorted(json.loads(stream.GetData()))
-        self.assertEqual(len(stats_json), 4)
-        self.assertIn("Number of expr evaluation failures", stats_json)
-        self.assertIn("Number of expr evaluation successes", stats_json)
-        self.assertIn("Number of frame var failures", stats_json)
-        self.assertIn("Number of frame var successes", stats_json)
+        stats_json = json.loads(stream.GetData())
+        self.assertEqual('expressionEvaluation' in stats_json, True,
+                'Make sure the "expressionEvaluation" key in in target.GetStatistics()')
+        self.assertEqual('frameVariable' in stats_json, True,
+                'Make sure the "frameVariable" key in in target.GetStatistics()')
+        expressionEvaluation = stats_json['expressionEvaluation']
+        self.assertEqual('successes' in expressionEvaluation, True,
+                'Make sure the "successes" key in in "expressionEvaluation" dictionary"')
+        self.assertEqual('failures' in expressionEvaluation, True,
+                'Make sure the "failures" key in in "expressionEvaluation" dictionary"')
+        frameVariable = stats_json['frameVariable']
+        self.assertEqual('successes' in frameVariable, True,
+                'Make sure the "successes" key in in "frameVariable" dictionary"')
+        self.assertEqual('failures' in frameVariable, True,
+                'Make sure the "failures" key in in "frameVariable" dictionary"')