[trace] [intel pt] Create a "thread trace dump stats" command

[hanbingwang]

Summary

When the user types that command 'thread trace dump info' and there's a running Trace session in LLDB, a raw trace in bytes should be printed; the command 'thread trace dump info all' should print the info for all the threads.

[wallace]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

118

the presentation of this line could be better. Something like this would look nicer

thread 1: tid = 123123
  
  - Tracing technology: Intel PT
  - Raw trace size: 1231232 bytes

[clayborg]

lldb/include/lldb/Target/Trace.h

155

Are any statistics being dumped here? Maybe DumpTraceSummary(...) or DumpTraceInfo(...) would be better?

lldb/source/Commands/CommandObjectThread.cpp

2230–2232

Since we are iterating on this new command, I am wondering if we should have just a "thread trace dump" command with options?

(lldb) thread trace dump --stats
(lldb) thread trace dump --instructions

This way the user could dump more than one thing at a time with a single command?:

(lldb) thread trace dump --stats --instructions

Just thinking out loud here, so no worries if you feel this is already correct and should stay this way

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

118

The "Tracing technology: Intel PT" should probably come before any of the thread infos if it is added:

Tracing technology: Intel PT
thread 1: tid = 111, size = 0x1000
thread 2: tid = 222, size = 0x1000

[wallace]

lldb/source/Commands/CommandObjectThread.cpp

2230–2232

I think that's a bit too much for this patch, but I'll keep it in mind if we end up having more dumpers.

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

118

That's a pretty good idea.

@hanbingwang , you can invoke trace_sp->GetPluginName() for getting the name of the tracing technology being used

[hanbingwang]

1. rename "thread trace dump stats" to "thread trace dump info"
2. Additionally, print out the tracing plugin name.

[hanbingwang]

lldb/include/lldb/Target/Trace.h

155

@clayborg Sorry I didn't turn on the notification! Just saw your comments yesterday.
I agree "DumpTraceInfo()" is a better name.

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

118

Hi @wallace @clayborg,

I'm wondering how to let the string "Tracing technology: Intel PT" printed out exactly once, when there are more than one threads? It looks like that HandleOneThread() will be called multiple times, which will then call DumpTraceInfo() which does the printout.

It seems that it'll be easy to print the string if we know if a thread is the *first* to be handled. However the threads are not indexed though?

[wallace]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

118

You need to override CommandObjectIterateOverThreads::DoExecute in your command object. Something like this (figure out the correct function names)

bool CommandObjectTraceDumpInfo::DoExecute(Args &command,
                                              CommandReturnObject &result) {
   Target &target = m_exe_ctx.GetTargetRef();
   result.GetOutput().Printf("trace technology: %s", target.GetTrace().GetPluginName().Data());
  
   return CommandObjectIterateOverThreads::DoExecute(command, result);
}

that way that piece of code is executed before iterating over the threads

[hanbingwang]

print out the tracing plugin name exactly once when there are one or more than one threads.

[wallace]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

110–129

add an empty line before this to clearly separate the output of each thread

110–129

Add 2 empty spaces before the text to have some sort of indentation

[hanbingwang]

add two spaces before the string "Raw trace size : xxx"; add new line between each thread.

[hanbingwang]

This comment has been deleted.

[hanbingwang]

• create a "thread trace dump info" command.
• the string "Trace technology: xxx" should be only printed out once when there are more than one thread.
• add two spaces before the string "Raw trace size : xxx"; add new line between each thread.

[JDevlieghere]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

125–126

No else after return. Seems like a good candidate for a ternary operator.

[wallace]

Committed on behalf of @hanbingwang. Commit hash 345ace026b6e5cdbc38d207291e4b399d72e62ee

lldb/include/lldb/Target/Trace.h

public:
   /// Get a \a TraceCursor for the given thread's trace.
   ///
   /// \return
   ///     A \a TraceCursorUP. If the thread is not traced or its trace
   ///     information failed to load, the corresponding error is embedded in the
   ///     trace.
   virtual lldb::TraceCursorUP GetCursor(Thread &thread) = 0;
 
+  /// Dump general info about a given thread's trace. Each Trace plug-in
+  /// decides which data to show.
+  ///
+  /// \param[in] thread
+  ///     The thread that owns the trace in question.
+  ///
+  /// \param[in] s
+  ///     The stream object where the info will be printed printed.
+  ///
+  /// \param[in] verbose
+  ///     If \b true, print detailed info
+  ///     If \b false, print compact info
+  virtual void DumpTraceInfo(Thread &thread, Stream &s, bool verbose) = 0;
+
   /// Check if a thread is currently traced by this object.
   ///
   /// \param[in] thread
   ///     The thread in question.
   ///
   /// \return
   ///     \b true if the thread is traced by this instance, \b false otherwise.
   virtual bool IsTraced(const Thread &thread) = 0;

lldb/source/Commands/CommandObjectThread.cpp

protected:
   CommandOptions m_options;
 
   // Repeat command helpers
   std::string m_repeat_command;
   bool m_create_repeat_command_just_invoked;
   std::map<lldb::tid_t, std::unique_ptr<TraceInstructionDumper>> m_dumpers;
 };
 
+// CommandObjectTraceDumpInfo
+#define LLDB_OPTIONS_thread_trace_dump_info
+#include "CommandOptions.inc"
+
+class CommandObjectTraceDumpInfo : public CommandObjectIterateOverThreads {
+public:
+  class CommandOptions : public Options {
+  public:
+    CommandOptions() : Options() { OptionParsingStarting(nullptr); }
+
+    ~CommandOptions() override = default;
+
+    Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_arg,
+                          ExecutionContext *execution_context) override {
+      Status error;
+      const int short_option = m_getopt_table[option_idx].val;
+
+      switch (short_option) {
+      case 'v': {
+        m_verbose = true;
+        break;
+      }
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+      return error;
+    }
+
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      m_verbose = false;
+    }
+
+    llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
+      return llvm::makeArrayRef(g_thread_trace_dump_info_options);
+    }
+
+    // Instance variables to hold the values for command options.
+    bool m_verbose;
+  };
+
+  bool DoExecute(Args &command, CommandReturnObject &result) override {
+    Target &target = m_exe_ctx.GetTargetRef();
+    result.GetOutputStream().Printf(
+        "Trace technology: %s\n",
+        target.GetTrace()->GetPluginName().AsCString());
+    return CommandObjectIterateOverThreads::DoExecute(command, result);
+  }
+
+  CommandObjectTraceDumpInfo(CommandInterpreter &interpreter)
+      : CommandObjectIterateOverThreads(
+            interpreter, "thread trace dump info",
+            "Dump the traced information for one or more threads.  If no "
+            "threads are specified, show the current thread.  Use the "
+            "thread-index \"all\" to see all threads.",
+            nullptr,
+            eCommandRequiresProcess | eCommandTryTargetAPILock |
+                eCommandProcessMustBeLaunched | eCommandProcessMustBePaused |
+                eCommandProcessMustBeTraced),
+        m_options() {}
+
+  ~CommandObjectTraceDumpInfo() override = default;
+
+  Options *GetOptions() override { return &m_options; }
+
+protected:
+  bool HandleOneThread(lldb::tid_t tid, CommandReturnObject &result) override {
+    const TraceSP &trace_sp = m_exe_ctx.GetTargetSP()->GetTrace();
+    ThreadSP thread_sp =
+        m_exe_ctx.GetProcessPtr()->GetThreadList().FindThreadByID(tid);
+    trace_sp->DumpTraceInfo(*thread_sp, result.GetOutputStream(),
+                            m_options.m_verbose);
+    return true;
+  }
+
+  CommandOptions m_options;
+};
+
 // CommandObjectMultiwordTraceDump
 class CommandObjectMultiwordTraceDump : public CommandObjectMultiword {
 public:
   CommandObjectMultiwordTraceDump(CommandInterpreter &interpreter)
       : CommandObjectMultiword(
             interpreter, "dump",
             "Commands for displaying trace information of the threads "
             "in the current process.",
             "thread trace dump <subcommand> [<subcommand objects>]") {
     LoadSubCommand(
         "instructions",
         CommandObjectSP(new CommandObjectTraceDumpInstructions(interpreter)));
+    LoadSubCommand(
+        "info", CommandObjectSP(new CommandObjectTraceDumpInfo(interpreter)));
   }
   ~CommandObjectMultiwordTraceDump() override = default;
 };
 
 // CommandObjectMultiwordTrace
 class CommandObjectMultiwordTrace : public CommandObjectMultiword {
 public:
   CommandObjectMultiwordTrace(CommandInterpreter &interpreter)
       : CommandObjectMultiword(

lldb/source/Commands/Options.td

let Command = "thread trace dump instructions" in {
   def thread_trace_dump_instructions_raw : Option<"raw", "r">,
     Group<1>,
     Desc<"Dump only instruction address without disassembly nor symbol information.">;
   def thread_trace_dump_instructions_show_tsc : Option<"tsc", "t">,
     Group<1>,
     Desc<"For each instruction, print the corresponding timestamp counter if available.">;
 }
 
+let Command = "thread trace dump info" in {
+  def thread_trace_dump_info_verbose : Option<"verbose", "v">, Group<1>,
+    Desc<"show verbose thread trace dump info">;
+}
+
 let Command = "type summary add" in {
   def type_summary_add_category : Option<"category", "w">, Arg<"Name">,
     Desc<"Add this to the given category instead of the default one.">;
   def type_summary_add_cascade : Option<"cascade", "C">, Arg<"Boolean">,
     Desc<"If true, cascade through typedef chains.">;
   def type_summary_add_no_value : Option<"no-value", "v">,
     Desc<"Don't show the value, just show the summary, for this type.">;
   def type_summary_add_skip_pointers : Option<"skip-pointers", "p">,

lldb/source/Plugins/Trace/intel-pt/DecodedThread.h

 /// decoding a trace, as well as a position cursor used when reverse debugging
 /// the trace.
 ///
 /// Each decoded thread contains a cursor to the current position the user is
 /// stopped at. See \a Trace::GetCursorPosition for more information.
 class DecodedThread : public std::enable_shared_from_this<DecodedThread> {
 public:
   DecodedThread(lldb::ThreadSP thread_sp,
-                std::vector<IntelPTInstruction> &&instructions);
+                std::vector<IntelPTInstruction> &&instructions,
+                size_t raw_trace_size);
 
   /// Constructor with a single error signaling a complete failure of the
   /// decoding process.
   DecodedThread(lldb::ThreadSP thread_sp, llvm::Error error);
 
   /// Get the instructions from the decoded trace. Some of them might indicate
   /// errors (i.e. gaps) in the trace.
   ///
   /// \return
   ///   The instructions of the trace.
   llvm::ArrayRef<IntelPTInstruction> GetInstructions() const;
 
   /// Get a new cursor for the decoded thread.
   lldb::TraceCursorUP GetCursor();
 
+  /// Get the size in bytes of the corresponding Intel PT raw trace
+  ///
+  /// \return
+  ///   The size of the trace.
+  size_t GetRawTraceSize() const;
+
 private:
   lldb::ThreadSP m_thread_sp;
   std::vector<IntelPTInstruction> m_instructions;
+  size_t m_raw_trace_size;
 };
 
 using DecodedThreadSP = std::shared_ptr<DecodedThread>;
 
 } // namespace trace_intel_pt
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_DECODEDTHREAD_H

lldb/source/Plugins/Trace/intel-pt/DecodedThread.cpp

Error IntelPTInstruction::ToError() const {
   if (!IsError())
     return Error::success();
 
   if (m_error->isA<IntelPTError>())
     return make_error<IntelPTError>(static_cast<IntelPTError &>(*m_error));
   return make_error<StringError>(m_error->message(),
                                  m_error->convertToErrorCode());
 }
+size_t DecodedThread::GetRawTraceSize() const { return m_raw_trace_size; }
 
 TraceInstructionControlFlowType
 IntelPTInstruction::GetControlFlowType(lldb::addr_t next_load_address) const {
   if (IsError())
     return (TraceInstructionControlFlowType)0;
 
   TraceInstructionControlFlowType mask =
       eTraceInstructionControlFlowTypeInstruction;

 }
 
 DecodedThread::DecodedThread(ThreadSP thread_sp, Error error)
     : m_thread_sp(thread_sp) {
   m_instructions.emplace_back(std::move(error));
 }
 
 DecodedThread::DecodedThread(ThreadSP thread_sp,
-                             std::vector<IntelPTInstruction> &&instructions)
-    : m_thread_sp(thread_sp), m_instructions(std::move(instructions)) {
+                             std::vector<IntelPTInstruction> &&instructions,
+                             size_t raw_trace_size)
+    : m_thread_sp(thread_sp), m_instructions(std::move(instructions)),
+      m_raw_trace_size(raw_trace_size) {
   if (m_instructions.empty())
     m_instructions.emplace_back(
         createStringError(inconvertibleErrorCode(), "empty trace"));
 }
 
 lldb::TraceCursorUP DecodedThread::GetCursor() {
   return std::make_unique<TraceCursorIntelPT>(m_thread_sp, shared_from_this());
 }

lldb/source/Plugins/Trace/intel-pt/IntelPTDecoder.cpp

DecodeInMemoryTrace(Process &process, TraceIntelPT &trace_intel_pt,
   std::vector<IntelPTInstruction> instructions = DecodeInstructions(*decoder);
 
   pt_insn_free_decoder(decoder);
   return instructions;
 }
 
 static Expected<std::vector<IntelPTInstruction>>
 DecodeTraceFile(Process &process, TraceIntelPT &trace_intel_pt,
-                const FileSpec &trace_file) {
+                const FileSpec &trace_file, size_t &raw_trace_size) {
   ErrorOr<std::unique_ptr<MemoryBuffer>> trace_or_error =
       MemoryBuffer::getFile(trace_file.GetPath());
   if (std::error_code err = trace_or_error.getError())
     return errorCodeToError(err);
 
   MemoryBuffer &trace = **trace_or_error;
   MutableArrayRef<uint8_t> trace_data(
       // The libipt library does not modify the trace buffer, hence the
       // following cast is safe.
       reinterpret_cast<uint8_t *>(const_cast<char *>(trace.getBufferStart())),
       trace.getBufferSize());
+  raw_trace_size = trace_data.size();
   return DecodeInMemoryTrace(process, trace_intel_pt, trace_data);
 }
 
 static Expected<std::vector<IntelPTInstruction>>
-DecodeLiveThread(Thread &thread, TraceIntelPT &trace) {
+DecodeLiveThread(Thread &thread, TraceIntelPT &trace, size_t &raw_trace_size) {
   Expected<std::vector<uint8_t>> buffer =
       trace.GetLiveThreadBuffer(thread.GetID());
   if (!buffer)
     return buffer.takeError();
+  raw_trace_size = buffer->size();
   if (Expected<pt_cpu> cpu_info = trace.GetCPUInfo())
     return DecodeInMemoryTrace(*thread.GetProcess(), trace,
                                MutableArrayRef<uint8_t>(*buffer));
   else
     return cpu_info.takeError();
 }
 
 DecodedThreadSP ThreadDecoder::Decode() {
   if (!m_decoded_thread.hasValue())
     m_decoded_thread = DoDecode();
   return *m_decoded_thread;
 }
 
 PostMortemThreadDecoder::PostMortemThreadDecoder(
     const lldb::ThreadPostMortemTraceSP &trace_thread, TraceIntelPT &trace)
     : m_trace_thread(trace_thread), m_trace(trace) {}
 
 DecodedThreadSP PostMortemThreadDecoder::DoDecode() {
+  size_t raw_trace_size = 0;
   if (Expected<std::vector<IntelPTInstruction>> instructions =
           DecodeTraceFile(*m_trace_thread->GetProcess(), m_trace,
-                          m_trace_thread->GetTraceFile()))
+                          m_trace_thread->GetTraceFile(), raw_trace_size))
     return std::make_shared<DecodedThread>(m_trace_thread->shared_from_this(),
-                                           std::move(*instructions));
+                                           std::move(*instructions),
+                                           raw_trace_size);
   else
     return std::make_shared<DecodedThread>(m_trace_thread->shared_from_this(),
                                            instructions.takeError());
 }
 
 LiveThreadDecoder::LiveThreadDecoder(Thread &thread, TraceIntelPT &trace)
     : m_thread_sp(thread.shared_from_this()), m_trace(trace) {}
 
 DecodedThreadSP LiveThreadDecoder::DoDecode() {
+  size_t raw_trace_size = 0;
   if (Expected<std::vector<IntelPTInstruction>> instructions =
-          DecodeLiveThread(*m_thread_sp, m_trace))
-    return std::make_shared<DecodedThread>(m_thread_sp,
-                                           std::move(*instructions));
+          DecodeLiveThread(*m_thread_sp, m_trace, raw_trace_size))
+    return std::make_shared<DecodedThread>(
+        m_thread_sp, std::move(*instructions), raw_trace_size);
   else
     return std::make_shared<DecodedThread>(m_thread_sp,
                                            instructions.takeError());
 }

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.h

public:
 
   lldb::CommandObjectSP
   GetThreadTraceStartCommand(CommandInterpreter &interpreter) override;
 
   llvm::StringRef GetSchema() override;
 
   lldb::TraceCursorUP GetCursor(Thread &thread) override;
 
+  void DumpTraceInfo(Thread &thread, Stream &s, bool verbose) override;
+
+  llvm::Optional<size_t> GetRawTraceSize(Thread &thread);
+
   void DoRefreshLiveProcessState(
       llvm::Expected<TraceGetStateResponse> state) override;
 
   bool IsTraced(const Thread &thread) override;
 
   const char *GetStartConfigurationHelp() override;
 
   /// Start tracing a live process.

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

return std::make_shared<DecodedThread>(
         thread.shared_from_this(),
         createStringError(inconvertibleErrorCode(), "thread not traced"));
   return it->second->Decode();
 }
 
 lldb::TraceCursorUP TraceIntelPT::GetCursor(Thread &thread) {
   return Decode(thread)->GetCursor();
 }
 
+void TraceIntelPT::DumpTraceInfo(Thread &thread, Stream &s, bool verbose) {
+  Optional<size_t> raw_size = GetRawTraceSize(thread);
+  s.Printf("\nthread #%u: tid = %" PRIu64, thread.GetIndexID(), thread.GetID());
+  if (!raw_size) {
+    s.Printf(", not traced\n");
+    return;
+  }
+  s.Printf("\n  Raw trace size: %zu bytes\n", *raw_size);
+  return;
+}
+
+Optional<size_t> TraceIntelPT::GetRawTraceSize(Thread &thread) {
+  if (IsTraced(thread))
+    return Decode(thread)->GetRawTraceSize();
+  else
+    return None;
+}
+
 Expected<pt_cpu> TraceIntelPT::GetCPUInfoForLiveProcess() {
   Expected<std::vector<uint8_t>> cpu_info = GetLiveProcessBinaryData("cpuInfo");
   if (!cpu_info)
     return cpu_info.takeError();
 
   int64_t cpu_family = -1;
   int64_t model = -1;
   int64_t stepping = -1;
   std::string vendor_id;

lldb/test/API/commands/trace/TestTraceDumpInfo.py

+import lldb
+from intelpt_testcase import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+from lldbsuite.test.decorators import *
+
+class TestTraceDumpInfo(TraceIntelPTTestCaseBase):
+    mydir = TestBase.compute_mydir(__file__)
+
+    def testErrorMessages(self):
+        # We first check the output when there are no targets
+        self.expect("thread trace dump info",
+            substrs=["error: invalid target, create a target using the 'target create' command"],
+            error=True)
+
+        # We now check the output when there's a non-running target
+        self.expect("target create " +
+            os.path.join(self.getSourceDir(), "intelpt-trace", "a.out"))
+
+        self.expect("thread trace dump info",
+            substrs=["error: invalid process"],
+            error=True)
+
+        # Now we check the output when there's a running target without a trace
+        self.expect("b main")
+        self.expect("run")
+
+        self.expect("thread trace dump info",
+            substrs=["error: Process is not being traced"],
+            error=True)
+
+    def testDumpRawTraceSize(self):
+        self.expect("trace load -v " +
+        os.path.join(self.getSourceDir(), "intelpt-trace", "trace.json"),
+        substrs=["intel-pt"])
+
+        self.expect("thread trace dump info",
+            substrs=['''Trace technology: intel-pt
+
+thread #1: tid = 3842849
+  Raw trace size: 4096 bytes'''])

lldb/test/API/commands/trace/TestTraceLoad.py

def testLoadTrace(self):
         self.assertGreater(module.GetNumSections(), 0)
         self.assertEqual(module.GetSectionAtIndex(0).GetFileAddress(), 0x400000)
 
         self.assertEqual("6AA9A4E2-6F28-2F33-377D-59FECE874C71-5B41261A", module.GetUUIDString())
 
         # check that the Process and Thread objects were created correctly
         self.expect("thread info", substrs=["tid = 3842849"])
         self.expect("thread list", substrs=["Process 1234 stopped", "tid = 3842849"])
+        self.expect("thread trace dump info", substrs=['''Trace technology: intel-pt
 
+thread #1: tid = 3842849
+  Raw trace size: 4096 bytes'''])
 
     def testLoadInvalidTraces(self):
         src_dir = self.getSourceDir()
         # We test first an invalid type
         self.expect("trace load -v " + os.path.join(src_dir, "intelpt-trace", "trace_bad.json"), error=True,
           substrs=['''error: expected object at traceSession.processes[0]
 
 Context: