[trace][intelpt] Added total memory usage by decoded trace

[zrthxn]

Summary

This fails currently but the basics are there

[zrthxn]

Clean up, works fine now

[zrthxn]

Fixed trace info test for this

[wallace]

Pretty nice! just some details i've noticed when reading your diff and good to go

lldb/source/Plugins/Trace/intel-pt/DecodedThread.cpp

123–128

you might be able to do something like

// As calculating the size used by error instructions is very difficult, because errors
// are dynamic objects, we just discard them from our calculations because errors
// are supposed to be rare and not really contribute much to the total memory usage.
size_t non_error_instructions = count(m_instructions.begin(), m_instructions.end(), [](const IntelPTInstruction& insn) {!insn.IsError();});

return IntelPTInstruction::GetNonErrorMemoryUsage() * non_error_instructions + GetRawTraceSize();

notice that we need the actual GetRawTraceSize() and not sizeof(m_raw_trace_size), because the latter will always give you 8 bytes on an x86_64 machine, regardless of how much data it is actually consuming.

lldb/source/Plugins/Trace/intel-pt/DecodedThread.h

85–95

I made a mistake. I didn't notice that there's an m_error member, whose size is dynamic, because it might hold a string message.
However, the number of errors should be negligible compared to the total number of correct instructions, so, for global calculations, let's assume that all instructions are fine and they are not errors.

120

158–162

We can put all the documentation in the \return section to avoid redundancy.

Let's also use the word Calculate instead of Get because Get is supposed to be used for cheap operations and Calculate for maybe not so trivial ones.

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

113–114

print this in KB, as it will be readability. You can assume that this number will always be an integer because the raw trace size by definition in the kernel is a multiple of 4KB.

115

better print KB instead of bytes. you can use a double with two digits of precision

[wallace]

lldb/source/Plugins/Trace/intel-pt/DecodedThread.cpp

123–128

sorry, it should be

return IntelPTInstruction::GetNonErrorMemoryUsage() * non_error_instructions + GetRawTraceSize() + sizeof(DecodedThread);

notice the sizeof(DecodedThread) portion that accounts for the the m_thread_sp. The actual storage of the Thread * is somewhere else, and DecodedThread is just keeping a reference to it, so sizeof() is enough for this.

lldb/source/Plugins/Trace/intel-pt/DecodedThread.h

162–165

[zrthxn]

Changed to KiB and fixed how caclulation is done

[wallace]

just some few remaining nits

lldb/source/Plugins/Trace/intel-pt/DecodedThread.h

85

86–88

move the implementation of this this method to the corresponding cpp file. We try to keep all headers just as declarations to reduce the binary size, unless we try to achieve an optimization

119

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

108–114

only calculate the memory usage if used, which happens after the early terminated if

118

use double instead of float. double has more precision

[zrthxn]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

118

But since we're showing only two decimal places, which gets rounded up, the precision eventually gets thrown away... doesn't it?

[zrthxn]

Cosmetic and minor changes

[wallace]

lldb/source/Plugins/Trace/intel-pt/DecodedThread.cpp

123–130

as discussed offline, this should just be

[zrthxn]

Ignore errored instructions

[wallace]

great!

lldb/source/Plugins/Trace/intel-pt/DecodedThread.h

public:
   /// \return
   ///     Whether this object represents an error.
   bool IsError() const;
 
   /// \return
   ///     The instruction pointer address, or \a LLDB_INVALID_ADDRESS if it is
   ///     an error.
   lldb::addr_t GetLoadAddress() const;
   
+  /// Get the size in bytes of a non-error instance of this class
+  static size_t GetNonErrorMemoryUsage();
+
   /// \return
   ///     An \a llvm::Error object if this class corresponds to an Error, or an
   ///     \a llvm::Error::success otherwise.
   llvm::Error ToError() const;
 
   /// Get the timestamp associated with the current instruction. The timestamp
   /// is similar to what a rdtsc instruction would return.
   ///
   /// \return
   ///     The timestamp or \b llvm::None if not available.
   llvm::Optional<uint64_t> GetTimestampCounter() const;
 
   /// Get the \a lldb::TraceInstructionControlFlowType categories of the
   /// instruction.
   ///
   /// \param[in] next_load_address
   ///     The address of the next instruction in the trace or \b
   ///     LLDB_INVALID_ADDRESS if not available.
   ///
   /// \return
   ///     The control flow categories, or \b 0 if the instruction is an error.
   lldb::TraceInstructionControlFlowType
   GetControlFlowType(lldb::addr_t next_load_address) const;
 
   IntelPTInstruction(IntelPTInstruction &&other) = default;
 
 private:
   IntelPTInstruction(const IntelPTInstruction &other) = delete;
   const IntelPTInstruction &operator=(const IntelPTInstruction &other) = delete;
 
+  // When adding new members to this class, make sure to update 
+  // IntelPTInstruction::GetNonErrorMemoryUsage() if needed.
   pt_insn m_pt_insn;
   llvm::Optional<uint64_t> m_timestamp;
   std::unique_ptr<llvm::ErrorInfoBase> m_error;
 };
 
 /// \class DecodedThread
 /// Class holding the instructions and function call hierarchy obtained from
 /// decoding a trace, as well as a position cursor used when reverse debugging
 /// the trace.
public:
   lldb::TraceCursorUP GetCursor();
 
   /// Get the size in bytes of the corresponding Intel PT raw trace
   ///
   /// \return
   ///   The size of the trace.
   size_t GetRawTraceSize() const;
 
+  /// The approximate size in bytes used by this instance, 
+  /// including all the already decoded instructions.
+  size_t CalculateApproximateMemoryUsage() const;
+
 private:
+  /// When adding new members to this class, make sure 
+  /// to update \a CalculateApproximateMemoryUsage() accordingly.
   lldb::ThreadSP m_thread_sp;
   std::vector<IntelPTInstruction> m_instructions;
   size_t m_raw_trace_size;
 };
 
 using DecodedThreadSP = std::shared_ptr<DecodedThread>;
 
 } // namespace trace_intel_pt
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_DECODEDTHREAD_H

lldb/source/Plugins/Trace/intel-pt/DecodedThread.cpp

IntelPTInstruction::IntelPTInstruction(llvm::Error err) {
   m_pt_insn.ip = LLDB_INVALID_ADDRESS;
   m_pt_insn.iclass = ptic_error;
 }
 
 bool IntelPTInstruction::IsError() const { return (bool)m_error; }
 
 lldb::addr_t IntelPTInstruction::GetLoadAddress() const { return m_pt_insn.ip; }
 
+size_t IntelPTInstruction::GetNonErrorMemoryUsage() { return sizeof(IntelPTInstruction); }
+
 Optional<uint64_t> IntelPTInstruction::GetTimestampCounter() const {
   return m_timestamp;
 }
 
 Error IntelPTInstruction::ToError() const {
   if (!IsError())
     return Error::success();
 
DecodedThread::DecodedThread(ThreadSP thread_sp,
   if (m_instructions.empty())
     m_instructions.emplace_back(
         createStringError(inconvertibleErrorCode(), "empty trace"));
 }
 
 lldb::TraceCursorUP DecodedThread::GetCursor() {
   return std::make_unique<TraceCursorIntelPT>(m_thread_sp, shared_from_this());
 }
+
+size_t DecodedThread::CalculateApproximateMemoryUsage() const {
+  return m_raw_trace_size
+    + IntelPTInstruction::GetNonErrorMemoryUsage() * m_instructions.size()
+    + sizeof(DecodedThread);
+}

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

 }
 
 lldb::TraceCursorUP TraceIntelPT::GetCursor(Thread &thread) {
   return Decode(thread)->GetCursor();
 }
 
 void TraceIntelPT::DumpTraceInfo(Thread &thread, Stream &s, bool verbose) {
   Optional<size_t> raw_size = GetRawTraceSize(thread);
   s.Printf("\nthread #%u: tid = %" PRIu64, thread.GetIndexID(), thread.GetID());
   if (!raw_size) {
     s.Printf(", not traced\n");
     return;
   }
   s.Printf("\n");
-  s.Printf("  Raw trace size: %zu bytes\n", *raw_size);
-  s.Printf("  Total number of instructions: %zu\n", 
+
+  size_t mem_used = Decode(thread)->CalculateApproximateMemoryUsage();
+  s.Printf("  Raw trace size: %zu KiB\n", *raw_size / 1024);
+  s.Printf("  Total number of instructions: %zu\n",
            Decode(thread)->GetInstructions().size());
+  s.Printf("  Total approximate memory usage: %0.2lf KiB\n",
+           (double)mem_used / 1024);
   return;
 }
 
 Optional<size_t> TraceIntelPT::GetRawTraceSize(Thread &thread) {
   if (IsTraced(thread.GetID()))
     return Decode(thread)->GetRawTraceSize();
   else
     return None;

lldb/test/API/commands/trace/TestTraceDumpInfo.py

def testDumpRawTraceSize(self):
         self.expect("trace load -v " +
         os.path.join(self.getSourceDir(), "intelpt-trace", "trace.json"),
         substrs=["intel-pt"])
 
         self.expect("thread trace dump info",
             substrs=['''Trace technology: intel-pt
 
 thread #1: tid = 3842849
-  Raw trace size: 4096 bytes
-  Total number of instructions: 21'''])
+  Raw trace size: 4 KiB
+  Total number of instructions: 21
+  Total approximate memory usage: 5.38 KiB'''])