[trace] Scaffold "thread trace dump instructions"

[wallace]

Summary

Depends on D88841

As per the discussion in the RFC, we'll implement both

thread trace dump [instructions | functions]

This is the first step in implementing the "instructions" dumping command.

It includes:

• A minimal ProcessTrace plugin for representing processes from a trace file. I noticed that it was a required step to mimic how core-based processes are initialized, e.g. ProcessElfCore and ProcessMinidump. I haven't had the need to create ThreadTrace yet, though. So far HistoryThread seems good enough.
• The command handling itself in CommandObjectThread, which outputs a placeholder text instead of the actual instructions. I'll do that part in the next diff.
• Tests

Depends on D88841.

[clayborg]

See inline comments about "thread trace dump instructions" as I am not quite sure what --offset means.

After we load a trace file, what is the default "offset" value? Is there a kind of "trace data position" that can be moved around? Does the offset default to the start of all trace instructions for a thread? The end of the trace data?

lldb/include/lldb/Target/Target.h

1122–1137

This should probably go on the lldb_private::Thread now if we are only going to ever dump information for a thread at a time.

lldb/source/Commands/Options.td

1011

This option text might need to be a bit more descriptive. What if the user types:

(lldb) thread trace dump --count 22

What would that display? 22 instructions forward from the current point we are at? If we load a trace file, will the position be at the end by default or at the beginning? I would think we would selec the end. Would this go 22 instructions back? That doesn't make much sense if we use in conjunction with the --offset.

1014–1015

So what happens if the user just types:

(lldb) thread trace dump

If the trace position offset defaults to zero, and the current position is the end of the data by default, would this display nothing? Everything? Or would it display the stats for this thread? like:

(lldb) thread trace dump
thread #1: tid = 1234, 4096 trace branches available, position = end
thread #2: tid = 2345, 2345 trace branches available, position = end

lldb/source/Plugins/Process/Trace/ProcessTrace.cpp

41–44

You must return "plugin_specified_by_name" here so that this plug-in only gets selected if it was specified by name. We don't want to have this ProcessTrace plug-in get selected by any real debug sessions. So this should be:

bool ProcessTrace::CanDebug(lldb::TargetSP target_sp,
                            bool plugin_specified_by_name) {
  return plugin_specified_by_name;
}

That way we can request this process plug-in by name.

If this is set to true, then when a target is shopping for a process plug-in by iterating through all installed Process plug-ins, if this function gets called before the ProcessGDBRemote or ProcessWindows it will take precedence over these other plug-ins. Typically a process plug-in will look at the target triple and look for an architecture, os, or vendor that matches and also look at the target's platform ("windows" for ProcessWindows) to determine if it can be used for debugging. In our case we should only be able to get a ProcessTrace instance back if we ask for it by name, it won't care about the target's triple, platform or anything else as it could be anything.

63–66

Probably shouldn't use the LoadCore functionality, see inline comments.

lldb/source/Plugins/Process/Trace/ProcessTrace.h

40

Seems weird to use LoadCore, see other inline comments.

lldb/source/Target/Target.cpp

3004–3013

move to lldb_private::Thread?

lldb/source/Target/TraceSettingsParser.cpp

68–73 ↗

(On Diff #295947)

We might need a ThreadTrace instead of a HistoryThread, and it might be a good idea to do that now. Soon we will end up wanting to step around and continue and reverse continue then we will need to muck with the thread and set its stop reason to things like "hit breakpoint" and change the stack frames.

114–115 ↗

(On Diff #295947)

We will need to make sure that this only happens when doing "trace load", and not when we do "process trace start" as we will have a real process at that point. We should talk about architecture as I have some ideas to make things work smoothly, but that can wait until we start to try and do more forward/reverse run/step stuff.

128–132 ↗

(On Diff #295947)

Seems weird to use LoadCore here as this isn't a core file. Since we loaded the Process plug-in by name, you can cast the a Process * to be a ProcessTrace and call a custom method on it if we need to do custom setup. DidAttach might be better to override instead of LoadCore?

lldb/test/API/commands/trace/TestTraceDumpInstructions.py

41

What will this do by default? I would assume it would dump all instructions starting from the beginning and dump all the way to the end. Where is the default "offset" for these instructions right after a trace is loaded? Is it set to the end of the trace instructions? If so, I am not sure what I would expect "thread trace dump instructions" to do. Again I would assume it would dump them all? Can you elaborate on what you are thinking here?

45

use the long options as this makes it much clearer what is happening.

Also what does --offset mean? Is it an offset in bytes within the trace data? Is it the number of branches from the start of the all of the trace data? Number of branches from the end of the trace data? Will this need to be specialized for each trace plug-in?

49–50

Add a test for an invalid thread index id?

[wallace]

Address comments:

• Created a basic ThreadTrace class instead of using HistoryThread. I'll modify this class later when I add the decoding functionality and refactor the json file parsing logic.
• Renamed offset to start_position, and added a clearer documentation about it. We'll have a current_position in each thread, and we'll print instructions in reverse order chronologically starting at current_position, unless start_position is specified. This matches gdb's implementation and works well with repeat commands, e.g. the user types "thread trace dump instructions" repeatedly showing a continous list of instructions from most recent to oldest.
• Now using DidAttach instead of LoadCore for setting the newly created procesess state to stopped.
• Moved the dump implementation from Target to Thread.
• Fixed the CanDebug function.
• Added a test for "thread trace dump instructions <invalid_thread_index>" and now using the long options instead of the short ones in the tests.

Some notes:

• I think that TraceSettingsParser should be used only when constructing a trace from a json file. In the case of a live process, we can skip it and directly create the trace objects accordingly. So the json parser can assume that the process is defunct.
• I haven't thought of making "thread trace dump" do anything. I'm thinking of having

thread trace info (for general information, including trace size, for example)
thread trace dump instructions
thread trace dump functions

[labath]

Before going into the code details, could you give a high-level overview of the intended interactions between the Trace and ProcessTrace classes?

I am thinking about how to organize this so that we don't have too many criss-cross dependencies. My main question is whether it's expected that each "trace" plugin will also create its own ProcessTrace subclass.

lldb/source/Target/TraceSettingsParser.cpp

13 ↗

(On Diff #296106)

This is a layering violation. We shouldn't have non-plugin code depend on plugin code. (I know there are some existing usages like that, but let's not add new ones -- people have been working for a long time to remove these.)

[wallace]

Rebased on top of https://reviews.llvm.org/D88841

• Comments on the architecture of classes are in that diff
• I was able to get rid of cross-plug-in dependencies except for RegisterContextHistory. The class is very simple, so I could move it to lldb/Core. What do you think?

[clayborg]

lldb/source/Commands/CommandObjectThread.cpp

2234–2243

This command does seem hard to abstract over all flavors of trace. Would it be better to have the trace plug-ins vend a command objects for "thread trace dump"? The idea would be something like "hey trace plug-in, do you have any commands you support for the 'thread trace dump' multiword command?". It could say "yes, I have one called 'instructions' and here is the CommandObjectSP

2248

We should override the repeat command method here:

const char *GetRepeatCommand(Args &current_command_args, uint32_t index) override;

This way if you type:

(lldb) thread trace dump instruction --offset 0 --count 32

And then hit enter, the next command it should run is:

(lldb) thread trace dump instruction --offset 32 --count 32

That you can keep dumping more instructions by just hitting ENTER.

lldb/source/Commands/Options.td

1008

Do we add a "--summary" option to do this command so that when this option is specified would dump the number of instructions that any threads from the might have? Like:

(lldb) thread trace dump instructions --summary
thread #1 has 4096 instructions
thread #2 has 234 instructions

1017–1020

Is there a "current position" that is maintained? Or is the current position where TraceThread currently is stopped?

lldb/source/Plugins/Process/Trace/ProcessTrace.h

19

So one issue is how do we eventually deal with debugging a live process that enables tracing. In that case we already have a real process class: ProcessGDBRemote most likely. We should avoid putting anything custom that is required from a process in this ProcessTrace class for when we actually have a real process class already. If we need to add anything, we will need to have virtual functions on the lldb_private::Process class that can call through to the Trace plug-in via its virtual functions as well to implement any functionality we might need.

Is this class solely going to be used for "trace load"?

lldb/test/API/commands/trace/TestTraceDumpInstructions.py

47

need to test the repeat command stuff here if we add support for:

const char *GetRepeatCommand(Args &current_command_args, uint32_t index) override {

[labath]

In D88769#2312482, @wallace wrote:

• Comments on the architecture of classes are in that diff

Where would that be? I couldn't find anything that would answer my question from the previous comment?

The reason I want to know that is that if each TraceXXX "plugin" is supposed to have/use/need a ProcessTraceXXX plugin, then I'd rather organize them such that they are closer together. OTOH, if ProcessTrace is supposed to work with all kinds of traces, then the current design makes perfect sense (though I am having trouble imagining how would that work).

• I was able to get rid of cross-plug-in dependencies except for RegisterContextHistory. The class is very simple, so I could move it to lldb/Core. What do you think?

IIUC, the only place which uses it the PT trace plugin. That is ok. Some plugin-to-plugin dependencies (though not cycles, ideally) are fine, particularly for something called "utility". That said, I don't think that moving this class to the core libraries would be a particularly bad choice either...

[labath]

In D88769#2314164, @labath wrote:

In D88769#2312482, @wallace wrote:

• Comments on the architecture of classes are in that diff

Where would that be? I couldn't find anything that would answer my question from the previous comment?

The reason I want to know that is that if each TraceXXX "plugin" is supposed to have/use/need a ProcessTraceXXX plugin, then I'd rather organize them such that they are closer together. OTOH, if ProcessTrace is supposed to work with all kinds of traces, then the current design makes perfect sense (though I am having trouble imagining how would that work).

I've now read the description of D88841, which kind of answers that. The reason I am not sure of this design is that I fear this will require a fairly big API surface between ProcessTrace and Trace classes in order to implement the more complicated commands like reverse-step. It also adds a bunch of dependency restrictions:

• Trace class (being a core class) should not know anything about ProcessTrace (a plugin) -- this bit could be removed if we make ProcessTrace a non-plugin
• ProcessTrace (being a generic plugin serving multiple traces) should not know anything about any specific trace plugin

OTOH, if ProcessTraceXXX is an internal affair of the TraceXXX plugin, then the two classes can freely cooperate (within reason), and can use any interfaces they see fit. And the TraceYYY plugin can use a completely different interface to communicate with its process class.

However, I don't think your design is bad, if you can pull it off, so feel free to carry on like you were planning. I'd just encourage you to, as you're working on it, to think about whether things can be made simpler with an different organization of things.

[tatyana-krasnukha]

lldb/source/Plugins/Process/Trace/ProcessTrace.h

19

One option is to implement btrace request in the ProcessGDBRemote and make remote stubs support it.

I'm also interested in live tracing for a custom process plugin which obtains instruction history in its own way. So, it would be good if a real process/thread provides data to the tracing plug-in.

[wallace]

lldb/source/Plugins/Process/Trace/ProcessTrace.h

19

Thanks @tatyana-krasnukha! I think I'll end up doing what you said. And definitely, step 2 of this project is to trace a live process.

19

Summarizing an offline conversation we had, we'd implement in this class only what is required for tracing a defunct process (i.e. from a json file). Otherwise, any trace logic should be in the main Process class, so that all trace plugins could reuse that logic.

Also, during a live debugging session, we don't want to create a new Process object to hold the trace. We'll try to keep a single process to keep the architecture simple.

[wallace]

Addressed comments, which includes:

• Implemented the requested repeat command logic, including tests

• Now showing the total number of instructions as part of the dump command, which is something like
  
  thread #1: tid = 3842849, total instructions = 1000 instruction 0 instruction 1 instruction 2 ...

• Moved some logic to the parent diff D88841, leaving this diff simpler.

Some notes:

• In my current design, TraceProcess will be shared by all trace plug-ins when loading json trace sessions. For live processes, we'll simply use the actual Process instance used by the process, which means that most of the trace logic regarding processes will end up being in the base Process.
• In my next diff I'll include the trace decoding functionality, which will remove the placeholder text printed when invoking the dump command.
• As discussed offline with Greg, each thread will hold a pointer to the currently stopped instruction, and dumping instructions, printing backtraces, e.g. will use that as starting point.

[clayborg]

LGTM. Pavel please chime in if you have any issues.

[wallace]

Rebase and made some cosmetic fixes.

[labath]

I was waiting for the dependant patch to take shape. This looks ok to me -- just one small design question.

lldb/include/lldb/Target/Target.h

1120

const TraceSP & ?

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

61

auto trace_instance = std::make_shared<TraceIntelPT>(...)

63

SetTrace(trace_instance)

lldb/source/Target/Thread.cpp

1617–1627 ↗

(On Diff #297127)

Given the intended design of having one process (and thread) class serve multiple trace types, can this function do anything useful besides calling into the Trace object to perform trace-specific dumping ?

And if it cannot, couldn't we just skip it have have callers call the relevant Trace method directly?

[wallace]

lldb/source/Target/Thread.cpp

1617–1627 ↗

(On Diff #297127)

That's a very good point! There's really no need to have this method here

[wallace]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

61

I can't do this because the constructor is private (specifically to std::shared_ptr).
I want to have this method as the main way to create instances because I need to use shared_ptr of this instance to set up everything correctly, and I can't do it from the constructor itself.
I've checked ways to make std::make_shared<TraceIntelPT>(...), but it looks like too much code that outweighs the benefit of the fancy allocator.

[wallace]

• Moved the thread dumping logic to Trace.h.
• Did the small fixes that were requested.
• Left a comment regarding the shared_ptr instantiation.

[labath]

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

61

Ah, cool. Sorry for the noise.

lldb/include/lldb/Target/Target.h

public:
 
   bool SetSectionUnloaded(const lldb::SectionSP &section_sp);
 
   bool SetSectionUnloaded(const lldb::SectionSP &section_sp,
                           lldb::addr_t load_addr);
 
   void ClearAllLoadedSections();
 
+  /// Set the \a Trace object containing processor trace information of this
+  /// target.
+  ///
+  /// \param[in] trace_sp
+  ///   The trace object.
+  void SetTrace(const lldb::TraceSP &trace_sp);
+
+  /// Get the \a Trace object containing processor trace information of this
+  /// target.
+  ///
+  /// \return
+  ///   The trace object. It might be undefined.
+  const lldb::TraceSP &GetTrace();
+
   // Since expressions results can persist beyond the lifetime of a process,
   // and the const expression results are available after a process is gone, we
   // provide a way for expressions to be evaluated from the Target itself. If
   // an expression is going to be run, then it should have a frame filled in in
   // the execution context.
   lldb::ExpressionResults EvaluateExpression(
       llvm::StringRef expression, ExecutionContextScope *exe_scope,
       lldb::ValueObjectSP &result_valobj_sp,
       const EvaluateExpressionOptions &options = EvaluateExpressionOptions(),
       std::string *fixed_expression = nullptr, ValueObject *ctx_obj = nullptr);
 
   lldb::ExpressionVariableSP GetPersistentVariable(ConstString name);
 
   lldb::addr_t GetPersistentSymbol(ConstString name);
 
   /// This method will return the address of the starting function for
   /// this binary, e.g. main() or its equivalent.  This can be used as
   /// an address of a function that is not called once a binary has
   /// started running - e.g. as a return address for inferior function
   /// calls that are unambiguous completion of the function call, not
   /// called during the course of the inferior function code running.
   ///
   /// If no entry point can be found, an invalid address is returned.
   ///
protected:
 
   typedef std::map<lldb::user_id_t, StopHookSP> StopHookCollection;
   StopHookCollection m_stop_hooks;
   lldb::user_id_t m_stop_hook_next_id;
   bool m_valid;
   bool m_suppress_stop_hooks;
   bool m_is_dummy_target;
   unsigned m_next_persistent_variable_index = 0;
+  /// An optional \a lldb_private::Trace object containing processor trace
+  /// information of this target.
+  lldb::TraceSP m_trace_sp;
   /// Stores the frame recognizers of this target.
   lldb::StackFrameRecognizerManagerUP m_frame_recognizer_manager_up;
 
   static void ImageSearchPathsChanged(const PathMappingList &path_list,
                                       void *baton);
 
   // Utilities for `statistics` command.
 private:

lldb/include/lldb/Target/Thread.h

virtual void ClearBackingThread() {
     // another thread. This is currently used for the OperatingSystem plug-ins
     // where they might have a thread that is in memory, yet its registers are
     // available through the lldb_private::Thread subclass for the current
     // lldb_private::Process class. Since each time the process stops the
     // backing threads for memory threads can change, we need a way to clear
     // the backing thread for all memory threads each time we stop.
   }
 
+  /// Dump \a count instructions of the thread's \a Trace starting at the \a
+  /// start_position position in reverse order.
+  ///
+  /// The instructions are indexed in reverse order, which means that the \a
+  /// start_position 0 represents the last instruction of the trace
+  /// chronologically.
+  ///
+  /// \param[in] s
+  ///   The stream object where the instructions are printed.
+  ///
+  /// \param[in] count
+  ///     The number of instructions to print.
+  ///
+  /// \param[in] start_position
+  ///     The position of the first instruction to print.
+  void DumpTraceInstructions(Stream &s, size_t count,
+                             size_t start_position = 0) const;
+
   // If stop_format is true, this will be the form used when we print stop
   // info. If false, it will be the form we use for thread list and co.
   void DumpUsingSettingsFormat(Stream &strm, uint32_t frame_idx,
                                bool stop_format);
 
   bool GetDescription(Stream &s, lldb::DescriptionLevel level,
                       bool print_json_thread, bool print_json_stopinfo);
 

lldb/include/lldb/Target/Trace.h

 ///
 /// Trace information can be loaded into a target without a process to allow
 /// introspection of the trace information during post mortem analysis, such as
 /// when loading core files.
 ///
 /// Processor trace information can also be fetched through the process
 /// interfaces during a live debug session if your process supports gathering
 /// this information.
-class Trace : public PluginInterface {
+class Trace : public PluginInterface,
+              public std::enable_shared_from_this<Trace> {
 public:
   /// Dump the trace data that this plug-in has access to.
   ///
   /// This function will dump all of the trace data for all threads in a user
   /// readable format. Options for dumping can be added as this API is iterated
   /// on.
   ///
   /// \param[in] s
public:
   /// \param[in] plugin_name
   ///     Name of the trace plugin.
   static llvm::Expected<llvm::StringRef>
   FindPluginSchema(llvm::StringRef plugin_name);
 
   /// \return
   ///     The JSON schema of this Trace plug-in.
   virtual llvm::StringRef GetSchema() = 0;
+
+  /// Dump \a count instructions of the given thread's \a Trace starting at the
+  /// \a start_position position in reverse order.
+  ///
+  /// The instructions are indexed in reverse order, which means that the \a
+  /// start_position 0 represents the last instruction of the trace
+  /// chronologically.
+  ///
+  /// \param[in] thread
+  ///     The thread whose trace will be dumped.
+  ///
+  /// \param[in] s
+  ///     The stream object where the instructions are printed.
+  ///
+  /// \param[in] count
+  ///     The number of instructions to print.
+  ///
+  /// \param[in] start_position
+  ///     The position of the first instruction to print.
+  void DumpTraceInstructions(Thread &thread, Stream &s, size_t count,
+                             size_t start_position) const;
 };
 
 } // namespace lldb_private
 
 #endif // LLDB_TARGET_TRACE_H

lldb/source/Commands/CommandObjectThread.cpp

 //===-- CommandObjectThread.cpp -------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "CommandObjectThread.h"
 
+#include <sstream>
+
 #include "lldb/Core/ValueObject.h"
 #include "lldb/Host/OptionParser.h"
 #include "lldb/Interpreter/CommandInterpreter.h"
 #include "lldb/Interpreter/CommandReturnObject.h"
 #include "lldb/Interpreter/OptionArgParser.h"
 #include "lldb/Interpreter/OptionGroupPythonClassWithDict.h"
 #include "lldb/Interpreter/Options.h"
 #include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/Function.h"
 #include "lldb/Symbol/LineEntry.h"
 #include "lldb/Symbol/LineTable.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/RegisterContext.h"
 #include "lldb/Target/SystemRuntime.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/Thread.h"
 #include "lldb/Target/ThreadPlan.h"
 #include "lldb/Target/ThreadPlanStepInRange.h"
+#include "lldb/Target/Trace.h"
 #include "lldb/Utility/State.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 // CommandObjectIterateOverThreads
 
 class CommandObjectIterateOverThreads : public CommandObjectParsed {
CommandObjectMultiwordThreadPlan(CommandInterpreter &interpreter)
     LoadSubCommand(
         "prune",
         CommandObjectSP(new CommandObjectThreadPlanPrune(interpreter)));
   }
 
   ~CommandObjectMultiwordThreadPlan() override = default;
 };
 
+// Next are the subcommands of CommandObjectMultiwordTrace
+
+// CommandObjectTraceDumpInstructions
+#define LLDB_OPTIONS_thread_trace_dump_instructions
+#include "CommandOptions.inc"
+
+class CommandObjectTraceDumpInstructions
+    : public CommandObjectIterateOverThreads {
+public:
+  class CommandOptions : public Options {
+  public:
+    CommandOptions() : Options() { OptionParsingStarting(nullptr); }
+
+    ~CommandOptions() override = default;
+
+    Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_arg,
+                          ExecutionContext *execution_context) override {
+      Status error;
+      const int short_option = m_getopt_table[option_idx].val;
+
+      switch (short_option) {
+      case 'c': {
+        int32_t count;
+        if (option_arg.empty() || option_arg.getAsInteger(0, count) ||
+            count < 0)
+          error.SetErrorStringWithFormat(
+              "invalid integer value for option '%s'",
+              option_arg.str().c_str());
+        else
+          m_count = count;
+        break;
+      }
+      case 's': {
+        int32_t start_position;
+        if (option_arg.empty() || option_arg.getAsInteger(0, start_position) ||
+            start_position < 0)
+          error.SetErrorStringWithFormat(
+              "invalid integer value for option '%s'",
+              option_arg.str().c_str());
+        else
+          m_start_position = start_position;
+        break;
+      }
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+      return error;
+    }
+
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      m_count = kDefaultCount;
+      m_start_position = kDefaultStartPosition;
+    }
+
+    llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
+      return llvm::makeArrayRef(g_thread_trace_dump_instructions_options);
+    }
+
+    static const uint32_t kDefaultCount = 20;
+    static const uint32_t kDefaultStartPosition = 0;
+
+    // Instance variables to hold the values for command options.
+    uint32_t m_count;
+    uint32_t m_start_position;
+  };
+
+  CommandObjectTraceDumpInstructions(CommandInterpreter &interpreter)
+      : CommandObjectIterateOverThreads(
+            interpreter, "thread trace dump instructions",
+            "Dump the traced instructions for one or more threads.  If no "
+            "threads are specified, show the current thread.  Use the "
+            "thread-index \"all\" to see all threads.",
+            nullptr,
+            eCommandRequiresProcess | eCommandTryTargetAPILock |
+                eCommandProcessMustBeLaunched | eCommandProcessMustBePaused),
+        m_options(), m_create_repeat_command_just_invoked(false) {}
+
+  ~CommandObjectTraceDumpInstructions() override = default;
+
+  Options *GetOptions() override { return &m_options; }
+
+  const char *GetRepeatCommand(Args &current_command_args,
+                               uint32_t index) override {
+    current_command_args.GetCommandString(m_repeat_command);
+    m_create_repeat_command_just_invoked = true;
+    return m_repeat_command.c_str();
+  }
+
+protected:
+  bool DoExecute(Args &args, CommandReturnObject &result) override {
+    bool status = CommandObjectIterateOverThreads::DoExecute(args, result);
+    PrepareRepeatArguments();
+    return status;
+  }
+
+  void PrepareRepeatArguments() {
+    m_repeat_start_position = m_options.m_count + GetStartPosition();
+    m_create_repeat_command_just_invoked = false;
+  }
+
+  bool IsRepeatCommand() {
+    return !m_repeat_command.empty() && !m_create_repeat_command_just_invoked;
+  }
+
+  uint32_t GetStartPosition() {
+    return IsRepeatCommand() ? m_repeat_start_position
+                             : m_options.m_start_position;
+  }
+
+  bool HandleOneThread(lldb::tid_t tid, CommandReturnObject &result) override {
+    const TraceSP &trace_sp = m_exe_ctx.GetTargetSP()->GetTrace();
+    if (!trace_sp) {
+      result.SetError("error: this thread is not being traced");
+      return false;
+    }
+
+    ThreadSP thread_sp =
+        m_exe_ctx.GetProcessPtr()->GetThreadList().FindThreadByID(tid);
+
+    trace_sp->DumpTraceInstructions(*thread_sp, result.GetOutputStream(),
+                                    m_options.m_count, GetStartPosition());
+    return true;
+  }
+
+  CommandOptions m_options;
+
+  // Repeat command helpers
+  std::string m_repeat_command;
+  bool m_create_repeat_command_just_invoked;
+  uint32_t m_repeat_start_position;
+};
+
+// CommandObjectMultiwordTraceDump
+class CommandObjectMultiwordTraceDump : public CommandObjectMultiword {
+public:
+  CommandObjectMultiwordTraceDump(CommandInterpreter &interpreter)
+      : CommandObjectMultiword(
+            interpreter, "dump",
+            "Commands for displaying trace information of the threads "
+            "in the current process.",
+            "thread trace dump <subcommand> [<subcommand objects>]") {
+    LoadSubCommand(
+        "instructions",
+        CommandObjectSP(new CommandObjectTraceDumpInstructions(interpreter)));
+  }
+  ~CommandObjectMultiwordTraceDump() override = default;
+};
+
+// CommandObjectMultiwordTrace
+class CommandObjectMultiwordTrace : public CommandObjectMultiword {
+public:
+  CommandObjectMultiwordTrace(CommandInterpreter &interpreter)
+      : CommandObjectMultiword(
+            interpreter, "trace",
+            "Commands for operating on traces of the threads in the current "
+            "process.",
+            "thread trace <subcommand> [<subcommand objects>]") {
+    LoadSubCommand("dump", CommandObjectSP(new CommandObjectMultiwordTraceDump(
+                               interpreter)));
+  }
+
+  ~CommandObjectMultiwordTrace() override = default;
+};
+
 // CommandObjectMultiwordThread
 
 CommandObjectMultiwordThread::CommandObjectMultiwordThread(
     CommandInterpreter &interpreter)
     : CommandObjectMultiword(interpreter, "thread",
                              "Commands for operating on "
                              "one or more threads in "
                              "the current process.",
LoadSubCommand(
           "You can also specify a dictionary of key (-k) and value (-v) pairs "
           "that will be used to populate an SBStructuredData Dictionary, which "
           "will be passed to the constructor of the class implementing the "
           "scripted step.  See the Python Reference for more details.",
           nullptr, eStepTypeScripted, eStepScopeSource)));
 
   LoadSubCommand("plan", CommandObjectSP(new CommandObjectMultiwordThreadPlan(
                              interpreter)));
+  LoadSubCommand("trace",
+                 CommandObjectSP(new CommandObjectMultiwordTrace(interpreter)));
 }
 
 CommandObjectMultiwordThread::~CommandObjectMultiwordThread() = default;

lldb/source/Commands/Options.td

def thread_plan_list_internal : Option<"internal", "i">, Group<1>,
     Desc<"Display internal as well as user thread plans">;
   def thread_plan_list_thread_id : Option<"thread-id", "t">, Group<1>,
     Arg<"ThreadID">, Desc<"List the thread plans for this TID, can be "
     "specified more than once.">;
   def thread_plan_list_unreported : Option<"unreported", "u">, Group<1>,
     Desc<"Display thread plans for unreported threads">;
 }
 
+let Command = "thread trace dump instructions" in {
+  def thread_trace_dump_instructions_count : Option<"count", "c">, Group<1>,
+    Arg<"Count">,
+    Desc<"The number of instructions to display starting at the current "
+    "position in reverse order chronologically.">;
+  def thread_trace_dump_instructions_start_position:
+    Option<"start-position", "s">,
+    Group<1>,
+    Arg<"Index">,
+    Desc<"The position of the first instruction to print. Defaults to the "
+    "current position, i.e. where the thread is stopped. The instructions are "
+    "indexed in reverse order, which means that a start position of 0 refers "
+    "to the last instruction chronologically.">;
+}
+
 let Command = "type summary add" in {
   def type_summary_add_category : Option<"category", "w">, Arg<"Name">,
     Desc<"Add this to the given category instead of the default one.">;
   def type_summary_add_cascade : Option<"cascade", "C">, Arg<"Boolean">,
     Desc<"If true, cascade through typedef chains.">;
   def type_summary_add_no_value : Option<"no-value", "v">,
     Desc<"Don't show the value, just show the summary, for this type.">;
   def type_summary_add_skip_pointers : Option<"skip-pointers", "p">,

lldb/source/Plugins/Process/CMakeLists.txt

 elseif (CMAKE_SYSTEM_NAME MATCHES "Darwin")
   add_subdirectory(MacOSX-Kernel)
 endif()
 add_subdirectory(gdb-remote)
 add_subdirectory(Utility)
 add_subdirectory(elf-core)
 add_subdirectory(mach-core)
 add_subdirectory(minidump)
+add_subdirectory(Trace)

lldb/source/Plugins/Process/Trace/CMakeLists.txt

+add_lldb_library(lldbPluginProcessTrace PLUGIN
+  ProcessTrace.cpp
+
+  LINK_LIBS
+    lldbCore
+    lldbTarget
+    lldbUtility
+    lldbPluginProcessUtility
+  LINK_COMPONENTS
+    BinaryFormat
+    Object
+    Support
+  )

lldb/source/Plugins/Process/Trace/ProcessTrace.h

+//===-- ProcessTrace.h ------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_SOURCE_PLUGINS_PROCESS_TRACE_PROCESSTRACE_H
+#define LLDB_SOURCE_PLUGINS_PROCESS_TRACE_PROCESSTRACE_H
+
+#include "lldb/Target/Process.h"
+#include "lldb/Utility/ConstString.h"
+#include "lldb/Utility/Status.h"
+
+namespace lldb_private {
+namespace process_trace {
+
+class ProcessTrace : public Process {
+public:
+  static lldb::ProcessSP CreateInstance(lldb::TargetSP target_sp,
+                                        lldb::ListenerSP listener_sp,
+                                        const FileSpec *crash_file_path);
+
+  static void Initialize();
+
+  static void Terminate();
+
+  static ConstString GetPluginNameStatic();
+
+  static const char *GetPluginDescriptionStatic();
+
+  ProcessTrace(lldb::TargetSP target_sp, lldb::ListenerSP listener_sp);
+
+  ~ProcessTrace() override;
+
+  bool CanDebug(lldb::TargetSP target_sp,
+                bool plugin_specified_by_name) override;
+
+  void DidAttach(ArchSpec &process_arch) override;
+
+  DynamicLoader *GetDynamicLoader() override { return nullptr; }
+
+  SystemRuntime *GetSystemRuntime() override { return nullptr; }
+
+  ConstString GetPluginName() override;
+
+  uint32_t GetPluginVersion() override;
+
+  Status DoDestroy() override;
+
+  void RefreshStateAfterStop() override;
+
+  Status WillResume() override {
+    Status error;
+    error.SetErrorStringWithFormat(
+        "error: %s does not support resuming processes",
+        GetPluginName().GetCString());
+    return error;
+  }
+
+  bool IsAlive() override;
+
+  bool WarnBeforeDetach() const override { return false; }
+
+  size_t ReadMemory(lldb::addr_t addr, void *buf, size_t size,
+                    Status &error) override;
+
+  size_t DoReadMemory(lldb::addr_t addr, void *buf, size_t size,
+                      Status &error) override;
+
+  ArchSpec GetArchitecture();
+
+  bool GetProcessInfo(ProcessInstanceInfo &info) override;
+
+protected:
+  void Clear();
+
+  bool UpdateThreadList(ThreadList &old_thread_list,
+                        ThreadList &new_thread_list) override;
+};
+
+} // namespace process_trace
+} // namespace lldb_private
+
+#endif // LLDB_SOURCE_PLUGINS_PROCESS_TRACE_PROCESSTRACE_H

lldb/source/Plugins/Process/Trace/ProcessTrace.cpp

+//===-- ProcessTrace.cpp --------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "ProcessTrace.h"
+
+#include <memory>
+
+#include "lldb/Core/Module.h"
+#include "lldb/Core/PluginManager.h"
+#include "lldb/Target/Target.h"
+
+using namespace lldb;
+using namespace lldb_private;
+using namespace lldb_private::process_trace;
+
+LLDB_PLUGIN_DEFINE(ProcessTrace)
+
+ConstString ProcessTrace::GetPluginNameStatic() {
+  static ConstString g_name("trace");
+  return g_name;
+}
+
+const char *ProcessTrace::GetPluginDescriptionStatic() {
+  return "Trace process plug-in.";
+}
+
+void ProcessTrace::Terminate() {
+  PluginManager::UnregisterPlugin(ProcessTrace::CreateInstance);
+}
+
+ProcessSP ProcessTrace::CreateInstance(TargetSP target_sp,
+                                       ListenerSP listener_sp,
+                                       const FileSpec *crash_file) {
+  return std::make_shared<ProcessTrace>(target_sp, listener_sp);
+}
+
+bool ProcessTrace::CanDebug(TargetSP target_sp, bool plugin_specified_by_name) {
+  return plugin_specified_by_name;
+}
+
+ProcessTrace::ProcessTrace(TargetSP target_sp, ListenerSP listener_sp)
+    : Process(target_sp, listener_sp) {}
+
+ProcessTrace::~ProcessTrace() {
+  Clear();
+  // We need to call finalize on the process before destroying ourselves to
+  // make sure all of the broadcaster cleanup goes as planned. If we destruct
+  // this class, then Process::~Process() might have problems trying to fully
+  // destroy the broadcaster.
+  Finalize();
+}
+
+ConstString ProcessTrace::GetPluginName() { return GetPluginNameStatic(); }
+
+uint32_t ProcessTrace::GetPluginVersion() { return 1; }
+
+void ProcessTrace::DidAttach(ArchSpec &process_arch) {
+  ListenerSP listener_sp(
+      Listener::MakeListener("lldb.process_trace.did_attach_listener"));
+  HijackProcessEvents(listener_sp);
+
+  SetCanJIT(false);
+  StartPrivateStateThread();
+  SetPrivateState(eStateStopped);
+
+  EventSP event_sp;
+  WaitForProcessToStop(llvm::None, &event_sp, true, listener_sp);
+
+  RestoreProcessEvents();
+
+  Process::DidAttach(process_arch);
+}
+
+bool ProcessTrace::UpdateThreadList(ThreadList &old_thread_list,
+                                    ThreadList &new_thread_list) {
+  return false;
+}
+
+void ProcessTrace::RefreshStateAfterStop() {}
+
+Status ProcessTrace::DoDestroy() { return Status(); }
+
+bool ProcessTrace::IsAlive() { return true; }
+
+size_t ProcessTrace::ReadMemory(addr_t addr, void *buf, size_t size,
+                                Status &error) {
+  // Don't allow the caching that lldb_private::Process::ReadMemory does since
+  // we have it all cached in the trace files.
+  return DoReadMemory(addr, buf, size, error);
+}
+
+void ProcessTrace::Clear() { m_thread_list.Clear(); }
+
+void ProcessTrace::Initialize() {
+  static llvm::once_flag g_once_flag;
+
+  llvm::call_once(g_once_flag, []() {
+    PluginManager::RegisterPlugin(GetPluginNameStatic(),
+                                  GetPluginDescriptionStatic(), CreateInstance);
+  });
+}
+
+ArchSpec ProcessTrace::GetArchitecture() {
+  return GetTarget().GetArchitecture();
+}
+
+bool ProcessTrace::GetProcessInfo(ProcessInstanceInfo &info) {
+  info.Clear();
+  info.SetProcessID(GetID());
+  info.SetArchitecture(GetArchitecture());
+  ModuleSP module_sp = GetTarget().GetExecutableModule();
+  if (module_sp) {
+    const bool add_exe_file_as_first_arg = false;
+    info.SetExecutableFile(GetTarget().GetExecutableModule()->GetFileSpec(),
+                           add_exe_file_as_first_arg);
+  }
+  return true;
+}
+
+size_t ProcessTrace::DoReadMemory(addr_t addr, void *buf, size_t size,
+                                  Status &error) {
+  return 0;
+}

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.h

public:
   ///     JSON data parsing.
   ///
   /// \return
   ///     A trace instance or an error in case of failures.
   static llvm::Expected<lldb::TraceSP>
   CreateInstance(const llvm::json::Value &trace_session_file,
                  llvm::StringRef session_file_dir, Debugger &debugger);
 
+  /// Create an instance of this class.
+  ///
+  /// \param[in] pt_cpu
+  ///     The libipt.h cpu information needed for decoding correctling the
+  ///     traces.
+  ///
+  /// \param[in] targets
+  ///     The list of targets to associate with this trace instance
+  ///
+  /// \return
+  ///     An intel-pt trace instance.
+  static lldb::TraceSP
+  CreateInstance(const pt_cpu &pt_cpu,
+                 const std::vector<lldb::TargetSP> &targets);
+
   static ConstString GetPluginNameStatic();
 
   uint32_t GetPluginVersion() override;
   /// \}
 
   llvm::StringRef GetSchema() override;
 
+private:
   TraceIntelPT(const pt_cpu &pt_cpu, const std::vector<lldb::TargetSP> &targets)
       : m_pt_cpu(pt_cpu) {
     for (const lldb::TargetSP &target_sp : targets)
       m_targets.push_back(target_sp);
   }
 
-private:
   pt_cpu m_pt_cpu;
   std::vector<std::weak_ptr<Target>> m_targets;
 };
 
 } // namespace trace_intel_pt
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_TRACEINTELPT_H

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

 //===-- TraceIntelPT.cpp --------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "TraceIntelPT.h"
 
 #include "TraceIntelPTSessionFileParser.h"
 #include "lldb/Core/PluginManager.h"
+#include "lldb/Target/Target.h"
 
 using namespace lldb;
 using namespace lldb_private;
 using namespace lldb_private::trace_intel_pt;
 using namespace llvm;
 
 LLDB_PLUGIN_DEFINE(TraceIntelPT)
 

 //------------------------------------------------------------------
 // PluginInterface protocol
 //------------------------------------------------------------------
 
 ConstString TraceIntelPT::GetPluginName() { return GetPluginNameStatic(); }
 
 uint32_t TraceIntelPT::GetPluginVersion() { return 1; }
 
-void TraceIntelPT::Dump(lldb_private::Stream *s) const {}
+void TraceIntelPT::Dump(Stream *s) const {}
 
-Expected<lldb::TraceSP>
+Expected<TraceSP>
 TraceIntelPT::CreateInstance(const json::Value &trace_session_file,
                              StringRef session_file_dir, Debugger &debugger) {
   return TraceIntelPTSessionFileParser(debugger, trace_session_file,
                                        session_file_dir)
       .Parse();
 }
+
+TraceSP TraceIntelPT::CreateInstance(const pt_cpu &pt_cpu,
+                                     const std::vector<TargetSP> &targets) {
+  TraceSP trace_instance(new TraceIntelPT(pt_cpu, targets));
+  for (const TargetSP &target_sp : targets)
+    target_sp->SetTrace(trace_instance);
+
+  return trace_instance;
+}

lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.cpp

 
 void TraceIntelPTSessionFileParser::ParseThread(
     ProcessSP &process_sp, const TraceSessionFileParser::JSONThread &thread) {
   lldb::tid_t tid = static_cast<lldb::tid_t>(thread.tid);
 
   FileSpec trace_file(thread.trace_file);
   NormalizePath(trace_file);
 
-  ThreadSP thread_sp(new ThreadIntelPT(*process_sp, tid, trace_file));
+  ThreadSP thread_sp =
+      std::make_shared<ThreadIntelPT>(*process_sp, tid, trace_file);
   process_sp->GetThreadList().AddThread(thread_sp);
 }
 
 Error TraceIntelPTSessionFileParser::ParseProcess(
     const TraceSessionFileParser::JSONProcess &process) {
   TargetSP target_sp;
   Status error = m_debugger.GetTargetList().CreateTarget(
       m_debugger, /*user_exe_path*/ StringRef(), process.triple,
       eLoadDependentsNo,
       /*platform_options*/ nullptr, target_sp);
 
   if (!target_sp)
     return error.ToError();
 
   m_targets.push_back(target_sp);
   m_debugger.GetTargetList().SetSelectedTarget(target_sp.get());
 
   ProcessSP process_sp(target_sp->CreateProcess(
-      /*listener*/ nullptr, /*plugin_name*/ StringRef(),
+      /*listener*/ nullptr, "trace",
       /*crash_file*/ nullptr));
   process_sp->SetID(static_cast<lldb::pid_t>(process.pid));
 
   for (const TraceSessionFileParser::JSONThread &thread : process.threads)
     ParseThread(process_sp, thread);
 
   for (const TraceSessionFileParser::JSONModule &module : process.modules) {
     if (Error err = ParseModule(target_sp, module))
       return err;
   }
-  return Error::success();
+
+  if (!process.threads.empty())
+    process_sp->GetThreadList().SetSelectedThreadByIndexID(0);
+
+  // We invoke DidAttach to create a correct stopped state for the process and
+  // its threads.
+  ArchSpec process_arch;
+  process_sp->DidAttach(process_arch);
+
+  return llvm::Error::success();
 }
 
 void TraceIntelPTSessionFileParser::ParsePTCPU(const JSONPTCPU &pt_cpu) {
   m_pt_cpu = {pt_cpu.vendor.compare("intel") == 0 ? pcv_intel : pcv_unknown,
               static_cast<uint16_t>(pt_cpu.family),
               static_cast<uint8_t>(pt_cpu.model),
               static_cast<uint8_t>(pt_cpu.stepping)};
 }
Expected<TraceSP> TraceIntelPTSessionFileParser::Parse() {
   if (Error err = ParseImpl()) {
     // Delete all targets that were created
     for (auto target_sp : m_targets)
       m_debugger.GetTargetList().DeleteTarget(target_sp);
     m_targets.clear();
     return std::move(err);
   }
 
-  return std::make_shared<TraceIntelPT>(m_pt_cpu, m_targets);
+  return TraceIntelPT::CreateInstance(m_pt_cpu, m_targets);
 }
 
 namespace llvm {
 namespace json {
 
 bool fromJSON(const Value &value,
               TraceIntelPTSessionFileParser::JSONPTCPU &pt_cpu, Path path) {
   ObjectMapper o(value, path);

lldb/source/Target/Target.cpp

Status Target::Launch(ProcessLaunchInfo &launch_info, Stream *stream) {
   default:
     error.SetErrorStringWithFormat("initial process state wasn't stopped: %s",
                                    StateAsCString(state));
     break;
   }
   return error;
 }
 
+void Target::SetTrace(const TraceSP &trace_sp) { m_trace_sp = trace_sp; }
+
+const TraceSP &Target::GetTrace() { return m_trace_sp; }
+
 Status Target::Attach(ProcessAttachInfo &attach_info, Stream *stream) {
   auto state = eStateInvalid;
   auto process_sp = GetProcessSP();
   if (process_sp) {
     state = process_sp->GetState();
     if (process_sp->IsAlive() && state != eStateConnected) {
       if (state == eStateAttaching)
         return Status("process attach is in progress");
       return Status("a process is already being debugged");
     }
   }
 
   const ModuleSP old_exec_module_sp = GetExecutableModule();
 
   // If no process info was specified, then use the target executable name as
   // the process to attach to by default
   if (!attach_info.ProcessInfoSpecified()) {
     if (old_exec_module_sp)

lldb/source/Target/Trace.cpp

 //===-- Trace.cpp ---------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Target/Trace.h"
 
 #include <sstream>
 
 #include "llvm/Support/Format.h"
 
 #include "lldb/Core/PluginManager.h"
+#include "lldb/Target/Thread.h"
+#include "lldb/Utility/Stream.h"
 
 using namespace lldb;
 using namespace lldb_private;
 using namespace llvm;
 
 // Helper structs used to extract the type of a trace session json without
 // having to parse the entire object.
 

 Expected<StringRef> Trace::FindPluginSchema(StringRef name) {
   ConstString plugin_name(name);
   StringRef schema = PluginManager::GetTraceSchema(plugin_name);
   if (!schema.empty())
     return schema;
 
   return createInvalidPlugInError(name);
 }
+
+void Trace::DumpTraceInstructions(Thread &thread, Stream &s, size_t count,
+                                  size_t start_position) const {
+  s.Printf("thread #%u: tid = %" PRIu64 ", total instructions = 1000\n",
+           thread.GetIndexID(), thread.GetID());
+  s.Printf("  would print %zu instructions from position %zu\n", count,
+           start_position);
+}

lldb/test/API/commands/trace/TestTraceDumpInstructions.py

+import lldb
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+from lldbsuite.test.decorators import *
+
+class TestTraceDumpInstructions(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+    NO_DEBUG_INFO_TESTCASE = True
+
+    def setUp(self):
+        TestBase.setUp(self)
+        if 'intel-pt' not in configuration.enabled_plugins:
+            self.skipTest("The intel-pt test plugin is not enabled")
+
+    def testErrorMessages(self):
+        # We first check the output when there are no targets
+        self.expect("thread trace dump instructions",
+            substrs=["error: invalid target, create a target using the 'target create' command"],
+            error=True)
+
+        # We now check the output when there's a non-running target
+        self.expect("target create " + os.path.join(self.getSourceDir(), "intelpt-trace", "a.out"))
+
+        self.expect("thread trace dump instructions",
+            substrs=["error: invalid process"],
+            error=True)
+
+        # Now we check the output when there's a running target without a trace
+        self.expect("b main")
+        self.expect("run")
+
+        self.expect("thread trace dump instructions",
+            substrs=["error: this thread is not being traced"],
+            error=True)
+
+    def testDumpInstructions(self):
+        self.expect("trace load -v " + os.path.join(self.getSourceDir(), "intelpt-trace", "trace.json"),
+            substrs=["intel-pt"])
+
+        self.expect("thread trace dump instructions",
+            substrs=['thread #1: tid = 3842849, total instructions = 1000',
+                     'would print 20 instructions from position 0'])
+
+        # We check if we can pass count and offset
+        self.expect("thread trace dump instructions --count 5 --start-position 10",
+            substrs=['thread #1: tid = 3842849, total instructions = 1000',
+                     'would print 5 instructions from position 10'])
+
+        # We check if we can access the thread by index id
+        self.expect("thread trace dump instructions 1",
+            substrs=['thread #1: tid = 3842849, total instructions = 1000',
+                     'would print 20 instructions from position 0'])
+
+        # We check that we get an error when using an invalid thread index id
+        self.expect("thread trace dump instructions 10", error=True,
+            substrs=['error: no thread with index: "10"'])
+
+    def testDumpInstructionsWithMultipleThreads(self):
+        # We load a trace with two threads
+        self.expect("trace load -v " + os.path.join(self.getSourceDir(), "intelpt-trace", "trace_2threads.json"))
+
+        # We print the instructions of two threads simultaneously
+        self.expect("thread trace dump instructions 1 2",
+            substrs=['''thread #1: tid = 3842849, total instructions = 1000
+  would print 20 instructions from position 0
+thread #2: tid = 3842850, total instructions = 1000
+  would print 20 instructions from position 0'''])
+
+        # We use custom --count and --start-position, saving the command to history for later
+        ci = self.dbg.GetCommandInterpreter()
+
+        result = lldb.SBCommandReturnObject()
+        ci.HandleCommand("thread trace dump instructions 1 2 --count 12 --start-position 5", result, True)
+        self.assertIn('''thread #1: tid = 3842849, total instructions = 1000
+  would print 12 instructions from position 5
+thread #2: tid = 3842850, total instructions = 1000
+  would print 12 instructions from position 5''', result.GetOutput())
+
+        # We use a repeat command and ensure the previous count is used and the start-position has moved to the next position
+        result = lldb.SBCommandReturnObject()
+        ci.HandleCommand("", result)
+        self.assertIn('''thread #1: tid = 3842849, total instructions = 1000
+  would print 12 instructions from position 17
+thread #2: tid = 3842850, total instructions = 1000
+  would print 12 instructions from position 17''', result.GetOutput())
+
+        ci.HandleCommand("", result)
+        self.assertIn('''thread #1: tid = 3842849, total instructions = 1000
+  would print 12 instructions from position 29
+thread #2: tid = 3842850, total instructions = 1000
+  would print 12 instructions from position 29''', result.GetOutput())

lldb/test/API/commands/trace/TestTraceLoad.py

def testLoadTrace(self):
         module = target.GetModuleAtIndex(0)
         path = module.GetFileSpec()
         self.assertEqual(path.fullpath, os.path.join(src_dir, "intelpt-trace", "a.out"))
         self.assertGreater(module.GetNumSections(), 0)
         self.assertEqual(module.GetSectionAtIndex(0).GetFileAddress(), 0x400000)
 
         self.assertEqual("6AA9A4E2-6F28-2F33-377D-59FECE874C71-5B41261A", module.GetUUIDString())
 
+        # check that the Process and Thread objects were created correctly
+        self.expect("thread info", substrs=["tid = 3842849"])
+        self.expect("thread list", substrs=["Process 1234 stopped", "tid = 3842849"])
+
 
     def testLoadInvalidTraces(self):
         src_dir = self.getSourceDir()
         # We test first an invalid type
         self.expect("trace load -v " + os.path.join(src_dir, "intelpt-trace", "trace_bad.json"), error=True,
           substrs=['''error: expected object at traceSession.processes[0]
 
 Context:

lldb/test/API/commands/trace/intelpt-trace/trace_2threads.json

+{
+  "trace": {
+    "type": "intel-pt",
+    "pt_cpu": {
+      "vendor": "intel",
+      "family": 6,
+      "model": 79,
+      "stepping": 1
+    }
+  },
+  "processes": [
+    {
+      "pid": 1234,
+      "triple": "x86_64-*-linux",
+      "threads": [
+        {
+          "tid": 3842849,
+          "traceFile": "3842849.trace"
+        },
+        {
+          "tid": 3842850,
+          "traceFile": "3842849.trace"
+        }
+      ],
+      "modules": [
+        {
+          "file": "a.out",
+          "systemPath": "a.out",
+          "loadAddress": "0x0000000000400000",
+          "uuid": "6AA9A4E2-6F28-2F33-377D-59FECE874C71-5B41261A"
+        }
+      ]
+    }
+  ]
+}