[trace] Add an option to save a compact trace bundle

[wallace]

Summary

A trace bundle contains many trace files, and, in the case of intel pt, the
largest files are often the context switch traces because they are not
compressed by default. As a way to improve this, I'm adding a --compact option
to the trace save command that filters out unwanted processes from the
context switch traces. Eventually we can do the same for intel pt traces as
well.

[wallace]

lldb/source/Commands/CommandObjectProcess.cpp

582

many formatting changes sneaked in. The actual changes are in the end of this file

lldb/source/Plugins/Trace/intel-pt/TraceIntelPTBundleSaver.cpp

273

this was a bug in the existing code that prevented copying a module that was loaded from a bundle

[wallace]

minor improvements for autocompletion in the CLI

[wallace]

make relative all paths being returned in the description file

[wallace]

improve command handling

[wallace]

fix tests

[wallace]

rename 'process trace save' to 'trace save' because it can actually dump the information of multiple processes.

[jj10306]

lgtm, two minor comments

lldb/source/Plugins/Trace/intel-pt/PerfContextSwitchDecoder.cpp

318

why do we want the error record?

323–325

consider using std::copy or .insert here instead?

[JDevlieghere]

lldb/source/Commands/CommandObjectProcess.cpp

582

Please separate the formatting changes into a separate patch/commit. That way they're much easier to identify in the git blame output instead of getting attributed to an unrelated change. What I usually do in this scenario is format the file, commit than, and then rebase my change on top.

[wallace]

I'll remove the unwanted formatting changes

[wallace]

lldb/source/Plugins/Trace/intel-pt/PerfContextSwitchDecoder.cpp

318

i don't want to lose this information because they can show contention issues

lldb/bindings/interface/SBTrace.i

 "Represents a processor trace."
 ) SBTrace;
 class LLDB_API SBTrace {
 public:
   SBTrace();
 
   const char *GetStartConfigurationHelp();
 
+  SBFileSpec SaveToDisk(SBError &error, const SBFileSpec &bundle_dir, bool compact = false);
+
   SBError Start(const SBStructuredData &configuration);
 
   SBError Start(const SBThread &thread, const SBStructuredData &configuration);
 
   SBError Stop();
 
   SBError Stop(const SBThread &thread);
 
   explicit operator bool() const;
 
   bool IsValid();
 };
 } // namespace lldb

lldb/include/lldb/API/SBTrace.h

public:
   SBTrace();
 
   SBTrace(const lldb::TraceSP &trace_sp);
 
   /// See SBDebugger::LoadTraceFromFile.
   static SBTrace LoadTraceFromFile(SBError &error, SBDebugger &debugger,
                                    const SBFileSpec &trace_description_file);
 
+  /// Save the trace to the specified directory, which will be created if
+  /// needed. This will also create a a file \a <directory>/trace.json with the
+  /// main properties of the trace session, along with others files which
+  /// contain the actual trace data. The trace.json file can be used later as
+  /// input for the "trace load" command to load the trace in LLDB, or for the
+  /// method \a SBDebugger.LoadTraceFromFile().
+  ///
+  /// \param[out] error
+  ///   This will be set with an error in case of failures.
+  ///
+  /// \param[in] directory
+  ///   The directory where the trace files will be saved.
+  ///
+  /// \param[in] compact
+  ///   Try not to save to disk information irrelevant to the traced processes.
+  ///   Each trace plug-in implements this in a different fashion.
+  ///
+  /// \return
+  ///   A \a SBFileSpec pointing to the bundle description file.
+  SBFileSpec SaveToDisk(SBError &error, const SBFileSpec &bundle_dir,
+                        bool compact = false);
+
   /// \return
   ///     A description of the parameters to use for the \a SBTrace::Start
   ///     method, or \b null if the object is invalid.
   const char *GetStartConfigurationHelp();
 
   /// Start tracing all current and future threads in a live process using a
   /// provided configuration. This is referred as "process tracing" in the
   /// documentation.

lldb/include/lldb/Target/Trace.h

public:
   /// This function will dump all of the trace data for all threads in a user
   /// readable format. Options for dumping can be added as this API is iterated
   /// on.
   ///
   /// \param[in] s
   ///     A stream object to dump the information to.
   virtual void Dump(Stream *s) const = 0;
 
-  /// Save the trace of a live process to the specified directory, which
-  /// will be created if needed.
-  /// This will also create a a file \a <directory>/trace.json with the main
-  /// properties of the trace session, along with others files which contain
-  /// the actual trace data. The trace.json file can be used later as input
-  /// for the "trace load" command to load the trace in LLDB.
-  /// The process being trace is not a live process, return an error.
+  /// Save the trace to the specified directory, which will be created if
+  /// needed. This will also create a a file \a <directory>/trace.json with the
+  /// main properties of the trace session, along with others files which
+  /// contain the actual trace data. The trace.json file can be used later as
+  /// input for the "trace load" command to load the trace in LLDB.
   ///
   /// \param[in] directory
   ///   The directory where the trace files will be saved.
   ///
+  /// \param[in] compact
+  ///   Try not to save to disk information irrelevant to the traced processes.
+  ///   Each trace plug-in implements this in a different fashion.
+  ///
   /// \return
-  ///   \a llvm::success if the operation was successful, or an \a llvm::Error
-  ///   otherwise.
-  virtual llvm::Error SaveLiveTraceToDisk(FileSpec directory) = 0;
+  ///   A \a FileSpec pointing to the bundle description file, or an \a
+  ///   llvm::Error otherwise.
+  virtual llvm::Expected<FileSpec> SaveToDisk(FileSpec directory,
+                                              bool compact) = 0;
 
   /// Find a trace plug-in using JSON data.
   ///
   /// When loading trace data from disk, the information for the trace data
   /// can be contained in multiple files and require plug-in specific
   /// information about the CPU. Using data like JSON provides an
   /// easy way to specify all of the settings and information that we will need
   /// to load trace data into LLDB. This structured data can include:

lldb/packages/Python/lldbsuite/test/tools/intelpt/intelpt_testcase.py

def traceStopThread(self, thread=None, error=False, substrs=None):
             if thread is not None:
                 command += " " + str(thread.GetIndexID())
             self.expect(command, error=error, substrs=substrs)
 
     def traceLoad(self, traceDescriptionFilePath="trace.json", error=False, substrs=None):
         if self.USE_SB_API:
             traceDescriptionFile = lldb.SBFileSpec(traceDescriptionFilePath, True)
             loadTraceError = lldb.SBError()
-            _trace = self.dbg.LoadTraceFromFile(loadTraceError, traceDescriptionFile)
+            self.dbg.LoadTraceFromFile(loadTraceError, traceDescriptionFile)
             self.assertSBError(loadTraceError, error)
         else:
             command = f"trace load -v {traceDescriptionFilePath}"
             self.expect(command, error=error, substrs=substrs)
+
+    def traceSave(self, traceBundleDir, compact=False, error=False, substrs=None):
+        if self.USE_SB_API:
+            save_error = lldb.SBError()
+            self.target().GetTrace().SaveToDisk(
+                save_error, lldb.SBFileSpec(traceBundleDir), compact)
+            self.assertSBError(save_error, error)
+        else:
+            command = f"process trace save -d {traceBundleDir}"
+            if compact:
+                command += " -c"
+            self.expect(command, error=error, substrs=substrs)

lldb/source/API/SBTrace.cpp

SBTrace SBTrace::LoadTraceFromFile(SBError &error, SBDebugger &debugger,
   if (!trace_or_err) {
     error.SetErrorString(toString(trace_or_err.takeError()).c_str());
     return SBTrace();
   }
 
   return SBTrace(trace_or_err.get());
 }
 
+SBFileSpec SBTrace::SaveToDisk(SBError &error, const SBFileSpec &bundle_dir,
+                               bool compact) {
+  LLDB_INSTRUMENT_VA(this, error, bundle_dir, compact);
+
+  error.Clear();
+  SBFileSpec file_spec;
+
+  if (!m_opaque_sp)
+    error.SetErrorString("error: invalid trace");
+  else if (Expected<FileSpec> desc_file =
+               m_opaque_sp->SaveToDisk(bundle_dir.ref(), compact))
+    file_spec.SetFileSpec(*desc_file);
+  else
+    error.SetErrorString(llvm::toString(desc_file.takeError()).c_str());
+
+  return file_spec;
+}
+
 const char *SBTrace::GetStartConfigurationHelp() {
   LLDB_INSTRUMENT_VA(this);
   return m_opaque_sp ? m_opaque_sp->GetStartConfigurationHelp() : nullptr;
 }
 
 SBError SBTrace::Start(const SBStructuredData &configuration) {
   LLDB_INSTRUMENT_VA(this, configuration);
   SBError error;

lldb/source/Commands/CommandObjectProcess.cpp

if (state == eStateStopped) {
                 if (!bp_ref.IsInternal()) {
                   bp_ref.SetIgnoreCount(m_options.m_ignore);
                 }
               }
             }
           }
         }
       }
-      
       Target *target = m_exe_ctx.GetTargetPtr();
       BreakpointIDList run_to_bkpt_ids;
       // Don't pass an empty run_to_breakpoint list, as Verify will look for the
       // default breakpoint.
       if (m_options.m_run_to_bkpt_args.GetArgumentCount() > 0)
         CommandObjectMultiwordBreakpoint::VerifyBreakpointOrLocationIDs(
-            m_options.m_run_to_bkpt_args, target, result, &run_to_bkpt_ids, 
+            m_options.m_run_to_bkpt_args, target, result, &run_to_bkpt_ids,
             BreakpointName::Permissions::disablePerm);
       if (!result.Succeeded()) {
         return false;
       }
       result.Clear();
       if (m_options.m_any_bkpts_specified && run_to_bkpt_ids.GetSize() == 0) {
         result.AppendError("continue-to breakpoints did not specify any actual "
                            "breakpoints or locations");
         return false;
       }
 
       // First figure out which breakpoints & locations were specified by the
       // user:
       size_t num_run_to_bkpt_ids = run_to_bkpt_ids.GetSize();
       std::vector<break_id_t> bkpts_disabled;
       std::vector<BreakpointID> locs_disabled;
       if (num_run_to_bkpt_ids != 0) {
-        // Go through the ID's specified, and separate the breakpoints from are 
+        // Go through the ID's specified, and separate the breakpoints from are
         // the breakpoint.location specifications since the latter require
         // special handling.  We also figure out whether there's at least one
         // specifier in the set that is enabled.
         BreakpointList &bkpt_list = target->GetBreakpointList();
         std::unordered_set<break_id_t> bkpts_seen;
         std::unordered_set<break_id_t> bkpts_with_locs_seen;
         BreakpointIDList with_locs;
         bool any_enabled = false;
-  
         for (size_t idx = 0; idx < num_run_to_bkpt_ids; idx++) {
           BreakpointID bkpt_id = run_to_bkpt_ids.GetBreakpointIDAtIndex(idx);
           break_id_t bp_id = bkpt_id.GetBreakpointID();
           break_id_t loc_id = bkpt_id.GetLocationID();
-          BreakpointSP bp_sp 
-              = bkpt_list.FindBreakpointByID(bp_id);
-          // Note, VerifyBreakpointOrLocationIDs checks for existence, so we 
+          BreakpointSP bp_sp = bkpt_list.FindBreakpointByID(bp_id);
+          // Note, VerifyBreakpointOrLocationIDs checks for existence, so we
           // don't need to do it again here.
           if (bp_sp->IsEnabled()) {
             if (loc_id == LLDB_INVALID_BREAK_ID) {
-              // A breakpoint (without location) was specified.  Make sure that 
+              // A breakpoint (without location) was specified.  Make sure that
               // at least one of the locations is enabled.
               size_t num_locations = bp_sp->GetNumLocations();
               for (size_t loc_idx = 0; loc_idx < num_locations; loc_idx++) {
-                BreakpointLocationSP loc_sp 
-                    = bp_sp->GetLocationAtIndex(loc_idx);
+                BreakpointLocationSP loc_sp =
+                    bp_sp->GetLocationAtIndex(loc_idx);
                 if (loc_sp->IsEnabled()) {
                   any_enabled = true;
                   break;
                 }
               }
             } else {
               // A location was specified, check if it was enabled:
               BreakpointLocationSP loc_sp = bp_sp->FindLocationByID(loc_id);
               if (loc_sp->IsEnabled())
                 any_enabled = true;
             }
-          
             // Then sort the bp & bp.loc entries for later use:
             if (bkpt_id.GetLocationID() == LLDB_INVALID_BREAK_ID)
               bkpts_seen.insert(bkpt_id.GetBreakpointID());
             else {
               bkpts_with_locs_seen.insert(bkpt_id.GetBreakpointID());
               with_locs.AddBreakpointID(bkpt_id);
             }
           }
         }
         // Do all the error checking here so once we start disabling we don't
         // have to back out half-way through.
-        
         // Make sure at least one of the specified breakpoints is enabled.
         if (!any_enabled) {
           result.AppendError("at least one of the continue-to breakpoints must "
                              "be enabled.");
           return false;
         }
-        
         // Also, if you specify BOTH a breakpoint and one of it's locations,
         // we flag that as an error, since it won't do what you expect, the
         // breakpoint directive will mean "run to all locations", which is not
         // what the location directive means...
         for (break_id_t bp_id : bkpts_with_locs_seen) {
           if (bkpts_seen.count(bp_id)) {
             result.AppendErrorWithFormatv("can't specify both a breakpoint and "
                                "one of its locations: {0}", bp_id);
           }
         }
-        
         // Now go through the breakpoints in the target, disabling all the ones
         // that the user didn't mention:
         for (BreakpointSP bp_sp : bkpt_list.Breakpoints()) {
           break_id_t bp_id = bp_sp->GetID();
           // Handle the case where no locations were specified.  Note we don't
           // have to worry about the case where a breakpoint and one of its
           // locations are both in the lists, we've already disallowed that.
           if (!bkpts_with_locs_seen.count(bp_id)) {
             if (!bkpts_seen.count(bp_id) && bp_sp->IsEnabled()) {
               bkpts_disabled.push_back(bp_id);
               bp_sp->SetEnabled(false);
             }
             continue;
           }
           // Next, handle the case where a location was specified:
           // Run through all the locations of this breakpoint and disable
           // the ones that aren't on our "with locations" BreakpointID list:
           size_t num_locations = bp_sp->GetNumLocations();
           BreakpointID tmp_id(bp_id, LLDB_INVALID_BREAK_ID);
           for (size_t loc_idx = 0; loc_idx < num_locations; loc_idx++) {
             BreakpointLocationSP loc_sp = bp_sp->GetLocationAtIndex(loc_idx);
             tmp_id.SetBreakpointLocationID(loc_idx);
             size_t position = 0;
-            if (!with_locs.FindBreakpointID(tmp_id, &position) 
-                && loc_sp->IsEnabled()) {
+            if (!with_locs.FindBreakpointID(tmp_id, &position) &&
+                loc_sp->IsEnabled()) {
               locs_disabled.push_back(tmp_id);
               loc_sp->SetEnabled(false);
             }
           }
         }
       }
 
       { // Scope for thread list mutex:
if (state == eStateStopped) {
       }
 
       const uint32_t iohandler_id = process->GetIOHandlerID();
 
       StreamString stream;
       Status error;
       // For now we can only do -b with synchronous:
       bool old_sync = GetDebugger().GetAsyncExecution();
-      
       if (run_to_bkpt_ids.GetSize() != 0) {
         GetDebugger().SetAsyncExecution(false);
         synchronous_execution = true;
-      } 
+      }
       if (synchronous_execution)
         error = process->ResumeSynchronous(&stream);
       else
         error = process->Resume();
-      
       if (run_to_bkpt_ids.GetSize() != 0) {
         GetDebugger().SetAsyncExecution(old_sync);
-      } 
-       
+      }
       // Now re-enable the breakpoints we disabled:
       BreakpointList &bkpt_list = target->GetBreakpointList();
       for (break_id_t bp_id : bkpts_disabled) {
         BreakpointSP bp_sp = bkpt_list.FindBreakpointByID(bp_id);
         if (bp_sp)
           bp_sp->SetEnabled(true);
       }
       for (const BreakpointID &bkpt_id : locs_disabled) {
-        BreakpointSP bp_sp 
-            = bkpt_list.FindBreakpointByID(bkpt_id.GetBreakpointID());
+        BreakpointSP bp_sp =
+            bkpt_list.FindBreakpointByID(bkpt_id.GetBreakpointID());
         if (bp_sp) {
-          BreakpointLocationSP loc_sp 
-              = bp_sp->FindLocationByID(bkpt_id.GetLocationID());
+          BreakpointLocationSP loc_sp =
+              bp_sp->FindLocationByID(bkpt_id.GetLocationID());
           if (loc_sp)
             loc_sp->SetEnabled(true);
         }
       }
 
       if (error.Success()) {
         // There is a race condition where this thread will return up the call
         // stack to the main command handler and show an (lldb) prompt before
if (num_valid_signals > 0) {
       }
     }
   }
 
 protected:
   bool DoExecute(Args &signal_args, CommandReturnObject &result) override {
     Target &target = GetSelectedOrDummyTarget();
 
-    // Any signals that are being set should be added to the Target's 
+    // Any signals that are being set should be added to the Target's
     // DummySignals so they will get applied on rerun, etc.
     // If we have a process, however, we can do a more accurate job of vetting
     // the user's options.
     ProcessSP process_sp = target.GetProcessSP();
 
     int stop_action = -1;   // -1 means leave the current setting alone
     int pass_action = -1;   // -1 means leave the current setting alone
     int notify_action = -1; // -1 means leave the current setting alone
bool DoExecute(Args &signal_args, CommandReturnObject &result) override {
     }
 
     if (!m_options.pass.empty() &&
         !VerifyCommandOptionValue(m_options.pass, pass_action)) {
       result.AppendError("Invalid argument for command option --pass; must be "
                          "true or false.\n");
       return false;
     }
-    
-    bool no_actions = (stop_action == -1 && pass_action == -1 
-        && notify_action == -1);
+    bool no_actions =
+        (stop_action == -1 && pass_action == -1 && notify_action == -1);
     if (m_options.only_target_values && !no_actions) {
       result.AppendError("-t is for reporting, not setting, target values.");
       return false;
     }
 
     size_t num_args = signal_args.GetArgumentCount();
     UnixSignalsSP signals_sp;
     if (process_sp)
if (num_args > 0) {
             result.AppendErrorWithFormat("Can't set signal handling by signal "
                                          "number with no process");
             return false;
           }
          num_signals_set = num_args;
         }
         auto set_lazy_bool = [] (int action) -> LazyBool {
           LazyBool lazy;
-          if (action == -1) 
+          if (action == -1)
             lazy = eLazyBoolCalculate;
-          else if (action) 
+          else if (action)
             lazy = eLazyBoolYes;
           else
             lazy = eLazyBoolNo;
           return lazy;
         };
 
         // If there were no actions, we're just listing, don't add the dummy:
         if (!no_actions)
if (num_args > 0) {
         }
       }
     }
 
     if (signals_sp)
       PrintSignalInformation(result.GetOutputStream(), signal_args,
                              num_signals_set, signals_sp);
     else
-      target.PrintDummySignals(result.GetOutputStream(), 
-          signal_args);
+      target.PrintDummySignals(result.GetOutputStream(), signal_args);
 
     if (num_signals_set > 0)
       result.SetStatus(eReturnStatusSuccessFinishResult);
     else
       result.SetStatus(eReturnStatusFailed);
 
     return result.Succeeded();
   }
Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_arg,
 
       switch (short_option) {
 
       case 'd': {
         m_directory.SetFile(option_arg, FileSpec::Style::native);
         FileSystem::Instance().Resolve(m_directory);
         break;
       }
+      case 'c': {
+        m_compact = true;
+        break;
+      }
       default:
         llvm_unreachable("Unimplemented option");
       }
       return error;
     }
 
-    void OptionParsingStarting(ExecutionContext *execution_context) override{};
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      m_compact = false;
+    };
 
     llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
       return llvm::makeArrayRef(g_process_trace_save_options);
     };
 
     FileSpec m_directory;
+    bool m_compact;
   };
 
   Options *GetOptions() override { return &m_options; }
   CommandObjectProcessTraceSave(CommandInterpreter &interpreter)
       : CommandObjectParsed(
             interpreter, "process trace save",
             "Save the trace of the current process in the specified directory. "
             "The directory will be created if needed. "
public:
   ~CommandObjectProcessTraceSave() override = default;
 
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
     ProcessSP process_sp = m_exe_ctx.GetProcessSP();
 
     TraceSP trace_sp = process_sp->GetTarget().GetTrace();
 
-    if (llvm::Error err = trace_sp->SaveLiveTraceToDisk(m_options.m_directory))
-      result.AppendError(toString(std::move(err)));
-    else
+    if (llvm::Expected<FileSpec> desc_file =
+            trace_sp->SaveToDisk(m_options.m_directory, m_options.m_compact)) {
+      result.AppendMessageWithFormatv(
+          "Trace bundle description file written to: {0}", *desc_file);
       result.SetStatus(eReturnStatusSuccessFinishResult);
+    } else {
+      result.AppendError(toString(desc_file.takeError()));
+    }
 
     return result.Succeeded();
   }
 
   CommandOptions m_options;
 };
 
 // CommandObjectProcessTraceStop

lldb/source/Commands/CommandObjectTrace.cpp

public:
     bool m_verbose; // Enable verbose logging for debugging purposes.
   };
 
   CommandObjectTraceLoad(CommandInterpreter &interpreter)
       : CommandObjectParsed(
             interpreter, "trace load",
             "Load a post-mortem processor trace session from a trace bundle.",
             "trace load") {
-    CommandArgumentData session_file_arg{eArgTypePath, eArgRepeatPlain};
+    CommandArgumentData session_file_arg{eArgTypeFilename, eArgRepeatPlain};
     m_arguments.push_back({session_file_arg});
   }
 
+  void
+  HandleArgumentCompletion(CompletionRequest &request,
+                           OptionElementVector &opt_element_vector) override {
+    CommandCompletions::InvokeCommonCompletionCallbacks(
+        GetCommandInterpreter(), CommandCompletions::eDiskFileCompletion,
+        request, nullptr);
+  }
+
   ~CommandObjectTraceLoad() override = default;
 
   Options *GetOptions() override { return &m_options; }
 
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
     if (command.size() != 1) {
       result.AppendError("a single path to a JSON file containing a the "

lldb/source/Commands/Options.td

let Command = "process save_core" in {
   def process_save_core_plugin_name : Option<"plugin-name", "p">,
     OptionalArg<"Plugin">, Desc<"Specify a plugin name to create the core file."
     "This allows core files to be saved in different formats.">;
 }
 
 let Command = "process trace save" in {
   def process_trace_save_directory: Option<"directory", "d">,
     Group<1>,
-    Arg<"Value">, Required,
+    Arg<"Filename">, Required,
     Desc<"The directory where the trace will be saved."
         "It will be created if it does not exist.">;
+  def process_trace_save_compact: Option<"compact", "c">,
+    Group<1>,
+    Desc<"Try not to save to disk information irrelevant to the traced "
+        "processes. Each trace plug-in implements this in a different "
+        "fashion.">;
 }
 
 let Command = "script import" in {
   def script_import_allow_reload : Option<"allow-reload", "r">, Group<1>,
     Desc<"Allow the script to be loaded even if it was already loaded before. "
     "This argument exists for backwards compatibility, but reloading is always "
     "allowed, whether you specify it or not.">;
   def relative_to_command_file : Option<"relative-to-command-file", "c">,

lldb/source/Plugins/Trace/intel-pt/PerfContextSwitchDecoder.h

 //===-- PerfContextSwitchDecoder.h --======----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_PERFCONTEXTSWITCHDECODER_H
 #define LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_PERFCONTEXTSWITCHDECODER_H
 
 #include "lldb/Utility/TraceIntelPTGDBRemotePackets.h"
 #include "lldb/lldb-types.h"
 
 #include "llvm/Support/Error.h"
 
+#include <set>
 #include <vector>
 
 namespace lldb_private {
 namespace trace_intel_pt {
 
 /// This class indicates the time interval in which a thread was running
 /// continuously on a cpu core.
 struct ThreadContinuousExecution {

 /// \return
 ///   A list of continuous executions recovered from the raw trace sorted by
 ///   time, or an \a llvm::Error if the data is malformed.
 llvm::Expected<std::vector<ThreadContinuousExecution>>
 DecodePerfContextSwitchTrace(llvm::ArrayRef<uint8_t> data,
                              lldb::cpu_id_t cpu_id,
                              const LinuxPerfZeroTscConversion &tsc_conversion);
 
+llvm::Expected<std::vector<uint8_t>>
+FilterProcessesFromContextSwitchTrace(llvm::ArrayRef<uint8_t> data,
+                                      const std::set<lldb::pid_t> &pids);
+
 } // namespace trace_intel_pt
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_PERFCONTEXTSWITCHDECODER_H

lldb/source/Plugins/Trace/intel-pt/PerfContextSwitchDecoder.cpp

 using namespace lldb_private;
 using namespace lldb_private::trace_intel_pt;
 using namespace llvm;
 
 /// Copied from <linux/perf_event.h> to avoid depending on perf_event.h on
 /// non-linux platforms.
 /// \{
 #define PERF_RECORD_MISC_SWITCH_OUT (1 << 13)
-#define PERF_RECORD_MAX 19
+
+#define PERF_RECORD_LOST 2
+#define PERF_RECORD_THROTTLE 5
+#define PERF_RECORD_UNTHROTTLE 6
+#define PERF_RECORD_LOST_SAMPLES 13
 #define PERF_RECORD_SWITCH_CPU_WIDE 15
+#define PERF_RECORD_MAX 19
 
 struct perf_event_header {
   uint32_t type;
   uint16_t misc;
   uint16_t size;
 
   /// \return
   ///   An \a llvm::Error if the record looks obviously wrong, or \a
if (type >= PERF_RECORD_MAX + 100)
           inconvertibleErrorCode(),
           formatv("Invalid record type {0} was found.", type));
     return Error::success();
   }
 
   bool IsContextSwitchRecord() const {
     return type == PERF_RECORD_SWITCH_CPU_WIDE;
   }
+
+  bool IsErrorRecord() const {
+    return type == PERF_RECORD_LOST || type == PERF_RECORD_THROTTLE ||
+           type == PERF_RECORD_UNTHROTTLE || type == PERF_RECORD_LOST_SAMPLES;
+  }
 };
 /// \}
 
 /// Record found in the perf_event context switch traces. It might contain
 /// additional fields in memory, but header.size should have the actual size
 /// of the record.
 struct PerfContextSwitchRecord {
   struct perf_event_header header;
lldb_private::trace_intel_pt::DecodePerfContextSwitchTrace(
   if (Error err = do_decode())
     return createStringError(inconvertibleErrorCode(),
                              formatv("Malformed perf context switch trace for "
                                      "cpu {0} at offset {1}. {2}",
                                      cpu_id, offset, toString(std::move(err))));
 
   return executions;
 }
+
+Expected<std::vector<uint8_t>>
+lldb_private::trace_intel_pt::FilterProcessesFromContextSwitchTrace(
+    llvm::ArrayRef<uint8_t> data, const std::set<lldb::pid_t> &pids) {
+  size_t offset = 0;
+  std::vector<uint8_t> out_data;
+
+  while (offset < data.size()) {
+    const perf_event_header &perf_record =
+        *reinterpret_cast<const perf_event_header *>(data.data() + offset);
+    if (Error err = perf_record.SanityCheck())
+      return std::move(err);
+    bool should_copy = false;
+    if (perf_record.IsContextSwitchRecord()) {
+      const PerfContextSwitchRecord &context_switch_record =
+          *reinterpret_cast<const PerfContextSwitchRecord *>(data.data() +
+                                                             offset);
+      if (pids.count(context_switch_record.pid))
+        should_copy = true;
+    } else if (perf_record.IsErrorRecord()) {
+      should_copy = true;
+    }
+
+    if (should_copy) {
+      for (size_t i = 0; i < perf_record.size; i++) {
+        out_data.push_back(data[offset + i]);
+      }
+    }
+
+    offset += perf_record.size;
+  }
+  return out_data;
+}

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.h

 
 namespace lldb_private {
 namespace trace_intel_pt {
 
 class TraceIntelPT : public Trace {
 public:
   void Dump(Stream *s) const override;
 
-  llvm::Error SaveLiveTraceToDisk(FileSpec directory) override;
+  llvm::Expected<FileSpec> SaveToDisk(FileSpec directory,
+                                      bool compact) override;
 
   ~TraceIntelPT() override = default;
 
   /// PluginInterface protocol
   /// \{
   llvm::StringRef GetPluginName() override { return GetPluginNameStatic(); }
 
   static void Initialize();

lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp

 }
 
 StringRef TraceIntelPT::GetSchema() {
   return TraceIntelPTBundleLoader::GetSchema();
 }
 
 void TraceIntelPT::Dump(Stream *s) const {}
 
-llvm::Error TraceIntelPT::SaveLiveTraceToDisk(FileSpec directory) {
+Expected<FileSpec> TraceIntelPT::SaveToDisk(FileSpec directory, bool compact) {
   RefreshLiveProcessState();
-  return TraceIntelPTBundleSaver().SaveToDisk(*this, directory);
+  return TraceIntelPTBundleSaver().SaveToDisk(*this, directory, compact);
 }
 
 Expected<TraceSP> TraceIntelPT::CreateInstanceForTraceBundle(
     const json::Value &bundle_description, StringRef bundle_dir,
     Debugger &debugger) {
   return TraceIntelPTBundleLoader(debugger, bundle_description,
                                        bundle_dir)
       .Load();

lldb/source/Plugins/Trace/intel-pt/TraceIntelPTBundleSaver.h

public:
   /// command to load the trace in LLDB.
   ///
   /// \param[in] trace_ipt
   ///     The Intel PT trace to be saved to disk.
   ///
   /// \param[in] directory
   ///     The directory where the trace bundle will be created.
   ///
+  /// \param[in] compact
+  ///     Filter out information irrelevant to the traced processes in the
+  ///     context switch and intel pt traces when using per-cpu mode. This
+  ///     effectively reduces the size of those traces.
+  ///
   /// \return
-  ///     \a llvm::success if the operation was successful, or an \a llvm::Error
-  ///     otherwise.
-  llvm::Error SaveToDisk(TraceIntelPT &trace_ipt, FileSpec directory);
+  ///   A \a FileSpec pointing to the bundle description file, or an \a
+  ///   llvm::Error otherwise.
+  llvm::Expected<FileSpec> SaveToDisk(TraceIntelPT &trace_ipt,
+                                      FileSpec directory, bool compact);
 };
 
 } // namespace trace_intel_pt
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_TRACE_INTEL_PT_TRACEINTELPTBUNDLESAVER_H

lldb/source/Plugins/Trace/intel-pt/TraceIntelPTBundleSaver.cpp

 //===-- TraceIntelPTBundleSaver.cpp ---------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "TraceIntelPTBundleSaver.h"
+
+#include "PerfContextSwitchDecoder.h"
 #include "TraceIntelPT.h"
 #include "TraceIntelPTJSONStructs.h"
+
 #include "lldb/Core/Module.h"
 #include "lldb/Core/ModuleList.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/SectionLoadList.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/ThreadList.h"
 #include "lldb/lldb-types.h"
 #include "llvm/ADT/None.h"

 ///
 /// \param[in] trace_bundle_description
 ///     The trace bundle description as JSON Object.
 ///
 /// \param[in] directory
 ///     The directory where the JSON file will be saved.
 ///
 /// \return
-///     \a llvm::Success if the operation was successful, or an \a llvm::Error
-///     otherwise.
-static llvm::Error
+///     A \a FileSpec pointing to the bundle description file, or an \a
+///     llvm::Error otherwise.
+static Expected<FileSpec>
 SaveTraceBundleDescription(const llvm::json::Value &trace_bundle_description,
                            const FileSpec &directory) {
   FileSpec trace_path = directory;
   trace_path.AppendPathComponent("trace.json");
   std::ofstream os(trace_path.GetPath());
   os << formatv("{0:2}", trace_bundle_description).str();
   os.close();
   if (!os)
     return createStringError(inconvertibleErrorCode(),
                              formatv("couldn't write to the file {0}",
                                      trace_path.GetPath().c_str()));
-  return Error::success();
+  return trace_path;
 }
 
 /// Build the threads sub-section of the trace bundle description file.
 /// Any associated binary files are created inside the given directory.
 ///
 /// \param[in] process
 ///     The process being traced.
 ///
if (trace_sp->GetTracedCpus().empty()) {
         return std::move(err);
     }
 
     json_threads.push_back(std::move(json_thread));
   }
   return json_threads;
 }
 
-static llvm::Expected<llvm::Optional<std::vector<JSONCpu>>>
-BuildCpusSection(TraceIntelPT &trace_ipt, FileSpec directory) {
-  if (trace_ipt.GetTracedCpus().empty())
-    return None;
+/// \return
+///   an \a llvm::Error in case of failures, \a None if the trace is not written
+///   to disk because the trace is empty and the \p compact flag is present, or
+///   the FileSpec of the trace file on disk.
+static Expected<Optional<FileSpec>>
+WriteContextSwitchTrace(TraceIntelPT &trace_ipt, lldb::cpu_id_t cpu_id,
+                        const FileSpec &cpus_dir, bool compact) {
+  FileSpec output_context_switch_trace = cpus_dir;
+  output_context_switch_trace.AppendPathComponent(std::to_string(cpu_id) +
+                                                  ".perf_context_switch_trace");
 
-  std::vector<JSONCpu> json_cpus;
-  FileSpec cpus_dir = directory;
-  cpus_dir.AppendPathComponent("cpus");
-  sys::fs::create_directories(cpus_dir.GetCString());
+  bool should_skip = false;
 
-  for (lldb::cpu_id_t cpu_id : trace_ipt.GetTracedCpus()) {
-    JSONCpu json_cpu;
-    json_cpu.id = cpu_id;
+  Error err = trace_ipt.OnCpuBinaryDataRead(
+      cpu_id, IntelPTDataKinds::kPerfContextSwitchTrace,
+      [&](llvm::ArrayRef<uint8_t> data) -> llvm::Error {
+        if (!compact)
+          return WriteBytesToDisk(output_context_switch_trace, data);
 
-    {
-      FileSpec output_trace = cpus_dir;
-      output_trace.AppendPathComponent(std::to_string(cpu_id) +
-                                       ".intelpt_trace");
-      json_cpu.ipt_trace = output_trace.GetPath();
+        std::set<lldb::pid_t> pids;
+        for (Process *process : trace_ipt.GetAllProcesses())
+          pids.insert(process->GetID());
+
+        Expected<std::vector<uint8_t>> compact_context_switch_trace =
+            FilterProcessesFromContextSwitchTrace(data, pids);
+        if (!compact_context_switch_trace)
+          return compact_context_switch_trace.takeError();
 
-      llvm::Error err = trace_ipt.OnCpuBinaryDataRead(
-          cpu_id, IntelPTDataKinds::kIptTrace,
-          [&](llvm::ArrayRef<uint8_t> data) -> llvm::Error {
-            return WriteBytesToDisk(output_trace, data);
-          });
-      if (err)
-        return std::move(err);
-    }
+        if (compact_context_switch_trace->empty()) {
+          should_skip = true;
+          return Error::success();
+        }
+
+        return WriteBytesToDisk(output_context_switch_trace,
+                                *compact_context_switch_trace);
+
+  if (should_skip)
+    return None;
+  return output_context_switch_trace;
 
-    {
-      FileSpec output_context_switch_trace = cpus_dir;
-      output_context_switch_trace.AppendPathComponent(
-          std::to_string(cpu_id) + ".perf_context_switch_trace");
-      json_cpu.context_switch_trace = output_context_switch_trace.GetPath();
+static Expected<FileSpec> WriteIntelPTTrace(TraceIntelPT &trace_ipt,
+                                            lldb::cpu_id_t cpu_id,
+                                            const FileSpec &cpus_dir) {
+  FileSpec output_trace = cpus_dir;
+  output_trace.AppendPathComponent(std::to_string(cpu_id) + ".intelpt_trace");
 
-      llvm::Error err = trace_ipt.OnCpuBinaryDataRead(
-          cpu_id, IntelPTDataKinds::kPerfContextSwitchTrace,
-          [&](llvm::ArrayRef<uint8_t> data) -> llvm::Error {
-            return WriteBytesToDisk(output_context_switch_trace, data);
-          });
-      if (err)
-        return std::move(err);
-    }
+  Error err = trace_ipt.OnCpuBinaryDataRead(
+      cpu_id, IntelPTDataKinds::kIptTrace,
+        return WriteBytesToDisk(output_trace, data);
+  return output_trace;
+
+static llvm::Expected<llvm::Optional<std::vector<JSONCpu>>>
+BuildCpusSection(TraceIntelPT &trace_ipt, FileSpec directory, bool compact) {
+  if (trace_ipt.GetTracedCpus().empty())
+    return None;
+
+  std::vector<JSONCpu> json_cpus;
+  FileSpec cpus_dir = directory;
+  cpus_dir.AppendPathComponent("cpus");
+  sys::fs::create_directories(cpus_dir.GetCString());
+
+  for (lldb::cpu_id_t cpu_id : trace_ipt.GetTracedCpus()) {
+    JSONCpu json_cpu;
+    json_cpu.id = cpu_id;
+    Expected<Optional<FileSpec>> context_switch_trace_path =
+        WriteContextSwitchTrace(trace_ipt, cpu_id, cpus_dir, compact);
+    if (!context_switch_trace_path)
+      return context_switch_trace_path.takeError();
+    if (!*context_switch_trace_path)
+      continue;
+    json_cpu.context_switch_trace = context_switch_trace_path.get()->GetPath();
+
+    if (Expected<FileSpec> ipt_trace_path =
+            WriteIntelPTTrace(trace_ipt, cpu_id, cpus_dir))
+      json_cpu.ipt_trace = ipt_trace_path->GetPath();
+    else
+      return ipt_trace_path.takeError();
+
     json_cpus.push_back(std::move(json_cpu));
   }
   return json_cpus;
 }
 
 /// Build modules sub-section of the trace bundle. The original modules
 /// will be copied over to the \a <directory/modules> folder. Invalid modules
 /// are skipped.
for (size_t i = 0; i < module_list.GetSize(); ++i) {
     if (load_addr == LLDB_INVALID_ADDRESS)
       continue;
 
     FileSpec path_to_copy_module = directory;
     path_to_copy_module.AppendPathComponent("modules");
     path_to_copy_module.AppendPathComponent(system_path);
     sys::fs::create_directories(path_to_copy_module.GetDirectory().AsCString());
 
-    if (std::error_code ec = llvm::sys::fs::copy_file(
-            system_path, path_to_copy_module.GetPath()))
+    if (std::error_code ec =
+            llvm::sys::fs::copy_file(file, path_to_copy_module.GetPath()))
       return createStringError(
           inconvertibleErrorCode(),
           formatv("couldn't write to the file. {0}", ec.message()));
 
     json_modules.push_back(
         JSONModule{system_path, path_to_copy_module.GetPath(),
                    JSONUINT64{load_addr}, module_sp->GetUUID().GetAsString()});
   }
if (llvm::Expected<JSONProcess> json_process =
             BuildProcessSection(*process, directory))
       processes.push_back(std::move(*json_process));
     else
       return json_process.takeError();
   }
   return processes;
 }
 
-Error TraceIntelPTBundleSaver::SaveToDisk(TraceIntelPT &trace_ipt,
-                                           FileSpec directory) {
+Expected<FileSpec> TraceIntelPTBundleSaver::SaveToDisk(TraceIntelPT &trace_ipt,
+                                                       FileSpec directory,
+                                                       bool compact) {
   if (std::error_code ec =
           sys::fs::create_directories(directory.GetPath().c_str()))
     return llvm::errorCodeToError(ec);
 
   Expected<pt_cpu> cpu_info = trace_ipt.GetCPUInfo();
   if (!cpu_info)
     return cpu_info.takeError();
 
   FileSystem::Instance().Resolve(directory);
 
   Expected<std::vector<JSONProcess>> json_processes =
       BuildProcessesSection(trace_ipt, directory);
 
   if (!json_processes)
     return json_processes.takeError();
 
   Expected<Optional<std::vector<JSONCpu>>> json_cpus =
-      BuildCpusSection(trace_ipt, directory);
+      BuildCpusSection(trace_ipt, directory, compact);
   if (!json_cpus)
     return json_cpus.takeError();
 
   JSONTraceBundleDescription json_intel_pt_bundle_desc{"intel-pt", *cpu_info, *json_processes,
                                          *json_cpus,
                                          trace_ipt.GetPerfZeroTscConversion()};
 
   return SaveTraceBundleDescription(toJSON(json_intel_pt_bundle_desc), directory);
 }

lldb/test/API/commands/trace/TestTraceLoad.py

def testLoadMultiCoreTrace(self):
           substrs=["19522: [tsc=40450075478109270] (error) expected tracing enabled event",
                    "m.out`foo() + 65 at multi_thread.cpp:12:21",
                    "19520: [tsc=40450075477657246] 0x0000000000400ba7    jg     0x400bb3"])
         self.expect("thread trace dump instructions 3 -t",
           substrs=["67911: [tsc=40450075477799536] 0x0000000000400bd7    addl   $0x1, -0x4(%rbp)",
                    "m.out`bar() + 26 at multi_thread.cpp:20:6"])
 
     @testSBAPIAndCommands
+    def testLoadCompactMultiCoreTrace(self):
+        src_dir = self.getSourceDir()
+        trace_description_file_path = os.path.join(src_dir, "intelpt-multi-core-trace", "trace.json")
+        self.traceLoad(traceDescriptionFilePath=trace_description_file_path, substrs=["intel-pt"])
+
+        self.expect("thread trace dump info 2", substrs=["Total number of continuous executions found: 153"])
+
+        # we'll save the trace in compact format
+        compact_trace_bundle_dir = os.path.join(self.getBuildDir(), "intelpt-multi-core-trace-compact")
+        self.traceSave(compact_trace_bundle_dir, compact=True)
+
+        # we'll delete the previous target and make sure it's trace object is deleted
+        self.dbg.DeleteTarget(self.dbg.GetTargetAtIndex(0))
+        self.expect("thread trace dump instructions 2 -t", substrs=["error: invalid target"], error=True)
+
+        # we'll load the compact trace and make sure it works
+        self.traceLoad(os.path.join(compact_trace_bundle_dir, "trace.json"), substrs=["intel-pt"])
+        self.expect("thread trace dump instructions 2 -t",
+          substrs=["19522: [tsc=40450075478109270] (error) expected tracing enabled event",
+                   "m.out`foo() + 65 at multi_thread.cpp:12:21",
+                   "19520: [tsc=40450075477657246] 0x0000000000400ba7    jg     0x400bb3"])
+        self.expect("thread trace dump instructions 3 -t",
+          substrs=["67911: [tsc=40450075477799536] 0x0000000000400bd7    addl   $0x1, -0x4(%rbp)",
+                   "m.out`bar() + 26 at multi_thread.cpp:20:6"])
+
+        # This reduced the number of continuous executions to look at
+        self.expect("thread trace dump info 2", substrs=["Total number of continuous executions found: 3"])
+
+        # We clean up for the next run of this test
+        self.dbg.DeleteTarget(self.dbg.GetTargetAtIndex(0))
+
+
+    @testSBAPIAndCommands
     def testLoadMultiCoreTraceWithStringNumbers(self):
         src_dir = self.getSourceDir()
         trace_description_file_path = os.path.join(src_dir, "intelpt-multi-core-trace", "trace_with_string_numbers.json")
         self.traceLoad(traceDescriptionFilePath=trace_description_file_path, substrs=["intel-pt"])
         self.expect("thread trace dump instructions 2 -t",
           substrs=["19522: [tsc=40450075478109270] (error) expected tracing enabled event",
                    "m.out`foo() + 65 at multi_thread.cpp:12:21",
                    "19520: [tsc=40450075477657246] 0x0000000000400ba7    jg     0x400bb3"])