[intel-pt] Disable/Enable tracing to guarantee the trace is correct

[wallace]

Summary

As mentioned in the comment inside the code, the Intel documentation states that the internal CPU buffer is forcefully flushed out to RAM right away when tracing is disabled. Otherwise, the buffer on RAM might be stale, as the CPU's automatic flushing happens only from time to time from time and the CPU decides when.

This diff disables tracing when the trace buffer is going to be read. This is a quite safe operation, as the reading is done when the inferior is paused at a breakpoint, so we are not losing any packets because there's no code being executed.

After the reading is finished, tracing is enabled back.

I run the following manual tests

(lldb) b main
Breakpoint 1: where = a.out`main + 15 at main.cpp:4:7, address = 0x000000000040050f
(lldb) r
Process 3078226 stopped
* thread #1, name = 'a.out', stop reason = breakpoint 1.1
    frame #0: 0x000000000040050f a.out`main at main.cpp:4:7
(lldb) processor-trace start
(lldb) b 5
Breakpoint 2: where = a.out`main + 22 at main.cpp:5:12, address = 0x0000000000400516
(lldb) c
Process 3078226 resuming
Process 3078226 stopped
* thread #1, name = 'a.out', stop reason = breakpoint 2.1
    frame #0: 0x0000000000400516 a.out`main at main.cpp:5:12
(lldb) processor-trace show-instr-log 
thread #1: tid=3078226
    0x40050f <+15>: movl   $0x0, -0x8(%rbp)

>>> Before, some runs of the script up to this point lead to empty traces

(lldb) b 6
Breakpoint 3: where = a.out`main + 42 at main.cpp:6:14, address = 0x000000000040052a
(lldb) c
Process 3092991 resuming
Process 3092991 stopped
* thread #1, name = 'a.out', stop reason = breakpoint 3.1
    frame #0: 0x000000000040052a a.out`main at main.cpp:6:14
(lldb) processor-trace show-instr-log                                                                                                                                                                                   thread #1: tid=3092991
    0x40050f <+15>: movl   $0x0, -0x8(%rbp)
    0x400516 <+22>: movl   $0x0, -0xc(%rbp)
    0x40051d <+29>: cmpl   $0x2710, -0xc(%rbp)       ; imm = 0x2710 
    0x400524 <+36>: jge    0x400546                  ; <+70> at main.cpp
    0x400524 <+36>: jge    0x400546                  ; <+70> at main.cpp

>>> The trace was re-enabled correctly and includes the instruction of the
first reading.

Those instructions correspond to these lines

3 int main() {
4  int z = 0;
5  for (int i = 0; i < 10000; i++) { 
6    z += fun(z)
...

[wallace]

remove an empty line

[labath]

Sounds reasonable.

lldb/source/Plugins/Process/Linux/ProcessorTrace.cpp

 //===-- ProcessorTrace.cpp ------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include <algorithm>
 #include <fstream>
 
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/MathExtras.h"
 
 #include "Plugins/Process/POSIX/ProcessPOSIXLog.h"
 #include "ProcessorTrace.h"
 #include "lldb/Host/linux/Support.h"
 
+#include <sys/ioctl.h>
 #include <sys/syscall.h>
 
 using namespace lldb;
 using namespace lldb_private;
 using namespace process_linux;
 using namespace llvm;
 
 lldb::user_id_t ProcessorTraceMonitor::m_trace_num = 1;

 }
 
 Status
 ProcessorTraceMonitor::ReadPerfTraceAux(llvm::MutableArrayRef<uint8_t> &buffer,
                                         size_t offset) {
 #ifndef PERF_ATTR_SIZE_VER5
   llvm_unreachable("perf event not supported");
 #else
+  // Disable the perf event to force a flush out of the CPU's internal buffer.
+  // Besides, we can guarantee that the CPU won't override any data as we are
+  // reading the buffer.
+  //
+  // The Intel documentation says:
+  //
+  // Packets are first buffered internally and then written out asynchronously.
+  // To collect packet output for postprocessing, a collector needs first to
+  // ensure that all packet data has been flushed from internal buffers.
+  // Software can ensure this by stopping packet generation by clearing
+  // IA32_RTIT_CTL.TraceEn (see “Disabling Packet Generation” in
+  // Section 35.2.7.2).
+  //
+  // This is achieved by the PERF_EVENT_IOC_DISABLE ioctl request, as mentioned
+  // in the man page of perf_event_open.
+  ioctl(*m_fd, PERF_EVENT_IOC_DISABLE);
+
   Log *log(ProcessPOSIXLog::GetLogIfAllCategoriesSet(POSIX_LOG_PTRACE));
   Status error;
   uint64_t head = m_mmap_meta->aux_head;
 
   LLDB_LOG(log, "Aux size -{0} , Head - {1}", m_mmap_meta->aux_size, head);
 
   /**
    * When configured as ring buffer, the aux buffer keeps wrapping around
    * the buffer and its not possible to detect how many times the buffer
    * wrapped. Initially the buffer is filled with zeros,as shown below
    * so in order to get complete buffer we first copy firstpartsize, followed
    * by any left over part from beginning to aux_head
    *
    * aux_offset [d,d,d,d,d,d,d,d,0,0,0,0,0,0,0,0,0,0,0] aux_size
    *                 aux_head->||<- firstpartsize  ->|
    *
    * */
 
   ReadCyclicBuffer(buffer, GetAuxBuffer(), static_cast<size_t>(head), offset);
   LLDB_LOG(log, "ReadCyclic BUffer Done");
+
+  // Reenable tracing now we have read the buffer
+  ioctl(*m_fd, PERF_EVENT_IOC_ENABLE);
   return error;
 #endif
 }
 
 Status
 ProcessorTraceMonitor::ReadPerfTraceData(llvm::MutableArrayRef<uint8_t> &buffer,
                                          size_t offset) {
 #ifndef PERF_ATTR_SIZE_VER5