This is an archive of the discontinued LLVM Phabricator instance.

[trace][intelpt] Server side changes for TSC to wall time conversion
ClosedPublic

Authored by jj10306 on Mar 22 2022, 11:16 AM.

Details

Summary
  • Add updated schema documentation to lldb-gdb-remote.txt
  • Add support for counter values in trace
  • Add TSC conversion logic: cache the perf
  • Move conversion calculation from Perf.h to TraceIntelPTGDBRemotePackets.h to remove dependency on linux specific headers
    • Update PerfTests to accommodate these changes
  • Add TraceGDBRemotePackets unittests

Third of the series of smaller diffs that split https://reviews.llvm.org/D120595 up

Diff Detail

Event Timeline

jj10306 created this revision.Mar 22 2022, 11:16 AM
Herald added a project: Restricted Project. · View Herald TranscriptMar 22 2022, 11:16 AM
jj10306 requested review of this revision.Mar 22 2022, 11:16 AM
wallace requested changes to this revision.Mar 22 2022, 1:33 PM

pretty good. Just some work to do with the documentation, some naming, and using a dictionary instead of a list for the counter info kinds.

lldb/docs/lldb-gdb-remote.txt
455–462

let's use a dictionary instead of an array

465

let's delete this so that we inline the documentation in the schema above

472

let's not call it kind, because tsc-perf-conversion is not a kind of counter, but kind of information of a counter

473

let's include the word zero to distinguish from the other formula that doesn't use the time_zero value.

475
482–488
lldb/include/lldb/Utility/TraceGDBRemotePackets.h
120
131

What about Convert?

140–141

nice

lldb/include/lldb/Utility/TraceIntelPTGDBRemotePackets.h
14–15

you can remove this one

55

let's include the word Zero in the name

71
75
88
lldb/source/Plugins/Process/Linux/IntelPTCollector.cpp
532–533

yes, you need to log it. Create a new entry in the enum POSIXLog called 'Trace` and log here the reason why we couldn't get the tsc conversion values

lldb/source/Plugins/Process/Linux/Perf.cpp
11

all good

lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp
12–13

i imagine you can remove these

47–49

what about just inlining ToWallTime here? This indirection is not really that useful because it's private

104

this will need to be an object now

This revision now requires changes to proceed.Mar 22 2022, 1:33 PM
wallace added inline comments.Mar 22 2022, 2:09 PM
lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp
68–96

We have to change these because of our new structure. I'll include an idea

"counter": {"tsc-perf-zero-conversion": {conversion values...}}

an important detail is that the json structure is not reflected in the struct

struct TraceIntelPTGetStateResponse : TraceGetStateResponse {
  /// TSC conversion params if they exists, otherwise `nullptr`.
  TraceTscConversionSP tsc_conversion;
} ;

We don't have an intermediate Counters structure, and it might not very useful to have it. That means that you have to do some manual things.

jj10306 updated this revision to Diff 417421.Mar 22 2022, 4:11 PM
jj10306 marked 20 inline comments as done.
  • update documentation
  • update schema to use dictionary instead of array
  • add Trace type to PosixLog
  • rename to LinuxPerfZeroTscConversion
  • clean up includes

Working on adding unittest to test the new schema's serialization/deserialization logic.

wallace added inline comments.Mar 22 2022, 4:22 PM
lldb/source/Plugins/Process/Linux/IntelPTCollector.cpp
532

TIL

clayborg added inline comments.Mar 22 2022, 4:28 PM
lldb/include/lldb/Utility/TraceGDBRemotePackets.h
142

std::unique_ptr<> instead? I am guessing we don't need to share this pointer with anyone else, just delete it when it goes out of scope? If so, std::unique_ptr is a better fit and TraceTscConversionUP would be the name to use.

lldb/include/lldb/Utility/TraceIntelPTGDBRemotePackets.h
89

TraceTscConversionUP is we switch to unique pointer

93

Might be nice to just pass in a reference to the TraceTscConversion since the value should never be NULL right?

lldb/source/Plugins/Process/Linux/IntelPTCollector.h
239

TraceTscConversionUP is we switch to unique pointer

jj10306 marked 4 inline comments as done.Mar 22 2022, 6:55 PM
jj10306 added inline comments.
lldb/include/lldb/Utility/TraceIntelPTGDBRemotePackets.h
93

This value is NULL when no TSC to wall time conversion exists.
Given this, what would your recommendation be for passing the TraceTscConversion data to this method to be populated?

davidca added inline comments.Mar 22 2022, 8:25 PM
lldb/include/lldb/Utility/TraceGDBRemotePackets.h
122

this can be a normal class and only have Convert to be templated. That's simpler and let's you use the same pointer to store any instance of this class. Simplifying code that uses this class.

lldb/source/Plugins/Process/Linux/Perf.cpp
22–23

nit on last diff that I probably missed. This is loading, or reading, not fetching :D . Fetching means getting the data from somewhere else, this is reading from the same host.

lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp
58

this casting is not safe, I know json sucks and has not unsigned, but you should assert that the casting will not cause overflow. You can use std::numeric_limits (https://en.cppreference.com/w/cpp/types/numeric_limits)

74

unsafe casting, check for overflow, alternatively, you could have LinuxPerfZeroTscConversion have a constructor that take all ints and checks for overflow and properly casts.

wallace added inline comments.Mar 22 2022, 9:16 PM
lldb/include/lldb/Utility/TraceIntelPTGDBRemotePackets.h
93

I think it's fine passing the shared pointer in this case, given what Jakob just explained

lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp
58

I think in this case we are fine with numeric overflows. Doing uint64_t -> int64_t -> uint64_t starting with a positive integer larger than what int64_t can represent correctly will end up having a negative int64_t that is later converted into a uint64_t exactly the same as the the initial number. We don't want to throw or anything in this case. We want to to be able to transmit the number in any representation possible.
Another possibility is to send the number as a string, which is what JS developers do for 64 bit ints, but in this case the unsafe cast is correct.

jj10306 updated this revision to Diff 417574.EditedMar 23 2022, 5:35 AM
jj10306 marked 6 inline comments as done.
  • use unique pointer instead of shared pointer, per @clayborg's suggestion
  • rename FetchPerfTscConversion to LoadPerfTscConversion per @davidca's suggestion
  • replace C-style casting with static_cast<>in fromJSON
jj10306 updated this revision to Diff 417629.Mar 23 2022, 8:29 AM
jj10306 retitled this revision from [wip][trace][intelpt] Server side changes for TSC to wall time conversion to [trace][intelpt] Server side changes for TSC to wall time conversion.
jj10306 edited the summary of this revision. (Show Details)

Add unittests for the serialization/deserialization of TraceIntelPTGetStateResponse. One tests an "empty" response and the other a non-empty response.
The non-empty response tests that the serialization to JSON isn't lossy when casting from unsigned values, followup to the discussions with @davidca and @wallace on this topic.

The newly added file currently only includes tests for the packet response this diff affects - future work could add unittests for all trace related packets to this file.

jj10306 added inline comments.Mar 23 2022, 8:34 AM
lldb/include/lldb/Utility/TraceGDBRemotePackets.h
122

From my understanding, virtual member functions cannot be templated so this forces us to make the class a template. Let me know if I'm misunderstanding your comment or if that is not the case!

jj10306 updated this revision to Diff 417638.Mar 23 2022, 8:37 AM

remove accidentally included patch file

wallace added inline comments.Mar 23 2022, 11:43 AM
lldb/source/Plugins/Process/Linux/IntelPTCollector.cpp
529–530

you don't need a unique_ptr for this variable in the collector. An Optional is enough

lldb/unittests/Utility/TraceGDBRemotePacketsTest.cpp
23–26

we can make it simple with some new functions

56–57

make_unique is able to pass the arguments and construct the pointer by itself in place

76

better do

ASSERT_EQ(pre_serialization_conversion, <actual value>);
ASSERT_EQ(post_serialization_conversion, <actual value>);

I just want to prevent something breaking and returning 0 for all conversions and passing this test

jj10306 updated this revision to Diff 417758.Mar 23 2022, 2:58 PM
jj10306 marked 5 inline comments as done.

address comments

wallace accepted this revision.Mar 23 2022, 3:26 PM

lgtm. Can we revisit the serialization of ints in a new diff if needed

This revision is now accepted and ready to land.Mar 23 2022, 3:26 PM
Herald added a project: Restricted Project. · View Herald TranscriptMar 24 2022, 5:36 AM