This is an archive of the discontinued LLVM Phabricator instance.

[lldb][LocateModuleCallback] Call locate module callback
ClosedPublic

Authored by splhack on Jun 25 2023, 6:16 PM.

Details

Summary

RFC https://discourse.llvm.org/t/rfc-python-callback-for-target-get-module/71580

Updated Target::GetOrCreateModule to call locate module callback if set.

  • include/lldb/Target/Platform.h, source/Target/Platform.cpp
    • Implemented SetLocateModuleCallback and GetLocateModuleCallback*
  • include/lldb/Target/Target.h, source/Target/Target.cpp
    • Implemented CallLocateModuleCallbackIfSet.
  • unittests/Target/LocateModuleCallbackTest.cpp
    • Added comprehensive GetOrCreateModule tests.

Diff Detail

Event Timeline

splhack created this revision.Jun 25 2023, 6:16 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 25 2023, 6:16 PM
splhack requested review of this revision.Jun 25 2023, 6:16 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 25 2023, 6:16 PM
splhack updated this revision to Diff 538741.Jul 10 2023, 11:08 AM

Fix comments

clayborg requested changes to this revision.Jul 10 2023, 4:30 PM

So the "SetTargetGetModuleCallback" functions in this patch need to pass both a callback _and_ a callback_baton. That way people can register native callbacks that can be used. The python interpreter, when SetTargetGetModuleCallback is called from python, can take care of registering a callback that will call through the interpreter. See the following function as a model to follow:

void SBDebugger::SetDestroyCallback(lldb::SBDebuggerDestroyCallback destroy_callback, void *baton);

See this patch for details:

$ git show b461398f1ce30
lldb/include/lldb/Target/Platform.h
888

You actually need a callback along with the baton here. We probably don't need the word "Target" in the function name?

Maybe better named as

void SetLocationModuleCallback(PlatformLocateModuleCallback callback, void *baton);
890

Usually there there are two functions:

  • one that returns the function callback
  • one that returns the callback baton
944

We need to store the callback itself here too. Remove "target" from the name

lldb/source/Core/Debugger.cpp
2170–2181 ↗(On Diff #538741)

I don't think we need any of this in debugger. when someone, through python calls the "SetTargetGetModuleCallback" that code should register a static function as the callback to use that will end up calling into the script interpreter.

We don't always want to call the script interpreter for all cases. Users should be able to register a native callback for this, and it shouldn't force us to always use the script interpreter.

This revision now requires changes to proceed.Jul 10 2023, 4:30 PM

@clayborg
Sure, I'll rename the type and function to Locate instead of Get.

typedef SBError (*SBPlatformLocateModuleCallback)(
    SBDebugger debugger,
    SBModuleSpec &module_spec,
    SBFileSpec &module_file_spec,
    SBFileSpec &symbol_file_spec);

This SBPlatformLocateModuleCallback typedef is required for the SWIG type conversion.

However it will introduce the SB* dependencies to Platform.h if we use the type. Is it ok?

void SetLocateModuleCallback(SBPlatformLocateModuleCallback callback, void *baton);

I think Platform.h should use void * instead of SBPlatformLocateModuleCallback. Is it correct?

void SetLocateModuleCallback(void *callback, void *baton);
lldb/source/Core/Debugger.cpp
2170–2181 ↗(On Diff #538741)

There was another reason why Debugger has this method. The reason was that Debugger is easily gMock-able but Target is not.
Please take a look at MockDebugger in lldb/unittests/Target/GetModuleCallbackTest.cpp in this diff.

I will update this diff to allow C function callback aside from Python function, but probably we need to keep this kind of method outside Target for gMock to write unit tests.

splhack updated this revision to Diff 539213.EditedJul 11 2023, 11:35 AM
  • Rename 'get module callback' to 'locate module callback'
  • Enable C++ callback aside from Python callback
  • Introduce LocateModuleCallback typedef to use std::function in order to capture callback and baton with lambda, to convert ModuleSpec/FileSpec from/to SBModuleSpec/SBFileSpec.
splhack retitled this revision from [lldb][TargetGetModuleCallback] Call get module callback to [lldb][LocateModuleCallback] Call locate module callback.Jul 11 2023, 11:36 AM
splhack edited the summary of this revision. (Show Details)
splhack marked 3 inline comments as done.Jul 11 2023, 12:14 PM

SBPlatform API and Python will use SBPlatformLocateModuleCallback type.

typedef SBError (*SBPlatformLocateModuleCallback)(
    void *baton,
    const SBModuleSpec &module_spec,
    SBFileSpec &module_file_spec,
    SBFileSpec &symbol_file_spec);

Platform.h will use LocateModuleCallback type.

typedef std::function<Status(
    const ModuleSpec &module_spec,
    FileSpec &module_file_spec,
    FileSpec &symbol_file_spec)> LocateModuleCallback;

So, Platform.h does not need to have SB* dependencies,
and SBPlatform can convert ModuleSpec/FileSpec from/to SBModuleSpec/SBFileSpec for the callback.

lldb/include/lldb/Target/Platform.h
888

Platform.h now only retains a std::function for the locate module callback.

In D153735, SBPlatform will use a lambda

  • to capture callback(SBPlatformLocateModuleCallback) and baton(void *)
  • to convert ModuleSpec/FileSpec from/to SBModuleSpec/SBFileSpec

Therefore, Platform.h does not need SB* dependencies.

splhack marked an inline comment as done.Jul 11 2023, 12:16 PM
clayborg added inline comments.Jul 11 2023, 3:53 PM
lldb/include/lldb/Target/Platform.h
884–887

I think we still need a baton for the callback so clients can register a callback + void *.

944

We probably need a baton still. In order to make this work for python, we need to be able to set a callback and a baton for the python callable?

splhack added inline comments.Jul 11 2023, 4:54 PM
lldb/include/lldb/Target/Platform.h
944

@clayborg Yes, in D153735, both the callback and the baton are captured and retained by SBPlatform. The reason of why SBPlatform captures and retains the callback and the baton is the SBPlatformLocateModuleCallback type contains SBModuleSpec type and SBFileSpec type in the argument. Which are not available in Platform.h.

typedef SBError (*SBPlatformLocateModuleCallback)(
    void *baton, const SBModuleSpec &module_spec, SBFileSpec &module_file_spec,
    SBFileSpec &symbol_file_spec);
splhack added inline comments.Jul 11 2023, 5:48 PM
lldb/include/lldb/Target/Platform.h
884–887

Yes, callback and baton(void *) are retained by SBPlatform.
The detailed flow is here https://reviews.llvm.org/D153735#4491892

944

commented the detailed callback and baton flow here.
https://reviews.llvm.org/D153735#4491892

clayborg accepted this revision.Jul 12 2023, 10:30 AM

LGTM. Thanks for the detailed explanations, it all makes sense.

This revision is now accepted and ready to land.Jul 12 2023, 10:30 AM
This revision was automatically updated to reflect the committed changes.

The unit test LocateModuleCallbackTest.cpp is very flaky and is failing on the lldb arm64 incremental bot

Notice build https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/2259/ passes
and https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/2260/ fails, even though the changes in 2260 have nothing to do with lldb.

rastogishubham added a comment.EditedJul 14 2023, 8:04 AM

I am reverting both fb087c17c8230 and 7f1028e9df52b to see if the bot becomes green again. I apologize for any incovenience

I am reverting both fb087c17c8230 and 7f1028e9df52b to see if the bot becomes green again. I apologize for any incovenience

Okay, so reverting those two changes cause other build failures, and I don't think it is a good idea to revert a bunch of changes, I am not sure what a good way to disable the test for arm64 is? Since these failures only appear on arm64

Would it be just adding a

#if !defined(__aarch64__)

to the top of the file work?

@rastogishubham let me check.

btw I'll commit D155124 to fix Windows tests.

ok, sounds like ASSERT_EQ FileSpec does not work on Linux aarch64.
will use FileSpec::Compare and double check on Linux aarch64.

D155333 should fix the flaky arm64 tests by replacing ASSERT_EQ(FileSpec, FileSpec) with ASSERT_EQ(FileSpec::Compare(FileSpec, FileSpec, true), 0).