This is an archive of the discontinued LLVM Phabricator instance.

Renamed Target::GetSharedModule to AddModule, allow for ModulesDidLoad to be delayed when batch adding Modules
ClosedPublic

Authored by jasonmolenda on Apr 2 2019, 7:49 PM.

Details

Summary

I'm addressing a perf issue where DynamicLoaderDarwin has been notified that a batch of solibs have been loaded. It adds them to the target one-by-one with Target::GetSharedModule(), and then calls Target::ModulesDidLoad() to give breakpoints a chance to evaluate for new locations, etc. One of the things we do on ModulesDidLoad is call ProcessGDBRemote::ModulesDidLoad which will send the qSymbol packet to the gdb remote serial protocol stub if it indicates it wants to know a symbol address. Currently, because Target::GetSharedModule() calls ModulesDidLoad, we will send the qSymbol packet many times - an app with hundreds of solibs are not unusual.

This patch renames Target::GetSharedModule to Target::AddModule() which better reflects what it actually does -- given a ModuleSpec set of criteria, it finds that binary on the local system and adds it to the Target, if it isn't already present. This method name has confused all of us at one point or another. As DynamicLoaderWindowsDYLD notes,

// Confusingly, there is no Target::AddSharedModule.  Instead, calling
// GetSharedModule() with a new module will add it to the module list and
// return a corresponding ModuleSP.

It adds a flag to Target::AddModule to suppress notification of the new Module (i.e. don't call ModulesDidLoad) - if the caller does this, the caller must call Target::ModulesDidLoad before resuming execution. I added a description of the method in Target.h to make this clear.

I also had to add flag to ModuleList::Append and ModuleList::AppendIfNeeded. I made these have default values because many uses of this are in a loop creating a standalone ModuleList, and forcing all of them to specify the notify boolean was nonintuitive for those users. When a ModuleList is a part of the Target, it has notifier callback functions, but when it is a standalone object, it doesn't.

I'm trying to think of how to test this -- but the problem I'm directly trying to address, the qSymbol packet being sent for every solib, instead of the # of groups of solib's that are loaded, is something we don't track today. I'll continue to think about it.

rdar://problem/48293064

Diff Detail

Repository
rLLDB LLDB

Event Timeline

jasonmolenda created this revision.Apr 2 2019, 7:49 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 2 2019, 7:49 PM
Herald added a subscriber: abidh. · View Herald Transcript
clayborg added inline comments.Apr 2 2019, 8:55 PM
include/lldb/Target/Target.h
535

Pavel had questions about this error. If we specify an error when we call this, is there a way to get a valid module shared pointer back and still get an error? Maybe this should be one of the llvm::ErrorOr return types?

source/Commands/CommandObjectTarget.cpp
399–400

Remove all "const bool notify = true; "statements and Inline with comment?

ModuleSP module_sp = target_sp->AddModule(main_module_spec, 
                                          true /*notify*/);

This would apply everywhere in this patch if so.

jasonmolenda marked 2 inline comments as done.Apr 2 2019, 9:15 PM

Thanks for looking the patch over.

include/lldb/Target/Target.h
535

There's only a handful of AddModule callers that pass in a Status object - the Windows ones, the CommandObjectTarget command, and the Minidump. I'll look through the Platform GetSharedModules implementations tomorrow to see who might put useful things in the error object, but in general either AddModule finds a binary that matches the ModuleSpec, and returns a shared pointer to it, or finds nothing and returns an empty shared pointer. The error object may have a message from a Platform indicating why the search failed ("could not find any SDK directories" or something) but I suspect this optional Status arg isn't doing anything.

source/Commands/CommandObjectTarget.cpp
399–400

Can do. I don't have a preference.

labath added a subscriber: labath.Apr 3 2019, 12:52 AM

Thank you for taking the time to rename this function. I think this has been long overdue. However, I believe that the name AddModule might cause confusion coming from the other direction, as now one may think that it unconditionally adds a new module to the target (which also isn't true, as it will just reuse the existing one if it's already there). Maybe something like GetOrCreateModule would be better (and also in line with how llvm names these kinds of things)?

As for testing, doesn't this cause a eBroadcastBitModulesLoaded event to be sent through some SB API. You could subscribe to that and check that it comes only ~once, and contains the right modules. This won't test exactly the thing you're interested in, but it might be close enough.

OK looked things over a bit.

I think there's a case for adopting llvm::ErrorOr here, and elsewhere across, but today the only place we're using that return type is when the error is encoded in a std::error_code (aka fancy errno). We should have an ErrorOr that uses a Status object to contain the error, I think that's possible? I've seen people use their own objects to return errors with llvm::Expected. In any event, I think a change like this is a little out of scope for what I'm doing in this patch right now.

After spending time looking at all the error handling codepaths related to AddOrCreateModule, if we simply fail to find a binary we return an empty ModuleSP and the platform may set a Status with an error string (e.g. PlatformPOSIX will say no such file) if that's non-nullptr. Or there may be a genuine error - a Target that doesn't have a Platform, we found a binary but it is an explicitly disallowed type (e.g. eTypeStubLibrary, a long-obsolete binary format on Darwin), we can return an error message explaining that to the caller if the Status ptr is non-nullptr.

The important wrinkle with AddOrCreateModule is that most of the callers are deep inside lldb internal mechanisms -- the DynamicLoaders. And for most of those, we're going to handle file-not-found in custom ways, we're not going to relay the error messages. For instance, the user-process DynamicLoaderDarwin, when it fails to find a file, it will call Process::ReadModuleFromMemory(). If the kernel debugging DynamicLoaderDarwinKernel fails to load a kext (a solib), it accumulates a list of kexts that it could not find, and at the end it will present a sorted list of them. These callers have no interest in the returned error message.

On the other hand, a caller like CommandObjectTarget, which is directly driven by user interaction, we must print the error message ("file not found" and such).

So in this particularly API, because only some callers want the error strings, leaving this as an optional argument seems reasonable. An llvm::ErrorOr which returns a ModuleSP or a Status would be a fine - and only the callers that want to print the error message would retrieve the Status object, the others would do their normal behavior when couldn't get the ModuleSP.

Remove const bool notify's. Rename method to Target::GetOrCreateModule. Refine the method headerdoc a bit.

labath added a comment.Apr 4 2019, 2:27 AM

To me, the greatest advantage in using Expected<T> (ErrorOr<T> is the old way of doing things; I wouldn't use that in new code) is greater clarity in what are the possible results of the function. If the function returns the object and the error in two pieces, then this leaves place for uncertainties like "but what is the state of the T object if the function returned an error", or "if the T is invalid (null), can I assume the error object will indicate failure too", etc. Using Expected<T> removes all of these, because if the function returns an error, there is no T object to speak of and vice-versa. (Ok, if T = std::shared_ptr<U>, then this still leaves the possibility of the function returning "success", but with an empty pointer; however, that's still a much smaller problem space).

Whether the caller is able to do something with that error is not that important here. If he doesn't want it, he can always explicitly ignore it (or maybe log it?). So, I would say that changing this function to use Expected<T> is definitely a good thing, but it doesn't have to happen in this patch.

BTW, if you have a Status object, you can convert it to an Expected<T> (via llvm::Error) by calling ToError on it. Also, you don't have to define your own error type if you just want to return some string. StringError (llvm::createStringError) should suffice for that.

Thanks Pavel, I'll convert this to use Expected<> while I'm working on it. I'm writing a test case right now, and looking at how DynamicPluginDarwin works more closely for adding & removing binaries, e.g. there's also no way to batch remove libraries and have a group broadcast notification be sent. There was also a bug in my original patch for calling LoadScriptingResourceForModule() to find a python resource file in a dSYM bundle - it is currently called from Target::ModuleUpdated when a binary is added to the target's ModuleList, but I'm suppressing that call with this change, so I need to move that over to ModulesDidLoad.

Thanks Pavel, I'll convert this to use Expected<> while I'm working on it. I'm writing a test case right now, and looking at how DynamicPluginDarwin works more closely for adding & removing binaries, e.g. there's also no way to batch remove libraries and have a group broadcast notification be sent. There was also a bug in my original patch for calling LoadScriptingResourceForModule() to find a python resource file in a dSYM bundle - it is currently called from Target::ModuleUpdated when a binary is added to the target's ModuleList, but I'm suppressing that call with this change, so I need to move that over to ModulesDidLoad.

Sounds good to me.

Added a testcase, TestModuleLoadedNotifys.

Renamed the pure virtual notifier methods in ModuleList to add 'Notify' at the start of their names, updated the names in subclass Target as well. Added a NotifyModulesRemoved and changed ModuleList::Remove(ModuleList&) so that it removes all of the modules one-by-one from the collection and then calls NotifyModulesRemoved() so they are broadcast as a group. I renamed the methods to make it clearer in Target that these are subclass implementations.

Changed Target::SetExecutableModule() so that it will load all dependent libraries without notifying, and send a group notification about all of them once they've been added.

Moved the call to LoadScriptingResourceForModule from Target::NotifyModuleAdded to Target::ModulesDidLoad so DynamicLoader plugins can call ModulesDidLoad with the full list of libraries and still get scripting files loaded.

I tried changing Target::GetOrCreateModule to return Expected<ModuleSP> but I wasn't thrilled with the double-dereference of the returned value when I started updating callers. It wasn't horrible, but it looked messy. I should go back and try again but my first shot at a few of them didn't look great.

I want to hand-test the scripting resource loading a bit on Monday but this is probably about what I'm doing for now. At this point, DynamicLoaderDarwin sends batched notifications on the CreateTarget() binary dependent binaries. It sends a batched notification when we start a real process and throw away all those binaries. It sends out batched notifications when we load the binaries back in via dyld notifications.

This revision was not accepted when it landed; it landed in state Needs Review.Apr 8 2019, 4:01 PM
This revision was automatically updated to reflect the committed changes.