Update patch to not use malloc/free for a temporary heap allocation; a std::string works fine. One caveat is that the c-strings we read in may have a nul byte terminator, and left alone the std::string will consider that part of the string; that can be preserved when using a string_view etc representation.

All of the feedback is addressed at this point. I had one decision I didn't like - the LC_NOTE has an explicit size in the load command, but I said that the JSON string needed to be nul byte ('\0') terminated, which is redundant. I've changed the implementation to (1) in the producer side, don't add a nul byte at the end of the string, and (2) in the consumer side, add the nul byte to the read buffer, before treating it as a c-string. To help emphasize that the size of the LC_NOTE is sufficient.

LGTM this is just mechanical passing around of the user's setting though the layers.

Oh, and I did change the LC_NOTE name that I'm adding to "process metadata" and specified that it *may* contain a threads key, instead of *shall* contain a threads key.

Address feedback from Alex and Jonas. Most significantly, add a ObjectFileMachO::FindLC_NOTEByName method which all the LC_NOTE readers in ObjectFileMachO call. One of them specifically needs to iterate over multiple LC_NOTEs with the same name ("load binary"), so it returns a vector of offset & sizes of matching LC_NOTEs which looks a little unusual for the "there will only be one of these" LC_NOTEs. I also simplified the tid_t creation for missing tid_t values in ProcessMachCore when only some threads were given specified tid_t values.

Thanks for the feedback Alex & Jonas. Jonas' comment that we have a lot of different methods manually stepping over load commands to find their LC_NOTEs was something I'd been meaning to deal with some day, but hadn't done yet. I shouldn't complicate this patch by also doing that but ... I did. Updating the patch in a minute.

Thanks for the feedback @bulbazord & @JDevlieghere , I will address those points.

One interesting question about the basic design of this LC_NOTE. If I call it "thread extrainfo" and it is a JSON dictionary with a threads array -- how long before someone wants to stick some non-thread specific piece of data in here. Should it be a proc metadata with a threads array, and it doesn't become confusing as it is extended over years.

After thinking about this more, and talking with @jingham I rewrote the patch so LC_NOTE "thread extrainfo" is a JSON dictionary with a key threads that has an array. The number of entries in the array must match the number of LC_THREADs in the Mach-O corefile. Each array entry may have a thread_id key with a thread id for that LC_THREAD, or lldb will create a thread_id for it. I expect we will add more per-thread keys in the future.

As soon as I started thinking about thread names and queue names and mach exception data, all variable length things, a binary format doesn't seem ideal. We do already send a lot of thread information about processes in JSON in gdb remote serial protocol from debugserver today. It's the first time we'd use JSON in a Mach-O LC_NOTE though, so I'm interested to hear other people's thoughts.

Or possibly we should just make "thread extrainfo" a JSON blob with a dictionary for each thread, which would make it easier to extend, instead of a binary format. I'm not sure I have a strong opinion.

LGTM, fwiw.

In D157764#4608265, @wallace wrote:

@jasonmolenda , yep. I have already reverted this patch. I'll figure out how to make that work nicely with these changes.

@wallace this is breaking TestIRInterpreter.py on macOS now; https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/ has been failing since it went in (there was another llvm issue causing failures right before this went in, so it was easy to miss). On my desktop this change results in TestIRInterpreter.py running without end; I think the bots kill tests that run for more than 5 minutes.

Hi, this is causing a regression on the greendragon x86_64 lldb bot, it took a little while to repo and narrow down today. https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/ is failing on TestDequeFromStdModule.py, I can repo it on an intel mac. We're crashing under

I'm going to land this one sans-approval to fix the CI bot. but I still think

self.runCmd("settings set target.process.extra-startup-command QSetLogging:bitmask=LOG_PROCESS|LOG_EXCEPTIONS|LOG_RNB_PACKETS|LOG_STEP;")

Looks good!

Updated back to my most recent version of this patch, thanks for the help all.

Update patch to remove the HasValue/IsValue and Clear methods from AddressableBits. Change methods that may fill in an AddressableBits object with values to return an AddressableBits object unconditionally. Change AddressableBits::SetProcessMasks so it can be called unconditionally, and it only sets mask values in Process if values were set.

ah dangit, I meant to update the patch in https://reviews.llvm.org/D158041 and accidentally blew away the last patch I had for this one. let's see if I can extract it from here somehow.

Update patch to remove the HasValue/IsValue and Clear methods from AddressableBits. Change methods that may fill in an AddressableBits object with values to return an AddressableBits object unconditionally. Change AddressableBits::SetProcessMasks so it can be called unconditionally, and it only sets mask values in Process if values were set.

Update patch to change the AddressableBits::IsValid to AddressableBits::HasValue which is a clearer description. The IsValid methods are used across the lldb_private classes everywhere. I tried changing this to operator bool but I thought that was less clear in use. I compromised by changing it to HasValue - this is a POD class so "IsValid" seemed a little questionable, but I didn't like operator bool. Let's try this out.

Incorporate Jonas' suggestion of a AddressableBits class that GDBRemoteCommunicationClient and ObjectFileMachO could use to store the zero/one/two addressable bits values back to a Process, and centralizing the logic for how those 0-2 values are used to set the Process address masks.

Fix size check when making sure there's enough data.

Incorporate Alex's suggestions, not sure about using llvm-dwarfdump yet tho.

Fix the problems Alex found on review, thanks Alex.

The lldb side looks good to me, and I think you've addressed the llvm parts.

update comments a tiny bit.

One more bit of explanation about the changes: There is an SBTarget::AddModule which takes individual parts of a module spec, and an SBTarget::AddModule which takes an SBModuleSpec. I was going to need to duplicate my code to force the expensive search & update the Target's arch if it didn't have one in both, so I had the first method create an SBModuleSpec with its information and call the second.

Maybe

const char *ent_name = 
#if TARGET_OS_OSX
"com.apple.security.get-task-allow";
#else
"get-task-allow";
#endif

Maybe get-task-allow is allowed (lol) on macOS too.

I'm fine with this but on macOS I believe it's com.apple.security.get-task-allow v. "Entitlements on macOS"
https://developer.apple.com/documentation/technotes/tn3125-inside-code-signing-provisioning-profiles

Rebase patch on current top of tree sources.

Rebase on current TOT sources, and add a comment to the ProcessMachCore::LoadBinariesViaMetadata prototype in the header documenting the meaning of the return value, I agree with Alex I should make this a little clearer.

Update patch to send these error messages to the ErrorStream not OutputStream. Also change LocateSymbolFileMacOSX.cpp so the full command that was used to find the binary is included in the error message, to make it clearer exactly where the error message originated.

fix typeo

Change calls to Log::Printf to use LLDB_LOGF().

Fix my script print's to just print, I can't repo a buffering problem that I thought I had seen in the past.

Incorporate Jonas' feedback about the naming of the new DynamicLoader method argument.