Complete support of loading a darwin kernel over a live gdb-remote connection given the address of a mach-o fileset

[jasonmolenda]

Summary

In an internal development environment, lldb will attach to a gdb stub that can provide the address of a Mach-O fileset including the kernel binary. I added some preliminary support for the binary-addresses key in qProcessInfo in https://reviews.llvm.org/D130813; this patch fixes that support now that we have a live stub to run against, and passes the addr_t it gets through the PlatformList to see if any Platform knows how to handle it.

PlatformDarwinKernel will test the addr_t to see if it is a Mach-O Fileset, using Jonas' mach-o fileset plugin (https://reviews.llvm.org/D132433 et al). If it is a fileset, it will search for a com.apple.kernel entry, and if found, it will set the Process DynamicLoader to be DynamicLoaderDarwinKernel, and register the address of the com.apple.kernel entry in the fileset, to be loaded later by the DynamicLoader plugin.

The big trick with this is that I needed to do several Darwin specific things -- parse a Mach-O fileset in memory, detect if this is a Darwin kernel, set the Process DynamicLoader -- when I'm given only an addr_t in ProcessGDBRemote. Going through the Platform plugins and asking them if they can do these things given the addr_t was a way of accomplishing this from generic code.

This does mean that I introduced cross-platform dependencies in PlatformDarwinKernel. It needs to directly create a ObjectContainerMachOFileset object to use the FindEntry methods which are only available to the subclass. I need to instantiate a DynamicLoaderDarwinKernel object and set the Process to use it as the m_dyld. I like isolating all of this very-darwin-specific logic down in PlatformDarwinKernel, but I didn't see a good way of working around the dependencies.

Testing this is a trick I haven't figured out yet; I'd need a remote stub that will return the qProcessInfo binary-addresses key, have a Mach-O fileset there with a com.apple.kernel entry pointing to a binary that is structured like a Darwin kernel binary enough to trick lldb into loading it. It's possible to hand-test against a live environment, but faking enough of that to create a test would be a lot.

There will be a second patch dealing with corefiles, where I ended up making a lot of changes to ProcessMachCore so it's a rather large diff, and it was possible to separate the two halves, so I did.

Any feedback appreciated, I've been working on this one for a little bit now & a fresh set of eyes is welcome.

[jingham]

So far just a bunch of random comments...

lldb/include/lldb/Target/Platform.h

1055

Are there any restrictions on when in the process lifecycle this should be called? If so it would be good to call that out.

lldb/source/Core/DynamicLoader.cpp

191

This isn't your addition, but it seems dangerous to assume process will always be non-null here...

233

It seems weird that we would get all the way here, with a live process and everything, but not have a target architecture. How does that happen?

lldb/source/Plugins/DynamicLoader/Darwin-Kernel/DynamicLoaderDarwinKernel.cpp

699–700

This second test is redundant, isn't it?

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.cpp

747

Is this comment accurate anymore? You only get to this code if module_spec.GetUUID is NOT valid, in which case DebugSymbols isn't going to find anything.

928

Do you really need to pass in a module_sp with a pretty much totally bogus module in it? That seems weird to me. The Module constructor does a bunch of work, all of which is going to fail with an empty ModuleSpec, so at some point CreateMemoryInstance is going to replace whatever is in this module_sp with something real. So what's the point of passing it something it's going to throw away?

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp

2214

llvm::to_integer(..., 16) doesn't handle strings that start with 0x? That seems weak.

lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp

597

I'm a little surprised we don't have a platform at this point?

What happens if we had a Platform at this point, but it wasn't the one that knew how to load this binary? That seems awkward.

[jasonmolenda]

Thanks Jim! Really helpful feedback and questions.

lldb/source/Core/DynamicLoader.cpp

191

Thanks, I don't even know why this is here, I don't seem to be calling the platform anywhere.

233

I hit this with a macho corefile debugging case where the Target arch *could* have been set by the corefile's cputype / cpusubtype, but it hadn't been yet (and that may not be the most accurate arch), and this crashed when it created a DataExtractor and tried to get addrsize. It's pretty early in the target's lifetime, so very little has been set up; it seemed simplest to make sure there's an architecture for the Target before I run code that I know may need it.

lldb/source/Plugins/DynamicLoader/Darwin-Kernel/DynamicLoaderDarwinKernel.cpp

699–700

This code is really not filling me with joy, I'm sure I wrote it a decade+ ago myself. The idea was that once I know the actual running UUID of the kernel in this process, I want to remove any non-matching modules that may have been added manually by a user specifying the wrong kernel binary. Otherwise we have extra kernel symbols loaded and symbol name lookups can get the wrong one. Let me rewrite this to do a clearer pass over all the Target's modules and remove any kernels with non-matching UUIDs, instead of just doing this with the 0th module in the list (the "executable module").

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.cpp

747

The old code behaved this way -- if the FileSpec in the ModuleSpec was empty or we had no UUID, we would call into the more generic PlatformDarwin::GetSharedModule. If I had a FileSpec, if it was the special name "mach_kernel", I would look in the list of local mach kernels I'd found by scanning; else it was a kext BundleID and I would look up the kexts I scanned locally by that bundle ID.

In the new code path, I might not be able to set "mach_kernel" filename for the kernel because I might only have gotten it by a bare address. So now I will check the UUIDs of all the kernels on the local system for a match. Else I'll try the kexts by bundle ID. If the "mach_kernel"/empty name finds a module, we return it directly. Kexts return directly.

Then we fall through to calling Platform::GetSharedModule with whatever we had in the ModuleSpec -- a FileSpec maybe, a UUID maybe. I believe this will be correct behavior.

928

Yeah, this is how the CreateMemoryInstance for ObjectFiles work too - Process::ReadModuleFromMemory creates a Module, possibly adding an ObjectFile from memory, and returns the ModuleSP. In my case here, I'm not going to use/return the ObjectContainer being constructed, but Jonas' ObjectContainerMachOFileset::CreateMemoryInstance requires a Module that it locks before starting its operations, and adds an ObjectFile to it. I think this is an artifact of following the ObjectFile API pattern, I checked with Jonas and he didn't want to deviate.

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp

2214

Looking at llvm::consumeUnsignedInteger in lib/Support/StringRef.cpp it will call GetAutoSenseRadix() if a radix isn't specified, it will see & advance past "0x" and work. But if you also want to accept base16 numbers that aren't 0x prefixed, you have to specify the radix to llvm::to_integer() and the GetAutoSenseRadix() won't be called. It only does one or the other, it doesn't accept a mix.

lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp

597

We do have a platform, but it's a Host platform. We've just connected to a gdb remote serial protocol port -- it could be debugserver, it could be a jtag device, it could be a kernel debugger. Once we get a real binary, we can start figuring out the best platform and dynamic linker, but we're starting with almost nothing here, so given this special binary address, we need to ask any of the platform plugins if they know what to do with it.

[jasonmolenda]

Update the patch to address Jim's comments/suggestions. Biggest change was to clean up how DynamicLoaderDarwinKernel filters out any kernels in the Target that don't match the UUID of the actually running kernel. There was a vague attempt at this before, checking the target's ExecutableModule, but that assumes there was only one module loaded and it was the wrong kernel. Easier to step through everything loaded in the target and eliminate these, and clearer what's being done. I had the same sequence for detecting if a module is a kernel in a bunch of places so I added module_is_a_kernel() as a static function there.

I also renamed the method in Platform and PlatformList to LoadSpecialBinaryAndSetDynamicLoader(), to help make it clear that this is an uncommon use case, so someone doesn't try to use it for loading general binaries.

[labath]

lldb/unittests/Interpreter/CMakeLists.txt

17–18 ↗

(On Diff #458970)

These dependencies should be added to the PlatformDarwinKernel CMakeLists.txt, not to the every file that depends on it.

[JDevlieghere]

The current patch didn't have context so I just left a bunch of nits.

lldb/include/lldb/Target/Platform.h

870–874

What makes the binary special? The comment above doesn't say. Can we give this a more descriptive name or omit the "Special" part?

lldb/source/Plugins/DynamicLoader/Darwin-Kernel/DynamicLoaderDarwinKernel.cpp

133

This seems needless verbose: is_kernel conveys the same information.

574–577

m_kernel_image = is_kernel(module_sp.get())

668–669

is_kernel = is_kernel(memory_module_sp.get())

747–752

The ModuleList has a Modules() function that returns a locking iterator. You can rewrite this as:

for(ModuleSP module : target.GetImages().Modules()) {
  ...
}

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.cpp

929

ModuleSP module_sp = std::make_shared<Module>()

970

if(platform_sp)

973–974

std::make_shared

[jasonmolenda]

Update patch to include more surrounding context.

[jasonmolenda]

lldb/include/lldb/Target/Platform.h

870–874

I'm still not happy with this method name, I agree this name isn't clear. I'll try to think of something better.

lldb/source/Plugins/DynamicLoader/Darwin-Kernel/DynamicLoaderDarwinKernel.cpp

133

Good point, will do.

668–669

ah yes, it made more sense previously when the conditional expression was larger.

747–752

Ach, thanks, missed that.

lldb/unittests/Interpreter/CMakeLists.txt

17–18 ↗

(On Diff #458970)

Thanks Pavel, I should have thought of that.

[jasonmolenda]

Update patch to incorporate Jonas & Pavel's feedback.

[JDevlieghere]

A small comment but otherwise this LGTM.

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.cpp

924

Looks like we're never checking this status below. Is there any chance that ReadMemory succeeds but the status is set to an error? If not, should we add an assert?

lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp

597–601

This is the crux of the patch. I think it would be helpful here to explain what's going on, i.e. that we give the platform a chance first, before deferring to the DynamicLoader.

[jasonmolenda]

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.cpp

924

Good point. Process::ReadMemoryFromInferior returns the number of bytes read, and a Status object. So if were were only able to read part of the mach header object, this would not return an error condition. (and we'd have uninitialized fields and could try to read too many load commands or something)

[thakis]

lldb/source/Plugins/Platform/MacOSX/CMakeLists.txt

47

This causes a dependency cycle:

//lldb/source/Plugins/Platform/MacOSX:MacOSX ->
//lldb/source/Plugins/DynamicLoader/Darwin-Kernel:Darwin-Kernel ->
//lldb/source/Plugins/Platform/MacOSX:MacOSX

[jasonmolenda]

lldb/source/Plugins/Platform/MacOSX/CMakeLists.txt

47

Ach, naturally. DynamicLoaderDarwinKernel has to create a PlatformDarwinKernel to set in the Target when it is initializing itself. :/ Maybe I'll just add DynamicLoaderDarwinKernel to the unit tests that have PlatformMacOSX in them.

[jasonmolenda]

lldb/source/Plugins/Platform/MacOSX/CMakeLists.txt

47

I removed the dependency in DynamicLoaderDarwinKernel, a very specialized plugin, and left the dependency in PlatformMacOSX which includes all of the darwin platforms and is a common one to import. I believe any target that is linking against DynamicLoaderDarwinKernel will also have a dependency on PlatformMacOSX already. I landed this as 30578c08568bc8de79dea72e41f49899ba10ea55 to make sure this causes no problems, we can fix it better if someone has a suggestion.

[thakis]

lldb/source/Plugins/Platform/MacOSX/CMakeLists.txt

47

Thanks!

I believe anything linking the darwin kernel DynamicLoader plugin
will already have lldbPluginPlatformMacOSX in its dependency list,
so not explicitly expressing this dependency is safe.

That sounds like it doesn't really remove the dependency, it just lies about the build system about it and things still happen to work :)

Normally, if you have a cycle between libraries A and B, you'd either:

• decide which of the two should depend on the other
• make the "lower" library expose some virtual delegate
• implement that in the "higher" library

…or move the needed shared bits to an even lower library that both A and B naturally depend on already.

https://chromium.googlesource.com/chromium/src/+/lkgr/docs/patterns/inversion-of-control.md has some (slightly, but not overly so) chromium-specific notes on this.

But layering in lldb is generally a mess, so I think your workaround is ok for now. It'd be nice to clean up lldb's layering at some point!

lldb/include/lldb/Target/Platform.h

public:
   ///     information type as the entry key and the matching  an array as the
   ///     entry value. \b nullptr if not implemented or  if the process has no
   ///     crash information entry. \b error if an error occured.
   virtual llvm::Expected<StructuredData::DictionarySP>
   FetchExtendedCrashInformation(lldb_private::Process &process) {
     return nullptr;
   }
 
+  /// Detect a binary in memory that will determine which Platform and
+  /// DynamicLoader should be used in this target/process, and update
+  /// the Platform/DynamicLoader.
+  /// The binary will be loaded into the Target, or will be registered with
+  /// the DynamicLoader so that it will be loaded at a later stage.  Returns
+  /// true to indicate that this is a platform binary and has been
+  /// loaded/registered, no further action should be taken by the caller.
+  ///
+  /// \param[in] process
+  ///     Process read memory from, a Process must be provided.
+  ///
+  /// \param[in] addr
+  ///     Address of a binary in memory.
+  ///
+  /// \param[in] notify
+  ///     Whether ModulesDidLoad should be called, if a binary is loaded.
+  ///     Caller may prefer to call ModulesDidLoad for multiple binaries
+  ///     that were loaded at the same time.
+  ///
+  /// \return
+  ///     Returns true if the binary was loaded in the target (or will be
+  ///     via a DynamicLoader).  Returns false if the binary was not
+  ///     loaded/registered, and the caller must load it into the target.
+  virtual bool LoadPlatformBinaryAndSetup(Process *process, lldb::addr_t addr,
+                                          bool notify) {
+    return false;
+  }
+
   virtual CompilerType GetSiginfoType(const llvm::Triple &triple);
   
   virtual Args GetExtraStartupCommands();
 
 protected:
   /// Create a list of ArchSpecs with the given OS and a architectures. The
   /// vendor field is left as an "unspecified unknown".
   static std::vector<ArchSpec>
public:
   /// If none of the above apply, this function returns a default platform. The
   /// candidates output argument differentiates between either no platforms
   /// supporting the given architecture or multiple platforms supporting the
   /// given architecture.
   lldb::PlatformSP GetOrCreate(llvm::ArrayRef<ArchSpec> archs,
                                const ArchSpec &process_host_arch,
                                std::vector<lldb::PlatformSP> &candidates);
 
   lldb::PlatformSP Create(llvm::StringRef name);
 
+  /// Detect a binary in memory that will determine which Platform and
+  /// DynamicLoader should be used in this target/process, and update
+  /// the Platform/DynamicLoader.
+  /// The binary will be loaded into the Target, or will be registered with
+  /// the DynamicLoader so that it will be loaded at a later stage.  Returns
+  /// true to indicate that this is a platform binary and has been
+  /// loaded/registered, no further action should be taken by the caller.
+  ///
+  /// \param[in] process
+  ///     Process read memory from, a Process must be provided.
+  ///
+  /// \param[in] addr
+  ///     Address of a binary in memory.
+  ///
+  /// \param[in] notify
+  ///     Whether ModulesDidLoad should be called, if a binary is loaded.
+  ///     Caller may prefer to call ModulesDidLoad for multiple binaries
+  ///     that were loaded at the same time.
+  ///
+  /// \return
+  ///     Returns true if the binary was loaded in the target (or will be
+  ///     via a DynamicLoader).  Returns false if the binary was not
+  ///     loaded/registered, and the caller must load it into the target.
+  bool LoadPlatformBinaryAndSetup(Process *process, lldb::addr_t addr,
+                                  bool notify);
+
 protected:
   typedef std::vector<lldb::PlatformSP> collection;
   mutable std::recursive_mutex m_mutex;
   collection m_platforms;
   lldb::PlatformSP m_selected_platform_sp;
 
 private:
   PlatformList(const PlatformList &) = delete;

lldb/include/lldb/Target/Process.h

/// UnregisterNotificationCallbacks (const Notifications&)
   ///
   /// The default action is to let the DynamicLoader plug-ins check the main
   /// executable and the DynamicLoader will select itself automatically.
   /// Subclasses can override this if inspecting the executable is not
   /// desired, or if Process subclasses can only use a specific DynamicLoader
   /// plug-in.
   virtual DynamicLoader *GetDynamicLoader();
 
+  void SetDynamicLoader(lldb::DynamicLoaderUP dyld);
+
   // Returns AUXV structure found in many ELF-based environments.
   //
   // The default action is to return an empty data buffer.
   //
   // \return
   //    A data extractor containing the contents of the AUXV data.
   virtual DataExtractor GetAuxvData();
 

lldb/source/Core/DynamicLoader.cpp

 
 ModuleSP DynamicLoader::LoadBinaryWithUUIDAndAddress(Process *process,
                                                      UUID uuid, addr_t value,
                                                      bool value_is_offset,
                                                      bool force_symbol_search,
                                                      bool notify) {
   ModuleSP memory_module_sp;
   ModuleSP module_sp;
   PlatformSP platform_sp = process->GetTarget().GetPlatform();
   Target &target = process->GetTarget();
   Status error;
   ModuleSpec module_spec;
   module_spec.GetUUID() = uuid;
 
   if (!uuid.IsValid() && !value_is_offset) {
     memory_module_sp = ReadUnnamedMemoryModule(process, value);
 
if (!module_sp.get() && value != LLDB_INVALID_ADDRESS && !value_is_offset) {
     if (!memory_module_sp)
       memory_module_sp = ReadUnnamedMemoryModule(process, value);
     if (memory_module_sp)
       module_sp = memory_module_sp;
   }
 
   Log *log = GetLog(LLDBLog::DynamicLoader);
   if (module_sp.get()) {
+    // Ensure the Target has an architecture set in case
+    // we need it while processing this binary/eh_frame/debug info.
+    if (!target.GetArchitecture().IsValid())
+      target.SetArchitecture(module_sp->GetArchitecture());
     target.GetImages().AppendIfNeeded(module_sp, false);
 
     bool changed = false;
     if (module_sp->GetObjectFile()) {
       if (value != LLDB_INVALID_ADDRESS) {
         LLDB_LOGF(log, "Loading binary UUID %s at %s 0x%" PRIx64,
                   uuid.GetAsString().c_str(),
                   value_is_offset ? "offset" : "address", value);

lldb/source/Plugins/DynamicLoader/Darwin-Kernel/DynamicLoaderDarwinKernel.cpp

public:
   }
 };
 
 static DynamicLoaderDarwinKernelProperties &GetGlobalProperties() {
   static DynamicLoaderDarwinKernelProperties g_settings;
   return g_settings;
 }
 
+static bool is_kernel(Module *module) {
+  if (!module)
+    return false;
+  ObjectFile *objfile = module->GetObjectFile();
+  if (!objfile)
+    return false;
+  if (objfile->GetType() != ObjectFile::eTypeExecutable)
+    return false;
+  if (objfile->GetStrata() != ObjectFile::eStrataKernel)
+    return false;
+
+  return true;
+}
+
 // Create an instance of this class. This function is filled into the plugin
 // info class that gets handed out by the plugin factory and allows the lldb to
 // instantiate an instance of this class.
 DynamicLoader *DynamicLoaderDarwinKernel::CreateInstance(Process *process,
                                                          bool force) {
   if (!force) {
     // If the user provided an executable binary and it is not a kernel, this
     // plugin should not create an instance.
-    Module *exe_module = process->GetTarget().GetExecutableModulePointer();
-    if (exe_module) {
-      ObjectFile *object_file = exe_module->GetObjectFile();
-      if (object_file) {
-        if (object_file->GetStrata() != ObjectFile::eStrataKernel) {
-          return nullptr;
-        }
-      }
-    }
+    if (!is_kernel(process->GetTarget().GetExecutableModulePointer()))
 
     // If the target's architecture does not look like an Apple environment,
     // this plugin should not create an instance.
     const llvm::Triple &triple_ref =
         process->GetTarget().GetArchitecture().GetTriple();
     switch (triple_ref.getOS()) {
     case llvm::Triple::Darwin:
     case llvm::Triple::MacOSX:
default:
       return nullptr;
       break;
     }
   }
 
   // At this point if there is an ExecutableModule, it is a kernel and the
   // Target is some variant of an Apple system. If the Process hasn't provided
   // the kernel load address, we need to look around in memory to find it.
-
   const addr_t kernel_load_address = SearchForDarwinKernel(process);
   if (CheckForKernelImageAtAddress(kernel_load_address, process).IsValid()) {
     process->SetCanRunCode(false);
     return new DynamicLoaderDarwinKernel(process, kernel_load_address);
   }
   return nullptr;
 }
 
 lldb::addr_t
 DynamicLoaderDarwinKernel::SearchForDarwinKernel(Process *process) {
   addr_t kernel_load_address = process->GetImageInfoAddress();
-  if (kernel_load_address == LLDB_INVALID_ADDRESS) {
+  if (kernel_load_address == LLDB_INVALID_ADDRESS)
     kernel_load_address = SearchForKernelAtSameLoadAddr(process);
-    if (kernel_load_address == LLDB_INVALID_ADDRESS) {
-      kernel_load_address = SearchForKernelWithDebugHints(process);
-      if (kernel_load_address == LLDB_INVALID_ADDRESS) {
-        kernel_load_address = SearchForKernelNearPC(process);
-        if (kernel_load_address == LLDB_INVALID_ADDRESS) {
-          kernel_load_address = SearchForKernelViaExhaustiveSearch(process);
-        }
-      }
-    }
-  }
+  if (kernel_load_address == LLDB_INVALID_ADDRESS)
+  if (kernel_load_address == LLDB_INVALID_ADDRESS)
+  if (kernel_load_address == LLDB_INVALID_ADDRESS)
+
   return kernel_load_address;
 }
 
 // Check if the kernel binary is loaded in memory without a slide. First verify
 // that the ExecutableModule is a kernel before we proceed. Returns the address
 // of the kernel if one was found, else LLDB_INVALID_ADDRESS.
 lldb::addr_t
 DynamicLoaderDarwinKernel::SearchForKernelAtSameLoadAddr(Process *process) {
   Module *exe_module = process->GetTarget().GetExecutableModulePointer();
-  if (exe_module == nullptr)
-    return LLDB_INVALID_ADDRESS;
 
-  ObjectFile *exe_objfile = exe_module->GetObjectFile();
-  if (exe_objfile == nullptr)
+  if (!is_kernel(process->GetTarget().GetExecutableModulePointer()))
     return LLDB_INVALID_ADDRESS;
 
-  if (exe_objfile->GetType() != ObjectFile::eTypeExecutable ||
-      exe_objfile->GetStrata() != ObjectFile::eStrataKernel)
-    return LLDB_INVALID_ADDRESS;
+  ObjectFile *exe_objfile = exe_module->GetObjectFile();
 
   if (!exe_objfile->GetBaseAddress().IsValid())
     return LLDB_INVALID_ADDRESS;
 
   if (CheckForKernelImageAtAddress(
           exe_objfile->GetBaseAddress().GetFileAddress(), process) ==
       exe_module->GetUUID())
     return exe_objfile->GetBaseAddress().GetFileAddress();
if (exe_objfile == nullptr) {
       LLDB_LOGF(log,
                 "DynamicLoaderDarwinKernel::CheckForKernelImageAtAddress "
                 "found a binary at 0x%" PRIx64
                 " but could not create an object file from memory",
                 addr);
       return UUID();
     }
 
-    if (exe_objfile->GetType() == ObjectFile::eTypeExecutable &&
-        exe_objfile->GetStrata() == ObjectFile::eStrataKernel) {
+    if (is_kernel(memory_module_sp.get())) {
       ArchSpec kernel_arch(eArchTypeMachO, header.cputype, header.cpusubtype);
       if (!process->GetTarget().GetArchitecture().IsCompatibleMatch(
               kernel_arch)) {
         process->GetTarget().SetArchitecture(kernel_arch);
       }
       if (log) {
         std::string uuid_str;
         if (memory_module_sp->GetUUID().IsValid()) {

 
 // Destructor
 DynamicLoaderDarwinKernel::~DynamicLoaderDarwinKernel() { Clear(true); }
 
 void DynamicLoaderDarwinKernel::UpdateIfNeeded() {
   LoadKernelModuleIfNeeded();
   SetNotificationBreakpointIfNeeded();
 }
-/// Called after attaching a process.
-///
-/// Allow DynamicLoader plug-ins to execute some code after
-/// attaching to a process.
+
+/// We've attached to a remote connection, or read a corefile.
+/// Now load the kernel binary and potentially the kexts, add
+/// them to the Target.
 void DynamicLoaderDarwinKernel::DidAttach() {
   PrivateInitialize(m_process);
   UpdateIfNeeded();
 }
 
 /// Called after attaching a process.
 ///
 /// Allow DynamicLoader plug-ins to execute some code after
if (m_module_sp) {
     if (m_module_sp->SetLoadAddress(process->GetTarget(), 0, true, changed))
       m_load_process_stop_id = process->GetStopID();
   }
   return false;
 }
 
 void DynamicLoaderDarwinKernel::KextImageInfo::SetModule(ModuleSP module_sp) {
   m_module_sp = module_sp;
-  if (module_sp.get() && module_sp->GetObjectFile()) {
-    if (module_sp->GetObjectFile()->GetType() == ObjectFile::eTypeExecutable &&
-        module_sp->GetObjectFile()->GetStrata() == ObjectFile::eStrataKernel) {
-      m_kernel_image = true;
-    } else {
-      m_kernel_image = false;
-    }
-  }
+  m_kernel_image = is_kernel(module_sp.get());
 }
 
 ModuleSP DynamicLoaderDarwinKernel::KextImageInfo::GetModule() {
   return m_module_sp;
 }
 
 void DynamicLoaderDarwinKernel::KextImageInfo::SetLoadAddress(
     addr_t load_addr) {
   m_load_address = load_addr;
 }
 
bool DynamicLoaderDarwinKernel::KextImageInfo::ReadMemoryModule(
   }
 
   ModuleSP memory_module_sp =
       process->ReadModuleFromMemory(file_spec, m_load_address, size_to_read);
 
   if (memory_module_sp.get() == nullptr)
     return false;
 
-  bool is_kernel = false;
-  if (memory_module_sp->GetObjectFile()) {
-    if (memory_module_sp->GetObjectFile()->GetType() ==
-            ObjectFile::eTypeExecutable &&
-        memory_module_sp->GetObjectFile()->GetStrata() ==
-            ObjectFile::eStrataKernel) {
-      is_kernel = true;
-    } else if (memory_module_sp->GetObjectFile()->GetType() ==
-               ObjectFile::eTypeSharedLibrary) {
-      is_kernel = false;
-    }
-  }
+  bool this_is_kernel = is_kernel(memory_module_sp.get());
 
   // If this is a kext, and the kernel specified what UUID we should find at
   // this load address, require that the memory module have a matching UUID or
   // something has gone wrong and we should discard it.
   if (m_uuid.IsValid()) {
     if (m_uuid != memory_module_sp->GetUUID()) {
       if (log) {
         LLDB_LOGF(log,
                   "KextImageInfo::ReadMemoryModule the kernel said to find "
                   "uuid %s at 0x%" PRIx64
                   " but instead we found uuid %s, throwing it away",
                   m_uuid.GetAsString().c_str(), m_load_address,
                   memory_module_sp->GetUUID().GetAsString().c_str());
       }
       return false;
     }
   }
 
   // If the in-memory Module has a UUID, let's use that.
   if (!m_uuid.IsValid() && memory_module_sp->GetUUID().IsValid()) {
     m_uuid = memory_module_sp->GetUUID();
   }
 
   m_memory_module_sp = memory_module_sp;
-  m_kernel_image = is_kernel;
-  if (is_kernel) {
+  m_kernel_image = this_is_kernel;
+  if (this_is_kernel) {
     if (log) {
       // This is unusual and probably not intended
       LLDB_LOGF(log,
                 "KextImageInfo::ReadMemoryModule read the kernel binary out "
                 "of memory");
     }
     if (memory_module_sp->GetArchitecture().IsValid()) {
       process->GetTarget().SetArchitecture(memory_module_sp->GetArchitecture());
     }
-    if (m_uuid.IsValid()) {
-      ModuleSP exe_module_sp = process->GetTarget().GetExecutableModule();
-      if (exe_module_sp.get() && exe_module_sp->GetUUID().IsValid()) {
-        if (m_uuid != exe_module_sp->GetUUID()) {
-          // The user specified a kernel binary that has a different UUID than
-          // the kernel actually running in memory.  This never ends well;
-          // clear the user specified kernel binary from the Target.
-
-          m_module_sp.reset();
-
-          ModuleList user_specified_kernel_list;
-          user_specified_kernel_list.Append(exe_module_sp);
-          process->GetTarget().GetImages().Remove(user_specified_kernel_list);
-        }
-      }
-    }
   }
 
   return true;
 }
 
 bool DynamicLoaderDarwinKernel::KextImageInfo::IsKernel() const {
   return m_kernel_image;
 }
 
if (ReadMemoryModule(process) == false) {
       return false;
     }
   }
 
   if (IsKernel() && m_uuid.IsValid()) {
     Stream &s = target.GetDebugger().GetOutputStream();
     s.Printf("Kernel UUID: %s\n", m_uuid.GetAsString().c_str());
     s.Printf("Load Address: 0x%" PRIx64 "\n", m_load_address);
+
+    // Start of a kernel debug session, we have the UUID of the kernel.
+    // Go through the target's list of modules and if there are any kernel
+    // modules with non-matching UUIDs, remove them.  The user may have added
+    // the wrong kernel binary manually and it will only confuse things.
+    ModuleList incorrect_kernels;
+    for (ModuleSP module_sp : target.GetImages().Modules()) {
+      if (is_kernel(module_sp.get()) && module_sp->GetUUID() != m_uuid)
+        incorrect_kernels.Append(module_sp);
+    }
+    target.GetImages().Remove(incorrect_kernels);
   }
 
   if (!m_module_sp) {
     // See if the kext has already been loaded into the target, probably by the
     // user doing target modules add.
     const ModuleList &target_images = target.GetImages();
     m_module_sp = target_images.FindModule(m_uuid);
 
     // Search for the kext on the local filesystem via the UUID
     if (!m_module_sp && m_uuid.IsValid()) {
       ModuleSpec module_spec;
       module_spec.GetUUID() = m_uuid;
       module_spec.GetArchitecture() = target.GetArchitecture();
if (!m_module_sp) {
     }
 
     // If we managed to find a module, append it to the target's list of
     // images. If we also have a memory module, require that they have matching
     // UUIDs
     if (m_module_sp) {
       if (m_uuid.IsValid() && m_module_sp->GetUUID() == m_uuid) {
         target.GetImages().AppendIfNeeded(m_module_sp, false);
-        if (IsKernel() &&
-            target.GetExecutableModulePointer() != m_module_sp.get()) {
-          target.SetExecutableModule(m_module_sp, eLoadDependentsNo);
-        }
       }
     }
   }
 
   // If we've found a binary, read the load commands out of memory so we
   // can set the segment load addresses.
   if (m_module_sp)
     ReadMemoryModule (process);

 }
 
 // Load the kernel module and initialize the "m_kernel" member. Return true
 // _only_ if the kernel is loaded the first time through (subsequent calls to
 // this function should return false after the kernel has been already loaded).
 void DynamicLoaderDarwinKernel::LoadKernelModuleIfNeeded() {
   if (!m_kext_summary_header_ptr_addr.IsValid()) {
     m_kernel.Clear();
-    m_kernel.SetModule(m_process->GetTarget().GetExecutableModule());
+    ModuleSP module_sp = m_process->GetTarget().GetExecutableModule();
+    if (is_kernel(module_sp.get())) {
+      m_kernel.SetModule(module_sp);
       m_kernel.SetIsKernel(true);
+    }
 
     ConstString kernel_name("mach_kernel");
     if (m_kernel.GetModule().get() && m_kernel.GetModule()->GetObjectFile() &&
         !m_kernel.GetModule()
              ->GetObjectFile()
              ->GetFileSpec()
              .GetFilename()
              .IsEmpty()) {

lldb/source/Plugins/Platform/MacOSX/CMakeLists.txt

add_lldb_library(lldbPluginPlatformMacOSX PLUGIN
   LINK_LIBS
     lldbBreakpoint
     lldbCore
     lldbHost
     lldbInterpreter
     lldbSymbol
     lldbTarget
     lldbUtility
+    lldbPluginDynamicLoaderDarwinKernel
+    lldbPluginObjectContainerMachOFileset
     lldbPluginPlatformPOSIX
     ${OBJC_LIBS}
   CLANG_LIBS
     clangBasic
   LINK_COMPONENTS
     Support
     )
 
 add_dependencies(lldbPluginPlatformMacOSX
   LLDBPluginPlatformMacOSXPropertiesGen
   LLDBPluginPlatformMacOSXPropertiesEnumGen)

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.h

Status GetSharedModuleKernel(
       const ModuleSpec &module_spec, Process *process,
       lldb::ModuleSP &module_sp, const FileSpecList *module_search_paths_ptr,
       llvm::SmallVectorImpl<lldb::ModuleSP> *old_modules, bool *did_create_ptr);
 
   Status ExamineKextForMatchingUUID(const FileSpec &kext_bundle_path,
                                     const UUID &uuid, const ArchSpec &arch,
                                     lldb::ModuleSP &exe_module_sp);
 
+  bool LoadPlatformBinaryAndSetup(Process *process, lldb::addr_t addr,
+                                  bool notify) override;
+
   // Most of the ivars are assembled under FileSystem::EnumerateDirectory calls
   // where the function being called for each file/directory must be static.
   // We'll pass a this pointer as a baton and access the ivars directly.
   // Toss-up whether this should just be a struct at this point.
 
 public:
   /// Multimap of CFBundleID to FileSpec on local filesystem, kexts with dSYMs
   /// next to them.

lldb/source/Plugins/Platform/MacOSX/PlatformDarwinKernel.cpp

 //===-- PlatformDarwinKernel.cpp ------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "PlatformDarwinKernel.h"
 
 #if defined(__APPLE__) // This Plugin uses the Mac-specific
                        // source/Host/macosx/cfcpp utilities
 
 #include "lldb/Breakpoint/BreakpointLocation.h"
+#include "lldb/Core/Debugger.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/ModuleList.h"
 #include "lldb/Core/ModuleSpec.h"
 #include "lldb/Core/PluginManager.h"
 #include "lldb/Host/Host.h"
 #include "lldb/Host/HostInfo.h"
 #include "lldb/Interpreter/OptionValueFileSpecList.h"
 #include "lldb/Interpreter/OptionValueProperties.h"
 #include "lldb/Interpreter/Property.h"
 #include "lldb/Symbol/ObjectFile.h"
 #include "lldb/Target/Platform.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/ArchSpec.h"
+#include "lldb/Utility/DataBufferHeap.h"
 #include "lldb/Utility/FileSpec.h"
 #include "lldb/Utility/LLDBLog.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Status.h"
 #include "lldb/Utility/StreamString.h"
 
 #include "llvm/Support/FileSystem.h"
 
 #include <CoreFoundation/CoreFoundation.h>
 
 #include <memory>
 
 #include "Host/macosx/cfcpp/CFCBundle.h"
+#include "Plugins/DynamicLoader/Darwin-Kernel/DynamicLoaderDarwinKernel.h"
+#include "Plugins/ObjectContainer/Mach-O-Fileset/ObjectContainerMachOFileset.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 // Static Variables
 static uint32_t g_initialize_count = 0;
 
 // Static Functions
Status PlatformDarwinKernel::GetSharedModule(
   Status error;
   module_sp.reset();
   const FileSpec &platform_file = module_spec.GetFileSpec();
 
   // Treat the file's path as a kext bundle ID (e.g.
   // "com.apple.driver.AppleIRController") and search our kext index.
   std::string kext_bundle_id = platform_file.GetPath();
 
-  if (!kext_bundle_id.empty() && module_spec.GetUUID().IsValid()) {
-    if (kext_bundle_id == "mach_kernel") {
-      return GetSharedModuleKernel(module_spec, process, module_sp,
+  if (module_spec.GetUUID().IsValid()) {
+    // DynamicLoaderDarwinKernel uses the magic name mach_kernel,
+    // UUID search can get here with no name - and it may be a kernel.
+    if (kext_bundle_id == "mach_kernel" || kext_bundle_id.empty()) {
+      error = GetSharedModuleKernel(module_spec, process, module_sp,
                                     module_search_paths_ptr, old_modules,
                                     did_create_ptr);
+      if (error.Success() && module_sp) {
+        return error;
+      }
     } else {
       return GetSharedModuleKext(module_spec, process, module_sp,
                                  module_search_paths_ptr, old_modules,
                                  did_create_ptr);
     }
-  } else {
-    // Give the generic methods, including possibly calling into  DebugSymbols
-    // framework on macOS systems, a chance.
-    return PlatformDarwin::GetSharedModule(module_spec, process, module_sp,
-                                           module_search_paths_ptr, old_modules,
-                                           did_create_ptr);
-  }
-}
+  }
+
+  // Give the generic methods, including possibly calling into DebugSymbols
 
 Status PlatformDarwinKernel::GetSharedModuleKext(
     const ModuleSpec &module_spec, Process *process, ModuleSP &module_sp,
     const FileSpecList *module_search_paths_ptr,
     llvm::SmallVectorImpl<ModuleSP> *old_modules, bool *did_create_ptr) {
   Status error;
   module_sp.reset();
   const FileSpec &platform_file = module_spec.GetFileSpec();
for (auto possible_kernel : m_kernel_binaries_with_dsyms) {
     if (FileSystem::Instance().Exists(possible_kernel)) {
       ModuleSpec kern_spec(possible_kernel);
       kern_spec.GetUUID() = module_spec.GetUUID();
       module_sp.reset(new Module(kern_spec));
       if (module_sp && module_sp->GetObjectFile() &&
           module_sp->MatchesModuleSpec(kern_spec)) {
         // module_sp is an actual kernel binary we want to add.
         if (process) {
-          process->GetTarget().GetImages().AppendIfNeeded(module_sp);
+          const bool notify = false;
+          process->GetTarget().GetImages().AppendIfNeeded(module_sp, notify);
           error.Clear();
           return error;
         } else {
           error = ModuleList::GetSharedModule(kern_spec, module_sp, nullptr,
                                               nullptr, nullptr);
           if (module_sp && module_sp->GetObjectFile() &&
               module_sp->GetObjectFile()->GetType() !=
                   ObjectFile::Type::eTypeCoreFile) {
for (FileSpec objfile : objfile_names) {
       kern_spec.GetUUID() = module_spec.GetUUID();
       kern_spec.GetSymbolFileSpec() = possible_kernel_dsym;
 
       module_sp.reset(new Module(kern_spec));
       if (module_sp && module_sp->GetObjectFile() &&
           module_sp->MatchesModuleSpec(kern_spec)) {
         // module_sp is an actual kernel binary we want to add.
         if (process) {
-          process->GetTarget().GetImages().AppendIfNeeded(module_sp);
+          const bool notify = false;
+          process->GetTarget().GetImages().AppendIfNeeded(module_sp, notify);
           error.Clear();
           return error;
         } else {
           error = ModuleList::GetSharedModule(kern_spec, module_sp, nullptr,
                                               nullptr, nullptr);
           if (module_sp && module_sp->GetObjectFile() &&
               module_sp->GetObjectFile()->GetType() !=
                   ObjectFile::Type::eTypeCoreFile) {
if (FileSystem::Instance().Exists(exe_file)) {
       }
       exe_module_sp.reset();
     }
   }
 
   return {};
 }
 
+static addr_t find_kernel_in_macho_fileset(Process *process,
+                                           addr_t input_addr) {
+  Status error;
+  WritableDataBufferSP header_data(new DataBufferHeap(512, 0));
+  if (!process->ReadMemory(input_addr, header_data->GetBytes(),
+                           header_data->GetByteSize(), error) ||
+      !error.Success())
+    return LLDB_INVALID_ADDRESS;
+  ModuleSP module_sp(new Module(ModuleSpec()));
+  ObjectContainerSP container_sp(
+      ObjectContainerMachOFileset::CreateMemoryInstance(
+          module_sp, header_data, process->shared_from_this(), input_addr));
+  if (!container_sp)
+    return LLDB_INVALID_ADDRESS;
+
+  ObjectContainerMachOFileset *fileset_container =
+      static_cast<ObjectContainerMachOFileset *>(container_sp.get());
+  ObjectContainerMachOFileset::Entry *entry =
+      fileset_container->FindEntry("com.apple.kernel");
+  if (entry)
+    return entry->vmaddr;
+  return LLDB_INVALID_ADDRESS;
+}
+
+bool PlatformDarwinKernel::LoadPlatformBinaryAndSetup(Process *process,
+                                                      lldb::addr_t input_addr,
+                                                      bool notify) {
+  Log *log =
+      GetLog(LLDBLog::Platform | LLDBLog::DynamicLoader | LLDBLog::Process);
+
+  if (!process)
+    return false;
+
+  addr_t actual_address = find_kernel_in_macho_fileset(process, input_addr);
+
+  LLDB_LOGF(log,
+            "PlatformDarwinKernel::%s check address 0x%" PRIx64 " for "
+            "a macho fileset, got back kernel address 0x%" PRIx64,
+            __FUNCTION__, input_addr, actual_address);
+
+  if (actual_address == LLDB_INVALID_ADDRESS)
+    return false;
+
+  // We have a xnu kernel binary, this is a kernel debug session.
+  // Set the Target's Platform to be PlatformDarwinKernel, and the
+  // Process' DynamicLoader to be DynamicLoaderDarwinKernel.
+
+  PlatformSP platform_sp =
+      process->GetTarget().GetDebugger().GetPlatformList().Create(
+          PlatformDarwinKernel::GetPluginNameStatic());
+  if (platform_sp)
+    process->GetTarget().SetPlatform(platform_sp);
+
+  DynamicLoaderUP dyld_up =
+      std::make_unique<DynamicLoaderDarwinKernel>(process, actual_address);
+  if (!dyld_up)
+    return false;
+
+  // Process owns it now
+  process->SetDynamicLoader(std::move(dyld_up));
+
+  return true;
+}
+
 std::vector<ArchSpec> PlatformDarwinKernel::GetSupportedArchitectures(
     const ArchSpec &process_host_arch) {
   std::vector<ArchSpec> result;
 #if defined(__arm__) || defined(__arm64__) || defined(__aarch64__)
   ARMGetSupportedArchitectures(result);
 #else
   x86GetSupportedArchitectures(result);
 #endif

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp

if (response.IsNormalResponse()) {
           StringExtractor extractor(value);
           m_process_standalone_value =
               extractor.GetU64(LLDB_INVALID_ADDRESS, 16);
           if (m_process_standalone_value != LLDB_INVALID_ADDRESS) {
             m_process_standalone_value_is_offset = false;
             ++num_keys_decoded;
           }
         } else if (name.equals("binary-addresses")) {
-          addr_t addr;
-          while (!value.empty()) {
-            llvm::StringRef addr_str;
-            std::tie(addr_str, value) = value.split(',');
-            if (!addr_str.getAsInteger(16, addr))
-              m_binary_addresses.push_back(addr);
+          m_binary_addresses.clear();
+          ++num_keys_decoded;
+          for (llvm::StringRef x : llvm::split(value, ',')) {
+            addr_t vmaddr;
+            x.consume_front("0x");
+            if (llvm::to_integer(x, vmaddr, 16))
+              m_binary_addresses.push_back(vmaddr);
           }
         }
       }
       if (num_keys_decoded > 0)
         m_qProcessInfo_is_valid = eLazyBoolYes;
       if (pid != LLDB_INVALID_PROCESS_ID) {
         m_curr_pid_is_valid = eLazyBoolYes;
         m_curr_pid_run = m_curr_pid = pid;

lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp

if (m_gdb_comm.GetStopReply(response)) {
       // a DynamicLoader plugin that knows how to find the loaded
       // binaries and will track updates as binaries are added.
 
       std::vector<addr_t> bin_addrs = m_gdb_comm.GetProcessStandaloneBinaries();
       if (bin_addrs.size()) {
         UUID uuid;
         const bool value_is_slide = false;
         for (addr_t addr : bin_addrs) {
-          const bool force_symbol_search = true;
           const bool notify = true;
+          // First see if this is a special platform
+          // binary that may determine the DynamicLoader and
+          // Platform to be used in this Process/Target in the
+          // process of loading it.
+          if (GetTarget()
+                  .GetDebugger()
+                  .GetPlatformList()
+                  .LoadPlatformBinaryAndSetup(this, addr, notify))
+            continue;
+
+          const bool force_symbol_search = true;
+          // Second manually load this binary into the Target.
           DynamicLoader::LoadBinaryWithUUIDAndAddress(
               this, uuid, addr, value_is_slide, force_symbol_search, notify);
         }
       }
 
       const StateType state = SetThreadStopInfo(response);
       if (state != eStateInvalid) {
         SetPrivateState(state);

lldb/source/Target/Platform.cpp

 }
 
 PlatformSP PlatformList::Create(llvm::StringRef name) {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
   PlatformSP platform_sp = Platform::Create(name);
   m_platforms.push_back(platform_sp);
   return platform_sp;
 }
+
+bool PlatformList::LoadPlatformBinaryAndSetup(Process *process,
+                                              lldb::addr_t addr, bool notify) {
+  std::lock_guard<std::recursive_mutex> guard(m_mutex);
+
+  PlatformCreateInstance create_callback;
+  for (int idx = 0;
+       (create_callback = PluginManager::GetPlatformCreateCallbackAtIndex(idx));
+       ++idx) {
+    ArchSpec arch;
+    PlatformSP platform_sp = create_callback(true, &arch);
+    if (platform_sp) {
+      if (platform_sp->LoadPlatformBinaryAndSetup(process, addr, notify))
+        return true;
+    }
+  }
+  return false;
+}

lldb/source/Target/Process.cpp

 }
 
 DynamicLoader *Process::GetDynamicLoader() {
   if (!m_dyld_up)
     m_dyld_up.reset(DynamicLoader::FindPlugin(this, ""));
   return m_dyld_up.get();
 }
 
+void Process::SetDynamicLoader(DynamicLoaderUP dyld_up) {
+  m_dyld_up = std::move(dyld_up);
+}
+
 DataExtractor Process::GetAuxvData() { return DataExtractor(); }
 
 llvm::Expected<bool> Process::SaveCore(llvm::StringRef outfile) {
   return false;
 }
 
 JITLoaderList &Process::GetJITLoaders() {
   if (!m_jit_loaders_up) {