Get register context for the 32-bit process in a WoW64 process minidump.

[amccarth]

Summary

32-bit processes on 64-bit Windows run in a layer called WoW64 (Windows-on-Windows64). If you capture a mini dump of such a process from a 32-bit debugger, you end up with a register context for the 64-bit WoW64 process rather than the 32-bit one you probably care about.

This detects WoW64 by looking to see if there's a module named wow64.dll loaded. For such processes, it then looks in the 64-bit Thread Environment Block (TEB) to locate a copy of the 32-bit CONTEXT record that the plugin needs for the register context.

I'll add a test before submitting this, but it seems to work in my manual experiments.

[zturner]

I'll have to look at this more carefully next week. I'm guessing the same logic can eventually be re-used to live debug a 32-bit process from a 64-bit LLDB? (Not that you have to address that now, just curious)

[zturner]

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

193

I think this structure should go in a different header file in Process/Windows/Common. Maybe it could be called NtStructures.h or something like that.

219–220

Are the other fields of this structure known what they mean?

[amccarth]

In D17465#357569, @zturner wrote:

I'll have to look at this more carefully next week.

Take your time, I'm still working on adding some tests for this.

I'm guessing the same logic can eventually be re-used to live debug a 32-bit process from a 64-bit LLDB? (Not that you have to address that now, just curious)

I think most of this patch is specific to minidump debugging. With a live process, you can get the 32-bit context through API calls like Wow64GetThreadContext function, which hopefully is less prone to breakage in future versions of Windows. That does mean, however, that we'll have to treat 32-bit on Wow64 as a distinct case from 32-bit on 32-bit and 64-bit on 64-bit.

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

219–220

Nope.

The fact that TLS slot 1 points to a structure that contains the context is documented (with the caveat that it may change in future versions of Windows) here: https://msdn.microsoft.com/en-us/library/ms681670.aspx

But there's not much other information.

I've updated the comment with that MSDN link.

[amccarth]

Addressed first round of comments.

[amccarth]

Adds some very basic tests using a check-in minidump captured with Task Manager.

What we can do in the test is limited because the minidump doesn't seem to have an exception record (thus the thread isn't stopped). I plan to look into this in the future. There's a chance we can get this information from the thread environment block (TEB).

[zturner]

source/Plugins/Process/Windows/Common/NtStructures.h

19

Just as a general tip for future reference, you can get better definitions of these structures in WinDbg. In this case, for example, dt ntdll!_TEB64 gives a fairly comprehensive definition of _TEB64. The offset of TlsSlots matches up with what you have here too, which at least shows it's correct.

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

238–240

Interesting, according to WinDbg's definition of _TEB64, the TEB has this field:

+0x180c WowTebOffset     : Int4B

Not sure what the difference is between this and wow64teb.Reserved1[0] (if there is one at all), but maybe we'll need this in the future (or maybe it's not what it sounds like)

[amccarth]

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

238–240

Yeah, I spotted that, too. The comment is based on documentation in MSDN.

source/Plugins/Process/Windows/Common/NtStructures.h

+//===-- NtStructures.h ------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef liblldb_Plugins_Process_Windows_Common_NtStructures_h_
+#define liblldb_Plugins_Process_Windows_Common_NtStructures_h_
+
+#include "lldb/Host/windows/windows.h"
+
+// This describes the layout of a TEB (Thread Environment Block) for a 64-bit
+// process.  It's adapted from the 32-bit TEB in winternl.h.  Currently, we care
+// only about the position of the TlsSlots.
+struct TEB64
+{
+    ULONG64 Reserved1[12];
+    ULONG64 ProcessEnvironmentBlock;
+    ULONG64 Reserved2[399];
+    BYTE Reserved3[1952];
+    ULONG64 TlsSlots[64];
+    BYTE Reserved4[8];
+    ULONG64 Reserved5[26];
+    ULONG64 ReservedForOle; // Windows 2000 only
+    ULONG64 Reserved6[4];
+    ULONG64 TlsExpansionSlots;
+};
+
+#endif

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

 #include "lldb/Target/StopInfo.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/UnixSignals.h"
 #include "lldb/Utility/LLDBAssert.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/raw_ostream.h"
 
+#include "Plugins/Process/Windows/Common/NtStructures.h"
+#include "Plugins/Process/Windows/Common/ProcessWindowsLog.h"
+
 #include "ExceptionRecord.h"
 #include "ThreadWinMiniDump.h"
 
 using namespace lldb_private;
 
 namespace
 {
 
public:
     Data();
     ~Data();
 
     FileSpec m_core_file;
     HANDLE m_dump_file;  // handle to the open minidump file
     HANDLE m_mapping;  // handle to the file mapping for the minidump file
     void * m_base_addr;  // base memory address of the minidump
     std::shared_ptr<ExceptionRecord> m_exception_sp;
+    bool m_is_wow64; // minidump is of a 32-bit process captured with a 64-bit debugger
 };
 
 ConstString
 ProcessWinMiniDump::GetPluginNameStatic()
 {
     static ConstString g_name("win-minidump");
     return g_name;
 }
if (m_dyld_ap.get() == NULL)
         m_dyld_ap.reset (DynamicLoader::FindPlugin(this, DynamicLoaderWindowsDYLD::GetPluginNameStatic().GetCString()));
     return m_dyld_ap.get();
 }
 
 bool
 ProcessWinMiniDump::UpdateThreadList(ThreadList &old_thread_list, ThreadList &new_thread_list)
 {
     size_t size = 0;
     auto thread_list_ptr = static_cast<const MINIDUMP_THREAD_LIST *>(FindDumpStream(ThreadListStream, &size));
     if (thread_list_ptr)
     {
         const ULONG32 thread_count = thread_list_ptr->NumberOfThreads;
         for (ULONG32 i = 0; i < thread_count; ++i) {
             const auto &mini_dump_thread = thread_list_ptr->Threads[i];
             auto thread_sp = std::make_shared<ThreadWinMiniDump>(*this, mini_dump_thread.ThreadId);
             if (mini_dump_thread.ThreadContext.DataSize >= sizeof(CONTEXT))
             {
-                const CONTEXT *context = reinterpret_cast<const CONTEXT *>(static_cast<const char *>(m_data_up->m_base_addr) + mini_dump_thread.ThreadContext.Rva);
+                const CONTEXT *context = reinterpret_cast<const CONTEXT *>(
+                    static_cast<const char *>(m_data_up->m_base_addr) + mini_dump_thread.ThreadContext.Rva);
+
+                if (m_data_up->m_is_wow64)
+                {
+                    // On Windows, a 32-bit process can run on a 64-bit machine under WOW64.
+                    // If the minidump was captured with a 64-bit debugger, then the CONTEXT
+                    // we just grabbed from the mini_dump_thread is the one for the 64-bit
+                    // "native" process rather than the 32-bit "guest" process we care about.
+                    // In this case, we can get the 32-bit CONTEXT from the TEB (Thread
+                    // Environment Block) of the 64-bit process.
+                    Error error;
+                    TEB64 wow64teb = {0};
+                    ReadMemory(mini_dump_thread.Teb, &wow64teb, sizeof(wow64teb), error);
+                    if (error.Success())
+                    {
+                        // Slot 1 of the thread-local storage in the 64-bit TEB points to a structure
+                        // that includes the 32-bit CONTEXT (after a ULONG).
+                        // See:  https://msdn.microsoft.com/en-us/library/ms681670.aspx
+                        const size_t addr = wow64teb.TlsSlots[1];
+                        Range range = {0};
+                        if (FindMemoryRange(addr, &range))
+                        {
+                            lldbassert(range.start <= addr);
+                            const size_t offset = addr - range.start + sizeof(ULONG);
+                            if (offset < range.size)
+                            {
+                                const size_t overlap = range.size - offset;
+                                if (overlap >= sizeof(CONTEXT))
+                                {
+                                    context = reinterpret_cast<const CONTEXT *>(range.ptr + offset);
+                                }
+                            }
+                        }
+                    }
+
+                    // NOTE:  We don't currently use the TEB for anything else.  If we need it in
+                    // the future, the 32-bit TEB is located according to the address stored in the
+                    // first slot of the 64-bit TEB (wow64teb.Reserved1[0]).
+                }
+
                 thread_sp->SetContext(context);
             }
             new_thread_list.AddThread(thread_sp);
         }
     }
 
     return new_thread_list.GetSize(false) > 0;
 }

 
 ArchSpec
 ProcessWinMiniDump::GetArchitecture()
 {
     // TODO
     return ArchSpec();
 }
 
-
-ProcessWinMiniDump::Data::Data() :
-    m_dump_file(INVALID_HANDLE_VALUE),
-    m_mapping(NULL),
-    m_base_addr(nullptr)
+ProcessWinMiniDump::Data::Data()
+    : m_dump_file(INVALID_HANDLE_VALUE), m_mapping(NULL), m_base_addr(nullptr), m_is_wow64(false)
 {
 }
 
 ProcessWinMiniDump::Data::~Data()
 {
     if (m_base_addr)
     {
         ::UnmapViewOfFile(m_base_addr);

 
 bool
 ProcessWinMiniDump::FindMemoryRange(lldb::addr_t addr, Range *range_out) const
 {
     size_t stream_size = 0;
     auto mem_list_stream = static_cast<const MINIDUMP_MEMORY_LIST *>(FindDumpStream(MemoryListStream, &stream_size));
     if (mem_list_stream)
     {
-        for (ULONG32 i = 0; i < mem_list_stream->NumberOfMemoryRanges; ++i) {
+        for (ULONG32 i = 0; i < mem_list_stream->NumberOfMemoryRanges; ++i)
+        {
             const MINIDUMP_MEMORY_DESCRIPTOR &mem_desc = mem_list_stream->MemoryRanges[i];
             const MINIDUMP_LOCATION_DESCRIPTOR &loc_desc = mem_desc.Memory;
             const lldb::addr_t range_start = mem_desc.StartOfMemoryRange;
             const size_t range_size = loc_desc.DataSize;
             if (range_start <= addr && addr < range_start + range_size)
             {
                 range_out->start = range_start;
                 range_out->size = range_size;
ProcessWinMiniDump::ReadModuleList()
     {
         return;
     }
 
     for (ULONG32 i = 0; i < module_list_ptr->NumberOfModules; ++i)
     {
         const auto &module = module_list_ptr->Modules[i];
         const auto file_name = GetMiniDumpString(m_data_up->m_base_addr, module.ModuleNameRva);
-        ModuleSpec module_spec = FileSpec(file_name, true);
+        const auto file_spec = FileSpec(file_name, true);
+        if (FileSpec::Compare(file_spec, FileSpec("wow64.dll", false), false) == 0)
+        {
+            WINLOG_IFALL(WINDOWS_LOG_PROCESS, "Minidump is for a WOW64 process.");
+            m_data_up->m_is_wow64 = true;
+        }
+        ModuleSpec module_spec = file_spec;
 
         lldb::ModuleSP module_sp = GetTarget().GetSharedModule(module_spec);
         if (!module_sp)
         {
             continue;
         }
         bool load_addr_changed = false;
         module_sp->SetLoadAddress(GetTarget(), module.BaseOfImage, false, load_addr_changed);