Get register context for the 32-bit process in a WoW64 process minidump.

[amccarth]

Summary

32-bit processes on 64-bit Windows run in a layer called WoW64 (Windows-on-Windows64). If you capture a mini dump of such a process from a 32-bit debugger, you end up with a register context for the 64-bit WoW64 process rather than the 32-bit one you probably care about.

This detects WoW64 by looking to see if there's a module named wow64.dll loaded. For such processes, it then looks in the 64-bit Thread Environment Block (TEB) to locate a copy of the 32-bit CONTEXT record that the plugin needs for the register context.

I'll add a test before submitting this, but it seems to work in my manual experiments.

[zturner]

I'll have to look at this more carefully next week. I'm guessing the same logic can eventually be re-used to live debug a 32-bit process from a 64-bit LLDB? (Not that you have to address that now, just curious)

[zturner]

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

192 ↗

(On Diff #48552)

I think this structure should go in a different header file in Process/Windows/Common. Maybe it could be called NtStructures.h or something like that.

236–237 ↗

(On Diff #48552)

Are the other fields of this structure known what they mean?

[amccarth]

In D17465#357569, @zturner wrote:

I'll have to look at this more carefully next week.

Take your time, I'm still working on adding some tests for this.

I'm guessing the same logic can eventually be re-used to live debug a 32-bit process from a 64-bit LLDB? (Not that you have to address that now, just curious)

I think most of this patch is specific to minidump debugging. With a live process, you can get the 32-bit context through API calls like Wow64GetThreadContext function, which hopefully is less prone to breakage in future versions of Windows. That does mean, however, that we'll have to treat 32-bit on Wow64 as a distinct case from 32-bit on 32-bit and 64-bit on 64-bit.

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

236–237 ↗

(On Diff #48552)

Nope.

The fact that TLS slot 1 points to a structure that contains the context is documented (with the caveat that it may change in future versions of Windows) here: https://msdn.microsoft.com/en-us/library/ms681670.aspx

But there's not much other information.

I've updated the comment with that MSDN link.

[amccarth]

Addressed first round of comments.

[amccarth]

Adds some very basic tests using a check-in minidump captured with Task Manager.

What we can do in the test is limited because the minidump doesn't seem to have an exception record (thus the thread isn't stopped). I plan to look into this in the future. There's a chance we can get this information from the thread environment block (TEB).

[zturner]

source/Plugins/Process/Windows/Common/NtStructures.h

18 ↗

(On Diff #48863)

Just as a general tip for future reference, you can get better definitions of these structures in WinDbg. In this case, for example, dt ntdll!_TEB64 gives a fairly comprehensive definition of _TEB64. The offset of TlsSlots matches up with what you have here too, which at least shows it's correct.

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

238–240 ↗

(On Diff #48863)

Interesting, according to WinDbg's definition of _TEB64, the TEB has this field:

+0x180c WowTebOffset     : Int4B

Not sure what the difference is between this and wow64teb.Reserved1[0] (if there is one at all), but maybe we'll need this in the future (or maybe it's not what it sounds like)

[amccarth]

source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

238–240 ↗

(On Diff #48863)

Yeah, I spotted that, too. The comment is based on documentation in MSDN.

lldb/trunk/packages/Python/lldbsuite/test/functionalities/postmortem/wow64_minidump/TestWow64MiniDump.py

+"""
+Test basics of a mini dump taken of a 32-bit process running in WoW64
+
+WoW64 is the subsystem that lets 32-bit processes run in 64-bit Windows.  If you
+capture a mini dump of a process running under WoW64 with a 64-bit debugger, you
+end up with a dump of the WoW64 layer.  In that case, LLDB must do extra work to
+get the 32-bit register contexts.
+"""
+
+from __future__ import print_function
+from six import iteritems
+
+
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+class Wow64MiniDumpTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    @skipUnlessWindows  # for now mini-dump debugging is limited to Windows hosts
+    @no_debug_info_test
+    def test_wow64_mini_dump(self):
+        """Test that lldb can read the process information from the minidump."""
+        # target create -c fizzbuzz_wow64.dmp
+        target = self.dbg.CreateTarget("")
+        process = target.LoadCore("fizzbuzz_wow64.dmp")
+        self.assertTrue(process, PROCESS_IS_VALID)
+        self.assertEqual(process.GetNumThreads(), 1)
+        self.assertEqual(process.GetProcessID(), 0x1E9C)
+
+    @skipUnlessWindows  # for now mini-dump debugging is limited to Windows hosts
+    @no_debug_info_test
+    def test_thread_info_in_wow64_mini_dump(self):
+        """Test that lldb can read the thread information from the minidump."""
+        # target create -c fizzbuzz_wow64.dmp
+        target = self.dbg.CreateTarget("")
+        process = target.LoadCore("fizzbuzz_wow64.dmp")
+        # This process crashed due to an access violation (0xc0000005), but the
+        # minidump doesn't have an exception record--perhaps the crash handler
+        # ate it.
+        # TODO:  See if we can recover the exception information from the TEB,
+        # which, according to Windbg, has a pointer to an exception list.
+
+        # In the dump, none of the threads are stopped, so we cannot use
+        # lldbutil.get_stopped_thread.
+        thread = process.GetThreadAtIndex(0)
+        self.assertEqual(thread.GetStopReason(), lldb.eStopReasonNone)
+
+    @skipUnlessWindows  # for now mini-dump debugging is limited to Windows hosts
+    @no_debug_info_test
+    def test_stack_info_in_wow64_mini_dump(self):
+        """Test that we can see a trivial stack in a VS-generate mini dump."""
+        # target create -c fizzbuzz_no_heap.dmp
+        target = self.dbg.CreateTarget("")
+        process = target.LoadCore("fizzbuzz_wow64.dmp")
+        self.assertGreaterEqual(process.GetNumThreads(), 1)
+        # This process crashed due to an access violation (0xc0000005), but the
+        # minidump doesn't have an exception record--perhaps the crash handler
+        # ate it.
+        # TODO:  See if we can recover the exception information from the TEB,
+        # which, according to Windbg, has a pointer to an exception list.
+
+        # In the dump, none of the threads are stopped, so we cannot use
+        # lldbutil.get_stopped_thread.
+        thread = process.GetThreadAtIndex(0)
+        # The crash is in main, so there should be at least one frame on the stack.
+        self.assertGreaterEqual(thread.GetNumFrames(), 1)
+        frame = thread.GetFrameAtIndex(0)
+        self.assertTrue(frame.IsValid())
+        pc = frame.GetPC()
+        eip = frame.FindRegister("pc")
+        self.assertTrue(eip.IsValid())
+        self.assertEqual(pc, eip.GetValueAsUnsigned())

lldb/trunk/packages/Python/lldbsuite/test/functionalities/postmortem/wow64_minidump/fizzbuzz.cpp

+// A sample program for getting minidumps on Windows.
+
+#include <iostream>
+
+bool
+fizz(int x)
+{
+    return x % 3 == 0;
+}
+
+bool
+buzz(int x)
+{
+    return x % 5 == 0;
+}
+
+int
+main()
+{
+    int *buggy = 0;
+
+    for (int i = 1; i <= 100; ++i)
+    {
+        if (fizz(i)) std::cout << "fizz";
+        if (buzz(i)) std::cout << "buzz";
+        if (!fizz(i) && !buzz(i)) std::cout << i;
+        std::cout << '\n';
+    }
+
+    return *buggy;
+}

lldb/trunk/source/Plugins/Process/Windows/Common/NtStructures.h

+//===-- NtStructures.h ------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef liblldb_Plugins_Process_Windows_Common_NtStructures_h_
+#define liblldb_Plugins_Process_Windows_Common_NtStructures_h_
+
+#include "lldb/Host/windows/windows.h"
+
+// This describes the layout of a TEB (Thread Environment Block) for a 64-bit
+// process.  It's adapted from the 32-bit TEB in winternl.h.  Currently, we care
+// only about the position of the TlsSlots.
+struct TEB64
+{
+    ULONG64 Reserved1[12];
+    ULONG64 ProcessEnvironmentBlock;
+    ULONG64 Reserved2[399];
+    BYTE Reserved3[1952];
+    ULONG64 TlsSlots[64];
+    BYTE Reserved4[8];
+    ULONG64 Reserved5[26];
+    ULONG64 ReservedForOle; // Windows 2000 only
+    ULONG64 Reserved6[4];
+    ULONG64 TlsExpansionSlots;
+};
+
+#endif

lldb/trunk/source/Plugins/Process/Windows/MiniDump/ProcessWinMiniDump.cpp

 #include "lldb/Target/StopInfo.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/UnixSignals.h"
 #include "lldb/Utility/LLDBAssert.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/raw_ostream.h"
 
+#include "Plugins/Process/Windows/Common/NtStructures.h"
+#include "Plugins/Process/Windows/Common/ProcessWindowsLog.h"
+
 #include "ExceptionRecord.h"
 #include "ThreadWinMiniDump.h"
 
 using namespace lldb_private;
 
 namespace
 {
 
public:
     Data();
     ~Data();
 
     FileSpec m_core_file;
     HANDLE m_dump_file;  // handle to the open minidump file
     HANDLE m_mapping;  // handle to the file mapping for the minidump file
     void * m_base_addr;  // base memory address of the minidump
     std::shared_ptr<ExceptionRecord> m_exception_sp;
+    bool m_is_wow64; // minidump is of a 32-bit process captured with a 64-bit debugger
 };
 
 ConstString
 ProcessWinMiniDump::GetPluginNameStatic()
 {
     static ConstString g_name("win-minidump");
     return g_name;
 }
ProcessWinMiniDump::UpdateThreadList(ThreadList &old_thread_list, ThreadList &new_thread_list)
     if (thread_list_ptr)
     {
         const ULONG32 thread_count = thread_list_ptr->NumberOfThreads;
         for (ULONG32 i = 0; i < thread_count; ++i) {
             const auto &mini_dump_thread = thread_list_ptr->Threads[i];
             auto thread_sp = std::make_shared<ThreadWinMiniDump>(*this, mini_dump_thread.ThreadId);
             if (mini_dump_thread.ThreadContext.DataSize >= sizeof(CONTEXT))
             {
-                const CONTEXT *context = reinterpret_cast<const CONTEXT *>(static_cast<const char *>(m_data_up->m_base_addr) + mini_dump_thread.ThreadContext.Rva);
+                const CONTEXT *context = reinterpret_cast<const CONTEXT *>(
+                    static_cast<const char *>(m_data_up->m_base_addr) + mini_dump_thread.ThreadContext.Rva);
+
+                if (m_data_up->m_is_wow64)
+                {
+                    // On Windows, a 32-bit process can run on a 64-bit machine under WOW64.
+                    // If the minidump was captured with a 64-bit debugger, then the CONTEXT
+                    // we just grabbed from the mini_dump_thread is the one for the 64-bit
+                    // "native" process rather than the 32-bit "guest" process we care about.
+                    // In this case, we can get the 32-bit CONTEXT from the TEB (Thread
+                    // Environment Block) of the 64-bit process.
+                    Error error;
+                    TEB64 wow64teb = {0};
+                    ReadMemory(mini_dump_thread.Teb, &wow64teb, sizeof(wow64teb), error);
+                    if (error.Success())
+                    {
+                        // Slot 1 of the thread-local storage in the 64-bit TEB points to a structure
+                        // that includes the 32-bit CONTEXT (after a ULONG).
+                        // See:  https://msdn.microsoft.com/en-us/library/ms681670.aspx
+                        const size_t addr = wow64teb.TlsSlots[1];
+                        Range range = {0};
+                        if (FindMemoryRange(addr, &range))
+                        {
+                            lldbassert(range.start <= addr);
+                            const size_t offset = addr - range.start + sizeof(ULONG);
+                            if (offset < range.size)
+                            {
+                                const size_t overlap = range.size - offset;
+                                if (overlap >= sizeof(CONTEXT))
+                                {
+                                    context = reinterpret_cast<const CONTEXT *>(range.ptr + offset);
+                                }
+                            }
+                        }
+                    }
+
+                    // NOTE:  We don't currently use the TEB for anything else.  If we need it in
+                    // the future, the 32-bit TEB is located according to the address stored in the
+                    // first slot of the 64-bit TEB (wow64teb.Reserved1[0]).
+                }
+
                 thread_sp->SetContext(context);
             }
             new_thread_list.AddThread(thread_sp);
         }
     }
 
     return new_thread_list.GetSize(false) > 0;
 }

 
 ArchSpec
 ProcessWinMiniDump::GetArchitecture()
 {
     // TODO
     return ArchSpec();
 }
 
-
-ProcessWinMiniDump::Data::Data() :
-    m_dump_file(INVALID_HANDLE_VALUE),
-    m_mapping(NULL),
-    m_base_addr(nullptr)
+ProcessWinMiniDump::Data::Data()
+    : m_dump_file(INVALID_HANDLE_VALUE), m_mapping(NULL), m_base_addr(nullptr), m_is_wow64(false)
 {
 }
 
 ProcessWinMiniDump::Data::~Data()
 {
     if (m_base_addr)
     {
         ::UnmapViewOfFile(m_base_addr);

 
 bool
 ProcessWinMiniDump::FindMemoryRange(lldb::addr_t addr, Range *range_out) const
 {
     size_t stream_size = 0;
     auto mem_list_stream = static_cast<const MINIDUMP_MEMORY_LIST *>(FindDumpStream(MemoryListStream, &stream_size));
     if (mem_list_stream)
     {
-        for (ULONG32 i = 0; i < mem_list_stream->NumberOfMemoryRanges; ++i) {
+        for (ULONG32 i = 0; i < mem_list_stream->NumberOfMemoryRanges; ++i)
+        {
             const MINIDUMP_MEMORY_DESCRIPTOR &mem_desc = mem_list_stream->MemoryRanges[i];
             const MINIDUMP_LOCATION_DESCRIPTOR &loc_desc = mem_desc.Memory;
             const lldb::addr_t range_start = mem_desc.StartOfMemoryRange;
             const size_t range_size = loc_desc.DataSize;
             if (range_start <= addr && addr < range_start + range_size)
             {
                 range_out->start = range_start;
                 range_out->size = range_size;

 ProcessWinMiniDump::ReadExceptionRecord()
 {
     size_t size = 0;
     auto exception_stream_ptr = static_cast<MINIDUMP_EXCEPTION_STREAM*>(FindDumpStream(ExceptionStream, &size));
     if (exception_stream_ptr)
     {
         m_data_up->m_exception_sp.reset(new ExceptionRecord(exception_stream_ptr->ExceptionRecord, exception_stream_ptr->ThreadId));
     }
+    else
+    {
+        WINLOG_IFALL(WINDOWS_LOG_PROCESS, "Minidump has no exception record.");
+        // TODO:  See if we can recover the exception from the TEB.
+    }
 }
 
 void
 ProcessWinMiniDump::ReadMiscInfo()
 {
     size_t size = 0;
     const auto misc_info_ptr = static_cast<MINIDUMP_MISC_INFO*>(FindDumpStream(MiscInfoStream, &size));
     if (!misc_info_ptr || size < sizeof(MINIDUMP_MISC_INFO)) {
ProcessWinMiniDump::ReadModuleList()
     {
         return;
     }
 
     for (ULONG32 i = 0; i < module_list_ptr->NumberOfModules; ++i)
     {
         const auto &module = module_list_ptr->Modules[i];
         const auto file_name = GetMiniDumpString(m_data_up->m_base_addr, module.ModuleNameRva);
-        ModuleSpec module_spec = FileSpec(file_name, true);
+        const auto file_spec = FileSpec(file_name, true);
+        if (FileSpec::Compare(file_spec, FileSpec("wow64.dll", false), false) == 0)
+        {
+            WINLOG_IFALL(WINDOWS_LOG_PROCESS, "Minidump is for a WOW64 process.");
+            m_data_up->m_is_wow64 = true;
+        }
+        ModuleSpec module_spec = file_spec;
 
         lldb::ModuleSP module_sp = GetTarget().GetSharedModule(module_spec);
         if (!module_sp)
         {
             continue;
         }
         bool load_addr_changed = false;
         module_sp->SetLoadAddress(GetTarget(), module.BaseOfImage, false, load_addr_changed);