[LLDB][Minidump] Merge executable module's architecture into target's architecture.

[zequanwu]

Summary

This allows minidump process ABI to match the PE/COFF file ABI.

[labath]

This dual environment thing is tricky.. MSVC is probably a better default, but it would still mean that the same bugs will appear if the situation is reversed (we assume MSVC environ, but the binary actually uses GNU). Have you looked at how this interacts with https://reviews.llvm.org/D127048 (and the setting introduced there) and whether you can achieve the same effect by changing that setting?

[zequanwu]

In D137873#3926821, @labath wrote:

This dual environment thing is tricky.. MSVC is probably a better default, but it would still mean that the same bugs will appear if the situation is reversed (we assume MSVC environ, but the binary actually uses GNU). Have you looked at how this interacts with https://reviews.llvm.org/D127048 (and the setting introduced there) and whether you can achieve the same effect by changing that setting?

Thanks, I didn't realize the existence of this setting.

[zequanwu]

Oh, they are not the same. I will work on a similar setting for minidump.

[zequanwu]

Add settings set plugin.process.minidump.abi msvc/gnu to override minidump abi triple.

[labath]

Adding a new setting isn't exactly what I had in mind. I was actually hoping that it would be enough to fiddle with that setting and leave the minidump environment blank (because we have code to fill in the missing pieces of a target ArchSpec from other sources.

If that doesn't work then (besides knowing why), I'd like us try some kind of a single-setting solution, as I don't think it makes sense to have two different settings like this (happy to hear counterarguments to that). I'd consider either moving that environment setting to some place more generic (so it can be accessed from both plugins) or just having the minidump plugin fetch the setting from inside ObjectFilePECOFF (I'd say that a process->object dependency is kinda OK, and we already have ProcessMachCore depending on ObjectFileMachO).

lldb/source/Plugins/Process/minidump/ProcessMinidump.cpp

392

Is "overrides" the correct word here? Are there any circumstances in which we are able to determine the environment from the minidump file? Because if it is, then I would expect this to be the other way around (that the environment from a specific file overrides the generic catch-all setting)...

[mstorsjo]

In D137873#3930683, @labath wrote:

If that doesn't work then (besides knowing why), I'd like us try some kind of a single-setting solution, as I don't think it makes sense to have two different settings like this (happy to hear counterarguments to that)

Overall, the ideal is if the user usually needs to twiddle one single setting. For a majority of cases, picking the default format (like is done for the objectfile setting, and afaik this one does too even though I didn't quite clearly see the codepath for it) will be what you'd want: If you're working with mingw tools, debugging with a mingw built lldb, then your app that you're debugging also probably uses mingw ABIs. But it's clearly not implausible to want to debug executables from the other ecosystem, so being able to change it with a single setting is good too.

And for the objectfile setting, it's possible to set it even on a per-DLL basis. This may sound contrieved, but it's actually totally reasonable. While the C++ ABIs are incompatible, such DLLs can interoperate over C APIs - this is not an uncommon scenario. Plus, every mingw executable links against msvcrt.dll or ucrtbase.dll (provided by the system) which use the MSVC C++ ABIs internally (if you use the versions of them with debug info).

So the ability to set it per DLL for the objectfile part is totally warranted. How that fits in towards minidump reading, I don't really know though (since the whole minidump contains stuff for the whole process). In which cases does the C++ ABI form matter for the minidump btw - since the actual interpretation of things still would be based on per-DLL debug info (either as DWARF or PDB)?

[zequanwu]

In D137873#3931888, @mstorsjo wrote:

In D137873#3930683, @labath wrote:

If that doesn't work then (besides knowing why), I'd like us try some kind of a single-setting solution, as I don't think it makes sense to have two different settings like this (happy to hear counterarguments to that)

Overall, the ideal is if the user usually needs to twiddle one single setting. For a majority of cases, picking the default format (like is done for the objectfile setting, and afaik this one does too even though I didn't quite clearly see the codepath for it) will be what you'd want: If you're working with mingw tools, debugging with a mingw built lldb, then your app that you're debugging also probably uses mingw ABIs. But it's clearly not implausible to want to debug executables from the other ecosystem, so being able to change it with a single setting is good too.

And for the objectfile setting, it's possible to set it even on a per-DLL basis. This may sound contrieved, but it's actually totally reasonable. While the C++ ABIs are incompatible, such DLLs can interoperate over C APIs - this is not an uncommon scenario. Plus, every mingw executable links against msvcrt.dll or ucrtbase.dll (provided by the system) which use the MSVC C++ ABIs internally (if you use the versions of them with debug info).

So the ability to set it per DLL for the objectfile part is totally warranted. How that fits in towards minidump reading, I don't really know though (since the whole minidump contains stuff for the whole process). In which cases does the C++ ABI form matter for the minidump btw - since the actual interpretation of things still would be based on per-DLL debug info (either as DWARF or PDB)?

lldb creates AST for user input expression which uses the abi from minidump process. That AST's class layout info is imported from another AST created from debug info that uses the abi specified in debug info. The current situation is that minidump process uses itanium ABI (because it's not specified in minidump, falls back to use itanium), but the debug info uses msvc ABI. The inconsistent causes crash when lowering class.

It looks like it's impossible to make it works with DLLs that have different ABIs, no matter which ABI we set for minidump. Unless we change how clang interacts with external layout: https://github.com/llvm/llvm-project/blob/main/clang/lib/AST/RecordLayoutBuilder.cpp#L1357. Maybe change it so that if using external layout failed due to mismatched ABI, try again with another ABI https://github.com/llvm/llvm-project/blob/main/clang/lib/AST/RecordLayoutBuilder.cpp#L3302?

Or maybe for now, we should just choose the default ABI for windows minidump to be MSVC to at least to prevent that crash happening in our case. That's what I did in first diff.

lldb/source/Plugins/Process/minidump/ProcessMinidump.cpp

392

Yeah, the word is not correct. It should be "sets".
I think minidump file just doesn't have that environment info: https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/BinaryFormat/Minidump.h#L161-L179. lldb sets the environment based on platform for android: https://github.com/llvm/llvm-project/blob/main/lldb/source/Plugins/Process/minidump/MinidumpParser.cpp#L195.

[alvinhochun]

I think the module spec of the modules should reflect the value of the plugin.object-file.pe-coff.abi setting. Can ProcessMinidump take the module spec of the executable module to use its target environment?

Mixing MSVC and MinGW DLLs is a tricky case so I will not be surprised if something doesn't work correctly. It would be nice to have it working but that would be outside the scope of this review. (I don't yet have a use case for this but I am getting close with some COM / WinRT experiments.)

[labath]

In D137873#3933171, @alvinhochun wrote:

I think the module spec of the modules should reflect the value of the plugin.object-file.pe-coff.abi setting. Can ProcessMinidump take the module spec of the executable module to use its target environment?

That's kinda what Target::MergeArchitecture is for.

Mixing MSVC and MinGW DLLs is a tricky case so I will not be surprised if something doesn't work correctly. It would be nice to have it working but that would be outside the scope of this review. (I don't yet have a use case for this but I am getting close with some COM / WinRT experiments.)

Sounds fun. :)

[zequanwu]

Use plugin.object-file.pe-coff.abi as minidump process abi when OS is Windows.

[labath]

Did you look at the Target::MergeArchitecture function? Is there a suitable place where we could insert a call to that function so that the executable object file environment is automatically propagated to the target?

[zequanwu]

Merge executable module's architecture into target's architecture.

[zequanwu]

lldb/test/Shell/SymbolFile/Breakpad/unwind-via-stack-win-no-memory-info.yaml

74

Processor Arch is X86, so this should be i386.

[labath]

Cool, great.

lldb/source/Plugins/Process/minidump/ProcessMinidump.cpp

Status ProcessMinidump::DoLoadCore() {
   GetTarget().SetArchitecture(arch, true /*set_platform*/);
 
   m_thread_list = m_minidump_parser->GetThreads();
   m_active_exception = m_minidump_parser->GetExceptionStream();
 
   SetUnixSignals(UnixSignals::Create(GetArchitecture()));
 
   ReadModuleList();
-
+  if (ModuleSP module = GetTarget().GetExecutableModule())
+    GetTarget().MergeArchitecture(module->GetArchitecture());
   llvm::Optional<lldb::pid_t> pid = m_minidump_parser->GetPid();
   if (!pid) {
     Debugger::ReportWarning("unable to retrieve process ID from minidump file, "
                             "setting process ID to 1",
                             GetTarget().GetDebugger().GetID());
     pid = 1;
   }
   SetID(*pid);
if (mem.empty()) {
     return 0;
   }
 
   std::memcpy(buf, mem.data(), mem.size());
   return mem.size();
 }
 
 ArchSpec ProcessMinidump::GetArchitecture() {
   if (!m_is_wow64) {
     return m_minidump_parser->GetArchitecture();
   }
 
   llvm::Triple triple;
   triple.setVendor(llvm::Triple::VendorType::UnknownVendor);
   triple.setArch(llvm::Triple::ArchType::x86);
   triple.setOS(llvm::Triple::OSType::Win32);
   return ArchSpec(triple);

lldb/test/Shell/Minidump/Windows/find-module.test

 Test that we correctly find a PE/COFF file in our executable search path, and
 use it when opening minidumps.
 
 RUN: yaml2obj %S/Inputs/find-module.exe.yaml -o %T/find-module.exe
 RUN: yaml2obj %S/Inputs/find-module.dmp.yaml -o %T/find-module.dmp
 RUN: %lldb -O "settings set target.exec-search-paths %T" \
-RUN:   -c %T/find-module.dmp -o "image dump objfile" -o exit | FileCheck %s
+RUN:   -c %T/find-module.dmp -o "image dump objfile" -o "target list" -o exit \
+RUN:   | FileCheck --check-prefix=DEFAULT %s
 
-CHECK-LABEL: image dump objfile
-CHECK: ObjectFilePECOFF, file = '{{.*}}find-module.exe', arch = i386
+RUN: %lldb -O "settings set plugin.object-file.pe-coff.abi msvc" \
+RUN:   -O "settings set target.exec-search-paths %T" -c %T/find-module.dmp \
+RUN:   -o "target list" -o exit | FileCheck --check-prefix=MSVC %s
+
+RUN: %lldb -O "settings set plugin.object-file.pe-coff.abi gnu" \
+RUN:   -O "settings set target.exec-search-paths %T" -c %T/find-module.dmp \
+RUN:   -o "target list" -o exit | FileCheck --check-prefix=GNU %s
+
+DEFAULT-LABEL: image dump objfile
+DEFAULT: ObjectFilePECOFF, file = '{{.*}}find-module.exe', arch = i386
+
+DEFAULT-LABEL: target list
+DEFAULT: arch=i386-pc-windows-{{msvc|gnu}}
+
+MSVC-LABEL: target list
+MSVC: arch=i386-pc-windows-msvc
+
+GNU-LABEL: target list
+GNU: arch=i386-pc-windows-gnu

lldb/test/Shell/SymbolFile/Breakpad/unwind-via-stack-win-no-memory-info.yaml

CPU:
       Vendor ID:       AuthenticAMD
       Version Info:    0x00800F82
       Feature Info:    0x178BFBFF
   - Type:            MiscInfo
     Content:         54050000F703000008290000C883495D0000000000000000AC0D000098080000AC0D00000200000002000000002000000D0000000000000002000000C4FFFFFF430065006E007400720061006C0020004500750072006F007000650020005300740061006E0064006100720064002000540069006D006500000000000000000000000A0000000500030000000000000000000000430065006E007400720061006C0020004500750072006F00700065002000530075006D006D00650072002000540069006D00650000000000000000000000000000000300000005000200000000000000C4FFFFFF310037003100330034002E0031002E007800380036006600720065002E007200730034005F00720065006C0065006100730065002E003100380030003400310030002D00310038003000340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000064006200670063006F00720065002E0069003300380036002C00310030002E0030002E00310037003100330034002E0031000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006C816B92
 ...
 
 --- !COFF
-OptionalHeader:  
+OptionalHeader:
   AddressOfEntryPoint: 0
   ImageBase:       720896
   SectionAlignment: 4096
   FileAlignment:   512
   MajorOperatingSystemVersion: 6
   MinorOperatingSystemVersion: 0
   MajorImageVersion: 0
   MinorImageVersion: 0
   MajorSubsystemVersion: 6
   MinorSubsystemVersion: 0
   Subsystem:       IMAGE_SUBSYSTEM_WINDOWS_GUI
   DLLCharacteristics: [ IMAGE_DLL_CHARACTERISTICS_HIGH_ENTROPY_VA, IMAGE_DLL_CHARACTERISTICS_DYNAMIC_BASE, IMAGE_DLL_CHARACTERISTICS_NX_COMPAT ]
   SizeOfStackReserve: 1048576
   SizeOfStackCommit: 4096
   SizeOfHeapReserve: 1048576
   SizeOfHeapCommit: 4096
-  Debug:           
+  Debug:
     RelativeVirtualAddress: 8192
     Size:            28
-header:          
-  Machine:         IMAGE_FILE_MACHINE_AMD64
+header:
+  Machine:         IMAGE_FILE_MACHINE_I386
   Characteristics: [ IMAGE_FILE_EXECUTABLE_IMAGE, IMAGE_FILE_LARGE_ADDRESS_AWARE, IMAGE_FILE_DLL ]
-sections:        
+sections:
   - Name:            .text
     Characteristics: [ IMAGE_SCN_CNT_CODE, IMAGE_SCN_MEM_EXECUTE, IMAGE_SCN_MEM_READ ]
     VirtualAddress:  4096
     VirtualSize:     512
   - Name:            .rdata
     Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ ]
     VirtualAddress:  8192
     VirtualSize:     236
     SectionData:     00000000A565B65C00000000020000006B0000001C2000001C060000525344533ED87D89C8A8184197F3A925EE4BF74101
 symbols:         []
 ...