[LLDB][MIPS] Fix Core file Architecture and OS information

[nitesh.jain]

Summary

The MIPS core file doesn't contain any Architecture revision, OS and ABI information. Hence we need to relied on executable to get that information. This patch also add core file test cases for MIPS.

Patch By:- Bhushan

[labath]

source/Plugins/Process/elf-core/ProcessElfCore.cpp

675

I'm not terribly happy with the default-to-host mode here, particularly as we already have some code which detects linux in ObjectFileELF::RefineModuleDetailsFromNote. I'm not terribly happy about that either, but I hope we could at least have just one dodgy detection code.

Did you check whether it's possible to extend that function to cover mips as well (probably the NT_FILE part, which searches for some i386-linux-gnu paths in the binary) ?

[labath]

BTW, thank you for adding the test. It's not obvious from the patch: how big are the core files?

[nitesh.jain]

They are around ~28KB

[nitesh.jain]

source/Plugins/Process/elf-core/ProcessElfCore.cpp

675

In our case , files path doesn't contain any Linux string.

nin@debian-co3-1:~/LLVM-new/llvm/tools/lldb/packages/Python/lldbsuite/test/functionalities/postmortem/elf-core$ readelf -a linux-mips64el-gnuabi64.core
ELF Header:

Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class:                             ELF64
Data:                              2's complement, little endian
Version:                           1 (current)
OS/ABI:                            UNIX - System V
ABI Version:                       0
Type:                              CORE (Core file)
Machine:                           MIPS R3000
Version:                           0x1
Entry point address:               0x0
Start of program headers:          64 (bytes into file)
Start of section headers:          0 (bytes into file)
Flags:                             0x0
Size of this header:               64 (bytes)
Size of program headers:           56 (bytes)
Number of program headers:         6
Size of section headers:           0 (bytes)
Number of section headers:         0
Section header string table index: 0

There are no sections in this file.

There are no sections to group in this file.

Program Headers:

Type           Offset             VirtAddr           PhysAddr
               FileSiz            MemSiz              Flags  Align
NOTE           0x0000000000000190 0x0000000000000000 0x0000000000000000
               0x0000000000000644 0x0000000000000000         0x0
LOAD           0x0000000000001000 0x0000000120000000 0x0000000000000000
               0x0000000000001000 0x0000000000001000  R E    0x1000
LOAD           0x0000000000002000 0x000000fff2a60000 0x0000000000000000
               0x0000000000001000 0x0000000000001000  R      0x1000
LOAD           0x0000000000003000 0x000000fff2a61000 0x0000000000000000
               0x0000000000001000 0x0000000000001000  R E    0x1000
LOAD           0x0000000000004000 0x000000ffffc01000 0x0000000000000000
               0x0000000000002000 0x0000000000002000  RWE    0x1000
LOAD           0x0000000000006000 0x000000fffffff000 0x0000000000000000
               0x0000000000000000 0x0000000000001000  RWE    0x1000

There is no dynamic section in this file.

There are no relocations in this file.

The decoding of unwind sections for machine type MIPS R3000 is not currently supported.

Dynamic symbol information is not available for displaying symbols.

No version information found in this file.

Displaying notes found at file offset 0x00000190 with length 0x00000644:

Owner                 Data size       Description
CORE                 0x000001e0       NT_PRSTATUS (prstatus structure)
CORE                 0x00000088       NT_PRPSINFO (prpsinfo structure)
CORE                 0x00000080       NT_SIGINFO (siginfo_t data)
CORE                 0x00000130       NT_AUXV (auxiliary vector)
CORE                 0x000000aa       NT_FILE (mapped files)
  Page size: 4096
               Start                 End         Page Offset
  0x0000000120000000  0x0000000120001000  0x0000000000000000
      /home/nin/LLVM-new/llvm/tools/lldb/packages/Python/lldbsuite/test/functionalities/postmortem/elf-core/linux-mips64el-gnuabi64.out

[labath]

source/Plugins/Process/elf-core/ProcessElfCore.cpp

675

That's a good point. In fact that is the case on the x86 core files as well we have checked in as well, as their executables were linked statically. Only, I guess in the x86 case we don't rely on the os that much.

Will this affect the ability of the tests to run reproducibly (on windows for example), or will the os be picked up from the executable if it is present? If it won't then we'll need to think hard about how to make the tests deterministic.

[nitesh.jain]

source/Plugins/Process/elf-core/ProcessElfCore.cpp

675

If Host OS is windows then this test case will fail. The MIPS support freeBSD, Linux and Android. The note.n_name == LLDB_NT_OWNER_GNU is valid for Linux platform. Hence we can used this information to set OS.

[labath]

source/Plugins/Process/elf-core/ProcessElfCore.cpp

675

That sounds better. Can we try that?

[nitesh.jain]

updated diff as per suggestion

[labath]

Thank you for that.

I just have one more concern. I see two places where you are copying the archspec from core file to the target and then back again. I must be missing something, because that looks completely unnecessary at first sight. What is the reason behind that? Why can't we leave ProcessElfCore::GetArchitecture as is ?

source/Plugins/Process/elf-core/ProcessElfCore.cpp

220

Here you copy the architecture from core to target.

668

And here you fetch the target instead of the core architecture.

[nitesh.jain]

source/Plugins/Process/elf-core/ProcessElfCore.cpp

220

Actually we are skipping it , if the arch is MIPS ( ! arch.IsMIPS()).

[nitesh.jain]

The core architecture doesn't have any revision, os and ABI information. Hence we need to relied on target architecture.

[labath]

Ah, that explains it, thanks.

Let's give Greg a chance to comment, but otherwise I am fine with this.

[clayborg]

Let me know what you think about the MergeFrom comment. I am generally ok with this, but just wanted to check in case the merge made any sense in this patch somewhere.

source/Plugins/Process/elf-core/ProcessElfCore.cpp

220

So the target has an incomplete architecture here? If that is true, you should probably get the architecture and merge it. You can merge any missing info with:

ArchSpec target_arch = GetTarget().GetArchitecture();
ArchSpec core_arch(m_core_module_sp->GetArchitecture());
target_arch.MergeFrom(core_arch);
GetTarget().SetArchitecture(target_arch);

Not sure how that would work here, but we should be using this. If we have a target whose arch is "x86_64-apple-macosx" and you have a core file that is "mips-..." an error should be produced.

[nitesh.jain]

Update diff as per suggestion.

Thanks

[clayborg]

Looks good

packages/Python/lldbsuite/test/functionalities/postmortem/elf-core/TestLinuxCore.py

 class LinuxCoreTestCase(TestBase):
     NO_DEBUG_INFO_TESTCASE = True
 
     mydir = TestBase.compute_mydir(__file__)
 
     _i386_pid = 32306
     _x86_64_pid = 32259
     _s390x_pid = 1045
+    _mips64_n64_pid = 25619
+    _mips64_n32_pid = 3670
+    _mips_o32_pid = 3532
 
     _i386_regions = 4
     _x86_64_regions = 5
     _s390x_regions = 2
+    _mips_regions = 5
 
     def setUp(self):
         super(LinuxCoreTestCase, self).setUp()
         self._initial_platform = lldb.DBG.GetSelectedPlatform()
 
     def tearDown(self):
         lldb.DBG.SetSelectedPlatform(self._initial_platform)
         super(LinuxCoreTestCase, self).tearDown()
 
     @skipIf(oslist=['windows'])
     @skipIf(triple='^mips')
     def test_i386(self):
         """Test that lldb can read the process information from an i386 linux core file."""
         self.do_test("linux-i386", self._i386_pid, self._i386_regions)
 
+    def test_mips_o32(self):
+        """Test that lldb can read the process information from an MIPS O32 linux core file."""
+        self.do_test("linux-mipsel-gnuabio32", self._mips_o32_pid, self._mips_regions)
+
+    def test_mips_n32(self):
+        """Test that lldb can read the process information from an MIPS N32 linux core file """
+        self.do_test("linux-mips64el-gnuabin32", self._mips64_n32_pid, self._mips_regions)
+
+    def test_mips_n64(self):
+        """Test that lldb can read the process information from an MIPS N64 linux core file """
+        self.do_test("linux-mips64el-gnuabi64", self._mips64_n64_pid, self._mips_regions)
+
     @skipIf(oslist=['windows'])
     @skipIf(triple='^mips')
     def test_x86_64(self):
         """Test that lldb can read the process information from an x86_64 linux core file."""
         self.do_test("linux-x86_64", self._x86_64_pid, self._x86_64_regions)
 
     @skipIf(oslist=['windows'])
     @skipIf(triple='^mips')

source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

case 20:
     kal_arch_variant = llvm::Triple::KalimbaSubArch_v5;
     break;
   default:
     break;
   }
   return kal_arch_variant;
 }
 
-static uint32_t mipsVariantFromElfFlags(const elf::elf_word e_flags,
-                                        uint32_t endian) {
-  const uint32_t mips_arch = e_flags & llvm::ELF::EF_MIPS_ARCH;
+static uint32_t mipsVariantFromElfFlags (const elf::ELFHeader &header) {
+  const uint32_t mips_arch = header.e_flags & llvm::ELF::EF_MIPS_ARCH;
+  uint32_t endian = header.e_ident[EI_DATA];
   uint32_t arch_variant = ArchSpec::eMIPSSubType_unknown;
+  uint32_t fileclass = header.e_ident[EI_CLASS];
+
+  // If there aren't any elf flags available (e.g core elf file) then return default 
+  // 32 or 64 bit arch (without any architecture revision) based on object file's class.
+  if (header.e_type == ET_CORE) {
+    switch (fileclass) {
+    case llvm::ELF::ELFCLASS32:
+      return (endian == ELFDATA2LSB) ? ArchSpec::eMIPSSubType_mips32el
+                                     : ArchSpec::eMIPSSubType_mips32;
+    case llvm::ELF::ELFCLASS64:
+      return (endian == ELFDATA2LSB) ? ArchSpec::eMIPSSubType_mips64el
+                                     : ArchSpec::eMIPSSubType_mips64;
+    default:
+      return arch_variant;
+    }
+  }
 
   switch (mips_arch) {
   case llvm::ELF::EF_MIPS_ARCH_1:
   case llvm::ELF::EF_MIPS_ARCH_2:
   case llvm::ELF::EF_MIPS_ARCH_32:
     return (endian == ELFDATA2LSB) ? ArchSpec::eMIPSSubType_mips32el
                                    : ArchSpec::eMIPSSubType_mips32;
   case llvm::ELF::EF_MIPS_ARCH_32R2:
default:
     break;
   }
 
   return arch_variant;
 }
 
 static uint32_t subTypeFromElfHeader(const elf::ELFHeader &header) {
   if (header.e_machine == llvm::ELF::EM_MIPS)
-    return mipsVariantFromElfFlags(header.e_flags, header.e_ident[EI_DATA]);
+    return mipsVariantFromElfFlags(header);
 
   return llvm::ELF::EM_CSR_KALIMBA == header.e_machine
              ? kalimbaVariantFromElfFlags(header.e_flags)
              : LLDB_INVALID_CPUTYPE;
 }
 
 //! The kalimba toolchain identifies a code section as being
 //! one with the SHT_PROGBITS set in the section sh_type and the top
else if (note.n_name == LLDB_NT_OWNER_GNU) {
             }
 
             // Save the build id as the UUID for the module.
             uuid.SetBytes(uuidbuf, note.n_descsz);
           }
         }
         break;
       }
+      if (arch_spec.IsMIPS() &&
+          arch_spec.GetTriple().getOS() == llvm::Triple::OSType::UnknownOS)
+        // The note.n_name == LLDB_NT_OWNER_GNU is valid for Linux platform
+        arch_spec.GetTriple().setOS(llvm::Triple::OSType::Linux);
     }
     // Process NetBSD ELF notes.
     else if ((note.n_name == LLDB_NT_OWNER_NETBSD) &&
              (note.n_type == LLDB_NT_NETBSD_ABI_TAG) &&
              (note.n_descsz == LLDB_NT_NETBSD_ABI_SIZE)) {
       // Pull out the min version info.
       uint32_t version_info;
       if (data.GetU32(&offset, &version_info, 1) == nullptr) {
else if ((note.n_type == LLDB_NT_GNU_ABI_TAG) &&
             return error;
           }
           llvm::StringRef path(cstr);
           if (path.contains("/lib/x86_64-linux-gnu") || path.contains("/lib/i386-linux-gnu")) {
             arch_spec.GetTriple().setOS(llvm::Triple::OSType::Linux);
             break;
           }
         }
+        if (arch_spec.IsMIPS() &&
+            arch_spec.GetTriple().getOS() == llvm::Triple::OSType::UnknownOS)
+          // In case of MIPSR6, the LLDB_NT_OWNER_GNU note is missing
+          // for some cases (e.g. compile with -nostdlib)
+          // Hence set OS to Linux
+          arch_spec.GetTriple().setOS(llvm::Triple::OSType::Linux); 
       }
     }
 
     // Calculate the offset of the next note just in case "offset" has been used
     // to poke at the contents of the note data
     offset = note_offset + note.GetByteSize();
   }
 

source/Plugins/Process/elf-core/ProcessElfCore.cpp

Error ProcessElfCore::DoLoadCore() {
   if (!ranges_are_sorted) {
     m_core_aranges.Sort();
     m_core_range_infos.Sort();
   }
 
   // Even if the architecture is set in the target, we need to override
   // it to match the core file which is always single arch.
   ArchSpec arch(m_core_module_sp->GetArchitecture());
-  if (arch.IsValid())
+
+  // In case of MIPS, Core file doesn't have architecture revision and ABI
+  // information. Hence we need to relied on target architecture for that.
+  if (arch.IsValid() && !arch.IsMIPS())
     GetTarget().SetArchitecture(arch);
  
   SetUnixSignals(UnixSignals::Create(GetArchitecture()));
 
   // Ensure we found at least one thread that was stopped on a signal.
   bool siginfo_signal_found = false;
   bool prstatus_signal_found = false;
   // Check we found a signal in a SIGINFO note.
   for (const auto &thread_data: m_thread_data) {
     if (thread_data.signo != 0)
size_t ProcessElfCore::DoReadMemory(lldb::addr_t addr, void *buf, size_t size,
   const lldb::addr_t file_start = address_range->data.GetRangeBase();
   const lldb::addr_t file_end = address_range->data.GetRangeEnd();
   size_t bytes_to_read = size; // Number of bytes to read from the core file
   size_t bytes_copied = 0;   // Number of bytes actually read from the core file
   size_t zero_fill_size = 0; // Padding
   lldb::addr_t bytes_left =
       0; // Number of bytes available in the core file from the given address
 
+  // Don't proceed if core file doesn't contain the actual data for this address range.
+  if (file_start == file_end)
+    return 0;
+
   // Figure out how many on-disk bytes remain in this segment
   // starting at the given offset
   if (file_end > file_start + offset)
     bytes_left = file_end - (file_start + offset);
 
   // Figure out how many bytes we need to zero-fill if we are
   // reading more bytes than available in the on-disk segment
   if (bytes_to_read > bytes_left) {
if (note.n_name == "FreeBSD") {
         thread_data->gpregset = DataExtractor(note_data, header_size, len);
         break;
       case NT_FPREGSET:
         // In a i386 core file NT_FPREGSET is present, but it's not the result
         // of the FXSAVE instruction like in 64 bit files.
         // The result from FXSAVE is in NT_PRXFPREG for i386 core files
         if (arch.GetCore() == ArchSpec::eCore_x86_64_x86_64)
           thread_data->fpregset = note_data;
+        else if(arch.IsMIPS())
+          thread_data->fpregset = note_data;
         break;
       case NT_PRPSINFO:
         have_prpsinfo = true;
         error = prpsinfo.Parse(note_data, arch);
         if (error.Fail())
           return error;
         thread_data->name = prpsinfo.pr_fname;
         SetID(prpsinfo.pr_pid);
uint32_t ProcessElfCore::GetNumThreadContexts() {
   return m_thread_data.size();
 }
 
 ArchSpec ProcessElfCore::GetArchitecture() {
   ObjectFileELF *core_file =
       (ObjectFileELF *)(m_core_module_sp->GetObjectFile());
   ArchSpec arch;
   core_file->GetArchitecture(arch);
+
+  ArchSpec target_arch = GetTarget().GetArchitecture();
+  
+  if (target_arch.IsMIPS())
+    return target_arch;
+
   return arch;
 }
 
 const lldb::DataBufferSP ProcessElfCore::GetAuxvData() {
   const uint8_t *start = m_auxv.GetDataStart();
   size_t len = m_auxv.GetByteSize();
   lldb::DataBufferSP buffer(new lldb_private::DataBufferHeap(start, len));
   return buffer;
 }
 
 bool ProcessElfCore::GetProcessInfo(ProcessInstanceInfo &info) {