Fix [vdso] handling on Android (x86 and aarch64)

[tberghammer]

Summary

Fix [vdso] handling on Android (x86 and aarch64)

• Add in-memory object file handling to the core dynamic loader
• Fix in memory object file handling in ObjectFileELF (previously only part of the file was loaded before parsing)
• Fix load address setting in ObjectFileELF for 32-bit targets when the load bias is negative
• Change hack in DYLDRendezvous.cpp to be more specific and not to interfere with object files with fixed load address

[tberghammer]

I investigated the problem on Linux a bit further and the issue causing us problems (with the missing path for the [vdso]) is not a bug, but a feature of the Linux linker. We have to add some code to LLDB in the future to handle the feature.

[labath]

The change looks reasonable to me, but i'll leave the decision to greg.

Would it be possible to add a test for this somehow? Check the output of image list? Look up a symbol in the vdso? Try to hit a breakpoint inside it?

[tberghammer]

In D10800#196243, @labath wrote:

The change looks reasonable to me, but i'll leave the decision to greg.

Would it be possible to add a test for this somehow? Check the output of image list? Look up a symbol in the vdso? Try to hit a breakpoint inside it?

It is quite complicated to test anything inside the [vdso] because as far as I know there is no specification about the list of functions what should be present in it. Checking the output of "image list" is possible but it won't give us too much coverage because it will be displayed there in several cases when it isn't loaded correctly.

[labath]

In D10800#196359, @tberghammer wrote:

In D10800#196243, @labath wrote:

Would it be possible to add a test for this somehow? Check the output of image list? Look up a symbol in the vdso? Try to hit a breakpoint inside it?

It is quite complicated to test anything inside the [vdso] because as far as I know there is no specification about the list of functions what should be present in it. Checking the output of "image list" is possible but it won't give us too much coverage because it will be displayed there in several cases when it isn't loaded correctly.

Yes, I suspect the contents of the vdso is highly platform-dependent. However, it (vdso) has to remain backward compatible with old binaries (and old versions of libc), therefore I think it's unlikely that a symbol will disappear from there overnight. I think the vdso support is very prone to accidental breakage at this point, so it would be good to get *any* coverage - it does not have to be exhaustive, but picking a symbol on one-or-two platforms and checking that should be possible.

[clayborg]

Looks good. I am very happy to see memory loading of ELF files.

[vharron]

If you've fixed vdso handling, shouldn't some i386 tests be marked as passing?

[tberghammer]

In D10800#196543, @vharron wrote:

If you've fixed vdso handling, shouldn't some i386 tests be marked as passing?

It is fixing it only on Android. There is a different between how the linker works on Linux and on Android and we need to solve one more additional issue to fix it on Linux also but it isn't trivial.

The problem is that the Linux linker don't fill in the file path in the rendezvous structure for the [vdso] because it isn't a file (it is treated as a feature in the code) but currently we use that path to find the load address of the file (based on /proc/<pid>/maps). To fix the issue on Linux also we need an other way to get the load address of an SO.

[vharron]

You can use the gdb remote get auxv command I think to get libvdso load
address.

source/Core/DynamicLoader.cpp

DynamicLoader::LoadModuleAtAddress(const FileSpec &file, addr_t link_map_addr, addr_t base_addr)
     if ((module_sp = modules.FindFirstModule (module_spec)))
     {
         UpdateLoadedSections(module_sp, link_map_addr, base_addr);
     }
     else if ((module_sp = target.GetSharedModule(module_spec)))
     {
         UpdateLoadedSections(module_sp, link_map_addr, base_addr);
     }
+    else
+    {
+        // Try to fetch the load address of the file from the process. It can be different from the
+        // address reported by the linker in case of a file with fixed load address because the
+        // linker reports the bias between the load address specified in the file and the actual
+        // load address it loaded the file.
+        bool is_loaded;
+        lldb::addr_t load_addr;
+        Error error = m_process->GetFileLoadAddress(file, is_loaded, load_addr);
+        if (error.Fail() || !is_loaded)
+            load_addr = base_addr;
+
+        if ((module_sp = m_process->ReadModuleFromMemory(file, load_addr)))
+        {
+            UpdateLoadedSections(module_sp, link_map_addr, base_addr);
+            target.GetImages().AppendIfNeeded(module_sp);
+        }
+    }
 
     return module_sp;
 }
 
 int64_t
 DynamicLoader::ReadUnsignedIntWithSizeInBytes(addr_t addr, int size_in_bytes)
 {
     Error error;

source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

if (!(addr = ReadPointer(addr, &entry.dyn_addr)))
         return false;
     
     if (!(addr = ReadPointer(addr, &entry.next)))
         return false;
     
     if (!(addr = ReadPointer(addr, &entry.prev)))
         return false;
 
-    entry.file_spec.SetFile(ReadStringFromMemory(entry.path_addr), false);
+    std::string file_path = ReadStringFromMemory(entry.path_addr);
+    entry.file_spec.SetFile(file_path, false);
 
-    // The base_addr is not filled in for some case.
-    // Try to figure it out based on the load address of the object file.
-    // The issue observed for '/system/bin/linker' on Android L (5.0, 5.1)
-    if (entry.base_addr == 0)
+    // On Android L (5.0, 5.1) the load address of the "/system/bin/linker" isn't filled in
+    // correctly. To get the correct load address we fetch the load address of the file from the
+    // proc file system.
+    if (arch.GetTriple().getEnvironment() == llvm::Triple::Android && entry.base_addr == 0 &&
+        (file_path == "/system/bin/linker" || file_path == "/system/bin/linker64"))
     {
         lldb::addr_t load_addr = LLDB_INVALID_ADDRESS;
         bool is_loaded = false;
         Error error = m_process->GetFileLoadAddress(entry.file_spec, is_loaded, load_addr);
         if (error.Success() && is_loaded)
             entry.base_addr = load_addr;
     }
 

source/Plugins/ObjectFile/ELF/ObjectFileELF.h

private:
     ObjectFileELF(const lldb::ModuleSP &module_sp,
                   lldb::DataBufferSP& data_sp,
                   lldb::offset_t data_offset,
                   const lldb_private::FileSpec* file,
                   lldb::offset_t offset,
                   lldb::offset_t length);
 
     ObjectFileELF (const lldb::ModuleSP &module_sp,
-                   lldb::DataBufferSP& data_sp,
+                   lldb::DataBufferSP& header_data_sp,
                    const lldb::ProcessSP &process_sp,
                    lldb::addr_t header_addr);
 
     typedef std::vector<elf::ELFProgramHeader>  ProgramHeaderColl;
     typedef ProgramHeaderColl::iterator         ProgramHeaderCollIter;
     typedef ProgramHeaderColl::const_iterator   ProgramHeaderCollConstIter;
 
     struct ELFSectionHeaderInfo : public elf::ELFSectionHeader

source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

ObjectFileELF::ObjectFileELF (const lldb::ModuleSP &module_sp,
     m_arch_spec()
 {
     if (file)
         m_file = *file;
     ::memset(&m_header, 0, sizeof(m_header));
 }
 
 ObjectFileELF::ObjectFileELF (const lldb::ModuleSP &module_sp,
-                              DataBufferSP& data_sp,
+                              DataBufferSP& header_data_sp,
                               const lldb::ProcessSP &process_sp,
                               addr_t header_addr) :
-    ObjectFile(module_sp, process_sp, LLDB_INVALID_ADDRESS, data_sp),
+    ObjectFile(module_sp, process_sp, header_addr, header_data_sp),
     m_header(),
     m_uuid(),
     m_gnu_debuglink_file(),
     m_gnu_debuglink_crc(0),
     m_program_headers(),
     m_section_headers(),
     m_dynamic_symbols(),
     m_filespec_ap(),
if (module_sp)
                 for (sect_idx = 0; sect_idx < num_sections; ++sect_idx)
                 {
                     // Iterate through the object file sections to find all
                     // of the sections that have SHF_ALLOC in their flag bits.
                     SectionSP section_sp (section_list->GetSectionAtIndex (sect_idx));
                     // if (section_sp && !section_sp->IsThreadSpecific())
                     if (section_sp && section_sp->Test(SHF_ALLOC))
                     {
-                        if (target.GetSectionLoadList().SetSectionLoadAddress (section_sp, section_sp->GetFileAddress() + value))
+                        lldb::addr_t load_addr = section_sp->GetFileAddress() + value;
+                        
+                        // On 32-bit systems the load address have to fit into 4 bytes. The rest of
+                        // the bytes are the overflow from the addition.
+                        if (GetAddressByteSize() == 4)
+                            load_addr &= 0xFFFFFFFF;
+
+                        if (target.GetSectionLoadList().SetSectionLoadAddress (section_sp, load_addr))
                             ++num_loaded_sections;
                     }
                 }
                 return num_loaded_sections > 0;
             }
             else
             {
                 // Not sure how to slide an ELF file given the base address

 {
     return std::distance(m_section_headers.begin(), I) + 1u;
 }
 
 bool
 ObjectFileELF::ParseHeader()
 {
     lldb::offset_t offset = 0;
-    return m_header.Parse(m_data, &offset);
+    if (!m_header.Parse(m_data, &offset))
+        return false;
+
+    if (!IsInMemory())
+        return true;
+
+    // For in memory object files m_data might not contain the full object file. Try to load it
+    // until the end of the "Section header table" what is at the end of the ELF file.
+    addr_t file_size = m_header.e_shoff + m_header.e_shnum * m_header.e_shentsize;
+    if (m_data.GetByteSize() < file_size)
+    {
+        ProcessSP process_sp (m_process_wp.lock());
+        if (!process_sp)
+            return false;
+
+        DataBufferSP data_sp = ReadMemory(process_sp, m_memory_addr, file_size);
+        if (!data_sp)
+            return false;
+        m_data.SetData(data_sp, 0, file_size);
+    }
+
+    return true;
 }
 
 bool
 ObjectFileELF::GetUUID(lldb_private::UUID* uuid)
 {
     // Need to parse the section list to get the UUIDs, so make sure that's been done.
     if (!ParseSectionHeaders() && GetType() != ObjectFile::eTypeCoreFile)
         return false;