[dsymutil] Force mmap'ing of binaries

[JDevlieghere]

Summary

After the recent refactoring that introduced parallel handling of different object, the binary holder became unique per object file. This defeats its optimization of caching archives, leading to an archive being opened for every binary it contains. This is obviously unfortunate and will need to be refactored soon.

Luckily in practice, the impact of this is limited as most files are mmap'ed instead of memcopy'd. There's a caveat however: when the memory buffer requires a null terminator and it's a multiple of the page size, we allocate instead of mmap'ing. If this happens for a static archive, we end up with N copies of it in memory, where N is the number of objects in the archive, leading to exuberant memory usage. This provided a stopgap solution to ensure that all the files it loads are mmap in memory by removing the requirement for a terminating null byte.

[friss]

If you have a local test case with an archive that triggers the malloc behavior, can you run an Asanified dsymutil on it with your patch applied? I'd like to make sure that nothing in dsymutil is going to access one past the end of the file.

[JDevlieghere]

In D48397#1138455, @friss wrote:

If you have a local test case with an archive that triggers the malloc behavior, can you run an Asanified dsymutil on it with your patch applied? I'd like to make sure that nothing in dsymutil is going to access one past the end of the file.

Done, no invalid memory access detected. This sounds right to me, as the MemoryBuffer sets the size of the buffer and libObject does a relatively good job at bounds checking.

[friss]

LGTM Thanks!

llvm/trunk/tools/dsymutil/BinaryHolder.cpp

if (auto ErrOrArchiveMembers =
             MapArchiveAndGetMemberBuffers(Filename, Timestamp))
       return *ErrOrArchiveMembers;
 
   // Otherwise, just try opening a standard file. If this is an
   // archive member specifiaction and any of the above didn't handle it
   // (either because the archive is not there anymore, or because the
   // archive doesn't contain the requested member), this will still
   // provide a sensible error message.
-  auto ErrOrFile = MemoryBuffer::getFileOrSTDIN(Filename);
+  auto ErrOrFile = MemoryBuffer::getFileOrSTDIN(Filename, -1, false);
   if (auto Err = ErrOrFile.getError())
     return Err;
 
   changeBackingMemoryBuffer(std::move(*ErrOrFile));
   if (Verbose)
     outs() << "\tloaded file.\n";
 
   auto ErrOrFat = object::MachOUniversalBinary::create(
ErrorOr<std::vector<MemoryBufferRef>> BinaryHolder::GetArchiveMemberBuffers(
   return Buffers;
 }
 
 ErrorOr<std::vector<MemoryBufferRef>>
 BinaryHolder::MapArchiveAndGetMemberBuffers(
     StringRef Filename, sys::TimePoint<std::chrono::seconds> Timestamp) {
   StringRef ArchiveFilename = Filename.substr(0, Filename.find('('));
 
-  auto ErrOrBuff = MemoryBuffer::getFileOrSTDIN(ArchiveFilename);
+  auto ErrOrBuff = MemoryBuffer::getFileOrSTDIN(ArchiveFilename, -1, false);
   if (auto Err = ErrOrBuff.getError())
     return Err;
 
   if (Verbose)
     outs() << "\topened new archive '" << ArchiveFilename << "'\n";
 
   changeBackingMemoryBuffer(std::move(*ErrOrBuff));
   std::vector<MemoryBufferRef> ArchiveBuffers;