[Support] Add MemoryBuffer::dontNeedIfMmap

[MaskRay]

Summary

On *NIX systems, this API calls madvise(MADV_DONTNEED) on read-only file mappings.
It should not be used on a writable buffer.
The API is used to implement ld.lld LTO memory saving trick (D116367).

Note: on read-only file mappings, Linux's MADV_DONTNEED semantics match POSIX
POSIX_MADV_DONTNEED and BSD systems' MADV_DONTNEED.

On Windows, VirtualAllocEx MEM_COMMIT/MEM_RESET have similar semantics
but are unfortunately not drop-in replacements. dontNeedIfMmap is currently a no-op.

[aganea]

LGTM.

llvm/lib/Support/Unix/Path.inc

874

Since MADV_DONTNEED will drop all written changes to the memory pages, is it worth adding assert(Mode == mapped_file_region::readonly); here?

llvm/lib/Support/Windows/Path.inc

962

Probably the closest we could do on Windows is:

void mapped_file_region::dontNeedImpl() {
  assert(Mode == mapped_file_region::readonly);
  VirtualFree(Mapping, Size, MEM_DECOMMIT);
}

...but that's "destructive", since VirtualAlloc(..MEM_COMMIT..) would be required to access the pages again. That would require another function mapped_file_region::needImpl() ...

There's also VirtualAlloc(..MEM_RESET..) which is closer to MADV_DONTNEED but that doesn't work with mmap'd files.

[MaskRay]

Add an assert

llvm/lib/Support/Unix/Path.inc

874

Good idea! Applied.

llvm/lib/Support/Windows/Path.inc

962

Ah, then what can be done with Windows? Probably not many use Windows ld.lld, but Windows lld-link with lower memory usage may be useful to some groups.

For this patch, I'll not touch the Windows implementation if a direct MADV_DONTNEED counterpart cannot be found.

[MaskRay]

Clarify that dontNeedIfMmap should not be called on a writable buffer (Linux MADV_DONTNEED has unusual destructive semantics)

[zibi]

llvm/lib/Support/Unix/Path.inc

876

Unfortunately this change broke AIX build since MADV_DONTNEED is not available. Is there any other OS without MADV_DONTNEED? Can this be reverted until we find alternative?

[daltenty]

Unfortunately this change broke AIX build since MADV_DONTNEED is not available. Is there any other OS without MADV_DONTNEED? Can this be reverted until we find alternative?

Maybe we can swap to posix_madvise if the semantics match what we need here? That should be available on AIX.

[aganea]

@zibi @daltenty Is madvise not available on AIX, or is it a specific version of the OS that doesn't have it? The doc suggests it is available: https://www.ibm.com/docs/en/aix/7.1?topic=m-madvise-subroutine
Is that specific machine missing a header? Perhaps HAVE_SYS_MMAN_H isn't defined? @simon_tatham

Unless any of you have a way to quickly test a solution, short term should we just #ifndef _AIX the call to madvise?

[zibi]

In D116366#3215212, @aganea wrote:

@zibi @daltenty Is madvise not available on AIX, or is it a specific version of the OS that doesn't have it? The doc suggests it is available: https://www.ibm.com/docs/en/aix/7.1?topic=m-madvise-subroutine
Is that specific machine missing a header? Perhaps HAVE_SYS_MMAN_H isn't defined? @simon_tatham

Unless any of you have a way to quickly test a solution, short term should we just #ifndef _AIX the call to madvise?

@daltenty, I guess the bots are not on 7.1, right?
FYI, we will have to take the same route on z/OS as Windows did since either madvise nor posix_madvise is available.

[simon_tatham]

In D116366#3215212, @aganea wrote:

Is that specific machine missing a header? Perhaps HAVE_SYS_MMAN_H isn't defined? @simon_tatham

Sorry, I think you might have confused me with someone else? I don't maintain an AIX buildbot, and the last time I used AIX at all was around 2000 and then only very briefly.

[daltenty]

In D116366#3215212, @aganea wrote:

@zibi @daltenty Is madvise not available on AIX, or is it a specific version of the OS that doesn't have it? The doc suggests it is available: https://www.ibm.com/docs/en/aix/7.1?topic=m-madvise-subroutine
Is that specific machine missing a header? Perhaps HAVE_SYS_MMAN_H isn't defined? @simon_tatham

Unless any of you have a way to quickly test a solution, short term should we just #ifndef _AIX the call to madvise?

@zibi @daltenty Is madvise not available on AIX, or is it a specific version of the OS that doesn't have it? The doc suggests it is available: https://www.ibm.com/docs/en/aix/7.1?topic=m-madvise-subroutine

Sorry you're right, the header on AIX does normally have it available under the default _ALL_SOURCE feature test macro. Unfortunately the CMake sets _XOPEN_SOURCE=700 (i.e. XOPEN/POSIX compat) when building LLVM on AIX because of other conflicting symbols when we use _ALL_SOURCE.

I think we should still prefer the posix_madvise solution here for AIX, since it's more portable in general (edit: perhaps not, I see a note for glibc that the semantic is different on Linux with POSIX_MADV_DONTNEED https://man7.org/linux/man-pages/man3/posix_madvise.3.html). For z/OS we can introduce a guard as proposed.

llvm/include/llvm/Support/FileSystem.h

#endif
   }
 
   void moveFromImpl(mapped_file_region &Moved) {
     copyFrom(Moved);
     Moved.copyFrom(mapped_file_region());
   }
 
   void unmapImpl();
+  void dontNeedImpl();
 
   std::error_code init(sys::fs::file_t FD, uint64_t Offset, mapmode Mode);
 
 public:
   mapped_file_region() = default;
   mapped_file_region(mapped_file_region &&Moved) { moveFromImpl(Moved); }
   mapped_file_region &operator=(mapped_file_region &&Moved) {
     unmap();
public:
   /// Check if this is a valid mapping.
   explicit operator bool() const { return Mapping; }
 
   /// Unmap.
   void unmap() {
     unmapImpl();
     copyFrom(mapped_file_region());
   }
+  void dontNeed() { dontNeedImpl(); }
 
   size_t size() const;
   char *data() const;
 
   /// Get a const view of the data. Modifying this memory has undefined
   /// behavior.
   const char *const_data() const;
 

llvm/include/llvm/Support/MemoryBuffer.h

public:
   StringRef getBuffer() const {
     return StringRef(BufferStart, getBufferSize());
   }
 
   /// Return an identifier for this buffer, typically the filename it was read
   /// from.
   virtual StringRef getBufferIdentifier() const { return "Unknown buffer"; }
 
+  /// For MemoryBuffer_MMap, mark the buffer as unused for an unspecified amount
+  /// of time and the kernel can free resources associated with it. Further
+  /// access is supported but may be expensive. This calls
+  /// madvise(MADV_DONTNEED) on Linux.
+  virtual void dontNeedIfMmap() {}
+
   /// Open the specified file as a MemoryBuffer, returning a new MemoryBuffer
   /// if successful, otherwise returning null.
   ///
   /// \param IsText Set to true to indicate that the file should be read in
   /// text mode.
   ///
   /// \param IsVolatile Set to true to indicate that the contents of the file
   /// can change outside the user's control, e.g. when libclang tries to parse

llvm/lib/Support/MemoryBuffer.cpp

public:
   StringRef getBufferIdentifier() const override {
     // The name is stored after the class itself.
     return StringRef(reinterpret_cast<const char *>(this + 1));
   }
 
   MemoryBuffer::BufferKind getBufferKind() const override {
     return MemoryBuffer::MemoryBuffer_MMap;
   }
+
+  void dontNeedIfMmap() override { MFR.dontNeed(); }
 };
 } // namespace
 
 static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
 getMemoryBufferForStream(sys::fs::file_t FD, const Twine &BufferName) {
   const ssize_t ChunkSize = 4096*4;
   SmallString<ChunkSize> Buffer;
 

llvm/lib/Support/Unix/Path.inc

if (ec)
     copyFrom(mapped_file_region());
 }
 
 void mapped_file_region::unmapImpl() {
   if (Mapping)
     ::munmap(Mapping, Size);
 }
 
+void mapped_file_region::dontNeedImpl() {
+  if (Mapping)
+    ::madvise(Mapping, Size, MADV_DONTNEED);
+}
+
 int mapped_file_region::alignment() {
   return Process::getPageSizeEstimate();
 }
 
 std::error_code detail::directory_iterator_construct(detail::DirIterState &it,
                                                      StringRef path,
                                                      bool follow_symlinks) {
   SmallString<128> path_null(path);

llvm/lib/Support/Windows/Path.inc

if (Mode == mapmode::readwrite && Exe && hasFlushBufferKernelBug()) {
       // after, under high I/O pressure.
       ::FlushFileBuffers(FileHandle);
     }
 
     ::CloseHandle(FileHandle);
   }
 }
 
+void mapped_file_region::dontNeedImpl() {}
+
 int mapped_file_region::alignment() {
   SYSTEM_INFO SysInfo;
   ::GetSystemInfo(&SysInfo);
   return SysInfo.dwAllocationGranularity;
 }
 
 static basic_file_status status_from_find_data(WIN32_FIND_DATAW *FindData) {
   return basic_file_status(file_type_from_attrs(FindData->dwFileAttributes),