[Support] Don't initialize compressed buffer allocated by zlib::compress

[MaskRay]

Summary

Initialization (zeroing) makes makes every allocated page resident. The actual size of the compressed buffer is usually much smaller. Making every page resident is wasteful.

When linking a test binary with ~1.9GiB uncompressed debug info with LLD, this optimization decreases max RSS by ~1.5GiB.

[MaskRay]

There are very few users of zlib::compress operating on SmallVectorImpl<char>. If the two overloads seem redundant, I can remove the SmallVectorImpl<char> overload in future revision.

[ruiu]

This is very interesting!

include/llvm/Support/Compression.h

36–37 ↗

(On Diff #158913)

I think you should replace this function entirely with the new one because this is LLVM-only function and you can update all callers.

[MaskRay]

Remove zlib::compress on SmallVectorImpl and change all callers to use std::unique_ptr<uint8_t[]>

[ruiu]

If you are using git-svn, can you combine this patch with your another patch for lld, so that we can review this API-changing patch as one patch?

[ruiu]

lib/Support/Compression.cpp

65–67

I think you should sync to the head. This line is already gone.

[ruiu]

If you are using git, you can actually combine this patch work your another patch as a single patch.

[MaskRay]

In D50223#1187829, @ruiu wrote:

If you are using git, you can actually combine this patch work your another patch as a single patch.

I use git but not mono-repo git..

git clone https://git.llvm.org/git/llvm.git
cd llvm/tools
git clone https://git.llvm.org/git/clang.git

I cannot combine this with changes in clang.

The change on clang's side is at https://reviews.llvm.org/D50267

[efriedma]

SmallVector has a method set_size to change the size of the vector without initializing the elements; you don't need to switch to unique_ptr just for that.

[MaskRay]

Use set_size

[ruiu]

LGTM

[MaskRay]

Committed in rL338913

This is not closed automatically because I amended the git commit and inserted

Differential Revision: https://reviews.llvm.org/50223  # but I missed a `D` here...

lib/Support/Compression.cpp

 }
 
 bool zlib::isAvailable() { return true; }
 
 Error zlib::compress(StringRef InputBuffer,
                      SmallVectorImpl<char> &CompressedBuffer,
                      CompressionLevel Level) {
   unsigned long CompressedSize = ::compressBound(InputBuffer.size());
-  CompressedBuffer.resize(CompressedSize);
+  CompressedBuffer.reserve(CompressedSize);
   int CLevel = encodeZlibCompressionLevel(Level);
   int Res = ::compress2((Bytef *)CompressedBuffer.data(), &CompressedSize,
                         (const Bytef *)InputBuffer.data(), InputBuffer.size(),
                         CLevel);
   // Tell MemorySanitizer that zlib output buffer is fully initialized.
   // This avoids a false report when running LLVM with uninstrumented ZLib.
   __msan_unpoison(CompressedBuffer.data(), CompressedSize);
-  CompressedBuffer.resize(CompressedSize);
+  CompressedBuffer.set_size(CompressedSize);
   return Res ? createError(convertZlibCodeToString(Res)) : Error::success();
 }
 
 Error zlib::uncompress(StringRef InputBuffer, char *UncompressedBuffer,
                        size_t &UncompressedSize) {
   int Res =
       ::uncompress((Bytef *)UncompressedBuffer, (uLongf *)&UncompressedSize,
                    (const Bytef *)InputBuffer.data(), InputBuffer.size());