Page MenuHomePhabricator

Use allocate_file in FileOutputBuffer to avoid possible bus error.
Needs ReviewPublic

Authored by ruiu on Nov 1 2017, 3:33 PM.

Details

Summary

This patch makes FileOutputBuffer to use fallocate(2) or equivalent
before mmap'ing a temporary file. If an operating system or a filesystem
do not support fallocate(2), FileOutputBuffer now creates an in-memory
buffer instead of a temporary file.

As a result, after committing this patch, all disk full errors are
gracefully handled by LLVM (including lld). Previously, we could be
killed by a bus error when a disk becomes full while writing to a sparse
file. That will never happen with this patch.

In return for better error handling, this patch disables mmap IO on
systems that don't support fallocate(2) or equivalent. That's most
Unix systems except macOS or Linux. On such systems, FileOutputBuffer
creates an in-memory buffer and write it to an output file using write(2)
on commit(). That is inefficient compared to mmap, but that is only
visible only when we are using FileOutputBuffer for lld and when lld is
creating a huge executable in practice, because in most cases IO speed
is not a limiting factor.

So, there's a tradeoff. Which we prefer, maximum IO performance or a
reliable error handling? I think I can be convinced in either way, but
I'm leaning towards a reliable error handling because the way how LLVM
tools that use FileOutputBuffer die when a disk is full is really bad
and pretty user-unfriendly.

I wish many more Unix systems start supporting fallocate(2).

Depends on https://reviews.llvm.org/D39464.

Event Timeline

ruiu created this revision.Nov 1 2017, 3:33 PM
andrewrk edited edge metadata.EditedNov 2 2017, 6:48 AM

Which we prefer, maximum IO performance or a reliable error handling?

Zig project prefers reliable error handling.

espindola edited reviewers, added: espindola; removed: rafael.Mar 15 2018, 9:06 AM