This is an archive of the discontinued LLVM Phabricator instance.

[Support] call FlushFileBuffers when closing raw_fd_ostream on Windows
AbandonedPublic

Authored by inglorion on May 8 2018, 7:27 PM.

Details

Reviewers
rnk
zturner
Summary

We have observed a problem where files we write on Windows are
sometimes read back with bytes having value 0, when that was not the
value that was written. D42925 works around this problem for files we
write to by memory-mapping them. This change applies a similar fix for
files we write using raw_fd_ostream, as we do for archives and in the
LTO cache.

Event Timeline

inglorion created this revision.May 8 2018, 7:27 PM
inglorion added a comment.EditedMay 8 2018, 7:31 PM

My hope is that this will help with https://crbug.com/786127. I haven't seen the undefined symbol errors in local builds for several weeks now, but they still happen on the bots.

This doesn’t seem necessary, as the problem in the referenced cl only
affects memory mapped files. Furthermore, this will probably adversely
impact performance.

pcc added a subscriber: pcc.May 8 2018, 7:33 PM

Is this a speculative fix or have you observed that this fixes the problem in practice?

It's a speculative fix. I'm not getting the error locally and it doesn't consistently reproduce anywhere I know of, which makes it hard to think of a way to ascertain that it actually fixed the problem. It's just that I noticed that a number of files we write when doing ThinLTO are written through raw_fd_ostream, and we then occasionally see bytes being read as 0 when they shouldn't be, which is a problem we fixed elsewhere by calling FlushFileBuffers. If we land this and the problem persists I think we should revert it (for the reasons zturner mentioned), but given the similarity to the problem we worked around with D42925, I would like to give this a shot and see if it helps.

I won’t block or approve, but I would be extraordinarily surprised if it
turns out to be the culprit since the Windows team already confirmed it to
only be a problem with mapped file i/o (and if it weren’t, it would
probably be so common that the whole world would have blown up by now).

So probably a good idea to think about next steps (eg obtaining one of the
“corrupt” files from a buildbot)

inglorion planned changes to this revision.May 9 2018, 10:50 AM

Bruce's blog post (https://randomascii.wordpress.com/2018/02/25/compiler-bug-linker-bug-windows-kernel-bug/) does seem to suggest that this issue should only ever be hit when files are written through memory maps. If that is the case, this change should not make a difference. I'll shelve it for now and see if I can think of something better. Thanks!

inglorion abandoned this revision.Aug 24 2018, 1:48 PM

We don't need this. Abandoning.