This takes care of cleaning up the temp files on crashes. It doesn't
handle cleanup when explicitly killed though.
I left this out from D111875 because I was worried it would end up too
messy and didn't want to hold up the rest of it on it, but it turns out
it should be pretty straightforward.