This is an archive of the discontinued LLVM Phabricator instance.

Reuse temporary files for print-changed=diff.
ClosedPublic

Authored by jamieschmeiser on Apr 8 2021, 8:33 AM.

Details

Summary

Make the file name and descriptors static so that they are reused by
print-changed=diff. This avoids errors about being unable to create
temporary files when doing the later comparisons in a large compile.

This was originally in the code but was removed during the review
because I didn't remember that it prevented this problem.

Diff Detail

Event Timeline

jamieschmeiser created this revision.Apr 8 2021, 8:33 AM
jamieschmeiser requested review of this revision.Apr 8 2021, 8:33 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 8 2021, 8:33 AM

I think I made the request because I thought that FileName and FD were being populated every time, but that's actually not the case.

This seems fine, but do you know why there are file issues? Are the files not being deleted?

I suspect it may have something to do with name collisions in the calls to create a unique file name. It says that it loops a set number of times trying permutations and I think it just runs out of permutations, partly because the name doesn't have special characters allowing extra permutations. The uses of the file cannot collide (unless the pass manager becomes multi-threaded) so static solves the problem.

Can we delete the files? That would fix this and also would fix us leaving around files after every invocation of opt

They already are removed; see the clean up section of code at the end of the function.

I took a quick look at the sys::fs code and it looks like The call to createTemporaryFile creates a random filename by randomly substituting the characters 0-9,a-f on the parameters to get a new name. I think it just runs out of permutations of names to try. A large compile could result in 10,000+ diffs/names and it has a limited number of attempts to find a name (128?) to avoid infinite loop/race conditions. It seems that once an error occurs, it gets nothing but errors afterward so it probably just gives up trying.

aeubanks accepted this revision.Apr 9 2021, 10:43 AM

I think if we're removing the file before doing another diff there shouldn't be any race conditions. But anyway this is fine.

This revision is now accepted and ready to land.Apr 9 2021, 10:43 AM
This revision was automatically updated to reflect the committed changes.