LTO: Create temporary cache files in the cache directory instead of $TMPDIR.

[pcc]

Summary

This fixes a race condition where another linker process can observe a
partially written file if we copy it from another file system, and allows
the link to be independent of the amount of free disk space in $TMPDIR.

[mehdi_amini]

That's nice! LGTM.

I wonder though if we shouldn't have the cache pruning take a "blacklist" of patterns and ignore these files. I wouldn't want one linker process to delete temporary files from another one (Unlikely to happen I guess since LRU though).

[pcc]

In D31045#702988, @mehdi_amini wrote:

That's nice! LGTM.

I wonder though if we shouldn't have the cache pruning take a "blacklist" of patterns and ignore these files. I wouldn't want one linker process to delete temporary files from another one (Unlikely to happen I guess since LRU though).

Probably the simplest thing we could do to avoid this problem is to create the temporaries in a subdirectory of the cache directory, and have the pruning ignore subdirectories. I agree with you that this is unlikely though.

llvm/trunk/lib/LTO/Caching.cpp

 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 using namespace llvm::lto;
 
-static void commitEntry(StringRef TempFilename, StringRef EntryPath) {
-  // Rename to final destination (hopefully race condition won't matter here)
-  auto EC = sys::fs::rename(TempFilename, EntryPath);
-  if (EC) {
-    // Renaming failed, probably not the same filesystem, copy and delete.
-    // FIXME: Avoid needing to do this by creating the temporary file in the
-    // cache directory.
-    {
-      auto ReloadedBufferOrErr = MemoryBuffer::getFile(TempFilename);
-      if (auto EC = ReloadedBufferOrErr.getError())
-        report_fatal_error(Twine("Failed to open temp file '") + TempFilename +
-                           "': " + EC.message() + "\n");
-
-      raw_fd_ostream OS(EntryPath, EC, sys::fs::F_None);
-      if (EC)
-        report_fatal_error(Twine("Failed to open ") + EntryPath +
-                           " to save cached entry\n");
-      // I'm not sure what are the guarantee if two processes are doing this
-      // at the same time.
-      OS << (*ReloadedBufferOrErr)->getBuffer();
-    }
-    sys::fs::remove(TempFilename);
-  }
-}
-
 Expected<NativeObjectCache> lto::localCache(StringRef CacheDirectoryPath,
                                             AddFileFn AddFile) {
   if (std::error_code EC = sys::fs::create_directories(CacheDirectoryPath))
     return errorCodeToError(EC);
 
   return [=](unsigned Task, StringRef Key) -> AddStreamFn {
     // First, see if we have a cache hit.
     SmallString<64> EntryPath;
struct CacheStream : NativeObjectStream {
                   unsigned Task)
           : NativeObjectStream(std::move(OS)), AddFile(std::move(AddFile)),
             TempFilename(std::move(TempFilename)),
             EntryPath(std::move(EntryPath)), Task(Task) {}
 
       ~CacheStream() {
         // Make sure the file is closed before committing it.
         OS.reset();
-        commitEntry(TempFilename, EntryPath);
+        // This is atomic on POSIX systems.
+        if (auto EC = sys::fs::rename(TempFilename, EntryPath))
+          report_fatal_error(Twine("Failed to rename temporary file ") +
+                             TempFilename + ": " + EC.message() + "\n");
         AddFile(Task, EntryPath);
       }
     };
 
     return [=](size_t Task) -> std::unique_ptr<NativeObjectStream> {
       // Write to a temporary to avoid race condition
       int TempFD;
-      SmallString<64> TempFilename;
+      SmallString<64> TempFilenameModel, TempFilename;
+      sys::path::append(TempFilenameModel, CacheDirectoryPath, "Thin-%%%%%%.tmp.o");
       std::error_code EC =
-          sys::fs::createTemporaryFile("Thin", "tmp.o", TempFD, TempFilename);
+          sys::fs::createUniqueFile(TempFilenameModel, TempFD, TempFilename,
+                                    sys::fs::owner_read | sys::fs::owner_write);
       if (EC) {
         errs() << "Error: " << EC.message() << "\n";
         report_fatal_error("ThinLTO: Can't get a temporary file");
       }
 
       // This CacheStream will move the temporary file into the cache when done.
       return llvm::make_unique<CacheStream>(
           llvm::make_unique<raw_fd_ostream>(TempFD, /* ShouldClose */ true),
           AddFile, TempFilename.str(), EntryPath.str(), Task);
     };
   };
 }