[LTO] Introduce an Output class to wrap the output stream creation (NFC)

[mehdi_amini]

Summary

While NFC for now, this will allow more flexibility on the client side
to hold state necessary to back up the stream.
Also when adding caching, this class will grow in complexity.

Note I blindly modified the gold-plugin as I can't compile it.

[mehdi_amini]

Missed a few renaming

[tejohnson]

Looks ok, let me download the patch and try with gold though. Will get back later today.

include/llvm/LTO/LTO.h

235 ↗

(On Diff #68135)

Maybe "NativeObjectOutput" or something like that to make it clear - one thing I would like to change eventually is the fact that this callback is passed to the WriteIndexesThinBackend, which doesn't write a native object file.

[davide]

Looks ok to me. I can't test the gold plugin easily (I'm on my macbook laptop without access to a Linux machine) either.

[tejohnson]

Building gold-plugin exposed a couple issues, one of which will require some changes to interface.

tools/gold/gold-plugin.cpp

657 ↗

(On Diff #68135)

FD was not declared, which exposes an issue - the interface needs to handle this being a temp file that we create here.

732 ↗

(On Diff #68135)

Needs a ';' at end

[mehdi_amini]

Fix gold-plugin build

tools/gold/gold-plugin.cpp

657 ↗

(On Diff #68135)

Does it? Can't we just not create the file open?

[tejohnson]

Will give the changes a try and report back.

tools/gold/gold-plugin.cpp

657 ↗

(On Diff #68135)

Ah, yes, that should be fine. Missed the fact that createTemporaryFile will return the new filename.

[tejohnson]

Do you have a companion clang change? BackendUtil.cpp needs an update for the new arg to thinBackend.

[mehdi_amini]

Rename to NativeObjectOutput and move declaration to Config.h

[tejohnson]

Confirmed that this builds and tests file with gold.

llvm/trunk/include/llvm/LTO/Config.h

 
 class Error;
 class Module;
 class ModuleSummaryIndex;
 class raw_pwrite_stream;
 
 namespace lto {
 
+/// Abstract class representing a single Task output to be implemented by the
+/// client of the LTO API.
+class NativeObjectOutput {
+public:
+  // Return an allocated stream for the output, or null in case of failure.
+  virtual std::unique_ptr<raw_pwrite_stream> getStream() = 0;
+  virtual ~NativeObjectOutput() = default;
+};
+
 /// LTO configuration. A linker can configure LTO by setting fields in this data
 /// structure and passing it to the lto::LTO constructor.
 struct Config {
   std::string CPU;
   std::string Features;
   TargetOptions Options;
   std::vector<std::string> MAttrs;
   Reloc::Model RelocModel = Reloc::PIC_;
struct Config {
   /// called, and writes the module to a bitcode file with a name prefixed by
   /// the given output file name, and (2) creates a resolution file whose name
   /// is prefixed by the given output file name and sets ResolutionFile to its
   /// file handle.
   Error addSaveTemps(std::string OutputFileName,
                      bool UseInputModulePath = false);
 };
 
-/// This type defines a stream callback. A stream callback is used to add a
-/// native object that is generated on the fly. The callee must set up and
-/// return a output stream to write the native object to.
-///
-/// Stream callbacks must be thread safe.
-typedef std::function<std::unique_ptr<raw_pwrite_stream>(unsigned Task)>
-    AddStreamFn;
+/// This type defines the callback to add a native object that is generated on
+/// the fly.
+///
+/// Output callbacks must be thread safe.
+typedef std::function<std::unique_ptr<NativeObjectOutput>(unsigned Task)>
+    AddOutputFn;
 
 /// A derived class of LLVMContext that initializes itself according to a given
 /// Config object. The purpose of this class is to tie ownership of the
 /// diagnostic handler to the context, as opposed to the Config object (which
 /// may be ephemeral).
 struct LTOLLVMContext : LLVMContext {
   static void funcDiagHandler(const DiagnosticInfo &DI, void *Context) {
     auto *Fn = static_cast<DiagnosticHandlerFunction *>(Context);

llvm/trunk/include/llvm/LTO/LTO.h

 };
 
 /// A ThinBackend defines what happens after the thin-link phase during ThinLTO.
 /// The details of this type definition aren't important; clients can only
 /// create a ThinBackend using one of the create*ThinBackend() functions below.
 typedef std::function<std::unique_ptr<ThinBackendProc>(
     Config &C, ModuleSummaryIndex &CombinedIndex,
     StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-    AddStreamFn AddStream)>
+    AddOutputFn AddOutput)>
     ThinBackend;
 
 /// This ThinBackend runs the individual backend jobs in-process.
 ThinBackend createInProcessThinBackend(unsigned ParallelismLevel);
 
 /// This ThinBackend writes individual module indexes to files, instead of
 /// running the individual backend jobs. This backend is for distributed builds
 /// where separate processes will invoke the real backends.

 /// - Create lto::InputFile objects using lto::InputFile::create(), then use
 ///   the symbols() function to enumerate its symbols and compute a resolution
 ///   for each symbol (see SymbolResolution below).
 /// - After the linker has visited each input file (and each regular object
 ///   file) and computed a resolution for each symbol, take each lto::InputFile
 ///   and pass it and an array of symbol resolutions to the add() function.
 /// - Call the getMaxTasks() function to get an upper bound on the number of
 ///   native object files that LTO may add to the link.
-/// - Call the run() function. This function will use the supplied AddStream
+/// - Call the run() function. This function will use the supplied AddOutput
 ///   function to add up to getMaxTasks() native object files to the link.
 class LTO {
   friend InputFile;
 
 public:
   /// Create an LTO object. A default constructed LTO object has a reasonable
   /// production configuration, but you can customize it by passing arguments to
   /// this constructor.
   /// FIXME: We do currently require the DiagHandler field to be set in Conf.
   /// Until that is fixed, a Config argument is required.
   LTO(Config Conf, ThinBackend Backend = nullptr,
       unsigned ParallelCodeGenParallelismLevel = 1);
 
   /// Add an input file to the LTO link, using the provided symbol resolutions.
   /// The symbol resolutions must appear in the enumeration order given by
   /// InputFile::symbols().
   Error add(std::unique_ptr<InputFile> Obj, ArrayRef<SymbolResolution> Res);
 
   /// Returns an upper bound on the number of tasks that the client may expect.
   /// This may only be called after all IR object files have been added. For a
   /// full description of tasks see LTOBackend.h.
   unsigned getMaxTasks() const;
 
-  /// Runs the LTO pipeline. This function calls the supplied AddStream function
+  /// Runs the LTO pipeline. This function calls the supplied AddOutput function
   /// to add native object files to the link.
-  Error run(AddStreamFn AddStream);
+  Error run(AddOutputFn AddOutput);
 
 private:
   Config Conf;
 
   struct RegularLTOState {
     RegularLTOState(unsigned ParallelCodeGenParallelismLevel, Config &Conf);
 
     unsigned ParallelCodeGenParallelismLevel;
void addSymbolToGlobalRes(object::IRObjectFile *Obj,
                             const InputFile::Symbol &Sym, SymbolResolution Res,
                             unsigned Partition);
 
   Error addRegularLTO(std::unique_ptr<InputFile> Input,
                       ArrayRef<SymbolResolution> Res);
   Error addThinLTO(std::unique_ptr<InputFile> Input,
                    ArrayRef<SymbolResolution> Res);
 
-  Error runRegularLTO(AddStreamFn AddStream);
-  Error runThinLTO(AddStreamFn AddStream);
+  Error runRegularLTO(AddOutputFn AddOutput);
+  Error runThinLTO(AddOutputFn AddOutput);
 
   mutable bool CalledGetMaxTasks = false;
 };
 
 /// The resolution for a symbol. The linker must provide a SymbolResolution for
 /// each global symbol based on its internal resolution of that symbol.
 struct SymbolResolution {
   SymbolResolution()

llvm/trunk/include/llvm/LTO/LTOBackend.h

 
 class Error;
 class Module;
 class Target;
 
 namespace lto {
 
 /// Runs a regular LTO backend.
-Error backend(Config &C, AddStreamFn AddStream,
+Error backend(Config &C, AddOutputFn AddStream,
               unsigned ParallelCodeGenParallelismLevel,
               std::unique_ptr<Module> M);
 
 /// Runs a ThinLTO backend.
-Error thinBackend(Config &C, unsigned Task, AddStreamFn AddStream, Module &M,
+Error thinBackend(Config &C, unsigned Task, AddOutputFn AddStream, Module &M,
                   ModuleSummaryIndex &CombinedIndex,
                   const FunctionImporter::ImportMapTy &ImportList,
                   const GVSummaryMapTy &DefinedGlobals,
                   MapVector<StringRef, MemoryBufferRef> &ModuleMap);
 }
 }
 
 #endif

llvm/trunk/lib/LTO/LTO.cpp

Error LTO::addThinLTO(std::unique_ptr<InputFile> Input,
   return Error();
 }
 
 unsigned LTO::getMaxTasks() const {
   CalledGetMaxTasks = true;
   return RegularLTO.ParallelCodeGenParallelismLevel + ThinLTO.ModuleMap.size();
 }
 
-Error LTO::run(AddStreamFn AddStream) {
+Error LTO::run(AddOutputFn AddOutput) {
   // Invoke regular LTO if there was a regular LTO module to start with,
   // or if there are any hooks that the linker may have used to add
   // its own resolved symbols to the combined module.
   if (RegularLTO.HasModule || Conf.PreOptModuleHook ||
       Conf.PostInternalizeModuleHook || Conf.PostOptModuleHook ||
       Conf.PreCodeGenModuleHook)
-    if (auto E = runRegularLTO(AddStream))
+    if (auto E = runRegularLTO(AddOutput))
       return E;
-  return runThinLTO(AddStream);
+  return runThinLTO(AddOutput);
 }
 
-Error LTO::runRegularLTO(AddStreamFn AddStream) {
+Error LTO::runRegularLTO(AddOutputFn AddOutput) {
   if (Conf.PreOptModuleHook &&
       !Conf.PreOptModuleHook(0, *RegularLTO.CombinedModule))
     return Error();
 
   for (const auto &R : GlobalResolutions) {
     if (R.second.IRName.empty())
       continue;
     if (R.second.Partition != 0 &&
for (const auto &R : GlobalResolutions) {
     if (R.second.Partition == 0)
       GV->setLinkage(GlobalValue::InternalLinkage);
   }
 
   if (Conf.PostInternalizeModuleHook &&
       !Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))
     return Error();
 
-  return backend(Conf, AddStream, RegularLTO.ParallelCodeGenParallelismLevel,
+  return backend(Conf, AddOutput, RegularLTO.ParallelCodeGenParallelismLevel,
                  std::move(RegularLTO.CombinedModule));
 }
 
 /// This class defines the interface to the ThinLTO backend.
 class lto::ThinBackendProc {
 protected:
   Config &Conf;
   ModuleSummaryIndex &CombinedIndex;
-  AddStreamFn AddStream;
+  AddOutputFn AddOutput;
   StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries;
 
 public:
   ThinBackendProc(Config &Conf, ModuleSummaryIndex &CombinedIndex,
-                  AddStreamFn AddStream,
+                  AddOutputFn AddOutput,
                   StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries)
-      : Conf(Conf), CombinedIndex(CombinedIndex), AddStream(AddStream),
+      : Conf(Conf), CombinedIndex(CombinedIndex), AddOutput(AddOutput),
         ModuleToDefinedGVSummaries(ModuleToDefinedGVSummaries) {}
 
   virtual ~ThinBackendProc() {}
   virtual Error start(unsigned Task, MemoryBufferRef MBRef,
                       const FunctionImporter::ImportMapTy &ImportList,
                       MapVector<StringRef, MemoryBufferRef> &ModuleMap) = 0;
   virtual Error wait() = 0;
 };
 
 class InProcessThinBackend : public ThinBackendProc {
   ThreadPool BackendThreadPool;
 
   Optional<Error> Err;
   std::mutex ErrMu;
 
 public:
   InProcessThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
                        unsigned ThinLTOParallelismLevel,
                        StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-                       AddStreamFn AddStream)
-      : ThinBackendProc(Conf, CombinedIndex, AddStream,
+                       AddOutputFn AddOutput)
+      : ThinBackendProc(Conf, CombinedIndex, AddOutput,
                         ModuleToDefinedGVSummaries),
         BackendThreadPool(ThinLTOParallelismLevel) {}
 
   Error
-  runThinLTOBackendThread(AddStreamFn AddStream, unsigned Task,
+  runThinLTOBackendThread(AddOutputFn AddOutput, unsigned Task,
                           MemoryBufferRef MBRef,
                           ModuleSummaryIndex &CombinedIndex,
                           const FunctionImporter::ImportMapTy &ImportList,
                           const GVSummaryMapTy &DefinedGlobals,
                           MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
     LLVMContext BackendContext;
 
     ErrorOr<std::unique_ptr<Module>> MOrErr =
         parseBitcodeFile(MBRef, BackendContext);
     assert(MOrErr && "Unable to load module in thread?");
 
-    return thinBackend(Conf, Task, AddStream, **MOrErr, CombinedIndex,
+    return thinBackend(Conf, Task, AddOutput, **MOrErr, CombinedIndex,
                        ImportList, DefinedGlobals, ModuleMap);
   }
 
   Error start(unsigned Task, MemoryBufferRef MBRef,
               const FunctionImporter::ImportMapTy &ImportList,
               MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
     StringRef ModulePath = MBRef.getBufferIdentifier();
     BackendThreadPool.async(
         [=](MemoryBufferRef MBRef, ModuleSummaryIndex &CombinedIndex,
             const FunctionImporter::ImportMapTy &ImportList,
             GVSummaryMapTy &DefinedGlobals,
             MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
           Error E =
-              runThinLTOBackendThread(AddStream, Task, MBRef, CombinedIndex,
+              runThinLTOBackendThread(AddOutput, Task, MBRef, CombinedIndex,
                                       ImportList, DefinedGlobals, ModuleMap);
           if (E) {
             std::unique_lock<std::mutex> L(ErrMu);
             if (Err)
               Err = joinErrors(std::move(*Err), std::move(E));
             else
               Err = std::move(E);
           }
Error wait() override {
     else
       return Error();
   }
 };
 
 ThinBackend lto::createInProcessThinBackend(unsigned ParallelismLevel) {
   return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
              StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-             AddStreamFn AddStream) {
+             AddOutputFn AddOutput) {
     return llvm::make_unique<InProcessThinBackend>(
         Conf, CombinedIndex, ParallelismLevel, ModuleToDefinedGVSummaries,
-        AddStream);
+        AddOutput);
   };
 }
 
 class WriteIndexesThinBackend : public ThinBackendProc {
   std::string OldPrefix, NewPrefix;
   bool ShouldEmitImportsFiles;
 
   std::string LinkedObjectsFileName;
   std::unique_ptr<llvm::raw_fd_ostream> LinkedObjectsFile;
 
 public:
   WriteIndexesThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
                           StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-                          AddStreamFn AddStream, std::string OldPrefix,
+                          AddOutputFn AddOutput, std::string OldPrefix,
                           std::string NewPrefix, bool ShouldEmitImportsFiles,
                           std::string LinkedObjectsFileName)
-      : ThinBackendProc(Conf, CombinedIndex, AddStream,
+      : ThinBackendProc(Conf, CombinedIndex, AddOutput,
                         ModuleToDefinedGVSummaries),
         OldPrefix(OldPrefix), NewPrefix(NewPrefix),
         ShouldEmitImportsFiles(ShouldEmitImportsFiles),
         LinkedObjectsFileName(LinkedObjectsFileName) {}
 
   /// Given the original \p Path to an output file, replace any path
   /// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
   /// resulting directory if it does not yet exist.

 };
 
 ThinBackend lto::createWriteIndexesThinBackend(std::string OldPrefix,
                                                std::string NewPrefix,
                                                bool ShouldEmitImportsFiles,
                                                std::string LinkedObjectsFile) {
   return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
              StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-             AddStreamFn AddStream) {
+             AddOutputFn AddOutput) {
     return llvm::make_unique<WriteIndexesThinBackend>(
-        Conf, CombinedIndex, ModuleToDefinedGVSummaries, AddStream, OldPrefix,
+        Conf, CombinedIndex, ModuleToDefinedGVSummaries, AddOutput, OldPrefix,
         NewPrefix, ShouldEmitImportsFiles, LinkedObjectsFile);
   };
 }
 
-Error LTO::runThinLTO(AddStreamFn AddStream) {
+Error LTO::runThinLTO(AddOutputFn AddOutput) {
   if (ThinLTO.ModuleMap.empty())
     return Error();
 
   if (Conf.CombinedIndexHook && !Conf.CombinedIndexHook(ThinLTO.CombinedIndex))
     return Error();
 
   // Collect for each module the list of function it defines (GUID ->
   // Summary).
return (ExportList != ExportLists.end() &&
            ExportedGUIDs.count(GUID);
   };
   thinLTOInternalizeAndPromoteInIndex(ThinLTO.CombinedIndex, isExported);
   thinLTOResolveWeakForLinkerInIndex(
       ThinLTO.CombinedIndex, isPrevailing,
       [](StringRef, GlobalValue::GUID, GlobalValue::LinkageTypes) {});
 
   std::unique_ptr<ThinBackendProc> BackendProc = ThinLTO.Backend(
-      Conf, ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries, AddStream);
+      Conf, ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries, AddOutput);
 
   // Partition numbers for ThinLTO jobs start at 1 (see comments for
   // GlobalResolution in LTO.h). Task numbers, however, start at
   // ParallelCodeGenParallelismLevel, as tasks 0 through
   // ParallelCodeGenParallelismLevel-1 are reserved for parallel code generation
   // partitions.
   unsigned Task = RegularLTO.ParallelCodeGenParallelismLevel;
   unsigned Partition = 1;

llvm/trunk/lib/LTO/LTOBackend.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/LTOBackend.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/ReaderWriter.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/LTO/LTO.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"
bool opt(Config &C, TargetMachine *TM, unsigned Task, Module &M,
   passes.run(M);
 
   if (C.PostOptModuleHook && !C.PostOptModuleHook(Task, M))
     return false;
 
   return true;
 }
 
-void codegen(Config &C, TargetMachine *TM, AddStreamFn AddStream, unsigned Task,
+void codegen(Config &C, TargetMachine *TM, AddOutputFn AddOutput, unsigned Task,
              Module &M) {
   if (C.PreCodeGenModuleHook && !C.PreCodeGenModuleHook(Task, M))
     return;
 
-  std::unique_ptr<raw_pwrite_stream> OS = AddStream(Task);
+  auto Output = AddOutput(Task);
+  std::unique_ptr<raw_pwrite_stream> OS = Output->getStream();
   legacy::PassManager CodeGenPasses;
   if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
                               TargetMachine::CGFT_ObjectFile))
     report_fatal_error("Failed to setup codegen");
   CodeGenPasses.run(M);
 }
 
-void splitCodeGen(Config &C, TargetMachine *TM, AddStreamFn AddStream,
+void splitCodeGen(Config &C, TargetMachine *TM, AddOutputFn AddOutput,
                   unsigned ParallelCodeGenParallelismLevel,
                   std::unique_ptr<Module> M) {
   ThreadPool CodegenThreadPool(ParallelCodeGenParallelismLevel);
   unsigned ThreadCount = 0;
   const Target *T = &TM->getTarget();
 
   SplitModule(
       std::move(M), ParallelCodeGenParallelismLevel,
SplitModule(
                   MemoryBufferRef(StringRef(BC.data(), BC.size()), "ld-temp.o"),
                   Ctx);
               if (!MOrErr)
                 report_fatal_error("Failed to read bitcode");
               std::unique_ptr<Module> MPartInCtx = std::move(MOrErr.get());
 
               std::unique_ptr<TargetMachine> TM =
                   createTargetMachine(C, MPartInCtx->getTargetTriple(), T);
-              codegen(C, TM.get(), AddStream, ThreadId, *MPartInCtx);
+              codegen(C, TM.get(), AddOutput, ThreadId, *MPartInCtx);
             },
             // Pass BC using std::move to ensure that it get moved rather than
             // copied into the thread's context.
             std::move(BC), ThreadCount++);
       },
       false);
 }
 
 Expected<const Target *> initAndLookupTarget(Config &C, Module &M) {
   if (!C.OverrideTriple.empty())
     M.setTargetTriple(C.OverrideTriple);
   else if (M.getTargetTriple().empty())
     M.setTargetTriple(C.DefaultTriple);
 
   std::string Msg;
   const Target *T = TargetRegistry::lookupTarget(M.getTargetTriple(), Msg);
   if (!T)
     return make_error<StringError>(Msg, inconvertibleErrorCode());
   return T;
 }
 
 }
 
-Error lto::backend(Config &C, AddStreamFn AddStream,
+Error lto::backend(Config &C, AddOutputFn AddOutput,
                    unsigned ParallelCodeGenParallelismLevel,
                    std::unique_ptr<Module> M) {
   Expected<const Target *> TOrErr = initAndLookupTarget(C, *M);
   if (!TOrErr)
     return TOrErr.takeError();
 
   std::unique_ptr<TargetMachine> TM =
       createTargetMachine(C, M->getTargetTriple(), *TOrErr);
 
   if (!opt(C, TM.get(), 0, *M, /*IsThinLto=*/false))
     return Error();
 
   if (ParallelCodeGenParallelismLevel == 1)
-    codegen(C, TM.get(), AddStream, 0, *M);
+    codegen(C, TM.get(), AddOutput, 0, *M);
   else
-    splitCodeGen(C, TM.get(), AddStream, ParallelCodeGenParallelismLevel,
+    splitCodeGen(C, TM.get(), AddOutput, ParallelCodeGenParallelismLevel,
                  std::move(M));
   return Error();
 }
 
-Error lto::thinBackend(Config &Conf, unsigned Task, AddStreamFn AddStream,
+Error lto::thinBackend(Config &Conf, unsigned Task, AddOutputFn AddOutput,
                        Module &Mod, ModuleSummaryIndex &CombinedIndex,
                        const FunctionImporter::ImportMapTy &ImportList,
                        const GVSummaryMapTy &DefinedGlobals,
                        MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
   Expected<const Target *> TOrErr = initAndLookupTarget(Conf, Mod);
   if (!TOrErr)
     return TOrErr.takeError();
 
Error lto::thinBackend(Config &Conf, unsigned Task, AddOutputFn AddOutput,
   Importer.importFunctions(Mod, ImportList);
 
   if (Conf.PostImportModuleHook && !Conf.PostImportModuleHook(Task, Mod))
     return Error();
 
   if (!opt(Conf, TM.get(), Task, Mod, /*IsThinLto=*/true))
     return Error();
 
-  codegen(Conf, TM.get(), AddStream, Task, Mod);
+  codegen(Conf, TM.get(), AddOutput, Task, Mod);
   return Error();
 }

llvm/trunk/tools/gold/gold-plugin.cpp

static void recordFile(std::string Filename, bool TempOutFile) {
   if (add_input_file(Filename.c_str()) != LDPS_OK)
     message(LDPL_FATAL,
             "Unable to add .o file to the link. File left behind in: %s",
             Filename.c_str());
   if (TempOutFile)
     Cleanup.push_back(Filename.c_str());
 }
 
-/// Open a file and return the new file descriptor given a base input
-/// file name, a flag indicating whether a temp file should be generated,
-/// and an optional task id. The new filename generated is
-/// returned in \p NewFilename.
-static int openOutputFile(SmallString<128> InFilename, bool TempOutFile,
+/// Return the desired output filename given a base input name, a flag
+/// indicating whether a temp file should be generated, and an optional task id.
+/// The new filename generated is returned in \p NewFilename.
+static void getOutputFileName(SmallString<128> InFilename, bool TempOutFile,
                               SmallString<128> &NewFilename, int TaskID = -1) {
-  int FD;
   if (TempOutFile) {
     std::error_code EC =
-        sys::fs::createTemporaryFile("lto-llvm", "o", FD, NewFilename);
+        sys::fs::createTemporaryFile("lto-llvm", "o", NewFilename);
     if (EC)
       message(LDPL_FATAL, "Could not create temporary file: %s",
               EC.message().c_str());
   } else {
     NewFilename = InFilename;
     if (TaskID >= 0)
       NewFilename += utostr(TaskID);
-    std::error_code EC =
-        sys::fs::openFileForWrite(NewFilename, FD, sys::fs::F_None);
-    if (EC)
-      message(LDPL_FATAL, "Could not open file: %s", EC.message().c_str());
   }
-  return FD;
 }
 
 /// Add all required common symbols to M, which is expected to be the first
 /// combined module.
 static void addCommons(Module &M) {
   for (auto &I : Commons) {
     if (!I.second.Prevailing)
       continue;
static void getThinLTOOldAndNewPrefix(std::string &OldPrefix,
                                       std::string &NewPrefix) {
   StringRef PrefixReplace = options::thinlto_prefix_replace;
   assert(PrefixReplace.empty() || PrefixReplace.find(";") != StringRef::npos);
   std::pair<StringRef, StringRef> Split = PrefixReplace.split(";");
   OldPrefix = Split.first.str();
   NewPrefix = Split.second.str();
 }
 
+namespace {
+// Define the LTOOutput handling
+class LTOOutput : public lto::NativeObjectOutput {
+  StringRef Path;
+
+public:
+  LTOOutput(StringRef Path) : Path(Path) {}
+  // Open the filename \p Path and allocate a stream.
+  std::unique_ptr<raw_pwrite_stream> getStream() override {
+    int FD;
+    std::error_code EC = sys::fs::openFileForWrite(Path, FD, sys::fs::F_None);
+    if (EC)
+      message(LDPL_FATAL, "Could not open file: %s", EC.message().c_str());
+    return llvm::make_unique<llvm::raw_fd_ostream>(FD, true);
+  }
+};
+}
+
 static std::unique_ptr<LTO> createLTO() {
   Config Conf;
   ThinBackend Backend;
   unsigned ParallelCodeGenParallelismLevel = 1;
 
   Conf.CPU = options::mcpu;
   Conf.Options = InitTargetOptionsFromCodeGenFlags();
 
for (claimed_file &F : Modules) {
     PluginInputFile InputFile(F.handle);
     const void *View = getSymbolsAndView(F);
     if (!View)
       continue;
     addModule(*Lto, F, View);
   }
 
   SmallString<128> Filename;
-  // Note that openOutputFile will append a unique ID for each task
+  // Note that getOutputFileName will append a unique ID for each task
   if (!options::obj_path.empty())
     Filename = options::obj_path;
   else if (options::TheOutputType == options::OT_SAVE_TEMPS)
     Filename = output_name + ".o";
   bool SaveTemps = !Filename.empty();
 
   MaxTasks = Lto->getMaxTasks();
   std::vector<uintptr_t> IsTemporary(MaxTasks);
   std::vector<SmallString<128>> Filenames(MaxTasks);
 
-  auto AddStream = [&](size_t Task) {
-    int FD = openOutputFile(Filename, /*TempOutFile=*/!SaveTemps,
-                            Filenames[Task], MaxTasks > 1 ? Task : -1);
+  auto AddOutput = [&](size_t Task) {
+    auto &OutputName = Filenames[Task];
+    getOutputFileName(Filename, /*TempOutFile=*/!SaveTemps, OutputName,
+                      MaxTasks > 1 ? Task : -1);
     IsTemporary[Task] = !SaveTemps;
-
-    return llvm::make_unique<llvm::raw_fd_ostream>(FD, true);
+    return llvm::make_unique<LTOOutput>(OutputName);
   };
 
-  check(Lto->run(AddStream));
+  check(Lto->run(AddOutput));
 
   if (options::TheOutputType == options::OT_DISABLE ||
       options::TheOutputType == options::OT_BC_ONLY)
     return LDPS_OK;
 
   if (options::thinlto_index_only) {
     cleanup_hook();
     exit(0);

llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp

 
 template <typename T> static T check(ErrorOr<T> E, std::string Msg) {
   if (E)
     return std::move(*E);
   check(E.getError(), Msg);
   return T();
 }
 
+namespace {
+// Define the LTOOutput handling
+class LTOOutput : public lto::NativeObjectOutput {
+  StringRef Path;
+
+public:
+  LTOOutput(StringRef Path) : Path(Path) {}
+  std::unique_ptr<raw_pwrite_stream> getStream() override {
+    std::error_code EC;
+    auto S = llvm::make_unique<raw_fd_ostream>(Path, EC, sys::fs::F_None);
+    check(EC, Path);
+    return std::move(S);
+  }
+};
+}
+
 int main(int argc, char **argv) {
   InitializeAllTargets();
   InitializeAllTargetMCs();
   InitializeAllAsmPrinters();
   InitializeAllAsmParsers();
 
   cl::ParseCommandLineOptions(argc, argv, "Resolution-based LTO test harness");
 
if (!CommandLineResolutions.empty()) {
     for (auto UnusedRes : CommandLineResolutions)
       llvm::errs() << argv[0] << ": unused symbol resolution for "
                    << UnusedRes.first.first << ',' << UnusedRes.first.second
                    << '\n';
   }
   if (HasErrors)
     return 1;
 
-  auto AddStream = [&](size_t Task) {
+  auto AddOutput = [&](size_t Task) {
     std::string Path = OutputFilename + "." + utostr(Task);
-    std::error_code EC;
-    auto S = llvm::make_unique<raw_fd_ostream>(Path, EC, sys::fs::F_None);
-    check(EC, Path);
-    return S;
+    return llvm::make_unique<LTOOutput>(Path);
   };
 
-  check(Lto.run(AddStream), "LTO::run failed");
+  check(Lto.run(AddOutput), "LTO::run failed");
 }