Output ELF files after ThinLTO is run.

[void]

Summary

The gold linker allowed you to output the ELF files after LTO was run. It did
it by using the 'obj-path' option. This replicates that behavior.

[void]

Ping? :-)

[MaskRay]

Is there a test case checking this behavior ?

[ruiu]

Replicating the gold behavior sounds fine, but does this patch work? This change seems to leave Buf uninitialized while Buf will be used later. Also, in gold LTO plugin, -obj-path overwrites -save-temps, but this change doesn't seem to replicate that behavior. Could you add tests?

[MaskRay]

The filenames of Task=0 and different. Is this preferable?

t.o t2.o were compiled with -module-summary

ld.lld --plugin-opt=obj-path=t3.o -shared t.o t2.o => creates t2.o0 t2.o1 t2.o2

gold --plugin=~/llvm/Release/lib/LLVMgold.so --plugin-opt=thinlto --plugin-opt=obj-path=t3.o -shared t.o t2.o => creates t2.o t2.o1 t2.o2

[void]

In D56046#1341491, @ruiu wrote:

Replicating the gold behavior sounds fine, but does this patch work? This change seems to leave Buf uninitialized while Buf will be used later. Also, in gold LTO plugin, -obj-path overwrites -save-temps, but this change doesn't seem to replicate that behavior. Could you add tests?

I'll add a test in a bit. I tested it and it did work, but I may have missed something. Let me get the test in and then you can take a look afterwards. Thanks!

[void]

Add testcase.

[void]

Don't append "0" to the object filename.

[void]

In D56046#1341515, @MaskRay wrote:

The filenames of Task=0 and different. Is this preferable?

t.o t2.o were compiled with -module-summary

ld.lld --plugin-opt=obj-path=t3.o -shared t.o t2.o => creates t2.o0 t2.o1 t2.o2

gold --plugin=~/llvm/Release/lib/LLVMgold.so --plugin-opt=thinlto --plugin-opt=obj-path=t3.o -shared t.o t2.o => creates t2.o t2.o1 t2.o2

I'm not sure it's important, but I modified it to retain the original behavior just in case.

[ruiu]

test/ELF/lto/obj-path.ll

12

Can you also verify %t3 to make sure that lld outputs a correct output file even if obj-path is given?

[void]

Add check that the object file is still valid after using "obj-path".

[ruiu]

ELF/LTO.cpp

225

I don't think we need this kind of sophisticated abstraction. I believe you can just write Buf contents to files after data is written to Buf. Please take a look at https://gist.github.com/rui314/836cc9dab97922271b63dba3d1d5a3f5. All tests still pass with that patch.

So, if my understanding is correct, the only difference between save-temps and obj-path is the basename of the created temporary files. Is this your expected behavior?

[void]

Implement Rui's suggestion for how to output the ELF files.

[void]

ELF/LTO.cpp

225

I implemented your suggestion here. The two options are similar, but have some important differences; save-temps outputs a lot of intermediate bitcode files. A new basename is good because it allows you to place the temps in a different directory. It might be possible to combine the two somehow?

[void]

Ping? :-)

[void]

Friendly ping. :-)

[void]

Another ping. Anyone? :-)

[MaskRay]

In D56046#1404904, @void wrote:

Another ping. Anyone? :-)

I think it is correct, but I'm not confident to press the "Accept Revision" button :) Added another expert

[void]

@pcc: Ping? :)

[pcc]

LGTM

test/ELF/lto/obj-path.ll

14

We don't normally split check lines into multiple files like this. Can you move the check line for SYMBOLS from the other file into this one (and remove the one for ELF since it is unused)?

[void]

Move checks over to the main LL file, not Input file.

ELF/LTO.cpp

static void createEmptyIndex(StringRef ModulePath) {
   WriteIndexToFile(M, *OS);
 
   if (Config->ThinLTOEmitImportsFiles)
     openFile(Path + ".imports");
 }
 
 // Merge all the bitcode files we have seen, codegen the result
 // and return the resulting ObjectFile(s).
 std::vector<InputFile *> BitcodeCompiler::compile() {
   unsigned MaxTasks = LTOObj->getMaxTasks();
   Buf.resize(MaxTasks);
   Files.resize(MaxTasks);
 
   // The --thinlto-cache-dir option specifies the path to a directory in which
   // to cache native object files for ThinLTO incremental builds. If a path was
   // specified, configure LTO to use it as the cache directory.
   lto::NativeObjectCache Cache;
   if (!Config->ThinLTOCacheDir.empty())
     Cache = check(
         lto::localCache(Config->ThinLTOCacheDir,
                         [&](size_t Task, std::unique_ptr<MemoryBuffer> MB) {
                           Files[Task] = std::move(MB);
                         }));
 
   checkError(LTOObj->run(
       [&](size_t Task) {
+        if (!Config->ThinLTOIndexOnly && !Config->LTOObjPath.empty()) {
+          std::string Filename = Config->LTOObjPath;
+          if (Task) Filename += utostr(Task);
+          return llvm::make_unique<lto::NativeObjectStream>(openFile(Filename));
+        }
         return llvm::make_unique<lto::NativeObjectStream>(
             llvm::make_unique<raw_svector_ostream>(Buf[Task]));
       },
       Cache));
 
   // Emit empty index files for non-indexed files
   for (StringRef S : ThinIndices) {
     std::string Path = getThinLTOOutputFile(S);

test/ELF/lto/Inputs/obj-path.ll

+; SYMBOLS: T g
+
+; CHECK: Format: ELF64-x86-64
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @g() {
+entry:
+  ret void
+}

test/ELF/lto/obj-path.ll

+; REQUIRES: x86
+
+; RUN: opt -module-summary %s -o %t1.o
+; RUN: opt -module-summary %p/Inputs/obj-path.ll -o %t2.o
+
+; Test to ensure that obj-path creates the ELF file.
+; RUN: rm -f %t4.o
+; RUN: ld.lld --plugin-opt=obj-path=%t4.o -shared %t1.o %t2.o -o %t3
+; RUN: llvm-readobj -h %t4.o1 | FileCheck %s
+; RUN: llvm-readobj -h %t4.o2 | FileCheck %s
+; RUN: llvm-nm %t4.o1 2>&1 | FileCheck %s -check-prefix=SYMBOLS
+; RUN: llvm-nm %t4.o2 2>&1 | FileCheck %p/Inputs/obj-path.ll -check-prefix=SYMBOLS
+
+; SYMBOLS: T f
+
+; CHECK: Format: ELF64-x86-64
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+  call void (...) @g()
+  ret void
+}