[ELF] Rewriting the path of sample profile file for --reproduce response.txt

[hoy]

Summary

Rewritting the path of the sample profile file in response.txt to be relative to the repro tar.

[MaskRay]

lld/ELF/DriverUtils.cpp

182

move to the block below

lld/test/ELF/reproduce-lto.s

13

I don't think the output of tar is portable.

FileCheck %s --check-prefix=RSP < repro1/response.txt

[hoy]

lld/test/ELF/reproduce-lto.s

13

I'd like to check the path in response.txt matches the path in the tar ball, or at least it is a relative path. Not sure what is a good way. Maybe I can limit the test to Linux only.

[MaskRay]

lld/test/ELF/reproduce-lto.s

13

Perhaps xvf is fine...

[hoy]

lld/ELF/DriverUtils.cpp

182

Done.

lld/test/ELF/reproduce-lto.s

13

Yeah, since tvf is also there.

[hoy]

Addressing MaskRay's comments.

[thakis]

The extraction seems to makes this test fail on Windows: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8855568452346719664/+/steps/package_clang/0/stdout?format=raw

Not quite clear why. Maybe path name lengths or something. Can you take a look, and revert for now if it takes a while to fix?

[hoy]

In D96193#2558020, @thakis wrote:

The extraction seems to makes this test fail on Windows: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8855568452346719664/+/steps/package_clang/0/stdout?format=raw

Not quite clear why. Maybe path name lengths or something. Can you take a look, and revert for now if it takes a while to fix?

Thanks for reporting this issue. I'll make the test limited to Linux only.

[hoy]

In D96193#2558021, @hoy wrote:

In D96193#2558020, @thakis wrote:

The extraction seems to makes this test fail on Windows: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8855568452346719664/+/steps/package_clang/0/stdout?format=raw

Not quite clear why. Maybe path name lengths or something. Can you take a look, and revert for now if it takes a while to fix?

Thanks for reporting this issue. I'll make the test limited to Linux only.

So it looks like tar -xvf doesn't work on Windows:

tar: Record size = 10 blocks
repro1/response.txt
repro1/version.txt
repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/empty_profile.txt
tar: repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/empty_profile.txt: Could not create file: No such file or directory
repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/foo.o
tar: repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/foo.o: Could not create file: No such file or directory
tar: Error exit delayed from previous errors
 
note: command had no output on stdout or stderr
error: command failed with exit status: 2

I'm not sure how to disable the test on Windows only so I'm undoing the test changes.

[thakis]

In D96193#2558053, @hoy wrote:

In D96193#2558021, @hoy wrote:

In D96193#2558020, @thakis wrote:

The extraction seems to makes this test fail on Windows: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8855568452346719664/+/steps/package_clang/0/stdout?format=raw

Not quite clear why. Maybe path name lengths or something. Can you take a look, and revert for now if it takes a while to fix?

Thanks for reporting this issue. I'll make the test limited to Linux only.

So it looks like tar -xvf doesn't work on Windows:

tar: Record size = 10 blocks
repro1/response.txt
repro1/version.txt
repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/empty_profile.txt
tar: repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/empty_profile.txt: Could not create file: No such file or directory
repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/foo.o
tar: repro1/C/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/tools/lld/test/ELF/Output/reproduce-lto.s.tmp.dir/build1/foo.o: Could not create file: No such file or directory
tar: Error exit delayed from previous errors
 
note: command had no output on stdout or stderr
error: command failed with exit status: 2

I'm not sure how to disable the test on Windows only so I'm undoing the test changes.

Thanks. My guess it's due to path lengths. Since you only need the rsp file for your test, maybe extracting just that single file (tar xvf foo.tar myfile) will work?

lld/ELF/DriverUtils.cpp

std::string elf::createResponseFile(const opt::InputArgList &args) {
   // Copy the command line to the output while rewriting paths.
   for (auto *arg : args) {
     switch (arg->getOption().getID()) {
     case OPT_reproduce:
       break;
     case OPT_INPUT:
       os << quote(rewritePath(arg->getValue())) << "\n";
       break;
     case OPT_o:
       // If -o path contains directories, "lld @response.txt" will likely
       // fail because the archive we are creating doesn't contain empty
       // directories for the output path (-o doesn't create directories).
       // Strip directories to prevent the issue.
       os << "-o " << quote(path::filename(arg->getValue())) << "\n";
       break;
+    case OPT_lto_sample_profile:
+      os << arg->getSpelling() << quote(rewritePath(arg->getValue())) << "\n";
+      break;
     case OPT_call_graph_ordering_file:
     case OPT_dynamic_list:
     case OPT_just_symbols:
     case OPT_library_path:
     case OPT_retain_symbols_file:
     case OPT_rpath:
     case OPT_script:
     case OPT_symbol_ordering_file:

lld/test/ELF/reproduce-lto.s

 # REQUIRES: x86
 ## Show that bitcode files and inputs to LTO-related options are stored in the
 ## reproduce package and that intermediate files (such as the LTO-compiled
 ## object) are not.
 
 # RUN: rm -rf %t.dir
 # RUN: mkdir -p %t.dir/build1
 # RUN: llvm-mc -filetype=obj -triple=x86_64 %s -o %t.dir/build1/foo.o
 # RUN: echo > %t.dir/build1/empty_profile.txt
 # RUN: cd %t.dir
 # RUN: ld.lld build1/foo.o -o /dev/null --reproduce repro1.tar --lto-sample-profile=%t.dir/build1/empty_profile.txt
 # RUN: tar tvf repro1.tar | FileCheck %s --implicit-check-not={{.}}
+# RUN: tar xvf repro1.tar &> %t2
+# RUN: cat repro1/response.txt >> %t2
+# RUN: FileCheck %s --check-prefix=RSP < %t2
 
 # CHECK-DAG: {{.*}} repro1/{{.*}}/empty_profile.txt
 # CHECK-DAG: {{.*}} repro1/response.txt
 # CHECK-DAG: {{.*}} repro1/version.txt
 # CHECK-DAG: {{.*}} repro1/{{.*}}/foo.o
+# RSP:       repro1/[[PATH:.*]]/empty_profile.txt
+# RSP:       --lto-sample-profile=[[PATH]]/empty_profile.txt