[BOLT] Restrict execution of tests that fail on Windows

[rafauler]

Summary

Turn off execution of tests that use UNIX-specific features.

[Amir]

I've recently added some magic to pass linux target triple to cflags in bolt/test/lit.local.cfg:

host_linux_triple = config.target_triple.split('-')[0]+'-linux'
common_linker_flags = '-fuse-ld=lld -Wl,--unresolved-symbols=ignore-all'
flags = f'--target={host_linux_triple} {common_linker_flags}'

This would take the arch from target triple (x86_64 or aarch64) and append -linux. I didn't test it on Windows but it works on macOS. Therefore I'm reluctant to add system-linux to these tests that would restrict them to linux.

Can you please share what's passed as cflags on windows? Let's try to fix it. Our tests don't depend on system headers, so there's a good chance they would work even on windows.

[rafauler]

In D126933#3555080, @Amir wrote:

I've recently added some magic to pass linux target triple to cflags in bolt/test/lit.local.cfg:

host_linux_triple = config.target_triple.split('-')[0]+'-linux'
common_linker_flags = '-fuse-ld=lld -Wl,--unresolved-symbols=ignore-all'
flags = f'--target={host_linux_triple} {common_linker_flags}'

This would take the arch from target triple (x86_64 or aarch64) and append -linux. I didn't test it on Windows but it works on macOS. Therefore I'm reluctant to add system-linux to these tests that would restrict them to linux.

Can you please share what's passed as cflags on windows? Let's try to fix it. Our tests don't depend on system headers, so there's a good chance they would work even on windows.

Sorry, you're right, I should at least update that comment, the triple is indeed being passed correctly. But the driver gets confused as it tries to locate libc stuff in an environment that lacks them:

$ "c:\users\rafael\llvm\build\minsizerel\bin\clang.exe" "--target=x86_64-linux" "-fuse-ld=lld" "-Wl,--unresolved-symbols=ignore-all" "-fPIC" "-shared" "C:\Users\Rafael\llvm\llvm-project\bolt\test\R_ABS.pic.lld.cpp" "-o" "C:\Users\Rafael\llvm\build\tools\bolt\test\Output\R_ABS.pic.lld.cpp.tmp.so" "-Wl,-q" "-fuse-ld=lld"
# command stderr:
ld.lld: error: cannot open crti.o: no such file or directory
ld.lld: error: cannot open crtbeginS.o: no such file or directory
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: cannot open crtendS.o: no such file or directory
ld.lld: error: cannot open crtn.o: no such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I imagine the driver is trying to look for stuff in /lib, /lib64 etc. but we are on windows, so it is never going to find these libc dependencies, even if it is aware of the target triple.

We would need to use some sort of -ffreestanding in order to be able to be truly independent of the system and avoid pulling in surprises into the final binary. I didn't bother modifying the tests as I didn't want to spend time trying to figure out if -ffreestanding would be creating an adequate test case. But if its working on MacOS, maybe we need to investigate where it is finding the object files with start symbols, because that's curious.

[rafauler]

Conceptually, the problem is that we need a full cross-compiler to build to a specific target (linux), but we only have the compiler, not the libs. That's why I thought it would make sense to restrict to environments where we are not cross compiling to a different system. Looking at "pie.test", for example, we are apparently building a .cpp file, which is even worse, as it requires libstdc++. Somehow I want to give test writers the idea that it is not OK to build C++ files unless you are not cross-compiling.

[Amir]

In D126933#3557106, @rafauler wrote:

In D126933#3555080, @Amir wrote:

I've recently added some magic to pass linux target triple to cflags in bolt/test/lit.local.cfg:

host_linux_triple = config.target_triple.split('-')[0]+'-linux'
common_linker_flags = '-fuse-ld=lld -Wl,--unresolved-symbols=ignore-all'
flags = f'--target={host_linux_triple} {common_linker_flags}'

This would take the arch from target triple (x86_64 or aarch64) and append -linux. I didn't test it on Windows but it works on macOS. Therefore I'm reluctant to add system-linux to these tests that would restrict them to linux.

Can you please share what's passed as cflags on windows? Let's try to fix it. Our tests don't depend on system headers, so there's a good chance they would work even on windows.

Sorry, you're right, I should at least update that comment, the triple is indeed being passed correctly. But the driver gets confused as it tries to locate libc stuff in an environment that lacks them:

$ "c:\users\rafael\llvm\build\minsizerel\bin\clang.exe" "--target=x86_64-linux" "-fuse-ld=lld" "-Wl,--unresolved-symbols=ignore-all" "-fPIC" "-shared" "C:\Users\Rafael\llvm\llvm-project\bolt\test\R_ABS.pic.lld.cpp" "-o" "C:\Users\Rafael\llvm\build\tools\bolt\test\Output\R_ABS.pic.lld.cpp.tmp.so" "-Wl,-q" "-fuse-ld=lld"
# command stderr:
ld.lld: error: cannot open crti.o: no such file or directory
ld.lld: error: cannot open crtbeginS.o: no such file or directory
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lgcc
ld.lld: error: unable to find library -lgcc_s
ld.lld: error: cannot open crtendS.o: no such file or directory
ld.lld: error: cannot open crtn.o: no such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I imagine the driver is trying to look for stuff in /lib, /lib64 etc. but we are on windows, so it is never going to find these libc dependencies, even if it is aware of the target triple.

We would need to use some sort of -ffreestanding in order to be able to be truly independent of the system and avoid pulling in surprises into the final binary. I didn't bother modifying the tests as I didn't want to spend time trying to figure out if -ffreestanding would be creating an adequate test case. But if its working on MacOS, maybe we need to investigate where it is finding the object files with start symbols, because that's curious.

Sorry for a long delay.
Tests are failing to link just the same. I agree it makes sense to require system-linux.

bolt/test/AArch64/asm-func-debug.test

 # Verify that we update DW_TAG_compile_unit' ranges and .debug_aranges
 # for assembly function that doesn't have corresponding DIE.
 #
 # The input test case foo() contains nops that we remove.
 
 RUN: %clang %cflags -no-pie -gdwarf-4 %p/../Inputs/asm_foo.s %p/../Inputs/asm_main.c -o %t.exe
 RUN: llvm-bolt %t.exe -o %t --update-debug-sections
 RUN: llvm-dwarfdump -all %t | FileCheck %s
 
 # Check ranges were created/updated for asm compile unit
 CHECK:       0x0000000b: DW_TAG_compile_unit
 CHECK-NEXT:       DW_AT_stmt_list (0x00000000)
 CHECK-NEXT:       DW_AT_low_pc (0x0000000000000000)
 CHECK-NEXT:       DW_AT_ranges
 CHECK-NEXT:          [0x0000000000[[#%x,ADDR:]],
 CHECK-SAME:                              0x0000000000[[#ADDR+4]]))
-CHECK-NEXT:       DW_AT_name ("{{.*}}/asm_foo.s")
+CHECK-NEXT:       DW_AT_name ("{{.*}}asm_foo.s")
 
 # Check .debug_aranges was updated for asm module
 CHECK: .debug_aranges contents:
 CHECK-NEXT:   Address Range Header: length = 0x0000002c, format = DWARF32, version = 0x0002, cu_offset = 0x00000000, addr_size = 0x08, seg_size = 0x00
 CHECK-NEXT:   [0x0000000000[[#ADDR]], 0x0000000000[[#ADDR+4]])
 
 # Check line number info was updated
 CHECK:       0x0000000000[[#ADDR]]   13      0      1   0             0  is_stmt
 CHECK-NEXT:  0x0000000000[[#ADDR+4]] 13      0      1   0             0  is_stmt end_sequence

bolt/test/R_ABS.pic.lld.cpp

 /*
  * Check that we don't assert on a duplicate static relocation added by lld
  * against _Z6myfuncv. The same address has a dynamic relocation against it.
  *
+ * This test uses the clang driver + lld and will only succeed on Linux systems
+ * with libc available.
+ * REQUIRES: system-linux
+ *
  * RUN: %clang %cflags -fPIC -shared %s -o %t.so -Wl,-q -fuse-ld=lld
  * RUN: llvm-bolt %t.so -o %t.so.bolt --relocs
  */
 
 unsigned long long myfunc();
 
 unsigned long long (*myglobal)() = myfunc;
 
 unsigned long long myfunc() {
   return reinterpret_cast<unsigned long long>(myglobal);
 }

bolt/test/X86/double-jump.test

 # Test the double jump removqal peephole.
 
+# This test has commands that rely on shell capabilities that won't execute
+# correctly on Windows e.g. subshell execution
+REQUIRES: shell
+
 RUN: %clang %cflags %p/Inputs/double_jump.cpp -o %t.exe
 RUN: (llvm-bolt %t.exe --peepholes=double-jumps \
 RUN:   --eliminate-unreachable -o %t 2>&1 \
 RUN:   && llvm-objdump -d %t --print-imm-hex --no-show-raw-insn) | FileCheck %s
 
 CHECK: BOLT-INFO: Peephole: 1 double jumps patched.
 
 CHECK: <_Z3foom>:

bolt/test/X86/jmp-optimization.test

 # Tests the optimization of functions that just do a tail call in the beginning.
 
+# This test has commands that rely on shell capabilities that won't execute
+# correctly on Windows e.g. unsupported parameter expansion
+REQUIRES: shell
+
 RUN: %clang %cflags -O2 %S/Inputs/jmp_opt{,2,3}.cpp -o %t
 RUN: llvm-bolt -inline-small-functions %t -o %t.bolt
 RUN: llvm-objdump -d %t.bolt --print-imm-hex | FileCheck %s
 
 CHECK:     <main>:
 CHECK-NOT: call
 CHECK:     xorl    %eax, %eax
 CHECK:     retq

bolt/test/X86/jump-table-icp.test

 RUN: llvm-mc -filetype=obj -triple x86_64-unknown-unknown \
 RUN:   %p/Inputs/jump_table_icp.s -o %t.o
 RUN: link_fdata %p/Inputs/jump_table_icp.s %t.o %t.fdata --nmtool llvm-nm
 RUN: llvm-strip --strip-unneeded %t.o
 RUN: %clang %cflags -no-pie %t.o -o %t.exe -Wl,-q
 
+# This test has commands that rely on shell capabilities that won't execute
+# correctly on Windows e.g. subshell execution
+REQUIRES: shell
+
 RUN: (llvm-bolt %t.exe --data %t.fdata -o %t --relocs \
 RUN:   --reorder-blocks=cache --split-functions --split-all-cold \
 RUN:   --use-gnu-stack --dyno-stats --indirect-call-promotion=jump-tables \
 RUN:   --print-icp -v=0 \
 RUN:   --icp-jt-remaining-percent-threshold=10 \
 RUN:   --icp-jt-total-percent-threshold=2 \
 RUN:   --indirect-call-promotion-topn=1 \
 RUN:   --icp-jump-tables-targets --align-functions-max-bytes=7 2>&1 && \

bolt/test/X86/shrinkwrapping.test

 # Verifies that llvm-bolt updates CFI correctly after
 # shrink-wrapping when optimizing a function without
 # frame pointers.
 
+# This test has commands that rely on shell capabilities that won't execute
+# correctly on Windows e.g. subshell execution to capture command output.
+REQUIRES: shell
+
 RUN: %clangxx %cxxflags -no-pie %S/Inputs/exc4sw.S -o %t.exe -Wl,-q
 RUN: llvm-bolt %t.exe -o %t --relocs --frame-opt=all \
 RUN:   --data=%p/Inputs/exc4sw.fdata --reorder-blocks=cache 2>&1 | \
 RUN:   FileCheck %s --check-prefix=CHECK-BOLT
 
 RUN: llvm-objdump --dwarf=frames %t.exe | grep -A20 -e \
 RUN:   `llvm-nm --numeric-sort %t.exe | grep main | tail -n 1 | \
 RUN:    cut -f1 -d' ' | tail -c9` 2>&1 | FileCheck %s --check-prefix=CHECK-INPUT

bolt/test/bad-exe.test

 # Check that llvm-bolt rejects input that is not a valid ELF executable
 # bzip2.debuginfo is the result of running "objcopy --only-keep-debug".
+
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %cflags %S/Inputs/icf-jump-tables.c -g -o %t
 RUN: llvm-objcopy --only-keep-debug %t %t.debuginfo
 RUN: not llvm-bolt %t.debuginfo -o /dev/null 2>&1 | FileCheck %s
 
 CHECK: input binary is not a valid ELF executable

bolt/test/bolt-info.test

 # Check that the .bolt_info section is generated properly.
+
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %cflags %S/Inputs/icf-jump-tables.c -o %t
 RUN: llvm-bolt %t -o %t.bolt && \
 RUN: llvm-objdump -s -j .note.bolt_info %t.bolt | grep -v "file format" | \
 RUN:   cut -c 44- | tr -d '\n' | FileCheck %s
 
 CHECK: BOLT revision: {{.*}}, command line: {{.*}}llvm-bolt {{.*}}/bolt-info.test.tmp -o {{.*}}/bolt-info.test.tmp.bolt

bolt/test/invalid-profile.test

 # Check that llvm-bolt detects bad profile data and aborts
 
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %S/Inputs/icf-jump-tables.c -o %t
 RUN: not llvm-bolt %t -o %t.bolt --data %t 2>&1 | FileCheck %s
 
 CHECK: no valid profile data found

bolt/test/no-relocs.test

 # Verifies that input without relocations is rejected in relocs mode.
 
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %cflags %S/Inputs/icf-jump-tables.c -o %t
 RUN: not llvm-bolt %t -o /dev/null --relocs 2>&1 | FileCheck %s
 
 CHECK: BOLT-ERROR: relocations against code are missing from the input file.

bolt/test/pie.test

 # Check that we do not reject position-independent executables (PIEs).
 
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %cflags -fPIC -pie %p/Inputs/jump_table_icp.cpp -o %t
 RUN: llvm-bolt %t -o /dev/null 2>&1 | FileCheck %s
 
 CHECK: BOLT-INFO: shared object or position-independent executable detected

bolt/test/re-optimize.test

 # Check that we detect re-optimization attempt.
 
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %cflags %S/Inputs/icf-jump-tables.c -o %t.exe
 RUN: llvm-bolt %t.exe -o %t 2>&1 > /dev/null
 RUN: not llvm-bolt %t -o %t.bolt 2>&1 | FileCheck %s
 
 CHECK: BOLT-ERROR: input file was processed by BOLT. Cannot re-optimize.

bolt/test/shared-object.test

 # Test that llvm-bolt processes *.so without a failure
+
+# This test uses the clang driver without target flags and will only succeed
+# on Linux systems where the host triple matches the target.
+REQUIRES: system-linux
+
 RUN: %clang %cflags %S/Inputs/icf-jump-tables.c -o %t.so -shared -fPIC -Wl,--build-id
 RUN: llvm-bolt %t.so -o %t | FileCheck %s
 
 CHECK: patched build-id