[libFuzzer] Enable on i386

[metzman]

Summary

Get libFuzzer to build on i386 and fix tests.

[metzman]

• fix comment

[metzman]

• change order

[metzman]

Please take a look.

compiler-rt/cmake/config-ix.cmake

233

Should I explicitly disable i386 on non-Linux if I don't really care about them?
When I tested this it didn't get built on Windows which I'm fine with.

[kcc]

Matt, please take a look.

compiler-rt/test/fuzzer/fork.test

15

no need for a regex, just remove extra zeroes.

compiler-rt/test/fuzzer/lit.cfg

28

That's not true:

% clang++ -m32 -fsanitize=address ~/misc-c/leak.cc && ./a.out

132478==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from:

#0 0x810cf89 in operator new(unsigned int) /usr/local/google/home/kcc/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:105:3
#1 0x8063e01 in __cxx_global_var_init (/usr/local/google/home/kcc/llvm-build/a.out+0x8063e01)
#2 0x8063e68 in _GLOBAL__sub_I_leak.cc (/usr/local/google/home/kcc/llvm-build/a.out+0x8063e68)
#3 0x810f63a in __libc_csu_init (/usr/local/google/home/kcc/llvm-build/a.out+0x810f63a)

SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).

[morehouse]

compiler-rt/lib/fuzzer/CMakeLists.txt

128

Why is this argument necessary?

compiler-rt/test/fuzzer/dataflow.test

2

Any idea why this fails on x86?

compiler-rt/test/fuzzer/only-some-bytes.test

2

Any idea why this doesn't work for x86?

compiler-rt/test/fuzzer/trace-malloc-threaded.test

4

Any idea why this doesn't work for x86?

[metzman]

• remove stale comment
• remove extra zeros
• fix broken test

[metzman]

compiler-rt/lib/fuzzer/CMakeLists.txt

128

Without it we get this error: /usr/bin/ld: Relocatable linking with relocations from format elf32-i386 (/home/user/llvm-build/lib/clang/9.0.0/lib/linux/libclang_rt.fuzzer-i386.a(FuzzerCrossOver.cpp.o)) to format elf64-x86-64 (fuzzer.o) is not supported

Interestingly, this has been a problem with libFuzzer before, see here.

compiler-rt/test/fuzzer/dataflow.test

2

Dataflow afaik only supports x86_64 Linux.

compiler-rt/test/fuzzer/fork.test

15

Done.

compiler-rt/test/fuzzer/lit.cfg

28

My bad, stale comment.

compiler-rt/test/fuzzer/only-some-bytes.test

2

Dataflow afaik only supports x86_64 Linux.

compiler-rt/test/fuzzer/trace-malloc-threaded.test

4

Got it working, it's because most of the i386 addresses begin with 0xf instead of 0x[0-9]

[morehouse]

compiler-rt/lib/fuzzer/CMakeLists.txt

128

The error sounds like we're trying to mix 32-bit and 64-bit. So maybe we should address that instead?

I also notice that the documentation for some linkers (gold) say that -m is obsolete.

[metzman]

compiler-rt/lib/fuzzer/CMakeLists.txt

128

The error sounds like we're trying to mix 32-bit and 64-bit.

I'm not sure this the case.

The command that causes this failure is

/usr/bin/ld --whole-archive /home/user/llvm-project/build-x862/lib/clang/9.0.0/lib/linux/libclang_rt.fuzzer-i386.a --no-whole-archive /home/user/llvm-project/build-x862/projects/compiler-rt/lib/fuzzer/libcxx_fuzzer_i386/lib/libc++.a -r -o fuzzer.o

Minimized:

/usr/bin/ld -r --whole-archive /home/user/llvm-project/build-x862/lib/clang/9.0.0/lib/linux/libclang_rt.fuzzer-i386.a -o fuzzer.o

But /home/user/llvm-project/build-x862/lib/clang/9.0.0/lib/linux/libclang_rt.fuzzer-i386.a only contains 32 bit code afaict:

objdump -d /home/user/llvm-project/build-x862/lib/clang/9.0.0/lib/linux/libclang_rt.fuzzer_no_main-i386.a | grep "\.o"
FuzzerCrossOver.cpp.o:     file format elf32-i386
FuzzerDataFlowTrace.cpp.o:     file format elf32-i386
FuzzerDriver.cpp.o:     file format elf32-i386
FuzzerExtFunctionsDlsym.cpp.o:     file format elf32-i386
FuzzerExtFunctionsWeak.cpp.o:     file format elf32-i386
FuzzerExtFunctionsWindows.cpp.o:     file format elf32-i386
FuzzerExtraCounters.cpp.o:     file format elf32-i386
FuzzerFork.cpp.o:     file format elf32-i386
FuzzerIO.cpp.o:     file format elf32-i386
FuzzerIOPosix.cpp.o:     file format elf32-i386
FuzzerIOWindows.cpp.o:     file format elf32-i386
FuzzerLoop.cpp.o:     file format elf32-i386
FuzzerMerge.cpp.o:     file format elf32-i386
FuzzerMutate.cpp.o:     file format elf32-i386
FuzzerSHA1.cpp.o:     file format elf32-i386
FuzzerTracePC.cpp.o:     file format elf32-i386
FuzzerUtil.cpp.o:     file format elf32-i386
FuzzerUtilDarwin.cpp.o:     file format elf32-i386
FuzzerUtilFuchsia.cpp.o:     file format elf32-i386
FuzzerUtilLinux.cpp.o:     file format elf32-i386
FuzzerUtilPosix.cpp.o:     file format elf32-i386
FuzzerUtilWindows.cpp.o:     file format elf32-i386

gold and lld can run /usr/bin/ld.bfd -r --whole-archive ~/llvm-project/build-x862/lib/clang/9.0.0/lib/linux/libclang_rt.fuzzer-i386.a -o fuzzer.o just fine by the way.
When I run file on the output, it says:

fuzzer.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), with debug_info, not stripped

Which would be surprising if I actually was including 64 bit code.
I think this is just a bug/quirk in GNU ld that needs -m elf_i386 to be solved.

You can verify ld needs this using this simple reproducer:

$ echo "int x() { return 0; }" > x.cc
$ g++ -m32 -c x.cc
$ ar qc x.a x.o
$ ld -r --whole-archive x.a -o x2.o
ld: Relocatable linking with relocations from format elf32-i386 (x.a(x.o)) to format elf64-x86-64 (x2.o) is not supported

[morehouse]

LGTM

compiler-rt/cmake/config-ix.cmake

if(APPLE)
   set(X86_64 x86_64 x86_64h)
 endif()
 
 set(ALL_SANITIZER_COMMON_SUPPORTED_ARCH ${X86} ${X86_64} ${PPC64}
     ${ARM32} ${ARM64} ${MIPS32} ${MIPS64} ${S390X})
 set(ALL_ASAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64}
     ${MIPS32} ${MIPS64} ${PPC64} ${S390X})
 set(ALL_DFSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
-set(ALL_FUZZER_SUPPORTED_ARCH ${X86_64} ${ARM64})
+set(ALL_FUZZER_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM64})
 
 if(APPLE)
   set(ALL_LSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${MIPS64} ${ARM64})
 else()
   set(ALL_LSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${MIPS64} ${ARM64} ${ARM32} ${PPC64})
 endif()
 set(ALL_MSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${PPC64})
 set(ALL_HWASAN_SUPPORTED_ARCH ${X86_64} ${ARM64})

compiler-rt/lib/fuzzer/CMakeLists.txt

add_compiler_rt_runtime(clang_rt.fuzzer_no_main
   OBJECT_LIBS RTfuzzer
   CFLAGS ${LIBFUZZER_CFLAGS}
   PARENT_TARGET fuzzer)
 
 if(OS_NAME MATCHES "Linux|Fuchsia" AND
    COMPILER_RT_LIBCXX_PATH AND
    COMPILER_RT_LIBCXXABI_PATH)
   macro(partially_link_libcxx name dir arch)
+    if(${arch} MATCHES "i386")
+      set(EMULATION_ARGUMENT "-m" "elf_i386")
+    else()
+      set(EMULATION_ARGUMENT "")
+    endif()
     set(cxx_${arch}_merge_dir "${CMAKE_CURRENT_BINARY_DIR}/cxx_${arch}_merge.dir")
     file(MAKE_DIRECTORY ${cxx_${arch}_merge_dir})
     add_custom_command(TARGET clang_rt.${name}-${arch} POST_BUILD
-      COMMAND ${CMAKE_LINKER} --whole-archive "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>" --no-whole-archive ${dir}/lib/libc++.a -r -o ${name}.o
+      COMMAND ${CMAKE_LINKER} ${EMULATION_ARGUMENT} --whole-archive "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>" --no-whole-archive ${dir}/lib/libc++.a -r -o ${name}.o
       COMMAND ${CMAKE_OBJCOPY} --localize-hidden ${name}.o
       COMMAND ${CMAKE_COMMAND} -E remove "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>"
       COMMAND ${CMAKE_AR} qcs "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>" ${name}.o
       WORKING_DIRECTORY ${cxx_${arch}_merge_dir}
     )
   endmacro()
 
   foreach(arch ${FUZZER_SUPPORTED_ARCH})

compiler-rt/test/fuzzer/CMakeLists.txt

foreach(part IN LISTS stdlib_list)
     string(TOUPPER ${first_letter} first_letter)
     string(REGEX REPLACE "^.(.*)" "${first_letter}\\1" part "${part}")
     set(STDLIB_CAPITALIZED "${STDLIB_CAPITALIZED}${part}")
   endforeach()
   foreach(arch ${FUZZER_SUPPORTED_ARCH})
     set(LIBFUZZER_TEST_COMPILER ${COMPILER_RT_TEST_COMPILER})
     get_test_cc_for_arch(${arch} LIBFUZZER_TEST_COMPILER LIBFUZZER_TEST_FLAGS)
 
+    set(LIBFUZZER_TEST_TARGET_ARCH ${arch})
     set(LIBFUZZER_TEST_APPLE_PLATFORM "osx")
 
     set(LIBFUZZER_TEST_STDLIB ${stdlib})
 
     string(TOUPPER ${arch} ARCH_UPPER_CASE)
     set(CONFIG_NAME ${ARCH_UPPER_CASE}${STDLIB_CAPITALIZED}${OS_NAME}Config)
 
     # LIT-based libFuzzer tests.

compiler-rt/test/fuzzer/compressed.test

 REQUIRES: linux
 REQUIRES: zlib
+# zlib is "supported" on i386 even when only for x86_64, explicitly make i386
+# unsupported by this test.
+UNSUPPORTED: i386
 # Custom mutator should find this bug, w/o custom -- no chance.
 RUN: %cpp_compiler %S/CompressedTest.cpp -o %t-CompressedTestCustom -DCUSTOM_MUTATOR -lz
 RUN: %cpp_compiler %S/CompressedTest.cpp -o %t-CompressedTestPlain -lz
 RUN: not %run %t-CompressedTestCustom -seed=1 -runs=1000000
 RUN:     %run %t-CompressedTestPlain  -seed=1 -runs=1000000

compiler-rt/test/fuzzer/dataflow.test

 # Tests the data flow tracer.
-REQUIRES: linux
-UNSUPPORTED: aarch64
+REQUIRES: linux, x86_64
 
 # Build the tracer and the test.
 RUN: %no_fuzzer_cpp_compiler -c -fno-sanitize=all -fsanitize=dataflow  %S/../../lib/fuzzer/dataflow/DataFlow.cpp -o  %t-DataFlow.o
 RUN: %no_fuzzer_cpp_compiler    -fno-sanitize=all -fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp   %S/ThreeFunctionsTest.cpp     %t-DataFlow.o -o %t-ThreeFunctionsTestDF
 RUN: %no_fuzzer_cpp_compiler    -fno-sanitize=all -fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp   %S/ExplodeDFSanLabelsTest.cpp %t-DataFlow.o -o %t-ExplodeDFSanLabelsTestDF
 RUN: %cpp_compiler %S/ThreeFunctionsTest.cpp -o %t-ThreeFunctionsTest
 
 # Dump the function list.

compiler-rt/test/fuzzer/fork.test

 # UNSUPPORTED: darwin, freebsd
 BINGO: BINGO
 RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-SimpleTest
 RUN: not %run %t-SimpleTest -fork=1 2>&1 | FileCheck %s --check-prefix=BINGO
 
 TIMEOUT: ERROR: libFuzzer: timeout
 RUN: %cpp_compiler %S/TimeoutTest.cpp -o %t-TimeoutTest
 RUN: not %run %t-TimeoutTest -fork=1 -timeout=1 -ignore_timeouts=0 2>&1 | FileCheck %s --check-prefix=TIMEOUT
 
 OOM: ERROR: libFuzzer: out-of-memory
 RUN: %cpp_compiler %S/OutOfMemoryTest.cpp -o %t-OutOfMemoryTest
 RUN: not %run %t-OutOfMemoryTest -fork=1 -ignore_ooms=0  -rss_limit_mb=128 2>&1 | FileCheck %s --check-prefix=OOM
 
-# access-violation is the error thrown on Windows.
-CRASH: {{SEGV|access-violation}} on unknown address 0x000000000000
+# access-violation is the error thrown on Windows. Address will be smaller on i386.
+CRASH: {{SEGV|access-violation}} on unknown address 0x00000000
 RUN: %cpp_compiler %S/ShallowOOMDeepCrash.cpp -o %t-ShallowOOMDeepCrash
 RUN: not %run %t-ShallowOOMDeepCrash -fork=1 -rss_limit_mb=128 2>&1 | FileCheck %s --check-prefix=CRASH
 
 MAX_TOTAL_TIME: INFO: fuzzed for {{.*}} seconds, wrapping up soon
 MAX_TOTAL_TIME: INFO: exiting: {{.*}} time:
 RUN: not %run %t-ShallowOOMDeepCrash -fork=1 -rss_limit_mb=128 -ignore_crashes=1 -max_total_time=5 2>&1 | FileCheck %s --check-prefix=MAX_TOTAL_TIME

compiler-rt/test/fuzzer/lit.cfg

 import lit.formats
 import sys
 import os
 
 config.name = "libFuzzer" + config.name_suffix
 config.test_format = lit.formats.ShTest(True)
 config.suffixes = ['.test']
 config.test_source_root = os.path.dirname(__file__)
+config.available_features.add(config.target_arch)
 
 # Choose between lit's internal shell pipeline runner and a real shell.  If
 # LIT_USE_INTERNAL_SHELL is in the environment, we use that as an override.
 use_lit_shell = os.environ.get("LIT_USE_INTERNAL_SHELL")
 if use_lit_shell:
     # 0 is external, "" is default, and everything else is internal.
     execute_external = (use_lit_shell == "0")
 else:
     # Otherwise we default to internal on Windows and external elsewhere, as
     # bash on Windows is usually very slow.
     execute_external = (not sys.platform in ['win32'])
 
 # testFormat: The test format to use to interpret tests.
 #
 # For now we require '&&' between commands, until they get globally killed and
 # the test runner updated.
 config.test_format = lit.formats.ShTest(execute_external)
 
 # LeakSanitizer is not supported on OSX or Windows right now.
 if (sys.platform.startswith('darwin') or
     sys.platform.startswith('freebsd') or
     sys.platform.startswith('netbsd') or
     sys.platform.startswith('win')):
   lit_config.note('lsan feature unavailable')
 else:
   lit_config.note('lsan feature available')
   config.available_features.add('lsan')
 
 # MemorySanitizer is not supported on OSX or Windows right now
-if sys.platform.startswith('darwin') or sys.platform.startswith('win'):
+if (sys.platform.startswith('darwin') or sys.platform.startswith('win') or
+    config.target_arch == 'i386'):
   lit_config.note('msan feature unavailable')
   assert 'msan' not in config.available_features
 else:
   lit_config.note('msan feature available')
   config.available_features.add('msan')
 
 if sys.platform.startswith('win') or sys.platform.startswith('cygwin'):
   config.available_features.add('windows')

compiler-rt/test/fuzzer/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 config.cpp_compiler = "@LIBFUZZER_TEST_COMPILER@"
 config.target_flags = "@LIBFUZZER_TEST_FLAGS@"
 config.c_compiler = "@LIBFUZZER_TEST_COMPILER@"
 config.stdlib = "@LIBFUZZER_TEST_STDLIB@"
 config.apple_platform = "@LIBFUZZER_TEST_APPLE_PLATFORM@"
 config.name_suffix = "@LIBFUZZER_TEST_CONFIG_SUFFIX@"
 
 config.osx_sysroot_flag = "@OSX_SYSROOT_FLAG@"
 config.cmake_binary_dir = "@CMAKE_BINARY_DIR@"
 config.llvm_library_dir = "@LLVM_LIBRARY_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
+config.target_arch = "@LIBFUZZER_TEST_TARGET_ARCH@"
 
 # Load common config for all compiler-rt lit tests.
 lit_config.load_config(config,
     "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")
 
 if config.enable_per_target_runtime_dir:
   config.runtime_library_dir = config.compiler_rt_libdir
 else:
   config.runtime_library_dir = "@LLVM_LIBRARY_DIR@"
 
 lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg")

compiler-rt/test/fuzzer/only-some-bytes.test

 # Tests the data flow tracer.
-REQUIRES: linux
-UNSUPPORTED: aarch64
+REQUIRES: linux, x86_64
 
 # Build the tracer and the test.
 RUN: %no_fuzzer_cpp_compiler -c -fno-sanitize=all -fsanitize=dataflow  %S/../../lib/fuzzer/dataflow/DataFlow.cpp -o  %t-DataFlow.o
 RUN: %no_fuzzer_cpp_compiler    -fno-sanitize=all -fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp   %S/OnlySomeBytesTest.cpp     %t-DataFlow.o -o %t-DFT
 RUN: %cpp_compiler %S/OnlySomeBytesTest.cpp -o %t-Fuzz
 
 # Prepare the inputs.
 RUN: rm -rf %t/*

compiler-rt/test/fuzzer/trace-malloc-threaded.test

 // FIXME: This test infinite loops on darwin because it crashes
 // printing a stack trace repeatedly
 // FIXME: Disabled on Windows because of a crash (possibly related to above).
 UNSUPPORTED: darwin, aarch64, windows
 
 RUN: %cpp_compiler %S/TraceMallocThreadedTest.cpp -o \
 RUN:   %t-TraceMallocThreadedTest
 
 RUN: %run %t-TraceMallocThreadedTest -trace_malloc=2 -runs=1 2>&1 | FileCheck %s
-CHECK: {{MALLOC\[[0-9]+] +0x[0-9]+ 5639}}
+CHECK: {{MALLOC\[[0-9]+] +0x[0-9a-f]+ 5639}}
 CHECK-NEXT: {{ +\#0 +}}
 CHECK-NEXT: {{ +\#1 +}}
 CHECK-NEXT: {{ +\#2 +}}
 
-CHECK: {{MALLOC\[[0-9]+] +0x[0-9]+ 5639}}
+CHECK: {{MALLOC\[[0-9]+] +0x[0-9a-f]+ 5639}}
 CHECK-NEXT: {{ +\#0 +}}
 CHECK-NEXT: {{ +\#1 +}}
 CHECK-NEXT: {{ +\#2 +}}
 
-CHECK: {{MALLOC\[[0-9]+] +0x[0-9]+ 5639}}
+CHECK: {{MALLOC\[[0-9]+] +0x[0-9a-f]+ 5639}}
 CHECK-NEXT: {{ +\#0 +}}
 CHECK-NEXT: {{ +\#1 +}}
 CHECK-NEXT: {{ +\#2 +}}
 
-CHECK: {{MALLOC\[[0-9]+] +0x[0-9]+ 5639}}
+CHECK: {{MALLOC\[[0-9]+] +0x[0-9a-f]+ 5639}}
 CHECK-NEXT: {{ +\#0 +}}
 CHECK-NEXT: {{ +\#1 +}}
 CHECK-NEXT: {{ +\#2 +}}
 
-CHECK: {{MALLOC\[[0-9]+] +0x[0-9]+ 5639}}
+CHECK: {{MALLOC\[[0-9]+] +0x[0-9a-f]+ 5639}}
 CHECK-NEXT: {{ +\#0 +}}
 CHECK-NEXT: {{ +\#1 +}}
 CHECK-NEXT: {{ +\#2 +}}
 
-CHECK: {{MALLOC\[[0-9]+] +0x[0-9]+ 5639}}
+CHECK: {{MALLOC\[[0-9]+] +0x[0-9a-f]+ 5639}}
 CHECK-NEXT: {{ +\#0 +}}
 CHECK-NEXT: {{ +\#1 +}}
 CHECK-NEXT: {{ +\#2 +}}