This is an archive of the discontinued LLVM Phabricator instance.

Differential D36295

Port libFuzzer tests to LIT. Do not require two-stage build for check-fuzzer.
ClosedPublic

Authored by george.karpenkov on Aug 3 2017, 4:32 PM.

Details

Reviewers
kcc
Commits
rG8ecdd7be154a: Port libFuzzer tests to LIT. Do not require two-stage build for check-fuzzer.
rL310075: Port libFuzzer tests to LIT. Do not require two-stage build for check-fuzzer.
Summary

This revision ports all libFuzzer tests apart from the unittest to LIT.
The advantages of doing so were already discussed, and include:

  • Tests being self-contained
  • Much easier debugging of a single test
  • No need to use two-stage compilation

The unit-test is still compiled using CMake, but it does not need a freshly built compiler.

Disadvantages require multiple test recompilation, but on my machine the final result is actually faster then the previous configuration (probably due to the fact that tests are compiled without optimizations).

NOTE: Now we can simply run check-fuzzer to run tests! However, the previous configuration will NOT work, as in the second stage build LLVM_USE_SANITIZER is set, which disables ASAN from being built. Thus bots would need to be reconfigured (but the new configuration is super-simple compared to the previous one).

Diff Detail

Repository
rL LLVM

Event Timeline

george.karpenkov created this revision.Aug 3 2017, 4:32 PM
Herald added subscribers: eraman, mgorny. · View Herald TranscriptAug 3 2017, 4:32 PM
george.karpenkov updated this revision to Diff 109654.Aug 3 2017, 4:39 PM

No need to change flags now that tests are ported to LIT.

george.karpenkov updated this revision to Diff 109655.Aug 3 2017, 4:42 PM

Add asan to libfuzzer tests dependencies.

george.karpenkov updated this revision to Diff 109657.Aug 3 2017, 4:46 PM

fix up isysroot flag

kcc edited edge metadata.Aug 3 2017, 4:47 PM

Lovely!

Once finalized and submitted, I'll take care of the linux buildbot.

lib/Fuzzer/test/lit.cfg
63 ↗(On Diff #109654)

Are these 4 lines correct?
Looks like the conditions should be reversed

george.karpenkov updated this revision to Diff 109658.Aug 3 2017, 4:54 PM

apply Kostya's fix

george.karpenkov marked an inline comment as done.Aug 3 2017, 4:55 PM
george.karpenkov added inline comments.
lib/Fuzzer/test/lit.cfg
63 ↗(On Diff #109654)

Good catch. Now I'm confused how the tests were passing before that, but seems passing now as well.

kcc added a comment.Aug 3 2017, 5:03 PM

Now, something is fishy here.
check-fuzzer indeed passes, but if I crippled the merge functionality (replace OUTER with OTTER in FuzzerMerge.cpp) it still passes, i.e. the tests don't work any more.

That's bad. If we silently disable tests we will break half of libFuzzer within a month before we notice.

BTW, please also disable check-fuzzer on windows ASAP -- it's causing us troubles

george.karpenkov updated this revision to Diff 109661.Aug 3 2017, 5:11 PM
george.karpenkov marked an inline comment as done.

add dependency on libFuzzer

george.karpenkov added a comment.Aug 3 2017, 5:12 PM

If we silently disable tests

there was simply a missing dependency on libFuzzer, so that it wasn't recompiled. Seems to work now.

george.karpenkov added a comment.Aug 3 2017, 5:12 PM

please also disable check-fuzzer on windows

OK

kcc added a comment.Aug 3 2017, 5:14 PM

Yep, this is ok now. Looking further.

kcc accepted this revision.Aug 3 2017, 5:24 PM

LGTM with a nit, and thanks again for doing this.
I guess you may want to submit this tomorrow morning, so that we don't spoil the tonight's LLVM social with whatever this is going to break.

lib/Fuzzer/test/lit.cfg
59 ↗(On Diff #109661)

Do you ever set asan_enabled to false?
If not, I guess you can remove this flag.

This revision is now accepted and ready to land.Aug 3 2017, 5:24 PM
george.karpenkov updated this revision to Diff 109665.Aug 3 2017, 5:45 PM

Applying Kostya's suggestion

Closed by commit rL310075: Port libFuzzer tests to LIT. Do not require two-stage build for check-fuzzer. (authored by george.karpenkov). · Explain WhyAug 4 2017, 10:20 AM
This revision was automatically updated to reflect the committed changes.
alekseyshl added a subscriber: alekseyshl.Aug 7 2017, 11:03 AM

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/14939 is unhappy about this change, please fix

george.karpenkov added a comment.Aug 7 2017, 1:15 PM

@alekseyshl I would assume the bot would need to be reconfigured. Previous CMake configuration was only building libFuzzer when LLVM_USE_SANITIZE_COVERAGE was set to true. In the current configuration it should be sufficient to add -DLIBFUZZER_ENABLE=ON to CMake invocation.
@kcc Would you be willing to reconfigure the bot, or should I do that?
Additionally, if we still want to be able to build libFuzzer correctly when LLVM_USE_SANITIZE_COVERAGE is on? If yes, I would need to add a command line flag which would avoid instrumenting libFuzzer itself.

Revision Contents

PathSize
llvm/
trunk/
docs/
9 lines
lib/
Fuzzer/
21 lines
test/
213 lines
14 lines
8 lines
13 lines
3 lines
3 lines
3 lines
9 lines
4 lines
3 lines
14 lines
5 lines
6 lines
3 lines
3 lines
4 lines
3 lines
3 lines
6 lines
8 lines
12 lines
5 lines
11 lines
24 lines
3 lines
10 lines
4 lines
7 lines
3 lines
5 lines
9 lines
9 lines
10 lines
19 lines
3 lines
53 lines
3 lines
inline-8bit-counters/
12 lines
31 lines
5 lines
6 lines
6 lines
16 lines
9 lines
6 lines
no-coverage/
29 lines
3 lines
8 lines
3 lines
8 lines
3 lines
6 lines
3 lines
4 lines
4 lines
3 lines
trace-pc/
12 lines
ubsan/
12 lines
3 lines
uninstrumented/
13 lines
3 lines
3 lines
3 lines
3 lines
3 lines
4 lines
3 lines
3 lines
3 lines
3 lines
6 lines

Diff 109770

llvm/trunk/docs/LibFuzzer.rst

Show First 20 Lines • Show All 618 Lines • ▼ Show 20 Lines
If your target has massive leaks and the leak detection is disabled If your target has massive leaks and the leak detection is disabled
you will eventually run out of RAM (see the ``-rss_limit_mb`` flag). you will eventually run out of RAM (see the ``-rss_limit_mb`` flag).
Developing libFuzzer Developing libFuzzer
==================== ====================
Building libFuzzer as a part of LLVM project and running its test requires LibFuzzer is built as a part of LLVM project by default on macos and Linux.
fresh clang as the host compiler and special CMake configuration: Users of other operating systems can explicitly request compilation using
``-DLIBFUZZER_ENABLE=YES`` flag.
Tests are run using ``check-fuzzer`` target from the build directory
(note that tests will take a long time to run if the compiler was compiled
without optimizations):
.. code-block:: console .. code-block:: console
cmake -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=YES -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON /path/to/llvm
ninja check-fuzzer ninja check-fuzzer
Fuzzing components of LLVM Fuzzing components of LLVM
========================== ==========================
.. contents:: .. contents::
:local: :local:
:depth: 1 :depth: 1
▲ Show 20 Lines • Show All 175 Lines • Show Last 20 Lines

llvm/trunk/lib/Fuzzer/CMakeLists.txt

include(CheckCXXSourceCompiles) include(CheckCXXSourceCompiles)
if( APPLE ) if( APPLE )
CHECK_CXX_SOURCE_COMPILES(" CHECK_CXX_SOURCE_COMPILES("
static thread_local int blah; static thread_local int blah;
int main() { int main() {
return 0; return 0;
} }
" HAS_THREAD_LOCAL) " HAS_THREAD_LOCAL)
if( NOT HAS_THREAD_LOCAL ) if( NOT HAS_THREAD_LOCAL )
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Dthread_local=__thread") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Dthread_local=__thread")
endif() endif()
endif() endif()
set(LIBFUZZER_FLAGS_BASE "${CMAKE_CXX_FLAGS}") if (CMAKE_SYSTEM_NAME MATCHES "Darwin|Linux")
if( LLVM_USE_SANITIZE_COVERAGE ) set(LIBFUZZER_ENABLED_CHECK ON)
if(NOT "${LLVM_USE_SANITIZER}" STREQUAL "Address") else()
message(FATAL_ERROR set(LIBFUZZER_ENABLED_CHECK OFF)
"LibFuzzer and its tests require LLVM_USE_SANITIZER=Address and "
"LLVM_USE_SANITIZE_COVERAGE=YES to be set."
)
endif()
# Disable the coverage and sanitizer instrumentation for the fuzzer itself.
set(CMAKE_CXX_FLAGS "${LIBFUZZER_FLAGS_BASE} -fno-sanitize-coverage=trace-pc-guard,edge,trace-cmp,indirect-calls,8bit-counters -Werror")
endif() endif()
# Compile libFuzzer if the compilation is specifically requested, OR # Compile libFuzzer if the compilation is specifically requested, OR
# if the platform is known to be working. # if the platform is known to be working.
if ( LLVM_USE_SANITIZE_COVERAGE OR CMAKE_SYSTEM_NAME MATCHES "Darwin|Linux" ) set(LIBFUZZER_ENABLE ${LIBFUZZER_ENABLED_CHECK} CACHE BOOL "Build libFuzzer and its tests")
if (LIBFUZZER_ENABLE)
add_library(LLVMFuzzerNoMainObjects OBJECT add_library(LLVMFuzzerNoMainObjects OBJECT
FuzzerCrossOver.cpp FuzzerCrossOver.cpp
FuzzerDriver.cpp FuzzerDriver.cpp
FuzzerExtFunctionsDlsym.cpp FuzzerExtFunctionsDlsym.cpp
FuzzerExtFunctionsDlsymWin.cpp FuzzerExtFunctionsDlsymWin.cpp
FuzzerExtFunctionsWeak.cpp FuzzerExtFunctionsWeak.cpp
FuzzerExtraCounters.cpp FuzzerExtraCounters.cpp
FuzzerIO.cpp FuzzerIO.cpp
Show All 25 Lines
if (MSVC) if (MSVC)
# Until bots are reconfigured, check-fuzzer on Windows is a no-OP. # Until bots are reconfigured, check-fuzzer on Windows is a no-OP.
add_custom_target(check-fuzzer) add_custom_target(check-fuzzer)
add_custom_command(TARGET check-fuzzer add_custom_command(TARGET check-fuzzer
COMMAND cmake -E echo "check-fuzzer is disalbed on Windows") COMMAND cmake -E echo "check-fuzzer is disalbed on Windows")
else() else()
if( LLVM_USE_SANITIZE_COVERAGE AND LLVM_INCLUDE_TESTS ) if (LLVM_INCLUDE_TESTS AND LIBFUZZER_ENABLE)
add_subdirectory(test) add_subdirectory(test)
endif() endif()
endif() endif()

llvm/trunk/lib/Fuzzer/test/CMakeLists.txt

# Build all these tests with -O0, otherwise optimizations may merge some
# basic blocks and we'll fail to discover the targets.
# We change the flags for every build type because we might be doing
# a multi-configuration build (e.g. Xcode) where CMAKE_BUILD_TYPE doesn't
# mean anything.
set(variables_to_filter
CMAKE_CXX_FLAGS_RELEASE
CMAKE_CXX_FLAGS_DEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO
CMAKE_CXX_FLAGS_MINSIZEREL
LIBFUZZER_FLAGS_BASE
)
foreach (VARNAME ${variables_to_filter})
string(REGEX REPLACE "([-/]O)[123s]" "\\10" ${VARNAME} "${${VARNAME}}")
endforeach()
# Enable the coverage instrumentation (it is disabled for the Fuzzer lib).
set(CMAKE_CXX_FLAGS "${LIBFUZZER_FLAGS_BASE} -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp,trace-div,trace-gep -gline-tables-only")
if(MSVC) if(MSVC)
# For tests use the CRT specified for release build # For tests use the CRT specified for release build
# (asan doesn't support MDd and MTd) # (asan doesn't support MDd and MTd)
if ("${LLVM_USE_CRT_RELEASE}" STREQUAL "") if ("${LLVM_USE_CRT_RELEASE}" STREQUAL "")
set(CRT_FLAG " /MD ") set(CRT_FLAG " /MD ")
else() else()
set(CRT_FLAG " /${LLVM_USE_CRT_RELEASE} ") set(CRT_FLAG " /${LLVM_USE_CRT_RELEASE} ")
endif() endif()
# In order to use the sanitizers in Windows, we need to link against many # In order to use the sanitizers in Windows, we need to link against many
# runtime libraries which will depend on the target being created # runtime libraries which will depend on the target being created
# (executable or dll) and the c runtime library used (MT/MD). # (executable or dll) and the c runtime library used (MT/MD).
# By default, cmake uses link.exe for linking, which fails because we don't # By default, cmake uses link.exe for linking, which fails because we don't
# specify the appropiate dependencies. # specify the appropiate dependencies.
# As we don't want to consider all of that possible situations which depends # As we don't want to consider all of that possible situations which depends
# on the implementation of the compiler-rt, the simplest option is to change # on the implementation of the compiler-rt, the simplest option is to change
# the rules for linking executables and shared libraries, using the compiler # the rules for linking executables and shared libraries, using the compiler
# instead of link.exe. Clang will consider the sanitizer flags, and # instead of link.exe. Clang will consider the sanitizer flags, and
# automatically provide the required libraries to the linker. # automatically provide the required libraries to the linker.
set(CMAKE_CXX_LINK_EXECUTABLE "<CMAKE_CXX_COMPILER> <FLAGS> ${CMAKE_CXX_FLAGS} ${CRT_FLAG} <OBJECTS> -o <TARGET> <LINK_LIBRARIES> /link <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS>") set(CMAKE_CXX_LINK_EXECUTABLE "<CMAKE_CXX_COMPILER> <FLAGS> ${CMAKE_CXX_FLAGS} ${CRT_FLAG} <OBJECTS> -o <TARGET> <LINK_LIBRARIES> /link <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS>")
set(CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_CXX_COMPILER> ${CMAKE_CXX_FLAGS} ${CRT_FLAG} /LD <CMAKE_SHARED_LIBRARY_CXX_FLAGS> <CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS> <CMAKE_SHARED_LIBRARY_SONAME_CXX_FLAG> <TARGET_SONAME> -o <TARGET> <OBJECTS> <LINK_LIBRARIES> /link <LINK_FLAGS>") set(CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_CXX_COMPILER> ${CMAKE_CXX_FLAGS} ${CRT_FLAG} /LD <CMAKE_SHARED_LIBRARY_CXX_FLAGS> <CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS> <CMAKE_SHARED_LIBRARY_SONAME_CXX_FLAG> <TARGET_SONAME> -o <TARGET> <OBJECTS> <LINK_LIBRARIES> /link <LINK_FLAGS>")
endif() endif()
add_custom_target(TestBinaries)
# add_libfuzzer_test(<name>
# SOURCES source0.cpp [source1.cpp ...]
# )
#
# Declares a LibFuzzer test executable with target name LLVMFuzzer-<name>.
#
# One or more source files to be compiled into the binary must be declared
# after the SOURCES keyword.
function(add_libfuzzer_test name)
set(multi_arg_options "SOURCES")
cmake_parse_arguments(
"add_libfuzzer_test" "" "" "${multi_arg_options}" ${ARGN})
if ("${add_libfuzzer_test_SOURCES}" STREQUAL "")
message(FATAL_ERROR "Source files must be specified")
endif()
add_executable(LLVMFuzzer-${name}
${add_libfuzzer_test_SOURCES}
)
target_link_libraries(LLVMFuzzer-${name} LLVMFuzzer)
# Place binary where llvm-lit expects to find it
set_target_properties(LLVMFuzzer-${name}
PROPERTIES RUNTIME_OUTPUT_DIRECTORY
"${CMAKE_BINARY_DIR}/lib/Fuzzer/test"
)
add_dependencies(TestBinaries LLVMFuzzer-${name})
endfunction()
###############################################################################
# Basic tests
###############################################################################
set(Tests
AbsNegAndConstantTest
AbsNegAndConstant64Test
AccumulateAllocationsTest
BadStrcmpTest
BogusInitializeTest
BufferOverflowOnInput
CallerCalleeTest
CleanseTest
CounterTest
CustomCrossOverAndMutateTest
CustomCrossOverTest
CustomMutatorTest
CxxStringEqTest
DeepRecursionTest
DivTest
EmptyTest
EquivalenceATest
EquivalenceBTest
FlagsTest
FourIndependentBranchesTest
FullCoverageSetTest
InitializeTest
Memcmp64BytesTest
MemcmpTest
LeakTest
LeakTimeoutTest
LoadTest
NullDerefTest
NullDerefOnEmptyTest
NthRunCrashTest
OneHugeAllocTest
OutOfMemoryTest
OutOfMemorySingleLargeMallocTest
OverwriteInputTest
RepeatedMemcmp
RepeatedBytesTest
SimpleCmpTest
SimpleDictionaryTest
SimpleHashTest
SimpleTest
SimpleThreadedTest
SingleByteInputTest
SingleMemcmpTest
SingleStrcmpTest
SingleStrncmpTest
SpamyTest
ShrinkControlFlowTest
ShrinkControlFlowSimpleTest
ShrinkValueProfileTest
StrcmpTest
StrncmpOOBTest
StrncmpTest
StrstrTest
SwapCmpTest
SwitchTest
Switch2Test
TableLookupTest
ThreadedLeakTest
ThreadedTest
TimeoutTest
TimeoutEmptyTest
TraceMallocTest
TwoDifferentBugsTest
)
if(APPLE OR MSVC) if(APPLE OR MSVC)
# LeakSanitizer is not supported on OSX and Windows right now # LeakSanitizer is not supported on OSX and Windows right now
set(HAS_LSAN 0) set(HAS_LSAN 0)
message(WARNING "LeakSanitizer is not supported." message(WARNING "LeakSanitizer is not supported."
" Building and running LibFuzzer LeakSanitizer tests is disabled." " Building and running LibFuzzer LeakSanitizer tests is disabled."
) )
else() else()
set(HAS_LSAN 1) set(HAS_LSAN 1)
endif() endif()
foreach(Test ${Tests})
add_libfuzzer_test(${Test} SOURCES ${Test}.cpp)
endforeach()
function(test_export_symbol target symbol)
if(MSVC)
set_target_properties(LLVMFuzzer-${target} PROPERTIES LINK_FLAGS
"-export:${symbol}")
endif()
endfunction()
test_export_symbol(FlagsTest "LLVMFuzzerInitialize")
test_export_symbol(InitializeTest "LLVMFuzzerInitialize")
test_export_symbol(BogusInitializeTest "LLVMFuzzerInitialize")
test_export_symbol(CustomCrossOverTest "LLVMFuzzerCustomCrossOver")
test_export_symbol(CustomMutatorTest "LLVMFuzzerCustomMutator")
############################################################################### ###############################################################################
# Unit tests # Unit tests
############################################################################### ###############################################################################
add_executable(LLVMFuzzer-Unittest add_custom_target(FuzzerUnitTests)
FuzzerUnittest.cpp set_target_properties(FuzzerUnitTests PROPERTIES FOLDER "libFuzzer tests")
)
add_executable(LLVMFuzzer-StandaloneInitializeTest add_executable(LLVMFuzzer-Unittest FuzzerUnittest.cpp)
InitializeTest.cpp
../standalone/StandaloneFuzzTargetMain.c
)
target_link_libraries(LLVMFuzzer-Unittest target_link_libraries(LLVMFuzzer-Unittest
gtest gtest
gtest_main gtest_main
LLVMFuzzerNoMain LLVMFuzzerNoMain
) )
target_include_directories(LLVMFuzzer-Unittest PRIVATE target_include_directories(LLVMFuzzer-Unittest PRIVATE
"${LLVM_MAIN_SRC_DIR}/utils/unittest/googletest/include" "${LLVM_MAIN_SRC_DIR}/utils/unittest/googletest/include"
) )
add_dependencies(TestBinaries LLVMFuzzer-Unittest)
set_target_properties(LLVMFuzzer-Unittest set_target_properties(LLVMFuzzer-Unittest
PROPERTIES RUNTIME_OUTPUT_DIRECTORY PROPERTIES RUNTIME_OUTPUT_DIRECTORY
"${CMAKE_CURRENT_BINARY_DIR}" "${CMAKE_CURRENT_BINARY_DIR}"
) )
add_dependencies(TestBinaries LLVMFuzzer-StandaloneInitializeTest)
set_target_properties(LLVMFuzzer-StandaloneInitializeTest
PROPERTIES RUNTIME_OUTPUT_DIRECTORY
"${CMAKE_CURRENT_BINARY_DIR}"
)
###############################################################################
# Additional tests
###############################################################################
include_directories(..) include_directories(..)
# add_subdirectory(uninstrumented)
add_subdirectory(no-coverage)
add_subdirectory(trace-pc)
add_subdirectory(ubsan)
if (NOT MSVC)
add_subdirectory(inline-8bit-counters)
endif()
add_library(LLVMFuzzer-DSO1 SHARED DSO1.cpp)
add_library(LLVMFuzzer-DSO2 SHARED DSO2.cpp)
add_executable(LLVMFuzzer-DSOTest
DSOTestMain.cpp
DSOTestExtra.cpp)
target_link_libraries(LLVMFuzzer-DSOTest
LLVMFuzzer-DSO1
LLVMFuzzer-DSO2
LLVMFuzzer
)
set_target_properties(LLVMFuzzer-DSOTest PROPERTIES RUNTIME_OUTPUT_DIRECTORY
"${CMAKE_BINARY_DIR}/lib/Fuzzer/test")
if(MSVC)
set_output_directory(LLVMFuzzer-DSO1
BINARY_DIR "${CMAKE_BINARY_DIR}/lib/Fuzzer/test"
LIBRARY_DIR "${CMAKE_BINARY_DIR}/lib/Fuzzer/test")
set_output_directory(LLVMFuzzer-DSO2
BINARY_DIR "${CMAKE_BINARY_DIR}/lib/Fuzzer/test"
LIBRARY_DIR "${CMAKE_BINARY_DIR}/lib/Fuzzer/test")
else(MSVC)
set_output_directory(LLVMFuzzer-DSO1
LIBRARY_DIR "${CMAKE_BINARY_DIR}/lib/Fuzzer/lib")
set_output_directory(LLVMFuzzer-DSO2
LIBRARY_DIR "${CMAKE_BINARY_DIR}/lib/Fuzzer/lib")
endif()
add_dependencies(TestBinaries LLVMFuzzer-DSOTest)
############################################################################### ###############################################################################
# Configure lit to run the tests # Configure lit to run the tests
# #
# Note this is done after declaring all tests so we can inform lit if any tests # Note this is done after declaring all tests so we can inform lit if any tests
# need to be disabled. # need to be disabled.
############################################################################### ###############################################################################
set(LIBFUZZER_POSIX 1) set(LIBFUZZER_POSIX 1)
if (MSVC) if (MSVC)
set(LIBFUZZER_POSIX 0) set(LIBFUZZER_POSIX 0)
endif() endif()
# Use just-built Clang to compile/link tests on all platforms, except for
# Windows where we need to use clang-cl instead.
if(NOT MSVC)
set(LIBFUZZER_TEST_COMPILER ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
set(LIBFUZZER_TEST_CXX_COMPILER ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang++)
else()
set(LIBFUZZER_TEST_COMPILER ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang.exe)
set(LIBFUZZER_TEST_CXX_COMPILER ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang++.exe)
endif()
# LIT-based libFuzzer tests.
configure_lit_site_cfg( configure_lit_site_cfg(
${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg
) )
# libFuzzer unit tests.
configure_lit_site_cfg( configure_lit_site_cfg(
${CMAKE_CURRENT_SOURCE_DIR}/unit/lit.site.cfg.in ${CMAKE_CURRENT_SOURCE_DIR}/unit/lit.site.cfg.in
${CMAKE_CURRENT_BINARY_DIR}/unit/lit.site.cfg ${CMAKE_CURRENT_BINARY_DIR}/unit/lit.site.cfg
) )
add_lit_testsuite(check-fuzzer "Running Fuzzer tests" add_lit_testsuite(check-fuzzer "Running Fuzzer tests"
${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR}
DEPENDS TestBinaries DEPENDS LLVMFuzzer-Unittest
) )
# Don't add dependencies on Windows. The linker step would fail on Windows, # Don't add dependencies on Windows. The linker step would fail on Windows,
# since cmake will use link.exe for linking and won't include compiler-rt libs. # since cmake will use link.exe for linking and won't include compiler-rt libs.
if(NOT MSVC) if(NOT MSVC)
add_dependencies(check-fuzzer FileCheck sancov not llvm-symbolizer) add_dependencies(check-fuzzer LLVMFuzzer asan clang llvm-symbolizer FileCheck sancov not)
endif() endif()

llvm/trunk/lib/Fuzzer/test/afl-driver-extra-stats.test

REQUIRES: posix REQUIRES: posix
RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %S/../afl/afl_driver.cpp -o %t-AFLDriverTest
; Test that not specifying an extra stats file isn't broken. ; Test that not specifying an extra stats file isn't broken.
RUN: unset AFL_DRIVER_EXTRA_STATS_FILENAME RUN: unset AFL_DRIVER_EXTRA_STATS_FILENAME
RUN: AFLDriverTest RUN: %t-AFLDriverTest
; Test that specifying an invalid extra stats file causes a crash. ; Test that specifying an invalid extra stats file causes a crash.
RUN: ASAN_OPTIONS= AFL_DRIVER_EXTRA_STATS_FILENAME=%T not --crash AFLDriverTest RUN: ASAN_OPTIONS= AFL_DRIVER_EXTRA_STATS_FILENAME=%T not --crash %t-AFLDriverTest
; Test that specifying a corrupted stats file causes a crash. ; Test that specifying a corrupted stats file causes a crash.
echo "peak_rss_mb :0" > %t echo "peak_rss_mb :0" > %t
ASAN_OPTIONS= AFL_DRIVER_EXTRA_STATS_FILENAME=%t not --crash AFLDriverTest ASAN_OPTIONS= AFL_DRIVER_EXTRA_STATS_FILENAME=%t not --crash %t-AFLDriverTest
; Test that specifying a valid nonexistent stats file works. ; Test that specifying a valid nonexistent stats file works.
RUN: rm -f %t RUN: rm -f %t
RUN: AFL_DRIVER_EXTRA_STATS_FILENAME=%t AFLDriverTest RUN: AFL_DRIVER_EXTRA_STATS_FILENAME=%t %t-AFLDriverTest
RUN: [[ $(grep "peak_rss_mb\|slowest_unit_time_sec" %t | wc -l) -eq 2 ]] RUN: [[ $(grep "peak_rss_mb\|slowest_unit_time_sec" %t | wc -l) -eq 2 ]]
; Test that specifying a valid preexisting stats file works. ; Test that specifying a valid preexisting stats file works.
RUN: printf "peak_rss_mb : 0\nslowest_unit_time_sec: 0\n" > %t RUN: printf "peak_rss_mb : 0\nslowest_unit_time_sec: 0\n" > %t
RUN: AFL_DRIVER_EXTRA_STATS_FILENAME=%t AFLDriverTest RUN: AFL_DRIVER_EXTRA_STATS_FILENAME=%t %t-AFLDriverTest
; Check that both lines were printed. ; Check that both lines were printed.
RUN: [[ $(grep "peak_rss_mb\|slowest_unit_time_sec" %t | wc -l) -eq 2 ]] RUN: [[ $(grep "peak_rss_mb\|slowest_unit_time_sec" %t | wc -l) -eq 2 ]]
; Test that peak_rss_mb and slowest_unit_time_in_secs are only updated when necessary. ; Test that peak_rss_mb and slowest_unit_time_in_secs are only updated when necessary.
; Check that both lines have 9999 since there's no way we have exceeded that ; Check that both lines have 9999 since there's no way we have exceeded that
; amount of time or virtual memory. ; amount of time or virtual memory.
RUN: printf "peak_rss_mb : 9999\nslowest_unit_time_sec: 9999\n" > %t RUN: printf "peak_rss_mb : 9999\nslowest_unit_time_sec: 9999\n" > %t
RUN: AFL_DRIVER_EXTRA_STATS_FILENAME=%t AFLDriverTest RUN: AFL_DRIVER_EXTRA_STATS_FILENAME=%t %t-AFLDriverTest
RUN: [[ $(grep "9999" %t | wc -l) -eq 2 ]] RUN: [[ $(grep "9999" %t | wc -l) -eq 2 ]]

llvm/trunk/lib/Fuzzer/test/afl-driver-stderr.test

REQUIRES: posix REQUIRES: posix
RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %S/../afl/afl_driver.cpp -o %t-AFLDriverTest
; Test that not specifying a stderr file isn't broken. ; Test that not specifying a stderr file isn't broken.
RUN: unset AFL_DRIVER_STDERR_DUPLICATE_FILENAME RUN: unset AFL_DRIVER_STDERR_DUPLICATE_FILENAME
RUN: AFLDriverTest RUN: %t-AFLDriverTest
; Test that specifying an invalid file causes a crash. ; Test that specifying an invalid file causes a crash.
RUN: ASAN_OPTIONS= AFL_DRIVER_STDERR_DUPLICATE_FILENAME="%T" not --crash AFLDriverTest RUN: ASAN_OPTIONS= AFL_DRIVER_STDERR_DUPLICATE_FILENAME="%T" not --crash %t-AFLDriverTest
; Test that a file is created when specified as the duplicate stderr. ; Test that a file is created when specified as the duplicate stderr.
RUN: AFL_DRIVER_STDERR_DUPLICATE_FILENAME=%t AFLDriverTest RUN: AFL_DRIVER_STDERR_DUPLICATE_FILENAME=%t %t-AFLDriverTest
RUN: stat %t RUN: stat %t

llvm/trunk/lib/Fuzzer/test/afl-driver.test

REQUIRES: linux REQUIRES: linux
RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %S/../afl/afl_driver.cpp -o %t-AFLDriverTest
RUN: echo -n "abc" > %t.file3 RUN: echo -n "abc" > %t.file3
RUN: echo -n "abcd" > %t.file4 RUN: echo -n "abcd" > %t.file4
RUN: AFLDriverTest < %t.file3 2>&1 | FileCheck %s --check-prefix=CHECK1 RUN: %t-AFLDriverTest < %t.file3 2>&1 | FileCheck %s --check-prefix=CHECK1
CHECK1: __afl_persistent_loop calle, Count = 1000 CHECK1: __afl_persistent_loop calle, Count = 1000
CHECK1: LLVMFuzzerTestOneInput called; Size = 3 CHECK1: LLVMFuzzerTestOneInput called; Size = 3
RUN: AFLDriverTest < %t.file3 -42 2>&1 | FileCheck %s --check-prefix=CHECK2 RUN: %t-AFLDriverTest < %t.file3 -42 2>&1 | FileCheck %s --check-prefix=CHECK2
CHECK2: __afl_persistent_loop calle, Count = 42 CHECK2: __afl_persistent_loop calle, Count = 42
CHECK2: LLVMFuzzerTestOneInput called; Size = 3 CHECK2: LLVMFuzzerTestOneInput called; Size = 3
RUN: AFLDriverTest < %t.file3 666 2>&1 | FileCheck %s --check-prefix=CHECK3 RUN: %t-AFLDriverTest < %t.file3 666 2>&1 | FileCheck %s --check-prefix=CHECK3
CHECK3: WARNING: using the deprecated call style CHECK3: WARNING: using the deprecated call style
CHECK3: __afl_persistent_loop calle, Count = 666 CHECK3: __afl_persistent_loop calle, Count = 666
CHECK3: LLVMFuzzerTestOneInput called; Size = 3 CHECK3: LLVMFuzzerTestOneInput called; Size = 3
RUN: AFLDriverTest %t.file3 2>&1 | FileCheck %s --check-prefix=CHECK4 RUN: %t-AFLDriverTest %t.file3 2>&1 | FileCheck %s --check-prefix=CHECK4
CHECK4: LLVMFuzzerTestOneInput called; Size = 3 CHECK4: LLVMFuzzerTestOneInput called; Size = 3
RUN: AFLDriverTest %t.file3 %t.file4 2>&1 | FileCheck %s --check-prefix=CHECK5 RUN: %t-AFLDriverTest %t.file3 %t.file4 2>&1 | FileCheck %s --check-prefix=CHECK5
CHECK5: LLVMFuzzerTestOneInput called; Size = 3 CHECK5: LLVMFuzzerTestOneInput called; Size = 3
CHECK5: LLVMFuzzerTestOneInput called; Size = 4 CHECK5: LLVMFuzzerTestOneInput called; Size = 4

llvm/trunk/lib/Fuzzer/test/bad-strcmp.test

RUN: LLVMFuzzer-BadStrcmpTest -runs=100000 RUN: %cpp_compiler %S/BadStrcmpTest.cpp -o %t-LLVMFuzzer-BadStrcmpTest
RUN: %t-LLVMFuzzer-BadStrcmpTest -runs=100000

llvm/trunk/lib/Fuzzer/test/caller-callee.test

RUN: %cpp_compiler %S/CallerCalleeTest.cpp -o %t-LLVMFuzzer-CallerCalleeTest
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-CallerCalleeTest -use_value_profile=1 -cross_over=0 -seed=1 -runs=10000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-CallerCalleeTest -use_value_profile=1 -cross_over=0 -seed=1 -runs=10000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/cleanse.test

RUN: %cpp_compiler %S/CleanseTest.cpp -o %t-LLVMFuzzer-CleanseTest
RUN: echo -n 0123456789ABCDEFGHIZ > %t-in RUN: echo -n 0123456789ABCDEFGHIZ > %t-in
RUN: LLVMFuzzer-CleanseTest -cleanse_crash=1 %t-in -exact_artifact_path=%t-out RUN: %t-LLVMFuzzer-CleanseTest -cleanse_crash=1 %t-in -exact_artifact_path=%t-out
RUN: echo -n ' 1 5 A Z' | diff - %t-out RUN: echo -n ' 1 5 A Z' | diff - %t-out

llvm/trunk/lib/Fuzzer/test/coverage.test

XFAIL: darwin XFAIL: darwin
RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-NullDerefTest
RUN: %cpp_compiler %S/DSO1.cpp -fPIC -shared -o %t-LLVMFuzzer-DSO1.so
RUN: %cpp_compiler %S/DSO2.cpp -fPIC -shared -o %t-LLVMFuzzer-DSO2.so
RUN: %cpp_compiler %S/DSOTestMain.cpp %S/DSOTestExtra.cpp -L. %t-LLVMFuzzer-DSO1.so %t-LLVMFuzzer-DSO2.so -o %t-LLVMFuzzer-DSOTest
CHECK: COVERAGE: CHECK: COVERAGE:
CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:13 CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:13
CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:14 CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:14
CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:16 CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:16
CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:19 CHECK-DAG: COVERED: {{.*}}in LLVMFuzzerTestOneInput {{.*}}NullDerefTest.cpp:19
CHECK: COVERED_DIRS: {{.*}}lib{{[/\\]}}Fuzzer{{[/\\]}}test CHECK: COVERED_DIRS: {{.*}}lib{{[/\\]}}Fuzzer{{[/\\]}}test
RUN: not LLVMFuzzer-NullDerefTest -print_coverage=1 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-NullDerefTest -print_coverage=1 2>&1 | FileCheck %s
RUN: LLVMFuzzer-DSOTest -print_coverage=1 -runs=0 2>&1 | FileCheck %s --check-prefix=DSO RUN: %t-LLVMFuzzer-DSOTest -print_coverage=1 -runs=0 2>&1 | FileCheck %s --check-prefix=DSO
DSO: COVERAGE: DSO: COVERAGE:
DSO-DAG: COVERED:{{.*}}DSO1{{.*}}DSO1.cpp DSO-DAG: COVERED:{{.*}}DSO1{{.*}}DSO1.cpp
DSO-DAG: COVERED:{{.*}}DSO2{{.*}}DSO2.cpp DSO-DAG: COVERED:{{.*}}DSO2{{.*}}DSO2.cpp
DSO-DAG: COVERED:{{.*}}LLVMFuzzerTestOneInput{{.*}}DSOTestMain DSO-DAG: COVERED:{{.*}}LLVMFuzzerTestOneInput{{.*}}DSOTestMain
DSO-DAG: UNCOVERED_LINE:{{.*}}DSO1{{.*}}DSO1.cpp DSO-DAG: UNCOVERED_LINE:{{.*}}DSO1{{.*}}DSO1.cpp
DSO-DAG: UNCOVERED_LINE:{{.*}}DSO2{{.*}}DSO2.cpp DSO-DAG: UNCOVERED_LINE:{{.*}}DSO2{{.*}}DSO2.cpp
DSO-DAG: UNCOVERED_FUNC: in Uncovered1 DSO-DAG: UNCOVERED_FUNC: in Uncovered1
DSO-DAG: UNCOVERED_FUNC: in Uncovered2 DSO-DAG: UNCOVERED_FUNC: in Uncovered2
DSO-DAG: UNCOVERED_LINE: in LLVMFuzzerTestOneInput DSO-DAG: UNCOVERED_LINE: in LLVMFuzzerTestOneInput
DSO-DAG: UNCOVERED_FILE:{{.*}}DSOTestExtra.cpp DSO-DAG: UNCOVERED_FILE:{{.*}}DSOTestExtra.cpp

llvm/trunk/lib/Fuzzer/test/cxxstring.test

UNSUPPORTED: windows UNSUPPORTED: windows
RUN: not LLVMFuzzer-CxxStringEqTest -seed=1 -runs=1000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/CxxStringEqTest.cpp -o %t-LLVMFuzzer-CxxStringEqTest
RUN: not %t-LLVMFuzzer-CxxStringEqTest -seed=1 -runs=1000000 2>&1 | FileCheck %s
CHECK: BINGO CHECK: BINGO

llvm/trunk/lib/Fuzzer/test/disable-leaks.test

REQUIRES: lsan REQUIRES: lsan
RUN: LLVMFuzzer-AccumulateAllocationsTest -detect_leaks=1 -runs=100000 2>&1 | FileCheck %s --check-prefix=ACCUMULATE_ALLOCS RUN: %cpp_compiler %S/AccumulateAllocationsTest.cpp -o %t-LLVMFuzzer-AccumulateAllocationsTest
RUN: %t-LLVMFuzzer-AccumulateAllocationsTest -detect_leaks=1 -runs=100000 2>&1 | FileCheck %s --check-prefix=ACCUMULATE_ALLOCS
ACCUMULATE_ALLOCS: INFO: libFuzzer disabled leak detection after every mutation ACCUMULATE_ALLOCS: INFO: libFuzzer disabled leak detection after every mutation

llvm/trunk/lib/Fuzzer/test/dump_coverage.test

RUN: %cpp_compiler %S/DSO1.cpp -fPIC -shared -o %t-LLVMFuzzer-DSO1.so
RUN: %cpp_compiler %S/DSO2.cpp -fPIC -shared -o %t-LLVMFuzzer-DSO2.so
RUN: %cpp_compiler %S/DSOTestMain.cpp %S/DSOTestExtra.cpp -L. %t-LLVMFuzzer-DSO1.so %t-LLVMFuzzer-DSO2.so -o %t-LLVMFuzzer-DSOTest
RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-NullDerefTest
RUN: rm -rf %t_workdir && mkdir -p %t_workdir RUN: rm -rf %t_workdir && mkdir -p %t_workdir
RUN: env ASAN_OPTIONS=coverage_dir='"%t_workdir"' not LLVMFuzzer-NullDerefTest -dump_coverage=1 2>&1 | FileCheck %s RUN: env ASAN_OPTIONS=coverage_dir='"%t_workdir"' not %t-LLVMFuzzer-NullDerefTest -dump_coverage=1 2>&1 | FileCheck %s
RUN: sancov -covered-functions LLVMFuzzer-NullDerefTest* %t_workdir/*.sancov | FileCheck %s --check-prefix=SANCOV RUN: sancov -covered-functions %t-LLVMFuzzer-NullDerefTest* %t_workdir/*.sancov | FileCheck %s --check-prefix=SANCOV
RUN: env ASAN_OPTIONS=coverage_dir='"%t_workdir"' LLVMFuzzer-DSOTest -dump_coverage=1 -runs=0 2>&1 | FileCheck %s --check-prefix=DSO RUN: env ASAN_OPTIONS=coverage_dir='"%t_workdir"' %t-LLVMFuzzer-DSOTest -dump_coverage=1 -runs=0 2>&1 | FileCheck %s --check-prefix=DSO
RUN: env ASAN_OPTIONS=coverage_dir='"%t_workdir"' not LLVMFuzzer-NullDerefTest -dump_coverage=0 2>&1 | FileCheck %s --check-prefix=NOCOV RUN: env ASAN_OPTIONS=coverage_dir='"%t_workdir"' not %t-LLVMFuzzer-NullDerefTest -dump_coverage=0 2>&1 | FileCheck %s --check-prefix=NOCOV
CHECK: SanitizerCoverage: {{.*}}LLVMFuzzer-NullDerefTest.{{.*}}.sancov: {{.*}} PCs written CHECK: SanitizerCoverage: {{.*}}LLVMFuzzer-NullDerefTest.{{.*}}.sancov: {{.*}} PCs written
SANCOV: LLVMFuzzerTestOneInput SANCOV: LLVMFuzzerTestOneInput
DSO: SanitizerCoverage: {{.*}}LLVMFuzzer-DSOTest.{{.*}}.sancov: {{.*}} PCs written DSO: SanitizerCoverage: {{.*}}LLVMFuzzer-DSOTest.{{.*}}.sancov: {{.*}} PCs written
DSO-DAG: SanitizerCoverage: {{.*}}LLVMFuzzer-DSO1.{{.*}}.sancov: {{.*}} PCs written DSO-DAG: SanitizerCoverage: {{.*}}LLVMFuzzer-DSO1.{{.*}}.sancov: {{.*}} PCs written
DSO-DAG: SanitizerCoverage: {{.*}}LLVMFuzzer-DSO2.{{.*}}.sancov: {{.*}} PCs written DSO-DAG: SanitizerCoverage: {{.*}}LLVMFuzzer-DSO2.{{.*}}.sancov: {{.*}} PCs written
NOCOV-NOT: SanitizerCoverage: {{.*}} PCs written NOCOV-NOT: SanitizerCoverage: {{.*}} PCs written

llvm/trunk/lib/Fuzzer/test/equivalence-signals.test

REQUIRES: posix REQUIRES: posix
# Run EquivalenceATest against itself with a small timeout # Run EquivalenceATest against itself with a small timeout
# to stress the signal handling and ensure that shmem doesn't mind # to stress the signal handling and ensure that shmem doesn't mind
# the signals. # the signals.
RUN: LLVMFuzzer-EquivalenceATest -timeout=1 -run_equivalence_server=EQUIV_SIG_TEST & export APID=$! RUN: %cpp_compiler %S/EquivalenceATest.cpp -o %t-LLVMFuzzer-EquivalenceATest
RUN: %t-LLVMFuzzer-EquivalenceATest -timeout=1 -run_equivalence_server=EQUIV_SIG_TEST & export APID=$!
RUN: sleep 3 RUN: sleep 3
RUN: LLVMFuzzer-EquivalenceATest -timeout=1 -use_equivalence_server=EQUIV_SIG_TEST -runs=500000 2>&1 RUN: %t-LLVMFuzzer-EquivalenceATest -timeout=1 -use_equivalence_server=EQUIV_SIG_TEST -runs=500000 2>&1
RUN: kill -9 $APID RUN: kill -9 $APID

llvm/trunk/lib/Fuzzer/test/equivalence.test

REQUIRES: posix REQUIRES: posix
RUN: %cpp_compiler %S/EquivalenceATest.cpp -o %t-LLVMFuzzer-EquivalenceATest
RUN: %cpp_compiler %S/EquivalenceBTest.cpp -o %t-LLVMFuzzer-EquivalenceBTest
RUN: LLVMFuzzer-EquivalenceATest -run_equivalence_server=EQUIV_TEST & export APID=$! RUN: %t-LLVMFuzzer-EquivalenceATest -run_equivalence_server=EQUIV_TEST & export APID=$!
RUN: sleep 3 RUN: sleep 3
RUN: not LLVMFuzzer-EquivalenceBTest -use_equivalence_server=EQUIV_TEST -max_len=4096 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-EquivalenceBTest -use_equivalence_server=EQUIV_TEST -max_len=4096 2>&1 | FileCheck %s
CHECK: ERROR: libFuzzer: equivalence-mismatch. Sizes: {{.*}}; offset 2 CHECK: ERROR: libFuzzer: equivalence-mismatch. Sizes: {{.*}}; offset 2
CHECK: SUMMARY: libFuzzer: equivalence-mismatch CHECK: SUMMARY: libFuzzer: equivalence-mismatch
RUN: kill -9 $APID RUN: kill -9 $APID

llvm/trunk/lib/Fuzzer/test/exit-report.test

RUN: not LLVMFuzzer-SimpleTest 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
RUN: not %t-LLVMFuzzer-SimpleTest 2>&1 | FileCheck %s
CHECK: ERROR: libFuzzer: fuzz target exited CHECK: ERROR: libFuzzer: fuzz target exited
CHECK: SUMMARY: libFuzzer: fuzz target exited CHECK: SUMMARY: libFuzzer: fuzz target exited
CHECK: Test unit written to CHECK: Test unit written to

llvm/trunk/lib/Fuzzer/test/extra-counters.test

REQUIRES: linux REQUIRES: linux
RUN: not LLVMFuzzer-TableLookupTest -print_final_stats=1 2>&1 | FileCheck %s RUN: %cpp_compiler %S/TableLookupTest.cpp -o %t-LLVMFuzzer-TableLookupTest
RUN: not %t-LLVMFuzzer-TableLookupTest -print_final_stats=1 2>&1 | FileCheck %s
CHECK: BINGO CHECK: BINGO
// Expecting >= 4096 new_units_added // Expecting >= 4096 new_units_added
CHECK: stat::new_units_added:{{.*[4][0-9][0-9][0-9]}} CHECK: stat::new_units_added:{{.*[4][0-9][0-9][0-9]}}

llvm/trunk/lib/Fuzzer/test/fuzzer-customcrossover.test

RUN: %cpp_compiler %S/CustomCrossOverTest.cpp -o %t-LLVMFuzzer-CustomCrossOverTest
RUN: rm -rf %t/CustomCrossover RUN: rm -rf %t/CustomCrossover
RUN: mkdir -p %t/CustomCrossover RUN: mkdir -p %t/CustomCrossover
RUN: echo "0123456789" > %t/CustomCrossover/digits RUN: echo "0123456789" > %t/CustomCrossover/digits
RUN: echo "abcdefghij" > %t/CustomCrossover/chars RUN: echo "abcdefghij" > %t/CustomCrossover/chars
RUN: not LLVMFuzzer-CustomCrossOverTest -seed=1 -runs=100000 %t/CustomCrossover 2>&1 | FileCheck %s --check-prefix=LLVMFuzzerCustomCrossover RUN: not %t-LLVMFuzzer-CustomCrossOverTest -seed=1 -runs=100000 %t/CustomCrossover 2>&1 | FileCheck %s --check-prefix=LLVMFuzzerCustomCrossover
RUN: rm -rf %t/CustomCrossover RUN: rm -rf %t/CustomCrossover
LLVMFuzzerCustomCrossover: In LLVMFuzzerCustomCrossover LLVMFuzzerCustomCrossover: In LLVMFuzzerCustomCrossover
LLVMFuzzerCustomCrossover: BINGO LLVMFuzzerCustomCrossover: BINGO

llvm/trunk/lib/Fuzzer/test/fuzzer-customcrossoverandmutate.test

RUN: LLVMFuzzer-CustomCrossOverAndMutateTest -seed=1 -runs=100000 RUN: %cpp_compiler %S/CustomCrossOverAndMutateTest.cpp -o %t-LLVMFuzzer-CustomCrossOverAndMutateTest
RUN: %t-LLVMFuzzer-CustomCrossOverAndMutateTest -seed=1 -runs=100000

llvm/trunk/lib/Fuzzer/test/fuzzer-custommutator.test

RUN: not LLVMFuzzer-CustomMutatorTest 2>&1 | FileCheck %s --check-prefix=LLVMFuzzerCustomMutator RUN: %cpp_compiler %S/CustomMutatorTest.cpp -o %t-LLVMFuzzer-CustomMutatorTest
RUN: not %t-LLVMFuzzer-CustomMutatorTest 2>&1 | FileCheck %s --check-prefix=LLVMFuzzerCustomMutator
LLVMFuzzerCustomMutator: In LLVMFuzzerCustomMutator LLVMFuzzerCustomMutator: In LLVMFuzzerCustomMutator
LLVMFuzzerCustomMutator: BINGO LLVMFuzzerCustomMutator: BINGO

llvm/trunk/lib/Fuzzer/test/fuzzer-dict.test

RUN: %cpp_compiler %S/SimpleDictionaryTest.cpp -o %t-LLVMFuzzer-SimpleDictionaryTest
CHECK: BINGO CHECK: BINGO
Done1000000: Done 1000000 runs in Done1000000: Done 1000000 runs in
RUN: not LLVMFuzzer-SimpleDictionaryTest -dict=%S/dict1.txt -seed=1 -runs=1000003 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-SimpleDictionaryTest -dict=%S/dict1.txt -seed=1 -runs=1000003 2>&1 | FileCheck %s
RUN: LLVMFuzzer-SimpleDictionaryTest -seed=1 -runs=1000000 2>&1 | FileCheck %s --check-prefix=Done1000000 RUN: %t-LLVMFuzzer-SimpleDictionaryTest -seed=1 -runs=1000000 2>&1 | FileCheck %s --check-prefix=Done1000000

llvm/trunk/lib/Fuzzer/test/fuzzer-dirs.test

RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
RUN: rm -rf %t/SUB1 RUN: rm -rf %t/SUB1
RUN: mkdir -p %t/SUB1/SUB2/SUB3 RUN: mkdir -p %t/SUB1/SUB2/SUB3
RUN: echo a > %t/SUB1/a RUN: echo a > %t/SUB1/a
RUN: echo b > %t/SUB1/SUB2/b RUN: echo b > %t/SUB1/SUB2/b
RUN: echo c > %t/SUB1/SUB2/SUB3/c RUN: echo c > %t/SUB1/SUB2/SUB3/c
RUN: LLVMFuzzer-SimpleTest %t/SUB1 -runs=0 2>&1 | FileCheck %s --check-prefix=SUBDIRS RUN: %t-LLVMFuzzer-SimpleTest %t/SUB1 -runs=0 2>&1 | FileCheck %s --check-prefix=SUBDIRS
SUBDIRS: READ units: 3 SUBDIRS: READ units: 3
RUN: echo -n zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz > %t/SUB1/f64 RUN: echo -n zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz > %t/SUB1/f64
RUN: cat %t/SUB1/f64 %t/SUB1/f64 %t/SUB1/f64 %t/SUB1/f64 > %t/SUB1/f256 RUN: cat %t/SUB1/f64 %t/SUB1/f64 %t/SUB1/f64 %t/SUB1/f64 > %t/SUB1/f256
RUN: cat %t/SUB1/f256 %t/SUB1/f256 %t/SUB1/f256 %t/SUB1/f256 > %t/SUB1/f1024 RUN: cat %t/SUB1/f256 %t/SUB1/f256 %t/SUB1/f256 %t/SUB1/f256 > %t/SUB1/f1024
RUN: cat %t/SUB1/f1024 %t/SUB1/f1024 %t/SUB1/f1024 %t/SUB1/f1024 > %t/SUB1/f4096 RUN: cat %t/SUB1/f1024 %t/SUB1/f1024 %t/SUB1/f1024 %t/SUB1/f1024 > %t/SUB1/f4096
RUN: cat %t/SUB1/f4096 %t/SUB1/f4096 > %t/SUB1/f8192 RUN: cat %t/SUB1/f4096 %t/SUB1/f4096 > %t/SUB1/f8192
RUN: LLVMFuzzer-SimpleTest %t/SUB1 -runs=0 2>&1 | FileCheck %s --check-prefix=LONG RUN: %t-LLVMFuzzer-SimpleTest %t/SUB1 -runs=0 2>&1 | FileCheck %s --check-prefix=LONG
LONG: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 8192 bytes LONG: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 8192 bytes
RUN: rm -rf %t/SUB1 RUN: rm -rf %t/SUB1
RUN: not LLVMFuzzer-SimpleTest NONEXISTENT_DIR 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR RUN: not %t-LLVMFuzzer-SimpleTest NONEXISTENT_DIR 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR
NONEXISTENT_DIR: No such directory: NONEXISTENT_DIR; exiting NONEXISTENT_DIR: No such directory: NONEXISTENT_DIR; exiting

llvm/trunk/lib/Fuzzer/test/fuzzer-fdmask.test

RUN: LLVMFuzzer-SpamyTest -runs=1 2>&1 | FileCheck %s --check-prefix=FD_MASK_0 RUN: %cpp_compiler %S/SpamyTest.cpp -o %t-LLVMFuzzer-SpamyTest
RUN: LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=0 2>&1 | FileCheck %s --check-prefix=FD_MASK_0
RUN: LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=1 2>&1 | FileCheck %s --check-prefix=FD_MASK_1 RUN: %t-LLVMFuzzer-SpamyTest -runs=1 2>&1 | FileCheck %s --check-prefix=FD_MASK_0
RUN: LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=2 2>&1 | FileCheck %s --check-prefix=FD_MASK_2 RUN: %t-LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=0 2>&1 | FileCheck %s --check-prefix=FD_MASK_0
RUN: LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=3 2>&1 | FileCheck %s --check-prefix=FD_MASK_3 RUN: %t-LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=1 2>&1 | FileCheck %s --check-prefix=FD_MASK_1
RUN: %t-LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=2 2>&1 | FileCheck %s --check-prefix=FD_MASK_2
RUN: %t-LLVMFuzzer-SpamyTest -runs=1 -close_fd_mask=3 2>&1 | FileCheck %s --check-prefix=FD_MASK_3
FD_MASK_0: PRINTF_STDOUT FD_MASK_0: PRINTF_STDOUT
FD_MASK_0: PRINTF_STDERR FD_MASK_0: PRINTF_STDERR
FD_MASK_0: STREAM_COUT FD_MASK_0: STREAM_COUT
FD_MASK_0: STREAM_CERR FD_MASK_0: STREAM_CERR
FD_MASK_0: INITED FD_MASK_0: INITED
FD_MASK_1-NOT: PRINTF_STDOUT FD_MASK_1-NOT: PRINTF_STDOUT
Show All 17 Lines

llvm/trunk/lib/Fuzzer/test/fuzzer-finalstats.test

RUN: LLVMFuzzer-SimpleTest -seed=1 -runs=77 -print_final_stats=1 2>&1 | FileCheck %s --check-prefix=FINAL_STATS RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
RUN: %t-LLVMFuzzer-SimpleTest -seed=1 -runs=77 -print_final_stats=1 2>&1 | FileCheck %s --check-prefix=FINAL_STATS
FINAL_STATS: stat::number_of_executed_units: 77 FINAL_STATS: stat::number_of_executed_units: 77
FINAL_STATS: stat::average_exec_per_sec: 0 FINAL_STATS: stat::average_exec_per_sec: 0
FINAL_STATS: stat::new_units_added: FINAL_STATS: stat::new_units_added:
FINAL_STATS: stat::slowest_unit_time_sec: 0 FINAL_STATS: stat::slowest_unit_time_sec: 0
FINAL_STATS: stat::peak_rss_mb: FINAL_STATS: stat::peak_rss_mb:
RUN: LLVMFuzzer-SimpleTest %S/dict1.txt -runs=33 -print_final_stats=1 2>&1 | FileCheck %s --check-prefix=FINAL_STATS1 RUN: %t-LLVMFuzzer-SimpleTest %S/dict1.txt -runs=33 -print_final_stats=1 2>&1 | FileCheck %s --check-prefix=FINAL_STATS1
FINAL_STATS1: stat::number_of_executed_units: 33 FINAL_STATS1: stat::number_of_executed_units: 33
FINAL_STATS1: stat::peak_rss_mb: FINAL_STATS1: stat::peak_rss_mb:

llvm/trunk/lib/Fuzzer/test/fuzzer-flags.test

RUN: not LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck %s --check-prefix=FOO_BAR RUN: %cpp_compiler %S/FlagsTest.cpp -o %t-LLVMFuzzer-FlagsTest
RUN: not %t-LLVMFuzzer-FlagsTest -foo_bar=1 2>&1 | FileCheck %s --check-prefix=FOO_BAR
FOO_BAR: WARNING: unrecognized flag '-foo_bar=1'; use -help=1 to list all flags FOO_BAR: WARNING: unrecognized flag '-foo_bar=1'; use -help=1 to list all flags
FOO_BAR: BINGO FOO_BAR: BINGO
RUN: not LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck %s --check-prefix=DASH_DASH RUN: not %t-LLVMFuzzer-FlagsTest -runs=10 --max_len=100 2>&1 | FileCheck %s --check-prefix=DASH_DASH
DASH_DASH: WARNING: did you mean '-max_len=100' (single dash)? DASH_DASH: WARNING: did you mean '-max_len=100' (single dash)?
DASH_DASH: INFO: A corpus is not provided, starting from an empty corpus DASH_DASH: INFO: A corpus is not provided, starting from an empty corpus
RUN: LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck %s --check-prefix=NO_INTERNAL RUN: %t-LLVMFuzzer-FlagsTest -help=1 2>&1 | FileCheck %s --check-prefix=NO_INTERNAL
NO_INTERNAL-NOT: internal flag NO_INTERNAL-NOT: internal flag
RUN: not LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU RUN: not %t-LLVMFuzzer-FlagsTest --foo-bar -runs=10 -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU
PASSTHRU: BINGO --foo-bar --baz -help=1 test PASSTHRU: BINGO --foo-bar --baz -help=1 test
RUN: mkdir -p %t/T0 %t/T1 RUN: mkdir -p %t/T0 %t/T1
RUN: touch %t/T1/empty RUN: touch %t/T1/empty
RUN: not LLVMFuzzer-FlagsTest --foo-bar -merge=1 %t/T0 %t/T1 -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU-MERGE RUN: not %t-LLVMFuzzer-FlagsTest --foo-bar -merge=1 %t/T0 %t/T1 -ignore_remaining_args=1 --baz -help=1 test 2>&1 | FileCheck %s --check-prefix=PASSTHRU-MERGE
PASSTHRU-MERGE: BINGO --foo-bar --baz -help=1 test PASSTHRU-MERGE: BINGO --foo-bar --baz -help=1 test

llvm/trunk/lib/Fuzzer/test/fuzzer-leak.test

REQUIRES: lsan REQUIRES: lsan
RUN: not LLVMFuzzer-LeakTest -runs=100000 -detect_leaks=1 2>&1 | FileCheck %s --check-prefix=LEAK_DURING RUN: %cpp_compiler %S/LeakTest.cpp -o %t-LLVMFuzzer-LeakTest
RUN: %cpp_compiler %S/ThreadedLeakTest.cpp -o %t-LLVMFuzzer-ThreadedLeakTest
RUN: %cpp_compiler %S/LeakTimeoutTest.cpp -o %t-LLVMFuzzer-LeakTimeoutTest
RUN: not %t-LLVMFuzzer-LeakTest -runs=100000 -detect_leaks=1 2>&1 | FileCheck %s --check-prefix=LEAK_DURING
LEAK_DURING: ERROR: LeakSanitizer: detected memory leaks LEAK_DURING: ERROR: LeakSanitizer: detected memory leaks
LEAK_DURING: Direct leak of 4 byte(s) in 1 object(s) allocated from: LEAK_DURING: Direct leak of 4 byte(s) in 1 object(s) allocated from:
LEAK_DURING: INFO: to ignore leaks on libFuzzer side use -detect_leaks=0 LEAK_DURING: INFO: to ignore leaks on libFuzzer side use -detect_leaks=0
LEAK_DURING: Test unit written to ./leak- LEAK_DURING: Test unit written to ./leak-
LEAK_DURING-NOT: DONE LEAK_DURING-NOT: DONE
LEAK_DURING-NOT: Done LEAK_DURING-NOT: Done
RUN: not LLVMFuzzer-LeakTest -runs=0 -detect_leaks=1 %S 2>&1 | FileCheck %s --check-prefix=LEAK_IN_CORPUS RUN: not %t-LLVMFuzzer-LeakTest -runs=0 -detect_leaks=1 %S 2>&1 | FileCheck %s --check-prefix=LEAK_IN_CORPUS
LEAK_IN_CORPUS: ERROR: LeakSanitizer: detected memory leaks LEAK_IN_CORPUS: ERROR: LeakSanitizer: detected memory leaks
LEAK_IN_CORPUS: INFO: a leak has been found in the initial corpus. LEAK_IN_CORPUS: INFO: a leak has been found in the initial corpus.
RUN: not LLVMFuzzer-LeakTest -runs=100000000 %S/hi.txt 2>&1 | FileCheck %s --check-prefix=MULTI_RUN_LEAK RUN: not %t-LLVMFuzzer-LeakTest -runs=100000000 %S/hi.txt 2>&1 | FileCheck %s --check-prefix=MULTI_RUN_LEAK
MULTI_RUN_LEAK-NOT: pulse MULTI_RUN_LEAK-NOT: pulse
MULTI_RUN_LEAK: LeakSanitizer: detected memory leaks MULTI_RUN_LEAK: LeakSanitizer: detected memory leaks
RUN: not LLVMFuzzer-LeakTest -runs=100000 -detect_leaks=0 2>&1 | FileCheck %s --check-prefix=LEAK_AFTER RUN: not %t-LLVMFuzzer-LeakTest -runs=100000 -detect_leaks=0 2>&1 | FileCheck %s --check-prefix=LEAK_AFTER
RUN: not LLVMFuzzer-LeakTest -runs=100000 2>&1 | FileCheck %s --check-prefix=LEAK_DURING RUN: not %t-LLVMFuzzer-LeakTest -runs=100000 2>&1 | FileCheck %s --check-prefix=LEAK_DURING
RUN: not LLVMFuzzer-ThreadedLeakTest -runs=100000 -detect_leaks=0 2>&1 | FileCheck %s --check-prefix=LEAK_AFTER RUN: not %t-LLVMFuzzer-ThreadedLeakTest -runs=100000 -detect_leaks=0 2>&1 | FileCheck %s --check-prefix=LEAK_AFTER
RUN: not LLVMFuzzer-ThreadedLeakTest -runs=100000 2>&1 | FileCheck %s --check-prefix=LEAK_DURING RUN: not %t-LLVMFuzzer-ThreadedLeakTest -runs=100000 2>&1 | FileCheck %s --check-prefix=LEAK_DURING
LEAK_AFTER: Done 100000 runs in LEAK_AFTER: Done 100000 runs in
LEAK_AFTER: ERROR: LeakSanitizer: detected memory leaks LEAK_AFTER: ERROR: LeakSanitizer: detected memory leaks
RUN: not LLVMFuzzer-LeakTest -runs=100000 -max_len=1 2>&1 | FileCheck %s --check-prefix=MAX_LEN_1 RUN: not %t-LLVMFuzzer-LeakTest -runs=100000 -max_len=1 2>&1 | FileCheck %s --check-prefix=MAX_LEN_1
MAX_LEN_1: Test unit written to ./leak-7cf184f4c67ad58283ecb19349720b0cae756829 MAX_LEN_1: Test unit written to ./leak-7cf184f4c67ad58283ecb19349720b0cae756829
RUN: not LLVMFuzzer-LeakTimeoutTest -timeout=1 2>&1 | FileCheck %s --check-prefix=LEAK_TIMEOUT RUN: not %t-LLVMFuzzer-LeakTimeoutTest -timeout=1 2>&1 | FileCheck %s --check-prefix=LEAK_TIMEOUT
LEAK_TIMEOUT: ERROR: libFuzzer: timeout after LEAK_TIMEOUT: ERROR: libFuzzer: timeout after
LEAK_TIMEOUT-NOT: LeakSanitizer LEAK_TIMEOUT-NOT: LeakSanitizer
RUN: LLVMFuzzer-LeakTest -error_exitcode=0 RUN: %t-LLVMFuzzer-LeakTest -error_exitcode=0

llvm/trunk/lib/Fuzzer/test/fuzzer-oom-with-profile.test

REQUIRES: linux REQUIRES: linux
RUN: not LLVMFuzzer-OutOfMemoryTest -rss_limit_mb=300 2>&1 | FileCheck %s RUN: %cpp_compiler %S/OutOfMemoryTest.cpp -o %t-LLVMFuzzer-OutOfMemoryTest
RUN: not %t-LLVMFuzzer-OutOfMemoryTest -rss_limit_mb=300 2>&1 | FileCheck %s
CHECK: ERROR: libFuzzer: out-of-memory (used: {{.*}}; limit: 300Mb) CHECK: ERROR: libFuzzer: out-of-memory (used: {{.*}}; limit: 300Mb)
CHECK: Live Heap Allocations CHECK: Live Heap Allocations
CHECK: Test unit written to ./oom- CHECK: Test unit written to ./oom-
SUMMARY: libFuzzer: out-of-memory SUMMARY: libFuzzer: out-of-memory

llvm/trunk/lib/Fuzzer/test/fuzzer-oom.test

RUN: not LLVMFuzzer-OutOfMemoryTest -rss_limit_mb=300 2>&1 | FileCheck %s RUN: %cpp_compiler %S/OutOfMemoryTest.cpp -o %t-LLVMFuzzer-OutOfMemoryTest
RUN: %cpp_compiler %S/OutOfMemorySingleLargeMallocTest.cpp -o %t-LLVMFuzzer-OutOfMemorySingleLargeMallocTest
RUN: %cpp_compiler %S/AccumulateAllocationsTest.cpp -o %t-LLVMFuzzer-AccumulateAllocationsTest
RUN: not %t-LLVMFuzzer-OutOfMemoryTest -rss_limit_mb=300 2>&1 | FileCheck %s
CHECK: ERROR: libFuzzer: out-of-memory (used: {{.*}}; limit: 300Mb) CHECK: ERROR: libFuzzer: out-of-memory (used: {{.*}}; limit: 300Mb)
CHECK: Test unit written to ./oom- CHECK: Test unit written to ./oom-
SUMMARY: libFuzzer: out-of-memory SUMMARY: libFuzzer: out-of-memory
RUN: not LLVMFuzzer-OutOfMemorySingleLargeMallocTest -rss_limit_mb=300 2>&1 | FileCheck %s --check-prefix=SINGLE_LARGE_MALLOC RUN: not %t-LLVMFuzzer-OutOfMemorySingleLargeMallocTest -rss_limit_mb=300 2>&1 | FileCheck %s --check-prefix=SINGLE_LARGE_MALLOC
We used to check for "out-of-memory (malloc(53{{.*}}))", but that would fail We used to check for "out-of-memory (malloc(53{{.*}}))", but that would fail
sometimes, so now we accept any OOM message. sometimes, so now we accept any OOM message.
SINGLE_LARGE_MALLOC: libFuzzer: out-of-memory SINGLE_LARGE_MALLOC: libFuzzer: out-of-memory
SINGLE_LARGE_MALLOC: in LLVMFuzzerTestOneInput SINGLE_LARGE_MALLOC: in LLVMFuzzerTestOneInput
# Check that -rss_limit_mb=0 means no limit. # Check that -rss_limit_mb=0 means no limit.
RUN: LLVMFuzzer-AccumulateAllocationsTest -runs=1000 -rss_limit_mb=0 RUN: %t-LLVMFuzzer-AccumulateAllocationsTest -runs=1000 -rss_limit_mb=0

llvm/trunk/lib/Fuzzer/test/fuzzer-printcovpcs.test

RUN: echo RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
DISABLED: not LLVMFuzzer-SimpleTest -print_pcs=1 -seed=1 2>&1 | FileCheck %s --check-prefix=PCS DISABLED: not %t-LLVMFuzzer-SimpleTest -print_pcs=1 -seed=1 2>&1 | FileCheck %s --check-prefix=PCS
PCS-NOT: NEW_PC PCS-NOT: NEW_PC
PCS:INITED PCS:INITED
PCS:NEW_PC: {{0x[a-f0-9]+}} PCS:NEW_PC: {{0x[a-f0-9]+}}
PCS:NEW_PC: {{0x[a-f0-9]+}} PCS:NEW_PC: {{0x[a-f0-9]+}}
PCS:NEW PCS:NEW
PCS:BINGO PCS:BINGO

llvm/trunk/lib/Fuzzer/test/fuzzer-runs.test

RUN: mkdir -p %t RUN: mkdir -p %t
RUN: %cpp_compiler %S/NthRunCrashTest.cpp -o %t-LLVMFuzzer-NthRunCrashTest
RUN: echo abcd > %t/NthRunCrashTest.in RUN: echo abcd > %t/NthRunCrashTest.in
RUN: LLVMFuzzer-NthRunCrashTest %t/NthRunCrashTest.in RUN: %t-LLVMFuzzer-NthRunCrashTest %t/NthRunCrashTest.in
RUN: LLVMFuzzer-NthRunCrashTest %t/NthRunCrashTest.in -runs=10 RUN: %t-LLVMFuzzer-NthRunCrashTest %t/NthRunCrashTest.in -runs=10
RUN: not LLVMFuzzer-NthRunCrashTest %t/NthRunCrashTest.in -runs=10000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-NthRunCrashTest %t/NthRunCrashTest.in -runs=10000 2>&1 | FileCheck %s
RUN: rm %t/NthRunCrashTest.in RUN: rm %t/NthRunCrashTest.in
CHECK: BINGO CHECK: BINGO

llvm/trunk/lib/Fuzzer/test/fuzzer-seed.test

RUN: LLVMFuzzer-SimpleCmpTest -seed=-1 -runs=0 2>&1 | FileCheck %s --check-prefix=CHECK_SEED_MINUS_ONE RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-SimpleCmpTest
RUN: %t-LLVMFuzzer-SimpleCmpTest -seed=-1 -runs=0 2>&1 | FileCheck %s --check-prefix=CHECK_SEED_MINUS_ONE
CHECK_SEED_MINUS_ONE: Seed: 4294967295 CHECK_SEED_MINUS_ONE: Seed: 4294967295

llvm/trunk/lib/Fuzzer/test/fuzzer-segv.test

RUN: env ASAN_OPTIONS=handle_segv=0 not LLVMFuzzer-NullDerefTest 2>&1 | FileCheck %s --check-prefix=LIBFUZZER_OWN_SEGV_HANDLER RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-NullDerefTest
RUN: env ASAN_OPTIONS=handle_segv=0 not %t-LLVMFuzzer-NullDerefTest 2>&1 | FileCheck %s --check-prefix=LIBFUZZER_OWN_SEGV_HANDLER
LIBFUZZER_OWN_SEGV_HANDLER: == ERROR: libFuzzer: deadly signal LIBFUZZER_OWN_SEGV_HANDLER: == ERROR: libFuzzer: deadly signal
LIBFUZZER_OWN_SEGV_HANDLER: SUMMARY: libFuzzer: deadly signal LIBFUZZER_OWN_SEGV_HANDLER: SUMMARY: libFuzzer: deadly signal
LIBFUZZER_OWN_SEGV_HANDLER: Test unit written to ./crash- LIBFUZZER_OWN_SEGV_HANDLER: Test unit written to ./crash-
RUN: env ASAN_OPTIONS=handle_segv=1 not LLVMFuzzer-NullDerefTest 2>&1 | FileCheck %s --check-prefix=LIBFUZZER_ASAN_SEGV_HANDLER RUN: env ASAN_OPTIONS=handle_segv=1 not %t-LLVMFuzzer-NullDerefTest 2>&1 | FileCheck %s --check-prefix=LIBFUZZER_ASAN_SEGV_HANDLER
LIBFUZZER_ASAN_SEGV_HANDLER: ERROR: AddressSanitizer: {{SEGV|access-violation}} on unknown address LIBFUZZER_ASAN_SEGV_HANDLER: ERROR: AddressSanitizer: {{SEGV|access-violation}} on unknown address

llvm/trunk/lib/Fuzzer/test/fuzzer-singleinputs.test

RUN: not LLVMFuzzer-NullDerefTest %S/hi.txt 2>&1 | FileCheck %s --check-prefix=SingleInput RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-NullDerefTest
RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
RUN: not %t-LLVMFuzzer-NullDerefTest %S/hi.txt 2>&1 | FileCheck %s --check-prefix=SingleInput
SingleInput-NOT: Test unit written to ./crash- SingleInput-NOT: Test unit written to ./crash-
RUN: rm -rf %tmp/SINGLE_INPUTS RUN: rm -rf %tmp/SINGLE_INPUTS
RUN: mkdir -p %tmp/SINGLE_INPUTS RUN: mkdir -p %tmp/SINGLE_INPUTS
RUN: echo aaa > %tmp/SINGLE_INPUTS/aaa RUN: echo aaa > %tmp/SINGLE_INPUTS/aaa
RUN: echo bbb > %tmp/SINGLE_INPUTS/bbb RUN: echo bbb > %tmp/SINGLE_INPUTS/bbb
RUN: LLVMFuzzer-SimpleTest %tmp/SINGLE_INPUTS/aaa %tmp/SINGLE_INPUTS/bbb 2>&1 | FileCheck %s --check-prefix=SINGLE_INPUTS RUN: %t-LLVMFuzzer-SimpleTest %tmp/SINGLE_INPUTS/aaa %tmp/SINGLE_INPUTS/bbb 2>&1 | FileCheck %s --check-prefix=SINGLE_INPUTS
RUN: LLVMFuzzer-SimpleTest -max_len=2 %tmp/SINGLE_INPUTS/aaa %tmp/SINGLE_INPUTS/bbb 2>&1 | FileCheck %s --check-prefix=SINGLE_INPUTS RUN: %t-LLVMFuzzer-SimpleTest -max_len=2 %tmp/SINGLE_INPUTS/aaa %tmp/SINGLE_INPUTS/bbb 2>&1 | FileCheck %s --check-prefix=SINGLE_INPUTS
RUN: rm -rf %tmp/SINGLE_INPUTS RUN: rm -rf %tmp/SINGLE_INPUTS
SINGLE_INPUTS: LLVMFuzzer-SimpleTest{{.*}}: Running 2 inputs 1 time(s) each. SINGLE_INPUTS: LLVMFuzzer-SimpleTest{{.*}}: Running 2 inputs 1 time(s) each.
SINGLE_INPUTS: aaa in SINGLE_INPUTS: aaa in
SINGLE_INPUTS: bbb in SINGLE_INPUTS: bbb in
SINGLE_INPUTS: NOTE: fuzzing was not performed, you have only SINGLE_INPUTS: NOTE: fuzzing was not performed, you have only
SINGLE_INPUTS: executed the target code on a fixed set of inputs. SINGLE_INPUTS: executed the target code on a fixed set of inputs.

llvm/trunk/lib/Fuzzer/test/fuzzer-threaded.test

CHECK: Done 1000 runs in CHECK: Done 1000 runs in
RUN: %cpp_compiler %S/ThreadedTest.cpp -o %t-LLVMFuzzer-ThreadedTest
RUN: LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s RUN: %t-LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s
RUN: LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s RUN: %t-LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s
RUN: LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s RUN: %t-LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s
RUN: LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s RUN: %t-LLVMFuzzer-ThreadedTest -use_traces=1 -runs=1000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/fuzzer-timeout.test

RUN: not LLVMFuzzer-TimeoutTest -timeout=1 2>&1 | FileCheck %s --check-prefix=TimeoutTest RUN: %cpp_compiler %S/TimeoutTest.cpp -o %t-LLVMFuzzer-TimeoutTest
RUN: %cpp_compiler %S/TimeoutEmptyTest.cpp -o %t-LLVMFuzzer-TimeoutEmptyTest
RUN: not %t-LLVMFuzzer-TimeoutTest -timeout=1 2>&1 | FileCheck %s --check-prefix=TimeoutTest
TimeoutTest: ALARM: working on the last Unit for TimeoutTest: ALARM: working on the last Unit for
TimeoutTest: Test unit written to ./timeout- TimeoutTest: Test unit written to ./timeout-
TimeoutTest: == ERROR: libFuzzer: timeout after TimeoutTest: == ERROR: libFuzzer: timeout after
TimeoutTest: #0 TimeoutTest: #0
TimeoutTest: #1 TimeoutTest: #1
TimeoutTest: #2 TimeoutTest: #2
TimeoutTest: SUMMARY: libFuzzer: timeout TimeoutTest: SUMMARY: libFuzzer: timeout
RUN: not LLVMFuzzer-TimeoutTest -timeout=1 %S/hi.txt 2>&1 | FileCheck %s --check-prefix=SingleInputTimeoutTest RUN: not %t-LLVMFuzzer-TimeoutTest -timeout=1 %S/hi.txt 2>&1 | FileCheck %s --check-prefix=SingleInputTimeoutTest
SingleInputTimeoutTest: ALARM: working on the last Unit for {{[1-3]}} seconds SingleInputTimeoutTest: ALARM: working on the last Unit for {{[1-3]}} seconds
SingleInputTimeoutTest-NOT: Test unit written to ./timeout- SingleInputTimeoutTest-NOT: Test unit written to ./timeout-
RUN: LLVMFuzzer-TimeoutTest -timeout=1 -timeout_exitcode=0 RUN: %t-LLVMFuzzer-TimeoutTest -timeout=1 -timeout_exitcode=0
RUN: not LLVMFuzzer-TimeoutEmptyTest -timeout=1 2>&1 | FileCheck %s --check-prefix=TimeoutEmptyTest RUN: not %t-LLVMFuzzer-TimeoutEmptyTest -timeout=1 2>&1 | FileCheck %s --check-prefix=TimeoutEmptyTest
TimeoutEmptyTest: ALARM: working on the last Unit for TimeoutEmptyTest: ALARM: working on the last Unit for
TimeoutEmptyTest: == ERROR: libFuzzer: timeout after TimeoutEmptyTest: == ERROR: libFuzzer: timeout after
TimeoutEmptyTest: SUMMARY: libFuzzer: timeout TimeoutEmptyTest: SUMMARY: libFuzzer: timeout

llvm/trunk/lib/Fuzzer/test/fuzzer-traces-hooks.test

// FIXME: Support for sanitizer hooks for memcmp and strcmp needs to // FIXME: Support for sanitizer hooks for memcmp and strcmp needs to
// be implemented in the sanitizer runtime for this test // be implemented in the sanitizer runtime for this test
UNSUPPORTED: windows UNSUPPORTED: windows
RUN: %cpp_compiler %S/MemcmpTest.cpp -o %t-LLVMFuzzer-MemcmpTest
RUN: %cpp_compiler %S/StrncmpTest.cpp -o %t-LLVMFuzzer-StrncmpTest
RUN: %cpp_compiler %S/StrcmpTest.cpp -o %t-LLVMFuzzer-StrcmpTest
RUN: %cpp_compiler %S/StrstrTest.cpp -o %t-LLVMFuzzer-StrstrTest
RUN: %cpp_compiler %S/Memcmp64BytesTest.cpp -o %t-LLVMFuzzer-Memcmp64BytesTest
RUN: %cpp_compiler %S/RepeatedMemcmp.cpp -o %t-LLVMFuzzer-RepeatedMemcmp
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-MemcmpTest -seed=1 -runs=10000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-MemcmpTest -seed=1 -runs=10000000 2>&1 | FileCheck %s
RUN: not LLVMFuzzer-StrncmpTest -seed=1 -runs=2000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-StrncmpTest -seed=2 -runs=2000000 2>&1 | FileCheck %s
RUN: not LLVMFuzzer-StrcmpTest -seed=1 -runs=2000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-StrcmpTest -seed=1 -runs=2000000 2>&1 | FileCheck %s
RUN: not LLVMFuzzer-StrstrTest -seed=1 -runs=2000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-StrstrTest -seed=1 -runs=2000000 2>&1 | FileCheck %s
RUN: not LLVMFuzzer-Memcmp64BytesTest -seed=1 -runs=1000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-Memcmp64BytesTest -seed=1 -runs=1000000 2>&1 | FileCheck %s
RUN: LLVMFuzzer-RepeatedMemcmp -seed=11 -runs=100000 -max_len=20 2>&1 | FileCheck %s --check-prefix=RECOMMENDED_DICT RUN: %t-LLVMFuzzer-RepeatedMemcmp -seed=11 -runs=100000 -max_len=20 2>&1 | FileCheck %s --check-prefix=RECOMMENDED_DICT
RECOMMENDED_DICT:###### Recommended dictionary. ###### RECOMMENDED_DICT:###### Recommended dictionary. ######
RECOMMENDED_DICT-DAG: "foo" RECOMMENDED_DICT-DAG: "foo"
RECOMMENDED_DICT-DAG: "bar" RECOMMENDED_DICT-DAG: "bar"
RECOMMENDED_DICT:###### End of recommended dictionary. ###### RECOMMENDED_DICT:###### End of recommended dictionary. ######

llvm/trunk/lib/Fuzzer/test/fuzzer-ubsan.test

RUN: not LLVMFuzzer-SignedIntOverflowTest-Ubsan 2>&1 | FileCheck %s RUN: %cpp_compiler -fsanitize=undefined -fno-sanitize-recover=all %S/SignedIntOverflowTest.cpp -o %t-LLVMFuzzer-SignedIntOverflowTest-Ubsan
RUN: not %t-LLVMFuzzer-SignedIntOverflowTest-Ubsan 2>&1 | FileCheck %s
CHECK: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int' CHECK: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
CHECK: Test unit written to ./crash- CHECK: Test unit written to ./crash-

llvm/trunk/lib/Fuzzer/test/fuzzer.test

CHECK: BINGO CHECK: BINGO
Done1000000: Done 1000000 runs in Done1000000: Done 1000000 runs in
RUN: %cpp_compiler %S/BogusInitializeTest.cpp -o %t-LLVMFuzzer-BogusInitializeTest
RUN: %cpp_compiler %S/BufferOverflowOnInput.cpp -o %t-LLVMFuzzer-BufferOverflowOnInput
RUN: %cpp_compiler %S/CounterTest.cpp -o %t-LLVMFuzzer-CounterTest
RUN: %cpp_compiler %S/DSO1.cpp -fPIC -shared -o %t-LLVMFuzzer-DSO1.so
RUN: %cpp_compiler %S/DSO2.cpp -fPIC -shared -o %t-LLVMFuzzer-DSO2.so
RUN: %cpp_compiler %S/DSOTestMain.cpp %S/DSOTestExtra.cpp -L. %t-LLVMFuzzer-DSO1.so %t-LLVMFuzzer-DSO2.so -o %t-LLVMFuzzer-DSOTest
RUN: %cpp_compiler %S/FullCoverageSetTest.cpp -o %t-LLVMFuzzer-FullCoverageSetTest
RUN: %cpp_compiler %S/InitializeTest.cpp -o %t-LLVMFuzzer-InitializeTest
RUN: %cpp_compiler %S/NotinstrumentedTest.cpp -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard -o %t-LLVMFuzzer-NotinstrumentedTest-NoCoverage
RUN: %cpp_compiler %S/NullDerefOnEmptyTest.cpp -o %t-LLVMFuzzer-NullDerefOnEmptyTest
RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-NullDerefTest
RUN: %cpp_compiler %S/ShrinkControlFlowTest.cpp -o %t-LLVMFuzzer-ShrinkControlFlowTest
RUN: %cpp_compiler %S/SimpleCmpTest.cpp -o %t-LLVMFuzzer-SimpleCmpTest
RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
RUN: %cpp_compiler %S/StrncmpOOBTest.cpp -o %t-LLVMFuzzer-StrncmpOOBTest
RUN: not LLVMFuzzer-SimpleTest 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-SimpleTest 2>&1 | FileCheck %s
# only_ascii mode. Will perform some minimal self-validation. # only_ascii mode. Will perform some minimal self-validation.
RUN: not LLVMFuzzer-SimpleTest -only_ascii=1 2>&1 RUN: not %t-LLVMFuzzer-SimpleTest -only_ascii=1 2>&1
RUN: LLVMFuzzer-SimpleCmpTest -max_total_time=1 -use_cmp=0 2>&1 | FileCheck %s --check-prefix=MaxTotalTime RUN: %t-LLVMFuzzer-SimpleCmpTest -max_total_time=1 -use_cmp=0 2>&1 | FileCheck %s --check-prefix=MaxTotalTime
MaxTotalTime: Done {{.*}} runs in {{.}} second(s) MaxTotalTime: Done {{.*}} runs in {{.}} second(s)
RUN: not LLVMFuzzer-NullDerefTest 2>&1 | FileCheck %s --check-prefix=NullDerefTest RUN: not %t-LLVMFuzzer-NullDerefTest 2>&1 | FileCheck %s --check-prefix=NullDerefTest
RUN: not LLVMFuzzer-NullDerefTest -close_fd_mask=3 2>&1 | FileCheck %s --check-prefix=NullDerefTest RUN: not %t-LLVMFuzzer-NullDerefTest -close_fd_mask=3 2>&1 | FileCheck %s --check-prefix=NullDerefTest
NullDerefTest: ERROR: AddressSanitizer: {{SEGV|access-violation}} on unknown address NullDerefTest: ERROR: AddressSanitizer: {{SEGV|access-violation}} on unknown address
NullDerefTest: Test unit written to ./crash- NullDerefTest: Test unit written to ./crash-
RUN: not LLVMFuzzer-NullDerefTest -artifact_prefix=ZZZ 2>&1 | FileCheck %s --check-prefix=NullDerefTestPrefix RUN: not %t-LLVMFuzzer-NullDerefTest -artifact_prefix=ZZZ 2>&1 | FileCheck %s --check-prefix=NullDerefTestPrefix
NullDerefTestPrefix: Test unit written to ZZZcrash- NullDerefTestPrefix: Test unit written to ZZZcrash-
RUN: not LLVMFuzzer-NullDerefTest -artifact_prefix=ZZZ -exact_artifact_path=FOOBAR 2>&1 | FileCheck %s --check-prefix=NullDerefTestExactPath RUN: not %t-LLVMFuzzer-NullDerefTest -artifact_prefix=ZZZ -exact_artifact_path=FOOBAR 2>&1 | FileCheck %s --check-prefix=NullDerefTestExactPath
NullDerefTestExactPath: Test unit written to FOOBAR NullDerefTestExactPath: Test unit written to FOOBAR
RUN: not LLVMFuzzer-NullDerefOnEmptyTest -print_final_stats=1 2>&1 | FileCheck %s --check-prefix=NULL_DEREF_ON_EMPTY RUN: not %t-LLVMFuzzer-NullDerefOnEmptyTest -print_final_stats=1 2>&1 | FileCheck %s --check-prefix=NULL_DEREF_ON_EMPTY
NULL_DEREF_ON_EMPTY: stat::number_of_executed_units: NULL_DEREF_ON_EMPTY: stat::number_of_executed_units:
#not LLVMFuzzer-FullCoverageSetTest -timeout=15 -seed=1 -mutate_depth=2 -use_full_coverage_set=1 2>&1 | FileCheck %s #not %t-LLVMFuzzer-FullCoverageSetTest -timeout=15 -seed=1 -mutate_depth=2 -use_full_coverage_set=1 2>&1 | FileCheck %s
RUN: not LLVMFuzzer-CounterTest -max_len=6 -seed=1 -timeout=15 2>&1 | FileCheck %s --check-prefix=COUNTERS RUN: not %t-LLVMFuzzer-CounterTest -max_len=6 -seed=1 -timeout=15 2>&1 | FileCheck %s --check-prefix=COUNTERS
COUNTERS: INITED {{.*}} {{bits:|ft:}} COUNTERS: INITED {{.*}} {{bits:|ft:}}
COUNTERS: NEW {{.*}} {{bits:|ft:}} {{[1-9]*}} COUNTERS: NEW {{.*}} {{bits:|ft:}} {{[1-9]*}}
COUNTERS: NEW {{.*}} {{bits:|ft:}} {{[1-9]*}} COUNTERS: NEW {{.*}} {{bits:|ft:}} {{[1-9]*}}
COUNTERS: BINGO COUNTERS: BINGO
# Don't run UninstrumentedTest for now since we build libFuzzer itself with asan. # Don't run UninstrumentedTest for now since we build libFuzzer itself with asan.
DISABLED: not LLVMFuzzer-UninstrumentedTest-Uninstrumented 2>&1 | FileCheck %s --check-prefix=UNINSTRUMENTED DISABLED: not %t-LLVMFuzzer-UninstrumentedTest-Uninstrumented 2>&1 | FileCheck %s --check-prefix=UNINSTRUMENTED
UNINSTRUMENTED: ERROR: __sanitizer_set_death_callback is not defined. Exiting. UNINSTRUMENTED: ERROR: __sanitizer_set_death_callback is not defined. Exiting.
RUN: not LLVMFuzzer-NotinstrumentedTest-NoCoverage 2>&1 | FileCheck %s --check-prefix=NO_COVERAGE RUN: not %t-LLVMFuzzer-NotinstrumentedTest-NoCoverage 2>&1 | FileCheck %s --check-prefix=NO_COVERAGE
NO_COVERAGE: ERROR: no interesting inputs were found. Is the code instrumented for coverage? Exiting NO_COVERAGE: ERROR: no interesting inputs were found. Is the code instrumented for coverage? Exiting
RUN: not LLVMFuzzer-BufferOverflowOnInput 2>&1 | FileCheck %s --check-prefix=OOB RUN: not %t-LLVMFuzzer-BufferOverflowOnInput 2>&1 | FileCheck %s --check-prefix=OOB
OOB: AddressSanitizer: heap-buffer-overflow OOB: AddressSanitizer: heap-buffer-overflow
OOB: is located 0 bytes to the right of 3-byte region OOB: is located 0 bytes to the right of 3-byte region
RUN: not LLVMFuzzer-InitializeTest -use_value_profile=1 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-InitializeTest -use_value_profile=1 2>&1 | FileCheck %s
RUN: not LLVMFuzzer-DSOTest 2>&1 | FileCheck %s --check-prefix=DSO RUN: not %t-LLVMFuzzer-DSOTest 2>&1 | FileCheck %s --check-prefix=DSO
DSO: INFO: Loaded 3 modules DSO: INFO: Loaded 3 modules
DSO: BINGO DSO: BINGO
RUN: LLVMFuzzer-SimpleTest -exit_on_src_pos=SimpleTest.cpp:18 2>&1 | FileCheck %s --check-prefix=EXIT_ON_SRC_POS RUN: %t-LLVMFuzzer-SimpleTest -exit_on_src_pos=SimpleTest.cpp:18 2>&1 | FileCheck %s --check-prefix=EXIT_ON_SRC_POS
RUN: LLVMFuzzer-ShrinkControlFlowTest -exit_on_src_pos=ShrinkControlFlowTest.cpp:23 2>&1 | FileCheck %s --check-prefix=EXIT_ON_SRC_POS RUN: %t-LLVMFuzzer-ShrinkControlFlowTest -exit_on_src_pos=ShrinkControlFlowTest.cpp:23 2>&1 | FileCheck %s --check-prefix=EXIT_ON_SRC_POS
EXIT_ON_SRC_POS: INFO: found line matching '{{.*}}', exiting. EXIT_ON_SRC_POS: INFO: found line matching '{{.*}}', exiting.
RUN: env ASAN_OPTIONS=strict_string_checks=1 not LLVMFuzzer-StrncmpOOBTest -seed=1 -runs=1000000 2>&1 | FileCheck %s --check-prefix=STRNCMP RUN: env ASAN_OPTIONS=strict_string_checks=1 not %t-LLVMFuzzer-StrncmpOOBTest -seed=1 -runs=1000000 2>&1 | FileCheck %s --check-prefix=STRNCMP
STRNCMP: AddressSanitizer: heap-buffer-overflow STRNCMP: AddressSanitizer: heap-buffer-overflow
STRNCMP-NOT: __sanitizer_weak_hook_strncmp STRNCMP-NOT: __sanitizer_weak_hook_strncmp
STRNCMP: in LLVMFuzzerTestOneInput STRNCMP: in LLVMFuzzerTestOneInput
RUN: not LLVMFuzzer-BogusInitializeTest 2>&1 | FileCheck %s --check-prefix=BOGUS_INITIALIZE RUN: not %t-LLVMFuzzer-BogusInitializeTest 2>&1 | FileCheck %s --check-prefix=BOGUS_INITIALIZE
BOGUS_INITIALIZE: argv[0] has been modified in LLVMFuzzerInitialize BOGUS_INITIALIZE: argv[0] has been modified in LLVMFuzzerInitialize

llvm/trunk/lib/Fuzzer/test/inline-8bit-counters.test

REQUIRES: linux REQUIRES: linux
RUN: %cpp_compiler %S/SimpleTest.cpp -fno-sanitize-coverage=trace-pc-guard -fsanitize-coverage=inline-8bit-counters -o %t-LLVMFuzzer-SimpleTest-Inline8bitCounters
CHECK: INFO: Loaded 1 modules ({{.*}} inline 8-bit counters) CHECK: INFO: Loaded 1 modules ({{.*}} inline 8-bit counters)
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SimpleTest-Inline8bitCounters -runs=1000000 -seed=1 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-SimpleTest-Inline8bitCounters -runs=1000000 -seed=1 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/inline-8bit-counters/CMakeLists.txt

# These tests are instrumented with -fsanitize-coverage=inline-8bit-counters
set(CMAKE_CXX_FLAGS
"${LIBFUZZER_FLAGS_BASE} -fno-sanitize-coverage=trace-pc-guard -fsanitize-coverage=inline-8bit-counters,pc-table")
set(Inline8bitCounterTests
SimpleTest
)
foreach(Test ${Inline8bitCounterTests})
add_libfuzzer_test(${Test}-Inline8bitCounters SOURCES ../${Test}.cpp)
endforeach()

llvm/trunk/lib/Fuzzer/test/lit.cfg

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
if sys.platform.startswith('linux'): if sys.platform.startswith('linux'):
# Note the value of ``sys.platform`` is not consistent # Note the value of ``sys.platform`` is not consistent
# between python 2 and 3, hence the use of ``.startswith()``. # between python 2 and 3, hence the use of ``.startswith()``.
lit_config.note('linux feature available') lit_config.note('linux feature available')
config.available_features.add('linux') config.available_features.add('linux')
else: else:
lit_config.note('linux feature unavailable') lit_config.note('linux feature unavailable')
config.substitutions.append(('%build_dir', config.cmake_binary_dir))
def generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True):
compiler_cmd = config.cpp_compiler if is_cpp else config.c_compiler
std_cmd = '-std=c++11' if is_cpp else ''
sanitizers = ['address']
if fuzzer_enabled:
sanitizers.append('fuzzer')
sanitizers_cmd = ('-fsanitize=%s' % ','.join(sanitizers))
isysroot_cmd = ('-isysroot %s' % config.osx_sysroot
) if 'darwin' in config.target_triple else ''
include_cmd = '-I%s/../.' % config.test_source_root
return '%s %s -gline-tables-only %s %s %s' % (
compiler_cmd, std_cmd, isysroot_cmd, sanitizers_cmd, include_cmd)
config.substitutions.append(('%cpp_compiler',
generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True)
))
config.substitutions.append(('%c_compiler',
generate_compiler_cmd(is_cpp=False, fuzzer_enabled=True)
))
config.substitutions.append(('%no_fuzzer_cpp_compiler',
generate_compiler_cmd(is_cpp=True, fuzzer_enabled=False)
))
config.substitutions.append(('%no_fuzzer_c_compiler',
generate_compiler_cmd(is_cpp=False, fuzzer_enabled=False)
))

llvm/trunk/lib/Fuzzer/test/lit.site.cfg.in

config.test_exec_root = "@CMAKE_CURRENT_BINARY_DIR@" config.test_exec_root = "@CMAKE_CURRENT_BINARY_DIR@"
config.llvm_tools_dir = "@LLVM_TOOLS_DIR@" config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
config.has_lsan = True if @HAS_LSAN@ == 1 else False config.has_lsan = True if @HAS_LSAN@ == 1 else False
config.is_posix = @LIBFUZZER_POSIX@ config.is_posix = @LIBFUZZER_POSIX@
config.cpp_compiler = "@LIBFUZZER_TEST_CXX_COMPILER@"
config.c_compiler = "@LIBFUZZER_TEST_COMPILER@"
config.osx_sysroot = "@CMAKE_OSX_SYSROOT@"
config.cmake_binary_dir = "@CMAKE_BINARY_DIR@"
config.target_triple = "@TARGET_TRIPLE@"
lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg") lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg")

llvm/trunk/lib/Fuzzer/test/merge-posix.test

REQUIRES: posix REQUIRES: posix
RUN: %cpp_compiler %S/FullCoverageSetTest.cpp -o %t-LLVMFuzzer-FullCoverageSetTest
RUN: rm -rf %tmp/T1 %tmp/T2 RUN: rm -rf %tmp/T1 %tmp/T2
RUN: mkdir -p %tmp/T1 %tmp/T2 RUN: mkdir -p %tmp/T1 %tmp/T2
RUN: echo F..... > %tmp/T1/1 RUN: echo F..... > %tmp/T1/1
RUN: echo .U.... > %tmp/T1/2 RUN: echo .U.... > %tmp/T1/2
RUN: echo ..Z... > %tmp/T1/3 RUN: echo ..Z... > %tmp/T1/3
RUN: echo .....F > %tmp/T2/1 RUN: echo .....F > %tmp/T2/1
RUN: echo ....U. > %tmp/T2/2 RUN: echo ....U. > %tmp/T2/2
RUN: echo ...Z.. > %tmp/T2/3 RUN: echo ...Z.. > %tmp/T2/3
RUN: echo ...Z.. > %tmp/T2/4 RUN: echo ...Z.. > %tmp/T2/4
RUN: echo ....E. > %tmp/T2/5 RUN: echo ....E. > %tmp/T2/5
RUN: echo .....R > %tmp/T2/6 RUN: echo .....R > %tmp/T2/6
# Check that we can report an error if file size exceeded # Check that we can report an error if file size exceeded
RUN: (ulimit -f 1; not LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=SIGXFSZ) RUN: (ulimit -f 1; not %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=SIGXFSZ)
SIGXFSZ: ERROR: libFuzzer: file size exceeded SIGXFSZ: ERROR: libFuzzer: file size exceeded
# Check that we honor TMPDIR # Check that we honor TMPDIR
RUN: TMPDIR=DIR_DOES_NOT_EXIST not LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=TMPDIR RUN: TMPDIR=DIR_DOES_NOT_EXIST not %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=TMPDIR
TMPDIR: MERGE-OUTER: failed to write to the control file: DIR_DOES_NOT_EXIST/libFuzzerTemp TMPDIR: MERGE-OUTER: failed to write to the control file: DIR_DOES_NOT_EXIST/libFuzzerTemp

llvm/trunk/lib/Fuzzer/test/merge-summary.test

RUN: %cpp_compiler %S/FullCoverageSetTest.cpp -o %t-LLVMFuzzer-FullCoverageSetTest
RUN: rm -rf %t/T1 %t/T2 RUN: rm -rf %t/T1 %t/T2
RUN: mkdir -p %t/T0 %t/T1 %t/T2 RUN: mkdir -p %t/T0 %t/T1 %t/T2
RUN: echo ...Z.. > %t/T2/1 RUN: echo ...Z.. > %t/T2/1
RUN: echo ....E. > %t/T2/2 RUN: echo ....E. > %t/T2/2
RUN: echo .....R > %t/T2/3 RUN: echo .....R > %t/T2/3
RUN: echo F..... > %t/T2/a RUN: echo F..... > %t/T2/a
RUN: echo .U.... > %t/T2/b RUN: echo .U.... > %t/T2/b
RUN: echo ..Z... > %t/T2/c RUN: echo ..Z... > %t/T2/c
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %t/T1 %t/T2 -save_coverage_summary=%t/SUMMARY 2>&1 | FileCheck %s --check-prefix=SAVE_SUMMARY RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %t/T1 %t/T2 -save_coverage_summary=%t/SUMMARY 2>&1 | FileCheck %s --check-prefix=SAVE_SUMMARY
SAVE_SUMMARY: MERGE-OUTER: writing coverage summary for 6 files to {{.*}}SUMMARY SAVE_SUMMARY: MERGE-OUTER: writing coverage summary for 6 files to {{.*}}SUMMARY
RUN: rm %t/T1/* RUN: rm %t/T1/*
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %t/T1 %t/T2 -load_coverage_summary=%t/SUMMARY 2>&1 | FileCheck %s --check-prefix=LOAD_SUMMARY RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %t/T1 %t/T2 -load_coverage_summary=%t/SUMMARY 2>&1 | FileCheck %s --check-prefix=LOAD_SUMMARY
LOAD_SUMMARY: MERGE-OUTER: coverage summary loaded from {{.*}}SUMMAR LOAD_SUMMARY: MERGE-OUTER: coverage summary loaded from {{.*}}SUMMAR
LOAD_SUMMARY: MERGE-OUTER: 0 new files with 0 new features added LOAD_SUMMARY: MERGE-OUTER: 0 new files with 0 new features added

llvm/trunk/lib/Fuzzer/test/merge.test

CHECK: BINGO CHECK: BINGO
RUN: %cpp_compiler %S/FullCoverageSetTest.cpp -o %t-LLVMFuzzer-FullCoverageSetTest
RUN: rm -rf %tmp/T0 %tmp/T1 %tmp/T2 RUN: rm -rf %tmp/T0 %tmp/T1 %tmp/T2
RUN: mkdir -p %tmp/T0 %tmp/T1 %tmp/T2 RUN: mkdir -p %tmp/T0 %tmp/T1 %tmp/T2
RUN: echo F..... > %tmp/T0/1 RUN: echo F..... > %tmp/T0/1
RUN: echo .U.... > %tmp/T0/2 RUN: echo .U.... > %tmp/T0/2
RUN: echo ..Z... > %tmp/T0/3 RUN: echo ..Z... > %tmp/T0/3
# T1 has 3 elements, T2 is empty. # T1 has 3 elements, T2 is empty.
RUN: cp %tmp/T0/* %tmp/T1/ RUN: cp %tmp/T0/* %tmp/T1/
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=CHECK1 RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=CHECK1
CHECK1: MERGE-OUTER: 3 files, 3 in the initial corpus CHECK1: MERGE-OUTER: 3 files, 3 in the initial corpus
CHECK1: MERGE-OUTER: 0 new files with 0 new features added CHECK1: MERGE-OUTER: 0 new files with 0 new features added
RUN: echo ...Z.. > %tmp/T2/1 RUN: echo ...Z.. > %tmp/T2/1
RUN: echo ....E. > %tmp/T2/2 RUN: echo ....E. > %tmp/T2/2
RUN: echo .....R > %tmp/T2/3 RUN: echo .....R > %tmp/T2/3
RUN: echo F..... > %tmp/T2/a RUN: echo F..... > %tmp/T2/a
RUN: echo .U.... > %tmp/T2/b RUN: echo .U.... > %tmp/T2/b
RUN: echo ..Z... > %tmp/T2/c RUN: echo ..Z... > %tmp/T2/c
# T1 has 3 elements, T2 has 6 elements, only 3 are new. # T1 has 3 elements, T2 has 6 elements, only 3 are new.
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=CHECK2 RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=CHECK2
CHECK2: MERGE-OUTER: 9 files, 3 in the initial corpus CHECK2: MERGE-OUTER: 9 files, 3 in the initial corpus
CHECK2: MERGE-OUTER: 3 new files with 3 new features added CHECK2: MERGE-OUTER: 3 new files with 3 new features added
# Now, T1 has 6 units and T2 has no new interesting units. # Now, T1 has 6 units and T2 has no new interesting units.
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=CHECK3 RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=CHECK3
CHECK3: MERGE-OUTER: 12 files, 6 in the initial corpus CHECK3: MERGE-OUTER: 12 files, 6 in the initial corpus
CHECK3: MERGE-OUTER: 0 new files with 0 new features added CHECK3: MERGE-OUTER: 0 new files with 0 new features added
# Check that we respect max_len during the merge and don't crash. # Check that we respect max_len during the merge and don't crash.
RUN: rm %tmp/T1/* RUN: rm %tmp/T1/*
RUN: cp %tmp/T0/* %tmp/T1/ RUN: cp %tmp/T0/* %tmp/T1/
RUN: echo looooooooong > %tmp/T2/looooooooong RUN: echo looooooooong > %tmp/T2/looooooooong
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 -max_len=6 2>&1 | FileCheck %s --check-prefix=MAX_LEN RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 -max_len=6 2>&1 | FileCheck %s --check-prefix=MAX_LEN
MAX_LEN: MERGE-OUTER: 3 new files MAX_LEN: MERGE-OUTER: 3 new files
# Check that merge tolerates failures. # Check that merge tolerates failures.
RUN: rm %tmp/T1/* RUN: rm %tmp/T1/*
RUN: cp %tmp/T0/* %tmp/T1/ RUN: cp %tmp/T0/* %tmp/T1/
RUN: echo 'FUZZER' > %tmp/T2/FUZZER RUN: echo 'FUZZER' > %tmp/T2/FUZZER
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=MERGE_WITH_CRASH RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=MERGE_WITH_CRASH
MERGE_WITH_CRASH: MERGE-OUTER: succesfull in 2 attempt(s) MERGE_WITH_CRASH: MERGE-OUTER: succesfull in 2 attempt(s)
MERGE_WITH_CRASH: MERGE-OUTER: 3 new files MERGE_WITH_CRASH: MERGE-OUTER: 3 new files
# Check that we actually limit the size with max_len # Check that we actually limit the size with max_len
RUN: LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 -max_len=5 2>&1 | FileCheck %s --check-prefix=MERGE_LEN5 RUN: %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 -max_len=5 2>&1 | FileCheck %s --check-prefix=MERGE_LEN5
MERGE_LEN5: MERGE-OUTER: succesfull in 1 attempt(s) MERGE_LEN5: MERGE-OUTER: succesfull in 1 attempt(s)
RUN: rm -rf %tmp/T1/* %tmp/T2/* RUN: rm -rf %tmp/T1/* %tmp/T2/*
RUN: not LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=EMPTY RUN: not %t-LLVMFuzzer-FullCoverageSetTest -merge=1 %tmp/T1 %tmp/T2 2>&1 | FileCheck %s --check-prefix=EMPTY
EMPTY: MERGE-OUTER: zero succesfull attempts, exiting EMPTY: MERGE-OUTER: zero succesfull attempts, exiting

llvm/trunk/lib/Fuzzer/test/minimize_crash.test

RUN: %cpp_compiler %S/NullDerefTest.cpp -o %t-LLVMFuzzer-NullDerefTest
RUN: %cpp_compiler %S/SingleByteInputTest.cpp -o %t-LLVMFuzzer-SingleByteInputTest
RUN: echo 'Hi!rv349f34t3gg' > not_minimal_crash RUN: echo 'Hi!rv349f34t3gg' > not_minimal_crash
RUN: LLVMFuzzer-NullDerefTest -minimize_crash=1 not_minimal_crash -max_total_time=2 2>&1 | FileCheck %s RUN: %t-LLVMFuzzer-NullDerefTest -minimize_crash=1 not_minimal_crash -max_total_time=2 2>&1 | FileCheck %s
CHECK: CRASH_MIN: failed to minimize beyond ./minimized-from-{{.*}} (3 bytes), exiting CHECK: CRASH_MIN: failed to minimize beyond ./minimized-from-{{.*}} (3 bytes), exiting
RUN: LLVMFuzzer-NullDerefTest -minimize_crash=1 not_minimal_crash -max_total_time=2 -exact_artifact_path=exact_minimized_path 2>&1 | FileCheck %s --check-prefix=CHECK_EXACT RUN: %t-LLVMFuzzer-NullDerefTest -minimize_crash=1 not_minimal_crash -max_total_time=2 -exact_artifact_path=exact_minimized_path 2>&1 | FileCheck %s --check-prefix=CHECK_EXACT
CHECK_EXACT: CRASH_MIN: failed to minimize beyond exact_minimized_path (3 bytes), exiting CHECK_EXACT: CRASH_MIN: failed to minimize beyond exact_minimized_path (3 bytes), exiting
RUN: rm not_minimal_crash minimized-from-* exact_minimized_path RUN: rm not_minimal_crash minimized-from-* exact_minimized_path
RUN: echo -n 'abcd*xyz' > not_minimal_crash RUN: echo -n 'abcd*xyz' > not_minimal_crash
RUN: LLVMFuzzer-SingleByteInputTest -minimize_crash=1 not_minimal_crash -exact_artifact_path=exact_minimized_path 2>&1 | FileCheck %s --check-prefix=MIN1 RUN: %t-LLVMFuzzer-SingleByteInputTest -minimize_crash=1 not_minimal_crash -exact_artifact_path=exact_minimized_path 2>&1 | FileCheck %s --check-prefix=MIN1
MIN1: Test unit written to exact_minimized_path MIN1: Test unit written to exact_minimized_path
MIN1: Test unit written to exact_minimized_path MIN1: Test unit written to exact_minimized_path
MIN1: INFO: The input is small enough, exiting MIN1: INFO: The input is small enough, exiting
MIN1: CRASH_MIN: failed to minimize beyond exact_minimized_path (1 bytes), exiting MIN1: CRASH_MIN: failed to minimize beyond exact_minimized_path (1 bytes), exiting

llvm/trunk/lib/Fuzzer/test/minimize_two_crashes.test

# Test that the minimizer stops when it sees a differe bug. # Test that the minimizer stops when it sees a differe bug.
RUN: %cpp_compiler %S/TwoDifferentBugsTest.cpp -o %t-LLVMFuzzer-TwoDifferentBugsTest
RUN: rm -rf %t && mkdir %t RUN: rm -rf %t && mkdir %t
RUN: echo H12345678901234667888090 > %t/long_crash RUN: echo H12345678901234667888090 > %t/long_crash
RUN: env ASAN_OPTIONS=dedup_token_length=3 LLVMFuzzer-TwoDifferentBugsTest -seed=1 -minimize_crash=1 %t/long_crash -exact_artifact_path=%t/result 2>&1 | FileCheck %s RUN: env ASAN_OPTIONS=dedup_token_length=3 %t-LLVMFuzzer-TwoDifferentBugsTest -seed=1 -minimize_crash=1 %t/long_crash -exact_artifact_path=%t/result 2>&1 | FileCheck %s
CHECK: DedupToken1: DEDUP_TOKEN: Bar CHECK: DedupToken1: DEDUP_TOKEN: Bar
CHECK: DedupToken2: DEDUP_TOKEN: Bar CHECK: DedupToken2: DEDUP_TOKEN: Bar
CHECK: DedupToken1: DEDUP_TOKEN: Bar CHECK: DedupToken1: DEDUP_TOKEN: Bar
CHECK: DedupToken2: DEDUP_TOKEN: Foo CHECK: DedupToken2: DEDUP_TOKEN: Foo
CHECK: CRASH_MIN: mismatch in dedup tokens CHECK: CRASH_MIN: mismatch in dedup tokens
RUN: not LLVMFuzzer-TwoDifferentBugsTest %t/result 2>&1 | FileCheck %s --check-prefix=VERIFY RUN: not %t-LLVMFuzzer-TwoDifferentBugsTest %t/result 2>&1 | FileCheck %s --check-prefix=VERIFY
VERIFY: ERROR: AddressSanitizer: VERIFY: ERROR: AddressSanitizer:
VERIFY: in Bar VERIFY: in Bar

llvm/trunk/lib/Fuzzer/test/no-coverage/CMakeLists.txt

# These tests are not instrumented with coverage,
# but have coverage rt in the binary.
set(CMAKE_CXX_FLAGS
"${LIBFUZZER_FLAGS_BASE} -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard")
set(NoCoverageTests
NotinstrumentedTest
)
foreach(Test ${NoCoverageTests})
add_libfuzzer_test(${Test}-NoCoverage SOURCES ../${Test}.cpp)
endforeach()
###############################################################################
# AFL Driver test
###############################################################################
if(NOT MSVC)
add_executable(AFLDriverTest
../AFLDriverTest.cpp ../../afl/afl_driver.cpp)
set_target_properties(AFLDriverTest
PROPERTIES RUNTIME_OUTPUT_DIRECTORY
"${CMAKE_BINARY_DIR}/lib/Fuzzer/test"
)
add_dependencies(TestBinaries AFLDriverTest)
endif()

llvm/trunk/lib/Fuzzer/test/overwrite-input.test

RUN: not LLVMFuzzer-OverwriteInputTest 2>&1 | FileCheck %s RUN: %cpp_compiler %S/OverwriteInputTest.cpp -o %t-LLVMFuzzer-OverwriteInputTest
RUN: not %t-LLVMFuzzer-OverwriteInputTest 2>&1 | FileCheck %s
CHECK: ERROR: libFuzzer: fuzz target overwrites it's const input CHECK: ERROR: libFuzzer: fuzz target overwrites it's const input

llvm/trunk/lib/Fuzzer/test/reduce_inputs.test

# Test -reduce_inputs=1 # Test -reduce_inputs=1
RUN: rm -rf %t/C RUN: rm -rf %t/C
RUN: mkdir -p %t/C RUN: mkdir -p %t/C
RUN: LLVMFuzzer-ShrinkControlFlowSimpleTest -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -runs=1000000 %t/C 2>&1 | FileCheck %s RUN: %cpp_compiler %S/ShrinkControlFlowSimpleTest.cpp -o %t-LLVMFuzzer-ShrinkControlFlowSimpleTest
RUN: %cpp_compiler %S/ShrinkControlFlowTest.cpp -o %t-LLVMFuzzer-ShrinkControlFlowTest
RUN: %t-LLVMFuzzer-ShrinkControlFlowSimpleTest -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -runs=1000000 %t/C 2>&1 | FileCheck %s
CHECK: INFO: found item with checksum '0eb8e4ed029b774d80f2b66408203801cb982a60' CHECK: INFO: found item with checksum '0eb8e4ed029b774d80f2b66408203801cb982a60'
# Test that reduce_inputs deletes redundant files in the corpus. # Test that reduce_inputs deletes redundant files in the corpus.
RUN: LLVMFuzzer-ShrinkControlFlowSimpleTest -runs=0 %t/C 2>&1 | FileCheck %s --check-prefix=COUNT RUN: %t-LLVMFuzzer-ShrinkControlFlowSimpleTest -runs=0 %t/C 2>&1 | FileCheck %s --check-prefix=COUNT
COUNT: READ units: 4 COUNT: READ units: 4
# a bit longer test # a bit longer test
RUN: LLVMFuzzer-ShrinkControlFlowTest -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -seed=1 -runs=1000000 2>&1 | FileCheck %s RUN: %t-LLVMFuzzer-ShrinkControlFlowTest -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -seed=1 -runs=1000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/repeated-bytes.test

RUN: %cpp_compiler %S/RepeatedBytesTest.cpp -o %t-LLVMFuzzer-RepeatedBytesTest
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-RepeatedBytesTest -seed=1 -runs=1000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-RepeatedBytesTest -seed=1 -runs=1000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/shrink.test

RUN: LLVMFuzzer-ShrinkControlFlowTest -seed=1 -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -runs=1000000 -shrink=1 -reduce_inputs=0 2>&1 | FileCheck %s --check-prefix=SHRINK1 RUN: %cpp_compiler %S/ShrinkControlFlowTest.cpp -o %t-LLVMFuzzer-ShrinkControlFlowTest
RUN: %cpp_compiler %S/ShrinkValueProfileTest.cpp -o %t-LLVMFuzzer-ShrinkValueProfileTest
RUN: %t-LLVMFuzzer-ShrinkControlFlowTest -seed=1 -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -runs=1000000 -shrink=1 -reduce_inputs=0 2>&1 | FileCheck %s --check-prefix=SHRINK1
# Limit max_len to run this negative test faster. # Limit max_len to run this negative test faster.
RUN: LLVMFuzzer-ShrinkControlFlowTest -seed=1 -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -runs=1000000 -shrink=0 -reduce_inputs=0 -max_len=64 2>&1 | FileCheck %s --check-prefix=SHRINK0 RUN: %t-LLVMFuzzer-ShrinkControlFlowTest -seed=1 -exit_on_item=0eb8e4ed029b774d80f2b66408203801cb982a60 -runs=1000000 -shrink=0 -reduce_inputs=0 -max_len=64 2>&1 | FileCheck %s --check-prefix=SHRINK0
RUN: LLVMFuzzer-ShrinkValueProfileTest -seed=1 -exit_on_item=aea2e3923af219a8956f626558ef32f30a914ebc -runs=100000 -shrink=1 -reduce_inputs=0 -use_value_profile=1 2>&1 | FileCheck %s --check-prefix=SHRINK1_VP RUN: %t-LLVMFuzzer-ShrinkValueProfileTest -seed=1 -exit_on_item=aea2e3923af219a8956f626558ef32f30a914ebc -runs=100000 -shrink=1 -reduce_inputs=0 -use_value_profile=1 2>&1 | FileCheck %s --check-prefix=SHRINK1_VP
SHRINK0: Done 1000000 runs in SHRINK0: Done 1000000 runs in
SHRINK1: INFO: found item with checksum '0eb8e4ed029b774d80f2b66408203801cb982a60', exiting. SHRINK1: INFO: found item with checksum '0eb8e4ed029b774d80f2b66408203801cb982a60', exiting.
SHRINK1_VP: INFO: found item with checksum 'aea2e3923af219a8956f626558ef32f30a914ebc', exiting SHRINK1_VP: INFO: found item with checksum 'aea2e3923af219a8956f626558ef32f30a914ebc', exiting

llvm/trunk/lib/Fuzzer/test/simple-cmp.test

RUN: %cpp_compiler %S/SimpleCmpTest.cpp -o %t-LLVMFuzzer-SimpleCmpTest
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SimpleCmpTest -seed=1 -runs=100000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-SimpleCmpTest -seed=1 -runs=100000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/standalone.test

RUN: LLVMFuzzer-StandaloneInitializeTest %S/hi.txt %S/dict1.txt 2>&1 | FileCheck %s RUN: %no_fuzzer_c_compiler %S/../standalone/StandaloneFuzzTargetMain.c -c -o %t_1.o
RUN: %no_fuzzer_cpp_compiler %S/InitializeTest.cpp -c -o %t_2.o
RUN: %no_fuzzer_cpp_compiler %t_1.o %t_2.o %build_dir/lib/libLLVMFuzzerNoMain.a -o %t-LLVMFuzzer-StandaloneInitializeTest
RUN: %t-LLVMFuzzer-StandaloneInitializeTest %S/hi.txt %S/dict1.txt 2>&1 | FileCheck %s
CHECK: StandaloneFuzzTargetMain: running 2 inputs CHECK: StandaloneFuzzTargetMain: running 2 inputs
CHECK: Done: {{.*}}hi.txt: (3 bytes) CHECK: Done: {{.*}}hi.txt: (3 bytes)
CHECK: Done: {{.*}}dict1.txt: (61 bytes) CHECK: Done: {{.*}}dict1.txt: (61 bytes)

llvm/trunk/lib/Fuzzer/test/swap-cmp.test

RUN: %cpp_compiler %S/SwapCmpTest.cpp -o %t-LLVMFuzzer-SwapCmpTest
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SwapCmpTest -seed=1 -runs=10000000 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-SwapCmpTest -seed=1 -runs=10000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/trace-malloc-2.test

// FIXME: This test infinite loops on darwin because it crashes // FIXME: This test infinite loops on darwin because it crashes
// printing a stack trace repeatedly // printing a stack trace repeatedly
UNSUPPORTED: darwin UNSUPPORTED: darwin
RUN: LLVMFuzzer-TraceMallocTest -seed=1 -trace_malloc=2 -runs=1000 2>&1 | FileCheck %s --check-prefix=TRACE2 RUN: %cpp_compiler %S/TraceMallocTest.cpp -o %t-LLVMFuzzer-TraceMallocTest
RUN: %t-LLVMFuzzer-TraceMallocTest -seed=1 -trace_malloc=2 -runs=1000 2>&1 | FileCheck %s --check-prefix=TRACE2
TRACE2-DAG: FREE[0] TRACE2-DAG: FREE[0]
TRACE2-DAG: MALLOC[0] TRACE2-DAG: MALLOC[0]
TRACE2-DAG: in LLVMFuzzerTestOneInput TRACE2-DAG: in LLVMFuzzerTestOneInput

llvm/trunk/lib/Fuzzer/test/trace-malloc.test

RUN: LLVMFuzzer-TraceMallocTest -seed=1 -trace_malloc=1 -runs=10000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/TraceMallocTest.cpp -o %t-LLVMFuzzer-TraceMallocTest
RUN: %t-LLVMFuzzer-TraceMallocTest -seed=1 -trace_malloc=1 -runs=10000 2>&1 | FileCheck %s
CHECK-DAG: MallocFreeTracer: STOP 0 0 (same) CHECK-DAG: MallocFreeTracer: STOP 0 0 (same)
CHECK-DAG: MallocFreeTracer: STOP 0 1 (DIFFERENT) CHECK-DAG: MallocFreeTracer: STOP 0 1 (DIFFERENT)
CHECK-DAG: MallocFreeTracer: STOP 1 0 (DIFFERENT) CHECK-DAG: MallocFreeTracer: STOP 1 0 (DIFFERENT)
CHECK-DAG: MallocFreeTracer: STOP 1 1 (same) CHECK-DAG: MallocFreeTracer: STOP 1 1 (same)

llvm/trunk/lib/Fuzzer/test/trace-pc.test

RUN: %cpp_compiler %S/SimpleTest.cpp -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard -fsanitize-coverage=trace-pc -o %t-LLVMFuzzer-SimpleTest-TracePC
CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SimpleTest-TracePC -runs=100000 -seed=1 2>&1 | FileCheck %s RUN: not %t-LLVMFuzzer-SimpleTest-TracePC -runs=100000 -seed=1 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/trace-pc/CMakeLists.txt

# These tests are instrumented with -fsanitize-coverage=trace-pc
set(CMAKE_CXX_FLAGS
"${LIBFUZZER_FLAGS_BASE} -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard -fsanitize-coverage=trace-pc")
set(TracePCTests
SimpleTest
)
foreach(Test ${TracePCTests})
add_libfuzzer_test(${Test}-TracePC SOURCES ../${Test}.cpp)
endforeach()

llvm/trunk/lib/Fuzzer/test/ubsan/CMakeLists.txt

# These tests are instrumented with ubsan in non-recovery mode.
set(CMAKE_CXX_FLAGS
"${LIBFUZZER_FLAGS_BASE} -fsanitize=undefined -fno-sanitize-recover=all")
set(UbsanTests
SignedIntOverflowTest
)
foreach(Test ${UbsanTests})
add_libfuzzer_test(${Test}-Ubsan SOURCES ../${Test}.cpp)
endforeach()

llvm/trunk/lib/Fuzzer/test/ulimit.test

REQUIRES: posix REQUIRES: posix
RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-LLVMFuzzer-SimpleTest
RUN: ulimit -s 1000 RUN: ulimit -s 1000
RUN: not LLVMFuzzer-SimpleTest RUN: not %t-LLVMFuzzer-SimpleTest

llvm/trunk/lib/Fuzzer/test/uninstrumented/CMakeLists.txt

# These tests are not instrumented with coverage and don't
# have coverage rt in the binary.
set(CMAKE_CXX_FLAGS
"${LIBFUZZER_FLAGS_BASE} -fno-sanitize=all -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard")
set(UninstrumentedTests
UninstrumentedTest
)
foreach(Test ${UninstrumentedTests})
add_libfuzzer_test(${Test}-Uninstrumented SOURCES ../${Test}.cpp)
endforeach()

llvm/trunk/lib/Fuzzer/test/value-profile-cmp.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SimpleCmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SimpleCmpTest.cpp -o %t-LLVMFuzzer-SimpleCmpTest
RUN: not %t-LLVMFuzzer-SimpleCmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-cmp2.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SimpleHashTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 -max_len=64 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SimpleHashTest.cpp -o %t-LLVMFuzzer-SimpleHashTest
RUN: not %t-LLVMFuzzer-SimpleHashTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 -max_len=64 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-cmp3.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-AbsNegAndConstantTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/AbsNegAndConstantTest.cpp -o %t-LLVMFuzzer-AbsNegAndConstantTest
RUN: not %t-LLVMFuzzer-AbsNegAndConstantTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-cmp4.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-AbsNegAndConstant64Test -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/AbsNegAndConstant64Test.cpp -o %t-LLVMFuzzer-AbsNegAndConstant64Test
RUN: not %t-LLVMFuzzer-AbsNegAndConstant64Test -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-div.test

CHECK: AddressSanitizer: {{FPE|int-divide-by-zero}} CHECK: AddressSanitizer: {{FPE|int-divide-by-zero}}
RUN: not LLVMFuzzer-DivTest -seed=1 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/DivTest.cpp -fsanitize-coverage=trace-div -o %t-LLVMFuzzer-DivTest
RUN: not %t-LLVMFuzzer-DivTest -seed=1 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-load.test

CHECK: AddressSanitizer: global-buffer-overflow CHECK: AddressSanitizer: global-buffer-overflow
RUN: not LLVMFuzzer-LoadTest -seed=2 -use_cmp=0 -use_value_profile=1 -runs=20000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/LoadTest.cpp -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-gep,trace-div,trace-cmp -o %t-LLVMFuzzer-LoadTest
RUN: not %t-LLVMFuzzer-LoadTest -seed=2 -use_cmp=0 -use_value_profile=1 -runs=20000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-mem.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SingleMemcmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SingleMemcmpTest.cpp -o %t-LLVMFuzzer-SingleMemcmpTest
RUN: not %t-LLVMFuzzer-SingleMemcmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-set.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-FourIndependentBranchesTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/FourIndependentBranchesTest.cpp -o %t-LLVMFuzzer-FourIndependentBranchesTest
RUN: not %t-LLVMFuzzer-FourIndependentBranchesTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-strcmp.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SingleStrcmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SingleStrcmpTest.cpp -o %t-LLVMFuzzer-SingleStrcmpTest
RUN: not %t-LLVMFuzzer-SingleStrcmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-strncmp.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SingleStrncmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SingleStrncmpTest.cpp -o %t-LLVMFuzzer-SingleStrncmpTest
RUN: not %t-LLVMFuzzer-SingleStrncmpTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=100000000 2>&1 | FileCheck %s

llvm/trunk/lib/Fuzzer/test/value-profile-switch.test

CHECK: BINGO CHECK: BINGO
RUN: not LLVMFuzzer-SwitchTest -use_cmp=0 -use_value_profile=1 -runs=100000000 -seed=1 2>&1 | FileCheck %s RUN: %cpp_compiler %S/SwitchTest.cpp -o %t-LLVMFuzzer-SwitchTest
RUN: not LLVMFuzzer-Switch2Test -use_cmp=0 -use_value_profile=1 -runs=100000000 -seed=1 2>&1 | FileCheck %s RUN: %cpp_compiler %S/Switch2Test.cpp -o %t-LLVMFuzzer-Switch2Test
RUN: not %t-LLVMFuzzer-SwitchTest -use_cmp=0 -use_value_profile=1 -runs=100000000 -seed=1 2>&1 | FileCheck %s
RUN: not %t-LLVMFuzzer-Switch2Test -use_cmp=0 -use_value_profile=1 -runs=100000000 -seed=1 2>&1 | FileCheck %s
llvm/trunk/lib/Fuzzer/test/trace-pc/CMakeLists.txt