Moving libFuzzer to compiler-rt

[george.karpenkov]

Summary

Any suggestions on how to make the patch more readable are welcome :)
Since for git-svn and all the git clones those are different repositories, I'm not sure whether it's possible to present the moves as version control moves.

This change gets libFuzzer into Clang's toolchain installation as a byproduct.

This patch is tightly coupled to https://reviews.llvm.org/D36909 in Clang.

[krytarowski]

cmake/config-ix.cmake

583

AND OS_NAME MATCHES "Darwin|Linux" ?

[george.karpenkov]

cmake/config-ix.cmake

583

Good point, the build fails on some other systems!

[george.karpenkov]

On Darwin build libFuzzer only for osx for now

[krytarowski]

cmake/config-ix.cmake

583

Thanks, I've ported it locally to NetBSD... however tests are broken for me.. I will hold on with merging this code.

[mgorny]

A side notice but wouldn't it be better to make libFuzzer a completely separate component (like openmp)? I've seen people take interest in it outside LLVM toolchain.

[krytarowski]

In D36908#846351, @mgorny wrote:

A side notice but wouldn't it be better to make libFuzzer a completely separate component (like openmp)? I've seen people take interest in it outside LLVM toolchain.

libfuzzer coexists with sanitizers, it's a natural place for this feature in compiler-rt.

[mgorny]

Ok, the first thing that I've noticed is that the compiler-rt version does not build with gcc:

c++: error: unrecognized command line option ‘-fno-sanitize-coverage=trace-pc-guard,edge,trace-cmp,indirect-calls,8bit-counters’

No clue how useful LLVMFuzzer built with gcc were but it certainly did build successfully.

[krytarowski]

I will move it here (from the mailing list):

This file test/fuzzer/lit.site.cfg.in contains redundant entries with the rest of infrastructure, notably config.cpp_compiler and config.c_compiler. This should be streamlined with the rest of the code

config.cpp_compiler = "@LIBFUZZER_TEST_COMPILER@"
config.target_flags = "@LIBFUZZER_TEST_FLAGS@"
config.c_compiler = "@LIBFUZZER_TEST_COMPILER@"

There are defines like COMPILER_RT_TEST_CXX_COMPILER and COMPILER_RT_TEST_COMPILER ready to use.

[krytarowski]

Please amend this patch with support for NetBSD:

http://www.netbsd.org/~kamil/llvm/libfuzzer-netbsd.txt

Not everything is ready, like missing lsan... but it's good enough for start and improve in future.

[kcc]

A side notice but wouldn't it be better to make libFuzzer a completely separate component (like openmp)? I've seen people take interest in it outside LLVM toolchain.

libFuzzer is already separate to a large extent.
You can build it outside of the LLVM tree (see build.sh) and use it w/o having LLVM (but you will need a compiler or some other tool to use libFuzzer-compatible coverage feedback).
But the libFuzzer development/testing is tightly integrated into the LLVM tool chain, and we don't expect this to change)

[kcc]

In D36908#846686, @krytarowski wrote:

Please amend this patch with support for NetBSD:

http://www.netbsd.org/~kamil/llvm/libfuzzer-netbsd.txt

Not everything is ready, like missing lsan... but it's good enough for start and improve in future.

Let's not overcomplicate the move to compiler-rt, it's pretty involved any way.
We would welcome support for any new platform in a separate patch once the dust settles (a week or two).
For any new supported platform will require a public build bot and a contact person.
Lacking that we can't promise to keep the port alive. (For further discussion, please start a new thread).

[kcc]

This is the compiler-rt part, right?
Where is the llvm part that removes the code (did I miss it)?

For the LLVM part, please

• delete all tests and cmake files
• keep the libFuzzer sources and build.sh
• add a README.txt explaining the temporary state (I'll remove the remnants once our internal users migrate, but will leave the README.txt for a while)

I am ok if you simply 'svn add' new files
Make sure the commit message contains instructions on how to obtain the svn history for the files prior to the move.

[krytarowski]

In D36908#846722, @kcc wrote:

In D36908#846686, @krytarowski wrote:

Please amend this patch with support for NetBSD:

http://www.netbsd.org/~kamil/llvm/libfuzzer-netbsd.txt

Not everything is ready, like missing lsan... but it's good enough for start and improve in future.

Let's not overcomplicate the move to compiler-rt, it's pretty involved any way.
We would welcome support for any new platform in a separate patch once the dust settles (a week or two).
For any new supported platform will require a public build bot and a contact person.
Lacking that we can't promise to keep the port alive. (For further discussion, please start a new thread).

There is a bot and I'm the contact person:
http://lab.llvm.org:8011/builders/lldb-amd64-ninja-netbsd8

Attaching compiler-rt builds is now premature (sanitizers aren't upstreamed). My goal is to enable tests in LLVM, Clang and LLDB first.

To make it happen earlier (to get a compiler-rt buildbot) please help to review the sanitizers, for start: D35554 and D36587.

There is another thread mentioned by @mgorny that compiler-rt is broken for GCC builds (reported on bugzilla). I've filed a simple patch in D33878.

I think amending the lubfuzzer patch isn't burden here. That way I can also review and test it for the generic benefit (I don't use !NetBSD systems).

Thanks for collaboration.

[kcc]

Any such change during a large refactoring is a burden, sorry.

[krytarowski]

In D36908#846732, @kcc wrote:

Any such change during a large refactoring is a burden, sorry.

The author asked me to elaborate on the patch and support issue (on a mail), this encouraged me to submit it.

[kcc]

Rule of thumb: never mix refactoring with functional changes (or porting to new platforms, etc) in a single change.

[krytarowski]

In D36908#846734, @kcc wrote:

Rule of thumb: never mix refactoring with functional changes (or porting to new platforms, etc) in a single change.

OK, thanks - I will wait for this code to be fully merged. I will provide a bot setup for compiler-rt/NetBSD to make the maintenance easier... just upstreaming and fixes take time - please be patient.

[george.karpenkov]

Ok, the first thing that I've noticed is that the compiler-rt version does not build with gcc:

@mgorny this is actually due to change proposed in https://reviews.llvm.org/D36887.
I am not sure what is the correct strategy here: previously, the flag was behind the guard, which was never set for gcc.
I don't think it is correct to rely on LLVM_* variable from a compiler-rt repository.
I can also test whether the flag is available first before compiling.

[kcc]

Once we move to compiler-rt, the test will be build with the fresh clang.
But what about libFuzzer itself? Will it build with fresh clang or with the host compiler?
If 'fresh clang', then this is not a big deal for the LLVM build tree.
(We do need to keep libFuzzer buildable with fresh GCC though, but this problem is not about GCC in general, just about using GCC in the LLVM tree build).

[george.karpenkov]

@kcc compiler-rt folder builds both from projects and from runtimes.
In former case, it is build using the host compiler, and in latter, using freshly built Clang.
The latter case is arguably more "correct", yet that's not how all systems are configured.

[mgorny]

In D36908#847787, @george.karpenkov wrote:

Ok, the first thing that I've noticed is that the compiler-rt version does not build with gcc:

@mgorny this is actually due to change proposed in https://reviews.llvm.org/D36887.
I am not sure what is the correct strategy here: previously, the flag was behind the guard, which was never set for gcc.
I don't think it is correct to rely on LLVM_* variable from a compiler-rt repository.
I can also test whether the flag is available first before compiling.

Ah, I see. Thanks for the explanation. However, I don't think the limitation on LLVM_* variables really applies here. After all, compiler-rt is built using LLVM CMake files that use such variables on their own. In fact, compiler-rt already uses LLVM_USE_SANITIZER in config-ix.cmake, so I see no reason why it wouldn't use the latter.

Checking flag is also fine here. However, I'd personally prefer if you tried to move stuff as 1:1 as possible in this patch, and only did the changes strictly necessary to keep things working. This change is not strictly necessary, so it'd be better to split it into a followup patch.

[vitalybuka]

Could you please spit this patch in two:

1. Move code from llvm to compile-rt as-is. No changes in cmake files and no hooking this into compiler-rt cmake files
2. cmake and other changes necessary to build

[george.karpenkov]

@vitalybuka: OK. Note that nothing would work in the just-moving patch though (ideally I would be able to simply upload my branch)

[george.karpenkov]

Update the DIFF not to include moving the actual files.

[george.karpenkov]

@vitalybuka unfortunately, the compilation system required moving tests around (e.g. splitting into lib/fuzzer/test and test/fuzzer).

This diff requires https://reviews.llvm.org/D36974 for testing application.

@mgorny should now compile under GCC. The change in question is already moved out into https://reviews.llvm.org/D36887: the diff attached to this PR is what needs to happen after the move itself is performed.

[kcc]

LGTM
Please update the commit message.

[george.karpenkov]

@krytarowski

There are defines like COMPILER_RT_TEST_CXX_COMPILER and COMPILER_RT_TEST_COMPILER ready to use

Yes. In fact, they are already being used: LIBFUZZER_TEST_COMPILER is populated by get_test_cc_for_arch, which uses COMPILER_RT_TEST_COMPILER.

[george.karpenkov]

@kcc @vitalybuka just found a bug: standalone.test no longer works, because the path to libLLVMFuzzerNoMain.a becomes OS- and ARCH-dependent.
I can see two possible solutions:
either

a) Add e.g. -fsanitize=fuzzer-no-main to the driver (links in libclang_rt.fuzzer_no_main*)
or
b) Rewrite standalone.test as a unittest, compiled using CMake.

Doing (a) is easier, but there's a question of how much should we pollute the driver.
Should this be exposed to users? Is it useful to be able to link in a library without overwriting main?

[kcc]

In D36908#848112, @george.karpenkov wrote:

@kcc @vitalybuka just found a bug: standalone.test no longer works, because the path to libLLVMFuzzerNoMain.a becomes OS- and ARCH-dependent.

This was introduced by https://reviews.llvm.org/D36295#db384381
In this test we should not link libFuzzer, so just remove %build_dir/lib/libLLVMFuzzerNoMain.a

[george.karpenkov]

Test fixes.

[kcc]

LGTM

CMakeLists.txt

 include(CompilerRTUtils)
 
 option(COMPILER_RT_BUILD_BUILTINS "Build builtins" ON)
 mark_as_advanced(COMPILER_RT_BUILD_BUILTINS)
 option(COMPILER_RT_BUILD_SANITIZERS "Build sanitizers" ON)
 mark_as_advanced(COMPILER_RT_BUILD_SANITIZERS)
 option(COMPILER_RT_BUILD_XRAY "Build xray" ON)
 mark_as_advanced(COMPILER_RT_BUILD_XRAY)
+option(COMPILER_RT_BUILD_LIBFUZZER "Build libFuzzer" ON)
+mark_as_advanced(COMPILER_RT_BUILD_LIBFUZZER)
 option(COMPILER_RT_BUILD_XRAY_NO_PREINIT "Build xray with no preinit patching" OFF)
 mark_as_advanced(COMPILER_RT_BUILD_XRAY_NO_PREINIT)
 
 set(COMPILER_RT_BAREMETAL_BUILD OFF CACHE BOOLEAN
   "Build for a bare-metal target.")
 
 if (COMPILER_RT_STANDALONE_BUILD)
   load_llvm_config()

cmake/Modules/AddCompilerRT.cmake

foreach(libname ${libnames})
     set_target_compile_flags(${libname} ${extra_cflags_${libname}})
     set_target_link_flags(${libname} ${extra_link_flags_${libname}})
     set_property(TARGET ${libname} APPEND PROPERTY
                 COMPILE_DEFINITIONS ${LIB_DEFS})
     set_target_output_directories(${libname} ${COMPILER_RT_LIBRARY_OUTPUT_DIR})
     set_target_properties(${libname} PROPERTIES
         OUTPUT_NAME ${output_name_${libname}})
     set_target_properties(${libname} PROPERTIES FOLDER "Compiler-RT Runtime")
-    if(${type} STREQUAL "SHARED")
-      if(LIB_LINK_LIBS)
-        target_link_libraries(${libname} ${LIB_LINK_LIBS})
-      endif()
+    if(${type} STREQUAL "SHARED")
       if(WIN32 AND NOT CYGWIN AND NOT MINGW)
         set_target_properties(${libname} PROPERTIES IMPORT_PREFIX "")
         set_target_properties(${libname} PROPERTIES IMPORT_SUFFIX ".lib")
       endif()
       if(APPLE)
         # Ad-hoc sign the dylibs
         add_custom_command(TARGET ${libname}
           POST_BUILD  
function(add_compiler_rt_test test_suite test_name arch)
   # link programs at all are included, but the changes needed for the test
   # trump. With MSVC we can't do that because CMake is set up to run link.exe
   # when linking, not the compiler. Here, we hack it to use the compiler
   # because we want to use -fsanitize flags.
   if(NOT MSVC)
     set(TEST_LINK_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${TEST_LINK_FLAGS}")
     separate_arguments(TEST_LINK_FLAGS)
   endif()
+
   add_custom_target(${test_name}
     COMMAND ${COMPILER_RT_TEST_COMPILER} ${TEST_OBJECTS}
             -o "${output_bin}"
             ${TEST_LINK_FLAGS}
     DEPENDS ${TEST_DEPS})
   set_target_properties(${test_name} PROPERTIES FOLDER "Compiler-RT Tests")
 
   # Make the test suite depend on the binary.

cmake/base-config-ix.cmake

 # The CompilerRT build system requires CMake version 2.8.8 or higher in order
 # to use its support for building convenience "libraries" as a collection of
 # .o files. This is particularly useful in producing larger, more complex
 # runtime libraries.
 
 include(CheckIncludeFile)
+include(CheckCXXSourceCompiles)
+
 check_include_file(unwind.h HAVE_UNWIND_H)
 
 # Top level target used to build all compiler-rt libraries.
 add_custom_target(compiler-rt ALL)
 add_custom_target(install-compiler-rt)
 set_target_properties(compiler-rt PROPERTIES FOLDER "Compiler-RT Misc")
 
 # Setting these variables from an LLVM build is sufficient that compiler-rt can
execute_process(COMMAND xcodebuild -version -sdk macosx Path
        ERROR_QUIET
        OUTPUT_STRIP_TRAILING_WHITESPACE)
     set(OSX_SYSROOT_FLAG "-isysroot${OSX_SYSROOT}")
   endif()
 
   option(COMPILER_RT_ENABLE_IOS "Enable building for iOS" On)
   option(COMPILER_RT_ENABLE_WATCHOS "Enable building for watchOS - Experimental" Off)
   option(COMPILER_RT_ENABLE_TVOS "Enable building for tvOS - Experimental" Off)
+
+  CHECK_CXX_SOURCE_COMPILES("
+      static thread_local int blah;
+      int main() {
+        return 0;
+      }
+      " HAS_THREAD_LOCAL)
+
+  if( NOT HAS_THREAD_LOCAL )
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Dthread_local=__thread")
+  endif()
 else()
   option(COMPILER_RT_DEFAULT_TARGET_ONLY "Build builtins only for the default target" Off)
 endif()
 
 if(WIN32 AND NOT MINGW AND NOT CYGWIN)
   set(CMAKE_SHARED_LIBRARY_PREFIX_C "")
   set(CMAKE_SHARED_LIBRARY_PREFIX_CXX "")
   set(CMAKE_STATIC_LIBRARY_PREFIX_C "")

cmake/config-ix.cmake

if(APPLE)
   set(X86_64 x86_64 x86_64h)
 endif()
 
 set(ALL_SANITIZER_COMMON_SUPPORTED_ARCH ${X86} ${X86_64} ${PPC64}
     ${ARM32} ${ARM64} ${MIPS32} ${MIPS64} ${S390X})
 set(ALL_ASAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64}
     ${MIPS32} ${MIPS64} ${PPC64} ${S390X})
 set(ALL_DFSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64})
+set(ALL_FUZZER_SUPPORTED_ARCH x86_64)
 
 if(APPLE)
   set(ALL_LSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${MIPS64} ${ARM64})
 else()
   set(ALL_LSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${MIPS64} ${ARM64} ${ARM32})
 endif()
 set(ALL_MSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${PPC64})
 set(ALL_PROFILE_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${PPC64}
if(APPLE)
   list_intersect(ESAN_SUPPORTED_ARCH
     ALL_ESAN_SUPPORTED_ARCH
     SANITIZER_COMMON_SUPPORTED_ARCH)
   list_intersect(SCUDO_SUPPORTED_ARCH
     ALL_SCUDO_SUPPORTED_ARCH
     SANITIZER_COMMON_SUPPORTED_ARCH)
   list_intersect(XRAY_SUPPORTED_ARCH
     ALL_XRAY_SUPPORTED_ARCH
-    SANITIZER_COMMON_SUPPORTED_ARCH)
+    SANITIZER_COMMON_SUPPORTED_ARCH)
+  list_intersect(FUZZER_SUPPORTED_ARCH
+    ALL_FUZZER_SUPPORTED_ARCH
+    ALL_SANITIZER_COMMON_SUPPORTED_ARCH)
+
 else()
   # Architectures supported by compiler-rt libraries.
   filter_available_targets(SANITIZER_COMMON_SUPPORTED_ARCH
     ${ALL_SANITIZER_COMMON_SUPPORTED_ARCH})
   # LSan and UBSan common files should be available on all architectures
   # supported by other sanitizers (even if they build into dummy object files).
   filter_available_targets(LSAN_COMMON_SUPPORTED_ARCH
     ${SANITIZER_COMMON_SUPPORTED_ARCH})
   filter_available_targets(UBSAN_COMMON_SUPPORTED_ARCH
     ${SANITIZER_COMMON_SUPPORTED_ARCH})
   filter_available_targets(ASAN_SUPPORTED_ARCH ${ALL_ASAN_SUPPORTED_ARCH})
+  filter_available_targets(FUZZER_SUPPORTED_ARCH ${ALL_FUZZER_SUPPORTED_ARCH})
   filter_available_targets(DFSAN_SUPPORTED_ARCH ${ALL_DFSAN_SUPPORTED_ARCH})
   filter_available_targets(LSAN_SUPPORTED_ARCH ${ALL_LSAN_SUPPORTED_ARCH})
   filter_available_targets(MSAN_SUPPORTED_ARCH ${ALL_MSAN_SUPPORTED_ARCH})
   filter_available_targets(PROFILE_SUPPORTED_ARCH ${ALL_PROFILE_SUPPORTED_ARCH})
   filter_available_targets(TSAN_SUPPORTED_ARCH ${ALL_TSAN_SUPPORTED_ARCH})
   filter_available_targets(UBSAN_SUPPORTED_ARCH ${ALL_UBSAN_SUPPORTED_ARCH})
   filter_available_targets(SAFESTACK_SUPPORTED_ARCH
     ${ALL_SAFESTACK_SUPPORTED_ARCH})

 endif()
 
 if (COMPILER_RT_HAS_SANITIZER_COMMON AND XRAY_SUPPORTED_ARCH AND
     OS_NAME MATCHES "Linux")
   set(COMPILER_RT_HAS_XRAY TRUE)
 else()
   set(COMPILER_RT_HAS_XRAY FALSE)
 endif()
+
+if (COMPILER_RT_HAS_SANITIZER_COMMON AND FUZZER_SUPPORTED_ARCH AND
+      OS_NAME MATCHES "Darwin|Linux")
+  set(COMPILER_RT_HAS_FUZZER TRUE)
+else()
+  set(COMPILER_RT_HAS_FUZZER FALSE)
+endif()

lib/CMakeLists.txt

if(COMPILER_RT_BUILD_SANITIZERS)
   endforeach()
 
   compiler_rt_build_runtime(profile)
 endif()
 
 if(COMPILER_RT_BUILD_XRAY)
   compiler_rt_build_runtime(xray)
 endif()
+
+if(COMPILER_RT_BUILD_LIBFUZZER)
+  compiler_rt_build_runtime(fuzzer)
+endif()

lib/fuzzer/CMakeLists.txt

-include(CheckCXXSourceCompiles)
-
-if( APPLE )
-  CHECK_CXX_SOURCE_COMPILES("
-      static thread_local int blah;
-      int main() {
-        return 0;
-      }
-      " HAS_THREAD_LOCAL)
-
-  if( NOT HAS_THREAD_LOCAL )
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Dthread_local=__thread")
-  endif()
-endif()
-
-if (CMAKE_SYSTEM_NAME MATCHES "Darwin|Linux")
-  set(LIBFUZZER_ENABLED_CHECK ON)
-else()
-  set(LIBFUZZER_ENABLED_CHECK OFF)
-endif()
-
-# Compile libFuzzer if the compilation is specifically requested, OR
-# if the platform is known to be working.
-set(LIBFUZZER_ENABLE ${LIBFUZZER_ENABLED_CHECK} CACHE BOOL "Build libFuzzer and its tests")
-set(LIBFUZZER_ENABLE_TESTS OFF CACHE BOOL "Build libFuzzer and its tests")
-
-if (LLVM_USE_SANITIZE_COVERAGE)
-  set(CMAKE_CXX_FLAGS
-    "${CMAKE_CXX_FLAGS} -fno-sanitize-coverage=trace-pc-guard,edge,trace-cmp,indirect-calls,8bit-counters -Werror")
-endif()
-
-if (LIBFUZZER_ENABLE)
-  add_library(LLVMFuzzerNoMainObjects OBJECT
-      FuzzerCrossOver.cpp
-      FuzzerDriver.cpp
-      FuzzerExtFunctionsDlsym.cpp
-      FuzzerExtFunctionsDlsymWin.cpp
-      FuzzerExtFunctionsWeak.cpp
-      FuzzerExtraCounters.cpp
-      FuzzerIO.cpp
-      FuzzerIOPosix.cpp
-      FuzzerIOWindows.cpp
-      FuzzerLoop.cpp
-      FuzzerMerge.cpp
-      FuzzerMutate.cpp
-      FuzzerSHA1.cpp
-      FuzzerShmemPosix.cpp
-      FuzzerShmemWindows.cpp
-      FuzzerTracePC.cpp
-      FuzzerUtil.cpp
-      FuzzerUtilDarwin.cpp
-      FuzzerUtilLinux.cpp
-      FuzzerUtilPosix.cpp
-      FuzzerUtilWindows.cpp
-      )
-  add_library(LLVMFuzzerNoMain STATIC
-      $<TARGET_OBJECTS:LLVMFuzzerNoMainObjects>
-      )
-  target_link_libraries(LLVMFuzzerNoMain ${LLVM_PTHREAD_LIB})
-  add_library(LLVMFuzzer STATIC
-      FuzzerMain.cpp
-      $<TARGET_OBJECTS:LLVMFuzzerNoMainObjects>
-      )
-  target_link_libraries(LLVMFuzzer ${LLVM_PTHREAD_LIB})
-endif()
+set(LIBFUZZER_SOURCES
 
-if (MSVC)
+if (CMAKE_CXX_FLAGS MATCHES "fsanitize-coverage")
+  set(LIBFUZZER_CFLAGS -fno-sanitize-coverage=trace-pc-guard,edge,trace-cmp,indirect-calls,8bit-counters)
+endif()
 
-  # Until bots are reconfigured, check-fuzzer on Windows is a no-OP.
-  add_custom_target(check-fuzzer)
-  add_custom_command(TARGET check-fuzzer
-    COMMAND cmake -E echo "check-fuzzer is disalbed on Windows")
-else()
-  if (LLVM_INCLUDE_TESTS AND LIBFUZZER_ENABLE_TESTS)
-    add_subdirectory(test)
-  endif()
+if(APPLE)
+  set(FUZZER_SUPPORTED_OS osx)
+
+add_compiler_rt_object_libraries(RTfuzzer
+  OS ${FUZZER_SUPPORTED_OS}
+  ARCHS ${FUZZER_SUPPORTED_ARCH}
+  SOURCES ${LIBFUZZER_SOURCES}
+  CFLAGS ${LIBFUZZER_CFLAGS})
+
+add_compiler_rt_object_libraries(RTfuzzer_main
+  OS ${FUZZER_SUPPORTED_OS}
+  ARCHS ${FUZZER_SUPPORTED_ARCH}
+  SOURCES FuzzerMain.cpp
+  CFLAGS ${LIBFUZZER_CFLAGS})
+
+add_compiler_rt_runtime(clang_rt.fuzzer
+  STATIC
+  OS ${FUZZER_SUPPORTED_OS}
+  ARCHS ${FUZZER_SUPPORTED_ARCH}
+  OBJECT_LIBS RTfuzzer RTfuzzer_main
+  CFLAGS ${LIBFUZZER_CFLAGS}
+  PARENT_TARGET fuzzer)
+
+add_compiler_rt_runtime(clang_rt.fuzzer_no_main
+  STATIC
+  OS ${FUZZER_SUPPORTED_OS}
+  ARCHS ${FUZZER_SUPPORTED_ARCH}
+  OBJECT_LIBS RTfuzzer
+  CFLAGS ${LIBFUZZER_CFLAGS}
+  PARENT_TARGET fuzzer)
+
+if(COMPILER_RT_INCLUDE_TESTS)
+  add_subdirectory(tests)
 endif()

lib/fuzzer/test/CMakeLists.txt

-
-if(APPLE)
-  # LeakSanitizer is not supported on OSX and Windows right now
-  set(HAS_LSAN 0)
-  message(WARNING "LeakSanitizer is not supported."
-    " Building and running LibFuzzer LeakSanitizer tests is disabled."
-    )
-else()
-  set(HAS_LSAN 1)
-endif()
-
-###############################################################################
-# Unit tests
-###############################################################################
-
-add_custom_target(FuzzerUnitTests)
-set_target_properties(FuzzerUnitTests PROPERTIES FOLDER "libFuzzer tests")
-
-add_executable(LLVMFuzzer-Unittest FuzzerUnittest.cpp)
-
-target_link_libraries(LLVMFuzzer-Unittest
-  gtest
-  gtest_main
-  LLVMFuzzerNoMain
-  )
-
-target_include_directories(LLVMFuzzer-Unittest PRIVATE
-  "${LLVM_MAIN_SRC_DIR}/utils/unittest/googletest/include"
-  )
-
-set_target_properties(LLVMFuzzer-Unittest
-  PROPERTIES RUNTIME_OUTPUT_DIRECTORY
-  "${CMAKE_CURRENT_BINARY_DIR}"
-)
-
-include_directories(..)
-
-###############################################################################
-# Configure lit to run the tests
-#
-# Note this is done after declaring all tests so we can inform lit if any tests
-# need to be disabled.
-###############################################################################
-
-# Use just-built Clang to compile/link tests on all platforms, except for
-# Windows where we need to use clang-cl instead.
-set(LIBFUZZER_TEST_COMPILER ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-set(LIBFUZZER_TEST_CXX_COMPILER ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang++)
-
-# LIT-based libFuzzer tests.
-configure_lit_site_cfg(
-  ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
-  ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg
-  )
-
-# libFuzzer unit tests.
-configure_lit_site_cfg(
-  ${CMAKE_CURRENT_SOURCE_DIR}/unit/lit.site.cfg.in
-  ${CMAKE_CURRENT_BINARY_DIR}/unit/lit.site.cfg
-  )
-
-add_lit_testsuite(check-fuzzer "Running Fuzzer tests"
-  ${CMAKE_CURRENT_BINARY_DIR}
-  DEPENDS LLVMFuzzer-Unittest)
-
-add_dependencies(check-fuzzer LLVMFuzzer asan clang llvm-symbolizer FileCheck sancov not)

lib/fuzzer/test/unit/lit.cfg

-import lit.formats
-
-config.name = "LLVMFuzzer-Unittest"
-print config.test_exec_root
-config.test_format = lit.formats.GoogleTest(".", "Unittest")
-config.suffixes = []
-config.test_source_root = config.test_exec_root

lib/fuzzer/test/unit/lit.site.cfg.in

-config.test_exec_root = "@CMAKE_CURRENT_BINARY_DIR@"
-lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/unit/lit.cfg")

lib/fuzzer/tests/CMakeLists.txt

+set(LIBFUZZER_UNITTEST_CFLAGS
+  ${COMPILER_RT_UNITTEST_CFLAGS}
+  ${COMPILER_RT_GTEST_CFLAGS}
+  -I${COMPILER_RT_SOURCE_DIR}/lib/fuzzer
+  -fno-rtti
+  -Werror
+  -O2)
+
+add_custom_target(FuzzerUnitTests)
+set_target_properties(FuzzerUnitTests PROPERTIES FOLDER "Compiler-RT Tests")
+
+set(LIBFUZZER_UNITTEST_LINK_FLAGS ${COMPILER_RT_UNITTEST_LINK_FLAGS})
+list(APPEND LIBFUZZER_UNITTEST_LINK_FLAGS --driver-mode=g++)
+
+if(APPLE)
+  list(APPEND LIBFUZZER_UNITTEST_LINK_FLAGS -lc++)
+else()
+  list(APPEND LIBFUZZER_UNITTEST_LINK_FLAGS -lstdc++)
+endif()
+
+foreach(arch ${FUZZER_SUPPORTED_ARCH})
+  set(LIBFUZZER_TEST_RUNTIME RTFuzzerTest.${arch})
+  if(APPLE)
+    set(LIBFUZZER_TEST_RUNTIME_OBJECTS
+      $<TARGET_OBJECTS:RTfuzzer.osx>)
+  else()
+    set(LIBFUZZER_TEST_RUNTIME_OBJECTS
+      $<TARGET_OBJECTS:RTfuzzer.${arch}>)
+  endif()
+  add_library(${LIBFUZZER_TEST_RUNTIME} STATIC
+    ${LIBFUZZER_TEST_RUNTIME_OBJECTS})
+  set_target_properties(${LIBFUZZER_TEST_RUNTIME} PROPERTIES
+    ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+    FOLDER "Compiler-RT Runtime tests")
+
+  set(FuzzerTestObjects)
+  generate_compiler_rt_tests(FuzzerTestObjects
+    FuzzerUnitTests "Fuzzer-${arch}-Test" ${arch}
+    SOURCES FuzzerUnittest.cpp ${COMPILER_RT_GTEST_SOURCE}
+    RUNTIME ${LIBFUZZER_TEST_RUNTIME}
+    DEPS gtest
+    CFLAGS ${LIBFUZZER_UNITTEST_CFLAGS}
+    LINK_FLAGS ${LIBFUZZER_UNITTEST_LINK_FLAGS})
+  set_target_properties(FuzzerUnitTests PROPERTIES
+    RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
+endforeach()

lib/fuzzer/tests/FuzzerUnittest.cpp

TEST(Fuzzer, ForEachNonZeroByte) {
   EXPECT_EQ(Res, Expected);
 
   Res.clear();
   ForEachNonZeroByte(Ar + 9, Ar + N - 9, 109, CB);
   Expected = {          {109, 2}, {118, 3}, {120, 4},
               {135, 5}, {137, 6}, {146, 7}};
   EXPECT_EQ(Res, Expected);
 }
+
+int main(int argc, char **argv) {
+  testing::InitGoogleTest(&argc, argv);
+  return RUN_ALL_TESTS();
+}

test/CMakeLists.txt

if(COMPILER_RT_CAN_EXECUTE_TESTS)
   endif()
   if(COMPILER_RT_BUILD_SANITIZERS)
     compiler_rt_test_runtime(interception)
 
     compiler_rt_test_runtime(lsan)
     # CFI tests require diagnostic mode, which is implemented in UBSan.
     compiler_rt_test_runtime(ubsan cfi)
     compiler_rt_test_runtime(sanitizer_common)
+    compiler_rt_test_runtime(fuzzer)
 
     foreach(sanitizer ${COMPILER_RT_SANITIZERS_TO_BUILD})
       # cfi testing is gated on ubsan
       if(NOT ${sanitizer} STREQUAL cfi)
         compiler_rt_test_runtime(${sanitizer})
       endif()
     endforeach()
 

test/fuzzer/CMakeLists.txt

+
+set(LIBFUZZER_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS})
+if (NOT COMPILER_RT_STANDALONE_BUILD)
+  list(APPEND LIBFUZZER_TEST_DEPS fuzzer)
+endif()
+
+if(COMPILER_RT_INCLUDE_TESTS)
+  list(APPEND LIBFUZZER_TEST_DEPS FuzzerUnitTests)
+endif()
+
+set(LIBFUZZER_TESTSUITES)
+
+
+if(COMPILER_RT_INCLUDE_TESTS)
+  # libFuzzer unit tests.
+  configure_lit_site_cfg(
+    ${CMAKE_CURRENT_SOURCE_DIR}/unit/lit.site.cfg.in
+    ${CMAKE_CURRENT_BINARY_DIR}/unit/lit.site.cfg)
+  list(APPEND LIBFUZZER_TESTSUITES ${CMAKE_CURRENT_BINARY_DIR}/unit)
+endif()
+
+foreach(arch ${FUZZER_SUPPORTED_ARCH})
+  get_test_cc_for_arch(${arch} LIBFUZZER_TEST_COMPILER LIBFUZZER_TEST_FLAGS)
+
+  string(TOUPPER ${arch} ARCH_UPPER_CASE)
+  set(CONFIG_NAME ${ARCH_UPPER_CASE}${OS_NAME}Config)
+
+   #LIT-based libFuzzer tests.
+  configure_lit_site_cfg(
+    ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
+    ${CMAKE_CURRENT_BINARY_DIR}/${CONFIG_NAME}/lit.site.cfg
+    )
+  list(APPEND LIBFUZZER_TESTSUITES ${CMAKE_CURRENT_BINARY_DIR}/${CONFIG_NAME})
+
+endforeach()
+
+add_lit_testsuite(check-fuzzer "Running Fuzzer tests"
+  ${LIBFUZZER_TESTSUITES}
+  DEPENDS ${LIBFUZZER_TEST_DEPS})
+set_target_properties(check-fuzzer PROPERTIES FOLDER "Compiler-RT Tests")

test/fuzzer/afl-driver-extra-stats.test

-RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %S/../afl/afl_driver.cpp -o %t-AFLDriverTest
+RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %libfuzzer_src/afl/afl_driver.cpp -o %t-AFLDriverTest
 
 ; Test that not specifying an extra stats file isn't broken.
 RUN: unset AFL_DRIVER_EXTRA_STATS_FILENAME
 RUN: %t-AFLDriverTest
 
 ; Test that specifying an invalid extra stats file causes a crash.
 RUN: ASAN_OPTIONS= AFL_DRIVER_EXTRA_STATS_FILENAME=%T not --crash %t-AFLDriverTest
 

test/fuzzer/afl-driver-stderr.test

-RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %S/../afl/afl_driver.cpp -o %t-AFLDriverTest
+RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %libfuzzer_src/afl/afl_driver.cpp -o %t-AFLDriverTest
 
 ; Test that not specifying a stderr file isn't broken.
 RUN: unset AFL_DRIVER_STDERR_DUPLICATE_FILENAME
 RUN: %t-AFLDriverTest
 
 ; Test that specifying an invalid file causes a crash.
 RUN: ASAN_OPTIONS= AFL_DRIVER_STDERR_DUPLICATE_FILENAME="%T" not --crash %t-AFLDriverTest
 
 ; Test that a file is created when specified as the duplicate stderr.
 RUN: AFL_DRIVER_STDERR_DUPLICATE_FILENAME=%t %t-AFLDriverTest
 RUN: stat %t

test/fuzzer/afl-driver.test

 REQUIRES: linux
 
-RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %S/../afl/afl_driver.cpp -o %t-AFLDriverTest
+RUN: %no_fuzzer_cpp_compiler -fno-sanitize-coverage=edge,trace-cmp,indirect-calls,8bit-counters,trace-pc-guard %S/AFLDriverTest.cpp %libfuzzer_src/afl/afl_driver.cpp -o %t-AFLDriverTest
 
 RUN: echo -n "abc" > %t.file3
 RUN: echo -n "abcd" > %t.file4
 
 RUN: %t-AFLDriverTest < %t.file3 2>&1 | FileCheck %s --check-prefix=CHECK1
 CHECK1: __afl_persistent_loop calle, Count = 1000
 CHECK1: LLVMFuzzerTestOneInput called; Size = 3
 

test/fuzzer/lit.cfg

 import lit.formats
 import sys
+import os
 
 config.name = "LLVMFuzzer"
 config.test_format = lit.formats.ShTest(True)
 config.suffixes = ['.test']
 config.test_source_root = os.path.dirname(__file__)
 
 # Choose between lit's internal shell pipeline runner and a real shell.  If
 # LIT_USE_INTERNAL_SHELL is in the environment, we use that as an override.
 use_lit_shell = os.environ.get("LIT_USE_INTERNAL_SHELL")
 if use_lit_shell:
     # 0 is external, "" is default, and everything else is internal.
     execute_external = (use_lit_shell == "0")
 else:
     # Otherwise we default to internal on Windows and external elsewhere, as
     # bash on Windows is usually very slow.
     execute_external = (not sys.platform in ['win32'])
 
 # testFormat: The test format to use to interpret tests.
 #
 # For now we require '&&' between commands, until they get globally killed and
 # the test runner updated.
 config.test_format = lit.formats.ShTest(execute_external)
 
-# Tweak PATH to include llvm tools dir and current exec dir.
-llvm_tools_dir = getattr(config, 'llvm_tools_dir', None)
-if (not llvm_tools_dir) or (not os.path.exists(llvm_tools_dir)):
-  lit_config.fatal("Invalid llvm_tools_dir config attribute: %r" % llvm_tools_dir)
-path = os.path.pathsep.join((llvm_tools_dir, config.test_exec_root,
-                             config.environment['PATH']))
-config.environment['PATH'] = path
-
-if config.has_lsan:
+# LeakSanitizer is not supported on OSX right now.
+if sys.platform.startswith('darwin'):
+  lit_config.note('lsan feature unavailable')
+else:
   lit_config.note('lsan feature available')
   config.available_features.add('lsan')
-else:
-  lit_config.note('lsan feature unavailable')
 
 if sys.platform.startswith('win') or sys.platform.startswith('cygwin'):
   config.available_features.add('windows')
 
 if sys.platform.startswith('darwin'):
   config.available_features.add('darwin')
 
 if sys.platform.startswith('linux'):
   # Note the value of ``sys.platform`` is not consistent
   # between python 2 and 3, hence the use of ``.startswith()``.
   lit_config.note('linux feature available')
   config.available_features.add('linux')
 else:
   lit_config.note('linux feature unavailable')
 
 config.substitutions.append(('%build_dir', config.cmake_binary_dir))
+libfuzzer_src_root = os.path.join(config.compiler_rt_src_root, "lib", "fuzzer")
+config.substitutions.append(('%libfuzzer_src', libfuzzer_src_root))
 
 def generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True):
-  compiler_cmd = config.cpp_compiler if is_cpp else config.c_compiler
+  compiler_cmd = config.c_compiler
+  link_cmd = '-lc++' if 'darwin' in config.target_triple else '-lstdc++'
   std_cmd = '-std=c++11' if is_cpp else ''
   sanitizers = ['address']
   if fuzzer_enabled:
     sanitizers.append('fuzzer')
   sanitizers_cmd = ('-fsanitize=%s' % ','.join(sanitizers))
   isysroot_cmd = ('-isysroot %s' % config.osx_sysroot
       ) if 'darwin' in config.target_triple else ''
-  include_cmd = '-I%s/../.' % config.test_source_root
-  return '%s %s -gline-tables-only %s %s %s' % (
-      compiler_cmd, std_cmd, isysroot_cmd, sanitizers_cmd, include_cmd)
+  include_cmd = '-I%s' % libfuzzer_src_root
+  return '%s %s %s -gline-tables-only %s %s %s' % (
+      compiler_cmd, std_cmd, link_cmd, isysroot_cmd, sanitizers_cmd, include_cmd)
 
 config.substitutions.append(('%cpp_compiler',
       generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True)
       ))
 
 config.substitutions.append(('%c_compiler',
       generate_compiler_cmd(is_cpp=False, fuzzer_enabled=True)
       ))
 
 config.substitutions.append(('%no_fuzzer_cpp_compiler',
       generate_compiler_cmd(is_cpp=True, fuzzer_enabled=False)
       ))
 
 config.substitutions.append(('%no_fuzzer_c_compiler',
       generate_compiler_cmd(is_cpp=False, fuzzer_enabled=False)
       ))

test/fuzzer/lit.site.cfg.in

+@LIT_SITE_CFG_IN_HEADER@
+
 config.test_exec_root = "@CMAKE_CURRENT_BINARY_DIR@"
-config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
-config.has_lsan = True if @HAS_LSAN@ == 1 else False
-config.cpp_compiler = "@LIBFUZZER_TEST_CXX_COMPILER@"
+
+config.cpp_compiler = "@LIBFUZZER_TEST_COMPILER@"
+config.target_flags = "@LIBFUZZER_TEST_FLAGS@"
 config.c_compiler = "@LIBFUZZER_TEST_COMPILER@"
+
 config.osx_sysroot = "@CMAKE_OSX_SYSROOT@"
 config.cmake_binary_dir = "@CMAKE_BINARY_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
+
+# Load common config for all compiler-rt lit tests.
+lit_config.load_config(config,
+    "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")
+
 lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg")

test/fuzzer/standalone.test

-RUN: %no_fuzzer_c_compiler %S/../standalone/StandaloneFuzzTargetMain.c -c -o %t_1.o
+RUN: %no_fuzzer_c_compiler %libfuzzer_src/standalone/StandaloneFuzzTargetMain.c -c -o %t_1.o
 RUN: %no_fuzzer_cpp_compiler %S/InitializeTest.cpp -c -o %t_2.o
 
 RUN: %no_fuzzer_cpp_compiler %t_1.o %t_2.o %build_dir/lib/libLLVMFuzzerNoMain.a -o %t-StandaloneInitializeTest
 RUN: %t-StandaloneInitializeTest %S/hi.txt %S/dict1.txt 2>&1 | FileCheck %s
 CHECK: StandaloneFuzzTargetMain: running 2 inputs
 CHECK: Done:    {{.*}}hi.txt: (3 bytes)
 CHECK: Done:    {{.*}}dict1.txt: (61 bytes)

test/fuzzer/unit/lit.site.cfg.in

+@LIT_SITE_CFG_IN_HEADER@
+
+config.name = "LLVMFuzzer-Unittest"
+# Load common config for all compiler-rt unit tests.
+lit_config.load_config(config, "@COMPILER_RT_BINARY_DIR@/unittests/lit.common.unit.configured")
+
+config.test_exec_root = os.path.join("@COMPILER_RT_BINARY_DIR@",
+                                     "lib", "fuzzer", "tests")
+config.test_source_root = config.test_exec_root