Add FuzzerRemote

[aarongreen]

Summary

This changes defines the FuzzerRemoteInterface, and implements it for remote processes. When combined with an IPC library, this library can be used to fuzz remote processes.

This is change 14 of (at least) 20 for cross-process fuzzing support.

[aarongreen]

Rebase

[morehouse]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

Is there an implicit assumption that Counters and PCs have matching indices? Is it a problem if we have a sequence of calls like this:

1. AddCoverage(Start1, Stop1)
2. AddCoverage(Start2, Stop2)
3. AddPCs(Beg2, End2)
4. AddPCs(Beg1, End1)

241

None of these implementations actually use PID. Do we need it here?

compiler-rt/lib/fuzzer/FuzzerRemoteInterface.h

25

If fuzzer can call FuzzerRemote* and fuzzer_remote can call FuzzerProxy*, aren't the arrows backwards?

32

Please expand this warning to explain why we need C interfaces.

153

We don't actually use this constant.

compiler-rt/lib/fuzzer/tests/FuzzerRemoteUnittest.cpp

2

85–88

These are superfluous.

138

Just wondering: are the lambdas required to satisfy the compiler, or could we just directly assign: EF->__lsan_disable = FakeLSan::Disable?

260

This is not a "leak" from LSan point of view since the pointer to the memory is never lost.

278

This also is not a leak that LSan would report.

328

Also not a leak.

[aarongreen]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

I got a bit stuck thinking about this. I'm answering here first before finishing and uploading another diff.

I think you're right; there's nothing preventing two threads from calling dlopen simultaneously, which IIUC leads to the calls added by ModuleSanitizerCoverage::instrumentModule in llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp. That function *does* imply __sanitizer_cov_8bit_counters_init will be called before a corresponding call to __sanitizer_cov_pcs_init, but I think I might try to limit myself to an even weaker assumption: the order of calls to __sanitizer_cov_8bit_counters_init matches then order of calls to __sanitizer_cov_pcs_init for a particular thread, but the order of combined calls is unspecified.

The wrinkle is that the changes for module-relative features files for fork mode and merging *also* made this assumption about matching indices. I'll need to tweak TracePC with some thread_locals to handle that correctly. I assume I can't use the c++ stdlib at that point either, because platforms other than Linux and Fuchsia don't have a hermetic libclang_rt.fuzzer.a, i.e. they don't statically link against libc++.a. That will make any solution a little less aesthetically pleasing.

[morehouse]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

I think this assumption exists in current libFuzzer as well. For example, TracePC::UpdateObservedPCs relies on indices into Modules and ModulePCTable matching. Presumably dlopen in the presence of threads is rare enough that we haven't had issues in practice.

So this may not be something you need to fix.

The main thing that threw me off was the locking in these member functions, which implies multiple threads, which implies races outside the lock here.

Maybe a class comment explaining the multithread scenario you're trying to guard against with the mutex would be helpful.

[aarongreen]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

Okay, sounds good. I've tried to make concurrency clearer here with a comment describing the two possible competing threads: the module loader thread (__sanitizer_cov_*_init) and the IPC receiver thread (FuzzerRemote*). I'd already adjusted AddCounters/AddPCs to use thread_locals, so I've kept that.

As far as Mutex goes, I've tried to lock it less frequently: Now it gets locked by FinishExecution and stays locked until the next StartExecution, ensuring coverage is only added during a fuzzing iteration (which is when we expected new processes to be started). In a perfect world, there'd by an easy way to know when all modules that will ever be loaded are loaded and locking can be ignored, but this is as close as I think I can reasonably get.

241

Yes. As I attempted to describe in FuzzerRemoteInterface.h, both the |FuzzerProxy*| and |FuzzerRemote*| interfaces are implemented twice, once by libFuzzer[Remote], and once by the OS-specific IPC transport library. The transport lib needs the PID to know where to send the IPC, and the interface needs to match between the transport lib and FuzzerRemote. As a result, we have an extra parameter here that looks superfluous.

I've tried to add a comment that says much the same.

compiler-rt/lib/fuzzer/FuzzerRemoteInterface.h

32

I'm not sure what you want here. I've added that it's an external interface (to/from the OS-specific IPC transport library). Beyond that, I don't think you want a discussion about C++'s lack of a consistent ABI, or the benefits of being able to FFI from Rust or Golang...

Honestly, I just copy/pased from FuzzerInterface.h. If you'd like more, let me know.

153

Yet. Note the reference to it in the comment for FuzzerProxyAddCoverage. If that method fails, it will return this, as it does in D94523, the change that adds the implementation of FuzzerProxy. Based on the comment, I'd like to leave this here, even if it goes unused for a change or two.

compiler-rt/lib/fuzzer/tests/FuzzerRemoteUnittest.cpp

138

I think I originally captured this and had non-static methods. When that obviously didn't compile, I changed made them static, but neglected to drop the lambdas. Done.

260

Ah, one of those little comments that ends up opening a can of worms...

You're right, of course, so I tried introducing an explicit leak just to make sure it was caught... only to discover two issues:

1. I was playing too fast and loose with the ExternalFunctions created by FuzzerRemotes constructor. That gets called from the __sanitizer_*_init functions, so I needed to remove any reliance on loadable modules. I can't quite wait until main to create EF, because I'm trying *very* hard not to require source-level changes in the remote process. This closest I can get is a late-prioritized global constructor, which seems to work.
2. I hadn't tested with different combinations of sanitizer instrumentation and runtimes. The behavior varied a bit between -fsanitize=fuzzer-no-link, -fsanitize=address, both, and neither. I've faked more of LSan out in order to make this (and other) unit tests pass in any of those configurations.

278

Same.

[aarongreen]

Crud... I just noticed the .o files. New diff incoming...

[aarongreen]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

One other note on this thread: I apologize that the code moved around so much. As noted below, when looking at LSan I realized the initialization around here was dangerous and wrong. While on Fuchsia (and Linux now?) we statically link libc++, this code shouldn't assume that and shouldn't rely on std:: in the module constructor invoked callbacks. The result of that is the small list implementation above.

[aarongreen]

Multiprocess fuzzing will not be supported by the libFuzzer maintainers. Fuchsia has implemented a new approach with their Component Fuzzing Framework (RFC-117).

compiler-rt/lib/fuzzer/CMakeLists.txt

 set(LIBFUZZER_BASE_SOURCES
   FuzzerDeprecated.cpp
   FuzzerExtFunctionsDlsym.cpp
   FuzzerExtFunctionsWeak.cpp
   FuzzerExtFunctionsWindows.cpp
   FuzzerIO.cpp
   FuzzerIOPosix.cpp
   FuzzerIOWindows.cpp
   FuzzerMonitor.cpp
+  FuzzerProxiedOptions.cpp
   FuzzerUtil.cpp
   FuzzerUtilDarwin.cpp
   FuzzerUtilFuchsia.cpp
   FuzzerUtilLinux.cpp
   FuzzerUtilPosix.cpp
   FuzzerUtilWindows.cpp)
 
 set(LIBFUZZER_BASE_HEADERS
   FuzzerDefs.h
   FuzzerExtFunctions.def
   FuzzerExtFunctions.h
   FuzzerLock.h
   FuzzerMonitor.h
   FuzzerIO.h
   FuzzerPlatform.h
+  FuzzerRemoteInterface.h
   FuzzerUtil.h)
 
 set(LIBFUZZER_SOURCES
   FuzzerCrossOver.cpp
   FuzzerDataFlowTrace.cpp
   FuzzerDriver.cpp
   FuzzerDSORelative.cpp
   FuzzerExtraCounters.cpp
set(LIBFUZZER_HEADERS
   FuzzerMerge.h
   FuzzerMutate.h
   FuzzerOptions.h
   FuzzerRandom.h
   FuzzerSHA1.h
   FuzzerTracePC.h
   FuzzerValueBitMap.h)
 
+
 include_directories(../../include)
 
 CHECK_CXX_SOURCE_COMPILES("
   static thread_local int blah;
   int main() {
   return 0;
   }
   " HAS_THREAD_LOCAL)
add_libfuzzer_objects(RTfuzzer
   HEADERS ${LIBFUZZER_HEADERS})
 
 add_libfuzzer_objects(RTfuzzer_main
   SOURCES FuzzerMain.cpp)
 
 add_libfuzzer_objects(RTfuzzer_interceptors
   SOURCES FuzzerInterceptors.cpp)
 
+add_libfuzzer_objects(RTfuzzer_remote
+  SOURCES FuzzerRemote.cpp)
+
 add_compiler_rt_component(fuzzer)
 
 function(add_libfuzzer_runtime name)
   cmake_parse_arguments(LOCAL "" "" "OBJECT_LIBS" "" ${ARGN})
   add_compiler_rt_runtime(clang_rt.${name}
     STATIC
     OS ${FUZZER_SUPPORTED_OS}
     ARCHS ${FUZZER_SUPPORTED_ARCH}

 
 add_libfuzzer_runtime(fuzzer_no_main
   OBJECT_LIBS RTfuzzer_base
               RTfuzzer)
 
 add_libfuzzer_runtime(fuzzer_interceptors
   OBJECT_LIBS RTfuzzer_interceptors)
 
+add_libfuzzer_runtime(fuzzer_remote
+  OBJECT_LIBS RTfuzzer_base
+              RTfuzzer_remote)
+
 if(COMPILER_RT_INCLUDE_TESTS)
   add_subdirectory(tests)
 endif()

compiler-rt/lib/fuzzer/FuzzerMonitor.h

 void AllocMonitorStartTracing(int TraceLevel);
 bool AllocMonitorStopTracing();
 void AllocMonitorPurge();
 
 } // namespace fuzzer
 
 extern "C" {
 
+// These functions are implemented for both the local process and for remote
+// processes, if supported. For the local process, they do not return. For
+// remote processes, they are sent asynchronously, and the remote process should
+// handle all further calls to the FuzzerRemoteInterface from the thread that
+// invoked the callback, e.g. FuzzerRemotePrintStackTrace should print the stack
+// of the callback-invoking thread.
 ATTRIBUTE_INTERFACE void FuzzerAlarmCallback();
 ATTRIBUTE_INTERFACE void FuzzerCrashSignalCallback(unsigned long PID);
 ATTRIBUTE_INTERFACE void FuzzerDeathCallback();
 ATTRIBUTE_INTERFACE void FuzzerExitCallback(unsigned long PID);
 ATTRIBUTE_INTERFACE void FuzzerFileSizeExceedCallback();
 ATTRIBUTE_INTERFACE void FuzzerGracefulExitCallback();
 ATTRIBUTE_INTERFACE void FuzzerInterruptCallback();
 ATTRIBUTE_INTERFACE void FuzzerLeakCallback(unsigned long PID);
 ATTRIBUTE_INTERFACE void FuzzerMallocLimitCallback(unsigned long PID,
                                                    size_t Size);
 ATTRIBUTE_INTERFACE void FuzzerRssLimitCallback(unsigned long PID);
 
 } // extern "C"
 
 #endif // LLVM_FUZZER_MONITOR_H

compiler-rt/lib/fuzzer/FuzzerProxiedOptions.cpp

+//===- FuzzerProxiedOptions.cpp ---------------------------------*- C++ -* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// Option de/serialization for FuzzerProxy and FuzzerRemote.
+//===----------------------------------------------------------------------===//
+
+#include "FuzzerRemoteInterface.h"
+
+namespace fuzzer {
+
+void ProxiedOptions::Pack(const FuzzingOptions &Options) {
+  FuzzingOptions Dup = Options;
+  Swap(&Dup);
+}
+
+void ProxiedOptions::Unpack(FuzzingOptions *Options) const {
+  ProxiedOptions Dup = *this;
+  Dup.Swap(Options);
+}
+
+void ProxiedOptions::Swap(FuzzingOptions *Options) {
+  std::swap(Options->Verbosity, Verbosity);
+  std::swap(Options->OOMExitCode, OOMExitCode);
+  std::swap(Options->InterruptExitCode, InterruptExitCode);
+  std::swap(Options->ErrorExitCode, ErrorExitCode);
+  std::swap(Options->RssLimitMb, RssLimitMb);
+  std::swap(Options->MallocLimitMb, MallocLimitMb);
+  std::swap(Options->UseValueProfile, UseValueProfile);
+  std::swap(Options->PurgeAllocatorIntervalSec, PurgeAllocatorIntervalSec);
+  std::swap(Options->TraceMalloc, TraceMalloc);
+  bool *BoolOptions[] = {
+      &Options->UseCounters,   &Options->UseMemmem,
+      &Options->UseCmp,        &Options->IgnoreOOMs,
+      &Options->IgnoreCrashes, &Options->IgnoreRemoteExits,
+      &Options->HandleAbrt,    &Options->HandleBus,
+      &Options->HandleFpe,     &Options->HandleIll,
+      &Options->HandleInt,     &Options->HandleSegv,
+      &Options->HandleTerm,
+  };
+  size_t i = 0;
+  for (bool *BoolOption : BoolOptions) {
+    uint32_t Bit = 1U << i++;
+    bool BitFlag = BitFlags & Bit;
+    if (*BoolOption)
+      BitFlags |= Bit;
+    else
+      BitFlags &= ~Bit;
+    *BoolOption = BitFlag;
+  }
+}
+
+} // namespace fuzzer

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

+//===- FuzzerRemoteInterface.h ----------------------------------*- C++ -* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// FuzzerRemoteInterface implementation for the remote process.
+//===----------------------------------------------------------------------===//
+
+// When fuzzing multiple processes communicating via IPC, the process linked
+// against libFuzzer is considered the "proxy", and the others are "remote"
+// processes.
+
+// The FuzzerProxy can be called directly by libFuzzer, and by the remote
+// processes via its implementations of the FuzzerProxy* interface methods. It
+// can also invoke remote methods using the FuzzerRemote* interface methods.
+
+#include "FuzzerExtFunctions.h"
+#include "FuzzerLock.h"
+#include "FuzzerMonitor.h"
+#include "FuzzerOptions.h"
+#include "FuzzerRemoteInterface.h"
+#include "FuzzerTracePC.h"
+#include "FuzzerUtil.h"
+#include <deque>
+
+namespace fuzzer {
+
+// Storage for global ExternalFunctions object.
+ExternalFunctions *EF = nullptr;
+
+namespace {
+
+// Singleton that keeps track of coverage initialization and fuzzer loop
+// execution.
+class Remote final {
+public:
+  static Remote &GetInstance() {
+    static Remote Singleton;
+    return Singleton;
+  }
+
+  ~Remote() = default;
+
+  void AddCounters(uint8_t *Start, uint8_t *Stop) EXCLUDES(Mutex) {
+    if (!Start || Stop <= Start)
+      return;
+    size_t CountersSize, PCsSize;
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      CountersSize = Counters.size();
+      PCsSize = PCs.size();
+      Counters.push_back({Start, Stop});
+    }
+    if (CountersSize < PCsSize)
+      AddCoverage(CountersSize);
+  }
+
+  void AddPCs(const uintptr_t *pcs_beg, const uintptr_t *pcs_end)
+      EXCLUDES(Mutex) {
+    if (!pcs_beg || pcs_end <= pcs_beg)
+      return;
+    size_t CountersSize, PCsSize;
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      CountersSize = Counters.size();
+      PCsSize = PCs.size();
+      PCs.push_back({pcs_beg, pcs_end});
+    }
+    if (PCsSize < CountersSize)
+      AddCoverage(PCsSize);
+  }
+
+  void StartExecution(uint32_t Flags) EXCLUDES(Mutex) {
+    if (FirstIteration) {
+      AllocMonitorConfigure(Options);
+      FirstIteration = false;
+    }
+    if ((Flags & kLeakDetection) != 0 && HasLSan) {
+      EF->__lsan_disable();
+      LSanEnabled = false;
+    }
+    if (EF->__msan_scoped_enable_interceptor_checks)
+      EF->__msan_scoped_enable_interceptor_checks();
+    AllocMonitorStartTracing(Options.TraceMalloc);
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      for (auto &C : Counters)
+        memset(C.first, 0, C.second - C.first);
+    }
+    FuzzerProxyExecutionStarted(PID);
+  }
+
+  void FinishExecution() EXCLUDES(Mutex) {
+    int HasMoreMallocsThanFrees = AllocMonitorStopTracing() ? 1 : 0;
+    if (EF->__msan_scoped_disable_interceptor_checks)
+      EF->__msan_scoped_disable_interceptor_checks();
+    bool LeakDetected = false;
+    if (HasLSan && !LSanEnabled) {
+      EF->__lsan_enable();
+      LSanEnabled = true;
+      if (HasMoreMallocsThanFrees)
+        LeakDetected = EF->__lsan_do_recoverable_leak_check();
+    }
+    if (LeakDetected) {
+      FuzzerLeakCallback(PID);
+    } else {
+      AllocMonitorPurge();
+      FuzzerProxyExecutionFinished(PID, HasMoreMallocsThanFrees);
+    }
+  }
+
+  void DetectLeaksAtExit() {
+    if (!HasLSan)
+      return;
+    if (!LSanEnabled) {
+      EF->__lsan_enable();
+      LSanEnabled = true;
+    }
+    if (EF->__lsan_do_recoverable_leak_check())
+      FuzzerLeakCallback(PID);
+  }
+
+private:
+  Remote() : PID(GetPid()) {
+    // Test environments may set EF up beforehand.
+    if (!EF)
+      EF = new ExternalFunctions();
+
+    ProxiedOptions Proxied;
+    FuzzerProxyConnect(PID, &Proxied, sizeof(Proxied));
+    Proxied.Unpack(&Options);
+
+    HasLSan = EF->__lsan_enable && EF->__lsan_disable &&
+              EF->__lsan_do_recoverable_leak_check;
+    LSanEnabled = true;
+
+    // Perform the shutdown check now and force it to be skipped later. There is
+    // no good way to get the shutdown check result back to the proxy when this
+    // process exits; use FuzzerRemoteDetectLeaksAtExit instead.
+    if (HasLSan)
+      EF->__lsan_do_leak_check();
+
+    // Start monitoring.
+    if (!Options.IgnoreRemoteExits)
+      std::atexit([]() { FuzzerExitCallback(GetPid()); });
+    if (EF->__sanitizer_set_death_callback)
+      EF->__sanitizer_set_death_callback(FuzzerDeathCallback);
+    SetSignalHandler(Options);
+    StartRssThread(Options.RssLimitMb);
+  }
+
+  void AddCoverage(size_t Idx) EXCLUDES(Mutex) {
+    uint8_t *Start, *Stop;
+    const uintptr_t *Begin, *End;
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      auto &C = Counters[Idx];
+      auto &P = PCs[Idx];
+      Start = C.first;
+      Stop = C.second;
+      Begin = P.first;
+      End = P.second;
+    }
+    FuzzerProxyAddCoverage(PID, Start, Stop, Begin, End);
+  }
+
+  // General parameters for the remote process.
+  unsigned long PID;
+  FuzzingOptions Options;
+  bool HasLSan;
+
+  // Per-fuzzing iteration options.
+  bool LSanEnabled;
+  bool FirstIteration = true;
+
+  // Used to synchronize the coverage-related variables, which are accessed and
+  // modified by both the sanitizer_common hooks and the receiver thread.
+  std::mutex Mutex;
+  std::vector<std::pair<uint8_t *, uint8_t *>> Counters GUARDED_BY(Mutex);
+  std::vector<std::pair<const uintptr_t *, const uintptr_t *>>
+      PCs GUARDED_BY(Mutex);
+
+  FUZZER_NO_COPY_OR_MOVE(Remote);
+};
+
+} // namespace
+} // namespace fuzzer
+
+// libFuzzerRemote's implementation of the FuzzerRemote* interface.
+
+extern "C" {
+
+ATTRIBUTE_INTERFACE
+void __sanitizer_cov_8bit_counters_init(uint8_t *Start, uint8_t *Stop) {
+  fuzzer::Remote::GetInstance().AddCounters(Start, Stop);
+}
+
+ATTRIBUTE_INTERFACE
+void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
+                              const uintptr_t *pcs_end) {
+  fuzzer::Remote::GetInstance().AddPCs(pcs_beg, pcs_end);
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteStartExecution(unsigned long PID,
+                                                    uint32_t Flags) {
+  fuzzer::Remote::GetInstance().StartExecution(Flags);
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteFinishExecution(unsigned long PID) {
+  fuzzer::Remote::GetInstance().FinishExecution();
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintPC(unsigned long PID,
+                                             const char *SymbolizedFMT,
+                                             const char *FallbackFMT,
+                                             uintptr_t PC) {
+  fuzzer::PrintPC(SymbolizedFMT, FallbackFMT, PC);
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDescribePC(unsigned long PID,
+                                                const char *SymbolizedFMT,
+                                                uintptr_t PC, char *Desc,
+                                                size_t DescLen) {
+  std::string Description = fuzzer::DescribePC(SymbolizedFMT, PC);
+  snprintf(Desc, DescLen, "%s", Description.c_str());
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintStackTrace(unsigned long PID) {
+  fuzzer::PrintStackTrace();
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintMemoryProfile(unsigned long PID) {
+  fuzzer::PrintMemoryProfile();
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDetectLeaksAtExit(unsigned long PID) {
+  fuzzer::Remote::GetInstance().DetectLeaksAtExit();
+}
+
+// These callbacks should never be triggered in a remote process.
+ATTRIBUTE_INTERFACE void FuzzerAlarmCallback() { _Exit(1); }
+
+ATTRIBUTE_INTERFACE void FuzzerFileSizeExceedCallback() { _Exit(1); }
+
+ATTRIBUTE_INTERFACE void FuzzerGracefulExitCallback() { _Exit(1); }
+
+ATTRIBUTE_INTERFACE void FuzzerInterruptCallback() { _Exit(1); }
+
+} // extern "C"

compiler-rt/lib/fuzzer/FuzzerRemoteInterface.h

+//===- FuzzerRemoteInterface.h ----------------------------------*- C++ -* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// Defines the interface between libFuzzer and a remote process being tested.
+//===----------------------------------------------------------------------===//
+
+// When Options.Remote=1, libFuzzer acts as a fuzzing "proxy" that can be used
+// to fuzz multiple processes ("remotes") communicating via IPC.
+
+// The interface methods below define method calls in both directions: libFuzzer
+// (or more specifically, the "fuzzer" runtime) can call FuzzerRemote* methods
+// on remote processes, and the remote processes (or more specifically, the
+// "fuzzer_remote" runtime) can call FuzzerProxy* methods on libFuzzer, i.e.:
+//
+//  +-- -----+ -P-> +-----------+       +------------+ -P-> +---------------+
+//  | fuzzer |      | ipc_proxy | <-I-> | ipc_remote |      | fuzzer_remote |
+//  +--------+ <-R- +-----------+       +------------+ <-R- +---------------+
+//
+//   (P) are calls to FuzzerProxy* interface methods.
+//   (R) are calls to FuzzerRemote* interface methods.
+//   (I) is platform-specific IPC.
+
+// The "ipc_proxy" and "ipc_remote" libraries above are platform-provided
+// libraries that should use platform-specific IPC mechanisms to implement these
+// same interfaces.
+
+// WARNING: keep the interfaces in C.
+
+#ifndef LLVM_FUZZER_REMOTE_INTERFACE_H
+#define LLVM_FUZZER_REMOTE_INTERFACE_H
+
+#include "FuzzerPlatform.h"
+#include <stddef.h>
+#include <stdint.h>
+
+#ifdef __cplusplus
+#include "FuzzerOptions.h"
+#include <limits>
+
+extern "C" {
+#endif // __cplusplus
+
+// Initializes the proxy side of the IPC layer to accept incoming connections
+// from remote prcoesses.
+ATTRIBUTE_INTERFACE void FuzzerAcceptRemotes();
+
+// Disconnect all remotes and tear down the IPC layer. This MUST be called
+// before the FuzzerProxy destructor is invoked.
+ATTRIBUTE_INTERFACE void FuzzerShutdownRemotes();
+
+// FuzzerProxy* functions are implemented twice:
+//  1. The platform-specific IPC front-end must provide an implementation that
+//     can be called by libFuzzerRemote. PIDs can be ignored.
+//  2. libFuzzer provides an implementation that can be called by the platform-
+//     specific IPC back-end. PIDs are used for identification.
+// Functions are synchronous unless otherwise noted; i.e. it can be safely
+// assumed that the remote method has returned when the function returns.
+
+ATTRIBUTE_INTERFACE void FuzzerProxyConnect(unsigned long PID, void *Options,
+                                            size_t OptionsLen);
+
+// Adds coverage memory regions and returns an index that only depends on the
+// relative offsets of the PCs, i.e. adding another set of PCs that match a
+// previous set except for a fixed offset will return the same index. If adding
+// coverage fails, it returns fuzzer::kInvalidIdx instead.
+//
+// NOTE: The counter memory is added to the TracePC and must remain mapped until
+// the fuzzer exits. It may be replaced by an equivalent memory region, but it
+// cannot be unmapped. The PC memory may be safely released after this call
+// completes. The returned index can be used by the IPC layer to track the
+// counter memory. The FuzzerRemote should ignore the index.
+ATTRIBUTE_INTERFACE uintptr_t FuzzerProxyAddCoverage(unsigned long PID,
+                                                     uint8_t *CountersBegin,
+                                                     uint8_t *CountersEnd,
+                                                     const uintptr_t *PCsBegin,
+                                                     const uintptr_t *PCsEnd);
+
+// See also FuzzerRemoteStartExecution, below.
+ATTRIBUTE_INTERFACE void FuzzerProxyExecutionStarted(unsigned long PID);
+
+// See also FuzzerRemoteFinishExecution, below.
+ATTRIBUTE_INTERFACE void
+FuzzerProxyExecutionFinished(unsigned long PID, int HasMoreMallocsThanFrees);
+
+ATTRIBUTE_INTERFACE void FuzzerProxyDisconnect(unsigned long PID);
+
+// FuzzerRemote* functions are implemented twice:
+//  1. The platform-specific IPC back-end must provide an implementation that
+//     can be called by libFuzzer. PIDs can be used for routing.
+//  2. libFuzzerRemote provides an implementation that can be called by the
+//     platform-specific IPC front-end. PIDs are ignored.
+// Functions are synchronous unless otherwise noted; i.e. it can be safely
+// assumed that the proxy method has returned when the function returns.
+
+// Asynchronous; callers should wait for a corresponding call to
+// FuzzerProxyExecutionStarted.
+ATTRIBUTE_INTERFACE void FuzzerRemoteStartExecution(unsigned long PID,
+                                                    uint32_t Flags);
+
+// Asynchronous; callers should wait for a corresponding call to
+// FuzzerProxyExecutionFinished.
+ATTRIBUTE_INTERFACE void FuzzerRemoteFinishExecution(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintPC(unsigned long PID,
+                                             const char *SymbolizedFMT,
+                                             const char *FallbackFMT,
+                                             uintptr_t PC);
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDescribePC(unsigned long PID,
+                                                const char *SymbolizedFMT,
+                                                uintptr_t PC, char *Desc,
+                                                size_t DescLen);
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintStackTrace(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintMemoryProfile(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDetectLeaksAtExit(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteTerminate(unsigned long PID);
+
+#ifdef __cplusplus
+} // extern "C"
+
+namespace fuzzer {
+
+// Subset of fuzzer::FuzzingOptions that is forwarded from the FuzzerProxy to
+// the FuzzerRemotes upon connection.
+struct ProxiedOptions final {
+public:
+  void Pack(const FuzzingOptions &Options);
+  void Unpack(FuzzingOptions *Options) const;
+
+private:
+  void Swap(FuzzingOptions *Options);
+
+  int Verbosity;
+  int OOMExitCode;
+  int InterruptExitCode;
+  int ErrorExitCode;
+  int RssLimitMb;
+  int MallocLimitMb;
+  int PurgeAllocatorIntervalSec;
+  int UseValueProfile;
+  int TraceMalloc;
+  uint32_t BitFlags;
+};
+
+// Index indicating a call to FuzzerProxyAddCoverage failed.
+constexpr uintptr_t kInvalidIdx = std::numeric_limits<uintptr_t>::max();
+
+// Flags that can be sent as part of FuzzerRemoteStartExecution.
+constexpr uint32_t kLeakDetection = 1 << 0;
+
+} // namespace fuzzer
+
+#endif // __cplusplus
+#endif // LLVM_FUZZER_REMOTE_INTERFACE_H

compiler-rt/lib/fuzzer/tests/CMakeLists.txt

OBJECT_LIBS RTfuzzer_base
                 RTfuzzer)
 
   generate_libfuzzer_unittests(FuzzedDataProviderUnitTests ${arch}
     NAME FuzzerUtils-${arch}-Test
     SOURCES FuzzedDataProviderUnittest.cpp
     OBJECT_LIBS RTfuzzer_base
                 RTfuzzer
     DEPS ${COMPILER_RT_SOURCE_DIR}/include/fuzzer/FuzzedDataProvider.h)
+
+  generate_libfuzzer_unittests(FuzzerRemoteUnitTests ${arch}
+    SOURCES FuzzerRemoteUnittest.cpp
+    OBJECT_LIBS RTfuzzer_base
+                RTfuzzer_remote)
 endif()

compiler-rt/lib/fuzzer/tests/FuzzerRemoteUnittest.cpp

+//===- FuzzerRemoteUnittest.cpph ------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// Unit tests of FuzzerRemote.
+//
+// This is kept separate from the other unit tests as it must link against the
+// fuzzer_remote runtime, which has symbols that conflict with the fuzzer
+// runtime, e.g. __sanitizer_cov_8bit_counters_init.
+//===----------------------------------------------------------------------===//
+
+#include "FuzzerExtFunctions.h"
+#include "FuzzerMonitor.h"
+#include "FuzzerRemoteInterface.h"
+#include "FuzzerTracePC.h"
+#include "FuzzerUtil.h"
+#include "tests/FuzzerTestUtil.h"
+#include "gtest/gtest.h"
+#include <cstddef>
+#include <cstdint>
+
+namespace fuzzer {
+
+// A collection of variables used to record the parameters to the interface
+// functions below.
+static struct {
+  unsigned long PID;
+  uint8_t *CountersBegin;
+  uint8_t *CountersEnd;
+  const uintptr_t *PCsBegin;
+  const uintptr_t *PCsEnd;
+  int HasMoreMallocsThanFrees;
+  int LeakCallbacks;
+} Last;
+
+// The remote library expects to be able to call the IPC layer's implementation
+// of these interface functions.
+extern "C" {
+
+void FuzzerProxyConnect(unsigned long PID, void *Options, size_t OptionsLen) {
+  FuzzingOptions Defaults; // default values.
+  ProxiedOptions Proxied;
+  Proxied.Pack(Defaults);
+  ASSERT_EQ(OptionsLen, sizeof(Proxied));
+  memcpy(Options, &Proxied, OptionsLen);
+}
+
+uintptr_t FuzzerProxyAddCoverage(unsigned long PID, uint8_t *CountersBegin,
+                                 uint8_t *CountersEnd,
+                                 const uintptr_t *PCsBegin,
+                                 const uintptr_t *PCsEnd) {
+  Last.PID = PID;
+  Last.CountersBegin = CountersBegin;
+  Last.CountersEnd = CountersEnd;
+  Last.PCsBegin = PCsBegin;
+  Last.PCsEnd = PCsEnd;
+  return fuzzer::kInvalidIdx;
+}
+
+void FuzzerProxyExecutionStarted(unsigned long PID) { Last.PID = PID; }
+
+void FuzzerProxyExecutionFinished(unsigned long PID,
+                                  int HasMoreMallocsThanFrees) {
+  Last.PID = PID;
+  Last.HasMoreMallocsThanFrees = HasMoreMallocsThanFrees;
+}
+
+// The monitoring callbacks should not be called as part of the unit tests...
+void FuzzerCrashSignalCallback(unsigned long PID) { FAIL(); }
+void FuzzerDeathCallback() { FAIL(); }
+void FuzzerMallocLimitCallback(unsigned long PID, size_t Size) { FAIL(); }
+void FuzzerRssLimitCallback(unsigned long PID) { FAIL(); }
+
+// ...with the exceptions of:
+// * The leak callback, invoked by LSan.
+// * The exit callback, invoked on normal exit.
+void FuzzerLeakCallback(unsigned long PID) { Last.LeakCallbacks++; }
+void FuzzerExitCallback(unsigned long PID) {}
+
+// Defined by the remote library, but not declared in any headers.
+extern "C" void __sanitizer_cov_8bit_counters_init(uint8_t *Start,
+                                                   uint8_t *Stop);
+extern "C" void __sanitizer_cov_pcs_init(const uintptr_t *Start,
+                                         const uintptr_t *Stop);
+
+} // extern "C"
+
+namespace {
+
+// Test fixtures
+
+class RemoteTest : public ::testing::Test {
+protected:
+  void SetUp() override {
+    PID = GetPid();
+    ResetLast();
+    // This ensures the remote singleton is instantiated.
+    __sanitizer_cov_pcs_init(nullptr, nullptr);
+  }
+
+  void ResetLast() {
+    Last.PID = 0;
+    Last.CountersBegin = nullptr;
+    Last.CountersEnd = nullptr;
+    Last.PCsBegin = nullptr;
+    Last.PCsEnd = nullptr;
+    Last.HasMoreMallocsThanFrees = -1;
+    Last.LeakCallbacks = 0;
+  }
+
+  bool LastIsChanged() {
+    return Last.PID != 0 || Last.CountersBegin != nullptr ||
+           Last.CountersEnd != nullptr || Last.PCsBegin != nullptr ||
+           Last.PCsEnd != nullptr || Last.HasMoreMallocsThanFrees != -1 ||
+           Last.LeakCallbacks != 0;
+  }
+
+  unsigned long PID;
+};
+
+// Instances of this class replace LSan, if present, with simple counters to
+// track calls. LSan function pointers are restored upon destruction.
+class FakeLSan {
+public:
+  FakeLSan()
+      : RealEnable(EF->__lsan_enable),
+        RealCheck(EF->__lsan_do_recoverable_leak_check),
+        RealDisable(EF->__lsan_disable) {
+    assert(!Current);
+    EF->__lsan_enable = []() { FakeLSan::Enable(); };
+    EF->__lsan_do_recoverable_leak_check = []() -> int {
+      return FakeLSan::Check();
+    };
+    EF->__lsan_disable = []() { FakeLSan::Disable(); };
+    Current = this;
+  }
+
+  ~FakeLSan() {
+    assert(Current == this);
+    EF->__lsan_enable = RealEnable;
+    EF->__lsan_do_recoverable_leak_check = RealCheck;
+    EF->__lsan_disable = RealDisable;
+    Current = nullptr;
+  }
+
+  bool IsPresent() const { return RealEnable && RealCheck && RealDisable; }
+
+  size_t EnableCalls() const { return Enables; }
+  size_t CheckCalls() const { return Checks; }
+  size_t DisableCalls() const { return Disables; }
+
+private:
+  static void Enable() { Current->Enables++; }
+  static int Check() {
+    Current->Checks++;
+    return Current->Enables ? Last.HasMoreMallocsThanFrees : 0;
+  }
+  static void Disable() { Current->Disables++; }
+
+  static FakeLSan *Current;
+
+  void (*RealEnable)();
+  int (*RealCheck)();
+  void (*RealDisable)();
+
+  size_t Enables = 0;
+  size_t Checks = 0;
+  size_t Disables = 0;
+};
+
+FakeLSan *FakeLSan::Current = nullptr;
+
+// Unit tests for the remote interface.
+TEST_F(RemoteTest, SanitizerCovInit) {
+  FakeDSO<0x100> DSO1;
+  FakeDSO<0x200> DSO2;
+  FakeDSO<0x300> DSO3;
+
+  // Invalid arguments are ignored.
+  ResetLast();
+  __sanitizer_cov_8bit_counters_init(nullptr, DSO1.CountersEnd());
+  __sanitizer_cov_pcs_init(nullptr, DSO1.PCsEnd());
+  EXPECT_FALSE(LastIsChanged());
+
+  ResetLast();
+  __sanitizer_cov_8bit_counters_init(DSO1.Counters, nullptr);
+  __sanitizer_cov_pcs_init(DSO1.PCs, nullptr);
+  EXPECT_FALSE(LastIsChanged());
+
+  ResetLast();
+  __sanitizer_cov_8bit_counters_init(DSO1.Counters, DSO1.Counters);
+  __sanitizer_cov_pcs_init(DSO1.PCs, DSO1.PCs);
+  EXPECT_FALSE(LastIsChanged());
+
+  ResetLast();
+  __sanitizer_cov_8bit_counters_init(DSO1.CountersEnd(), DSO1.Counters);
+  __sanitizer_cov_pcs_init(DSO1.PCsEnd(), DSO1.PCs);
+  EXPECT_FALSE(LastIsChanged());
+
+  // Counters and PCs are added together.
+  // Send two counter regions; nothing should happen without PC regions.
+  ResetLast();
+  __sanitizer_cov_8bit_counters_init(DSO1.Counters, DSO1.CountersEnd());
+  __sanitizer_cov_8bit_counters_init(DSO2.Counters, DSO2.CountersEnd());
+  EXPECT_FALSE(LastIsChanged());
+
+  // Should match the first counter region.
+  __sanitizer_cov_pcs_init(DSO1.PCs, DSO1.PCsEnd());
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.CountersBegin, DSO1.Counters);
+  EXPECT_EQ(Last.CountersEnd, DSO1.CountersEnd());
+  EXPECT_EQ(Last.PCsBegin, DSO1.PCs);
+  EXPECT_EQ(Last.PCsEnd, DSO1.PCsEnd());
+
+  // Should match the second counter region.
+  ResetLast();
+  __sanitizer_cov_pcs_init(DSO2.PCs, DSO2.PCsEnd());
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.CountersBegin, DSO2.Counters);
+  EXPECT_EQ(Last.CountersEnd, DSO2.CountersEnd());
+  EXPECT_EQ(Last.PCsBegin, DSO2.PCs);
+  EXPECT_EQ(Last.PCsEnd, DSO2.PCsEnd());
+
+  // Nothing should happen without another counter region.
+  ResetLast();
+  __sanitizer_cov_pcs_init(DSO3.PCs, DSO3.PCsEnd());
+  EXPECT_FALSE(LastIsChanged());
+
+  // Should match the third PC region.
+  __sanitizer_cov_8bit_counters_init(DSO3.Counters, DSO3.CountersEnd());
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.CountersBegin, DSO3.Counters);
+  EXPECT_EQ(Last.CountersEnd, DSO3.CountersEnd());
+  EXPECT_EQ(Last.PCsBegin, DSO3.PCs);
+  EXPECT_EQ(Last.PCsEnd, DSO3.PCsEnd());
+}
+
+TEST_F(RemoteTest, StartAndFinishExecution) {
+  FakeLSan LSan;
+
+  // No leaks
+  ResetLast();
+  FuzzerRemoteStartExecution(PID, 0);
+  EXPECT_EQ(Last.PID, PID);
+
+  ResetLast();
+  FuzzerRemoteFinishExecution(PID);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.HasMoreMallocsThanFrees, 0);
+
+  // With leak, but no leak detection
+  ResetLast();
+  FuzzerRemoteStartExecution(PID, 0);
+  EXPECT_EQ(Last.PID, PID);
+
+  std::unique_ptr<uint8_t[]> Leaked(new uint8_t[1]);
+  ResetLast();
+  FuzzerRemoteFinishExecution(PID);
+  EXPECT_EQ(Last.PID, PID);
+  if (EF->__sanitizer_install_malloc_and_free_hooks)
+    EXPECT_EQ(Last.HasMoreMallocsThanFrees, 1);
+  else
+    EXPECT_EQ(Last.HasMoreMallocsThanFrees, 0);
+  EXPECT_EQ(LSan.EnableCalls(), 0U);
+  EXPECT_EQ(LSan.CheckCalls(), 0U);
+  EXPECT_EQ(Last.LeakCallbacks, 0);
+  EXPECT_EQ(LSan.DisableCalls(), 0U);
+  Leaked.reset();
+
+  // With leak and detection.
+  ResetLast();
+  FuzzerRemoteStartExecution(PID, kLeakDetection);
+  EXPECT_EQ(Last.PID, PID);
+  Leaked.reset(new uint8_t[2]);
+
+  ResetLast();
+  FuzzerRemoteFinishExecution(PID);
+  EXPECT_EQ(Last.PID, PID);
+  if (EF->__sanitizer_install_malloc_and_free_hooks)
+    EXPECT_EQ(Last.HasMoreMallocsThanFrees, 1);
+  else
+    EXPECT_EQ(Last.HasMoreMallocsThanFrees, 0);
+
+  if (LSan.IsPresent()) {
+    EXPECT_EQ(LSan.EnableCalls(), 1U);
+    EXPECT_EQ(LSan.CheckCalls(), 1U);
+    EXPECT_EQ(Last.LeakCallbacks, 1);
+    EXPECT_EQ(LSan.DisableCalls(), 1U);
+  } else {
+    EXPECT_EQ(LSan.EnableCalls(), 0U);
+    EXPECT_EQ(LSan.CheckCalls(), 0U);
+    EXPECT_EQ(Last.LeakCallbacks, 0);
+    EXPECT_EQ(LSan.DisableCalls(), 0U);
+  }
+}
+
+TEST_F(RemoteTest, DescribePC) {
+  // "Placeholders" from sanitizer_common/sanitizer_stacktrace_printer.h
+  char SymbolizedFMT[3] = {'%', '_', '\0'};
+  constexpr const char FormatChars[] = "npmofqslcFSLM";
+
+  for (size_t i = 0; i < strlen(FormatChars); ++i) {
+    SCOPED_TRACE(FormatChars[i]);
+    SymbolizedFMT[1] = FormatChars[i];
+    uintptr_t PC = 0xdeadbeef;
+
+    std::string Expected = DescribePC(SymbolizedFMT, PC);
+    char Actual[256];
+    FuzzerRemoteDescribePC(PID, SymbolizedFMT, PC, Actual, sizeof(Actual));
+
+    EXPECT_STREQ(Expected.c_str(), Actual);
+  }
+}
+
+TEST_F(RemoteTest, DetectLeaksAtExit) {
+  FakeLSan LSan;
+
+  // No leaks
+  ResetLast();
+  FuzzerRemoteDetectLeaksAtExit(PID);
+
+  // With leaks
+  ResetLast();
+  std::unique_ptr<uint8_t[]> Leaked(new uint8_t[1]);
+  FuzzerRemoteDetectLeaksAtExit(PID);
+  if (LSan.IsPresent()) {
+    EXPECT_EQ(LSan.EnableCalls(), 1U);
+    EXPECT_EQ(LSan.CheckCalls(), 1U);
+    EXPECT_EQ(Last.LeakCallbacks, 1);
+    EXPECT_EQ(LSan.DisableCalls(), 1U);
+  } else {
+    EXPECT_EQ(LSan.EnableCalls(), 0U);
+    EXPECT_EQ(LSan.CheckCalls(), 0U);
+    EXPECT_EQ(Last.LeakCallbacks, 0);
+    EXPECT_EQ(LSan.DisableCalls(), 0U);
+  }
+}
+
+} // namespace
+} // namespace fuzzer
+
+int main(int argc, char **argv) {
+  testing::InitGoogleTest(&argc, argv);
+  testing::AddGlobalTestEnvironment(new fuzzer::TestEnvironment());
+  return RUN_ALL_TESTS();
+}

compiler-rt/test/fuzzer/CMakeLists.txt

 set(FUZZER_TEST_ARCH ${FUZZER_SUPPORTED_ARCH})
 if (APPLE)
   darwin_filter_host_archs(FUZZER_SUPPORTED_ARCH FUZZER_TEST_ARCH)
 endif()
 
 if(COMPILER_RT_INCLUDE_TESTS)
   list(APPEND LIBFUZZER_TEST_DEPS FuzzerUnitTests)
   list(APPEND LIBFUZZER_TEST_DEPS FuzzedDataProviderUnitTests)
+  list(APPEND LIBFUZZER_TEST_DEPS FuzzerRemoteUnitTests)
 endif()
 
 add_custom_target(check-fuzzer)
 
 if(COMPILER_RT_INCLUDE_TESTS)
   # libFuzzer unit tests.
   configure_lit_site_cfg(
     ${CMAKE_CURRENT_SOURCE_DIR}/unit/lit.site.cfg.py.in