Add FuzzerRemote

[aarongreen]

Summary

This changes defines the FuzzerRemoteInterface, and implements it for remote processes. When combined with an IPC library, this library can be used to fuzz remote processes.

This is change 14 of (at least) 20 for cross-process fuzzing support.

[aarongreen]

Rebase

[morehouse]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

Is there an implicit assumption that Counters and PCs have matching indices? Is it a problem if we have a sequence of calls like this:

1. AddCoverage(Start1, Stop1)
2. AddCoverage(Start2, Stop2)
3. AddPCs(Beg2, End2)
4. AddPCs(Beg1, End1)

241

None of these implementations actually use PID. Do we need it here?

compiler-rt/lib/fuzzer/FuzzerRemoteInterface.h

25

If fuzzer can call FuzzerRemote* and fuzzer_remote can call FuzzerProxy*, aren't the arrows backwards?

32

Please expand this warning to explain why we need C interfaces.

153

We don't actually use this constant.

compiler-rt/lib/fuzzer/tests/FuzzerRemoteUnittest.cpp

2

85–88

These are superfluous.

138

Just wondering: are the lambdas required to satisfy the compiler, or could we just directly assign: EF->__lsan_disable = FakeLSan::Disable?

260

This is not a "leak" from LSan point of view since the pointer to the memory is never lost.

278

This also is not a leak that LSan would report.

328

Also not a leak.

[aarongreen]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

I got a bit stuck thinking about this. I'm answering here first before finishing and uploading another diff.

I think you're right; there's nothing preventing two threads from calling dlopen simultaneously, which IIUC leads to the calls added by ModuleSanitizerCoverage::instrumentModule in llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp. That function *does* imply __sanitizer_cov_8bit_counters_init will be called before a corresponding call to __sanitizer_cov_pcs_init, but I think I might try to limit myself to an even weaker assumption: the order of calls to __sanitizer_cov_8bit_counters_init matches then order of calls to __sanitizer_cov_pcs_init for a particular thread, but the order of combined calls is unspecified.

The wrinkle is that the changes for module-relative features files for fork mode and merging *also* made this assumption about matching indices. I'll need to tweak TracePC with some thread_locals to handle that correctly. I assume I can't use the c++ stdlib at that point either, because platforms other than Linux and Fuchsia don't have a hermetic libclang_rt.fuzzer.a, i.e. they don't statically link against libc++.a. That will make any solution a little less aesthetically pleasing.

[morehouse]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

I think this assumption exists in current libFuzzer as well. For example, TracePC::UpdateObservedPCs relies on indices into Modules and ModulePCTable matching. Presumably dlopen in the presence of threads is rare enough that we haven't had issues in practice.

So this may not be something you need to fix.

The main thing that threw me off was the locking in these member functions, which implies multiple threads, which implies races outside the lock here.

Maybe a class comment explaining the multithread scenario you're trying to guard against with the mutex would be helpful.

[aarongreen]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

Okay, sounds good. I've tried to make concurrency clearer here with a comment describing the two possible competing threads: the module loader thread (__sanitizer_cov_*_init) and the IPC receiver thread (FuzzerRemote*). I'd already adjusted AddCounters/AddPCs to use thread_locals, so I've kept that.

As far as Mutex goes, I've tried to lock it less frequently: Now it gets locked by FinishExecution and stays locked until the next StartExecution, ensuring coverage is only added during a fuzzing iteration (which is when we expected new processes to be started). In a perfect world, there'd by an easy way to know when all modules that will ever be loaded are loaded and locking can be ignored, but this is as close as I think I can reasonably get.

241

Yes. As I attempted to describe in FuzzerRemoteInterface.h, both the |FuzzerProxy*| and |FuzzerRemote*| interfaces are implemented twice, once by libFuzzer[Remote], and once by the OS-specific IPC transport library. The transport lib needs the PID to know where to send the IPC, and the interface needs to match between the transport lib and FuzzerRemote. As a result, we have an extra parameter here that looks superfluous.

I've tried to add a comment that says much the same.

compiler-rt/lib/fuzzer/FuzzerRemoteInterface.h

32

I'm not sure what you want here. I've added that it's an external interface (to/from the OS-specific IPC transport library). Beyond that, I don't think you want a discussion about C++'s lack of a consistent ABI, or the benefits of being able to FFI from Rust or Golang...

Honestly, I just copy/pased from FuzzerInterface.h. If you'd like more, let me know.

153

Yet. Note the reference to it in the comment for FuzzerProxyAddCoverage. If that method fails, it will return this, as it does in D94523, the change that adds the implementation of FuzzerProxy. Based on the comment, I'd like to leave this here, even if it goes unused for a change or two.

compiler-rt/lib/fuzzer/tests/FuzzerRemoteUnittest.cpp

138

I think I originally captured this and had non-static methods. When that obviously didn't compile, I changed made them static, but neglected to drop the lambdas. Done.

260

Ah, one of those little comments that ends up opening a can of worms...

You're right, of course, so I tried introducing an explicit leak just to make sure it was caught... only to discover two issues:

1. I was playing too fast and loose with the ExternalFunctions created by FuzzerRemotes constructor. That gets called from the __sanitizer_*_init functions, so I needed to remove any reliance on loadable modules. I can't quite wait until main to create EF, because I'm trying *very* hard not to require source-level changes in the remote process. This closest I can get is a late-prioritized global constructor, which seems to work.
2. I hadn't tested with different combinations of sanitizer instrumentation and runtimes. The behavior varied a bit between -fsanitize=fuzzer-no-link, -fsanitize=address, both, and neither. I've faked more of LSan out in order to make this (and other) unit tests pass in any of those configurations.

278

Same.

[aarongreen]

Crud... I just noticed the .o files. New diff incoming...

[aarongreen]

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

58

One other note on this thread: I apologize that the code moved around so much. As noted below, when looking at LSan I realized the initialization around here was dangerous and wrong. While on Fuchsia (and Linux now?) we statically link libc++, this code shouldn't assume that and shouldn't rely on std:: in the module constructor invoked callbacks. The result of that is the small list implementation above.

[aarongreen]

Multiprocess fuzzing will not be supported by the libFuzzer maintainers. Fuchsia has implemented a new approach with their Component Fuzzing Framework (RFC-117).

compiler-rt/lib/fuzzer/CMakeLists.txt

 set(LIBFUZZER_BASE_SOURCES
   FuzzerDeprecated.cpp
   FuzzerExtFunctionsDlsym.cpp
   FuzzerExtFunctionsWeak.cpp
   FuzzerExtFunctionsWindows.cpp
   FuzzerIO.cpp
   FuzzerIOPosix.cpp
   FuzzerIOWindows.cpp
   FuzzerMonitor.cpp
+  FuzzerProxiedOptions.cpp
   FuzzerUtil.cpp
   FuzzerUtilDarwin.cpp
   FuzzerUtilFuchsia.cpp
   FuzzerUtilLinux.cpp
   FuzzerUtilPosix.cpp
   FuzzerUtilWindows.cpp)
 
 set(LIBFUZZER_BASE_HEADERS
   FuzzerDefs.h
   FuzzerExtFunctions.def
   FuzzerExtFunctions.h
   FuzzerLock.h
   FuzzerMonitor.h
   FuzzerIO.h
   FuzzerPlatform.h
+  FuzzerRemoteInterface.h
   FuzzerUtil.h)
 
 set(LIBFUZZER_SOURCES
   FuzzerCrossOver.cpp
   FuzzerDataFlowTrace.cpp
   FuzzerDriver.cpp
   FuzzerExtraCounters.cpp
   FuzzerFork.cpp
set(LIBFUZZER_HEADERS
   FuzzerModuleRelative.h
   FuzzerMutate.h
   FuzzerOptions.h
   FuzzerRandom.h
   FuzzerSHA1.h
   FuzzerTracePC.h
   FuzzerValueBitMap.h)
 
+
 include_directories(../../include)
 
 CHECK_CXX_SOURCE_COMPILES("
   static thread_local int blah;
   int main() {
   return 0;
   }
   " HAS_THREAD_LOCAL)
add_libfuzzer_objects(RTfuzzer
   HEADERS ${LIBFUZZER_HEADERS})
 
 add_libfuzzer_objects(RTfuzzer_main
   SOURCES FuzzerMain.cpp)
 
 add_libfuzzer_objects(RTfuzzer_interceptors
   SOURCES FuzzerInterceptors.cpp)
 
+add_libfuzzer_objects(RTfuzzer_remote
+  SOURCES FuzzerRemote.cpp)
+
 add_compiler_rt_component(fuzzer)
 
 function(add_libfuzzer_runtime name)
   cmake_parse_arguments(LOCAL "" "" "OBJECT_LIBS" "" ${ARGN})
   add_compiler_rt_runtime(clang_rt.${name}
     STATIC
     OS ${FUZZER_SUPPORTED_OS}
     ARCHS ${FUZZER_SUPPORTED_ARCH}

 
 add_libfuzzer_runtime(fuzzer_no_main
   OBJECT_LIBS RTfuzzer_base
               RTfuzzer)
 
 add_libfuzzer_runtime(fuzzer_interceptors
   OBJECT_LIBS RTfuzzer_interceptors)
 
+add_libfuzzer_runtime(fuzzer_remote
+  OBJECT_LIBS RTfuzzer_base
+              RTfuzzer_remote)
+
 if(COMPILER_RT_INCLUDE_TESTS)
   add_subdirectory(tests)
 endif()

compiler-rt/lib/fuzzer/FuzzerMonitor.h

 void AllocMonitorStartTracing(int TraceLevel);
 bool AllocMonitorStopTracing();
 void AllocMonitorPurge();
 
 } // namespace fuzzer
 
 extern "C" {
 
+// These functions are implemented for both the local process and for remote
+// processes, if supported. For the local process, they do not return. For
+// remote processes, they are sent asynchronously, and the remote process should
+// handle all further calls to the FuzzerRemoteInterface from the thread that
+// invoked the callback, e.g. FuzzerRemotePrintStackTrace should print the stack
+// of the callback-invoking thread.
 ATTRIBUTE_INTERFACE void FuzzerAlarmCallback();
 ATTRIBUTE_INTERFACE void FuzzerCrashSignalCallback(unsigned long PID);
 ATTRIBUTE_INTERFACE void FuzzerDeathCallback();
 ATTRIBUTE_INTERFACE void FuzzerExitCallback(unsigned long PID);
 ATTRIBUTE_INTERFACE void FuzzerFileSizeExceedCallback();
 ATTRIBUTE_INTERFACE void FuzzerGracefulExitCallback();
 ATTRIBUTE_INTERFACE void FuzzerInterruptCallback();
 ATTRIBUTE_INTERFACE void FuzzerLeakCallback(unsigned long PID);
 ATTRIBUTE_INTERFACE void FuzzerMallocLimitCallback(unsigned long PID,
                                                    size_t Size);
 ATTRIBUTE_INTERFACE void FuzzerRssLimitCallback(unsigned long PID);
 
 } // extern "C"
 
 #endif // LLVM_FUZZER_MONITOR_H

compiler-rt/lib/fuzzer/FuzzerMonitor.cpp

 private:
   std::lock_guard<RecursiveMutex> Lock;
 };
 
 ATTRIBUTE_NO_SANITIZE_MEMORY
 void MallocHook(const volatile void *ptr, size_t size) {
   size_t N = AllocMonitor.Mallocs++;
   if (AllocMonitor.MallocLimit && size >= AllocMonitor.MallocLimit)
-    FuzzerMallocLimitCallback(0, size);
+    FuzzerMallocLimitCallback(GetPid(), size);
   if (int TraceLevel = AllocMonitor.TraceLevel) {
     TraceLock Lock;
     if (Lock.IsDisabled())
       return;
     if (ptr)
       Printf("MALLOC[%zd] %p %zd\n", N, ptr, size);
     if (TraceLevel >= 2 && EF)
       PrintStackTrace();

 void StartRssThread(size_t RssLimitMb) {
   if (!RssLimitMb)
     return;
   std::thread T([RssLimitMb]() {
     while (true) {
       SleepSeconds(1);
       size_t Peak = GetPeakRSSMb();
       if (Peak > RssLimitMb)
-        FuzzerRssLimitCallback(0);
+        FuzzerRssLimitCallback(GetPid());
     }
   });
   T.detach();
 }
 
 void AllocMonitorConfigure(const FuzzingOptions &Options) {
   AllocMonitor.Configure(Options);
 }

compiler-rt/lib/fuzzer/FuzzerProxiedOptions.cpp

+//===- FuzzerProxiedOptions.cpp ---------------------------------*- C++ -* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// Option de/serialization for FuzzerProxy and FuzzerRemote.
+//===----------------------------------------------------------------------===//
+
+#include "FuzzerRemoteInterface.h"
+
+namespace fuzzer {
+
+void ProxiedOptions::Pack(const FuzzingOptions &Options) {
+  FuzzingOptions Dup = Options;
+  Swap(&Dup);
+}
+
+void ProxiedOptions::Unpack(FuzzingOptions *Options) const {
+  ProxiedOptions Dup = *this;
+  Dup.Swap(Options);
+}
+
+void ProxiedOptions::Swap(FuzzingOptions *Options) {
+  std::swap(Options->Verbosity, Verbosity);
+  std::swap(Options->OOMExitCode, OOMExitCode);
+  std::swap(Options->InterruptExitCode, InterruptExitCode);
+  std::swap(Options->ErrorExitCode, ErrorExitCode);
+  std::swap(Options->RssLimitMb, RssLimitMb);
+  std::swap(Options->MallocLimitMb, MallocLimitMb);
+  std::swap(Options->UseValueProfile, UseValueProfile);
+  std::swap(Options->PurgeAllocatorIntervalSec, PurgeAllocatorIntervalSec);
+  std::swap(Options->TraceMalloc, TraceMalloc);
+  bool *BoolOptions[] = {
+      &Options->UseCounters,   &Options->UseMemmem,
+      &Options->UseCmp,        &Options->IgnoreOOMs,
+      &Options->IgnoreCrashes, &Options->IgnoreRemoteExits,
+      &Options->HandleAbrt,    &Options->HandleBus,
+      &Options->HandleFpe,     &Options->HandleIll,
+      &Options->HandleInt,     &Options->HandleSegv,
+      &Options->HandleTerm,
+  };
+  size_t i = 0;
+  for (bool *BoolOption : BoolOptions) {
+    uint32_t Bit = 1U << i++;
+    bool BitFlag = BitFlags & Bit;
+    if (*BoolOption)
+      BitFlags |= Bit;
+    else
+      BitFlags &= ~Bit;
+    *BoolOption = BitFlag;
+  }
+}
+
+} // namespace fuzzer

compiler-rt/lib/fuzzer/FuzzerRemote.cpp

+//===- FuzzerRemoteInterface.h ----------------------------------*- C++ -* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// FuzzerRemoteInterface implementation for the remote process.
+//===----------------------------------------------------------------------===//
+
+// When fuzzing multiple processes communicating via IPC, the process linked
+// against the libFuzzer fuzzing engine is considered the "proxy", and the
+// others are "remote" processes.
+
+// The FuzzerProxy can be called directly by libFuzzer, and by the remote
+// processes via its implementations of the FuzzerProxy* interface methods. It
+// can also invoke remote methods using the FuzzerRemote* interface methods.
+
+#include "FuzzerExtFunctions.h"
+#include "FuzzerLock.h"
+#include "FuzzerMonitor.h"
+#include "FuzzerOptions.h"
+#include "FuzzerRemoteInterface.h"
+#include "FuzzerTracePC.h"
+#include "FuzzerUtil.h"
+#include <cstdlib>
+#include <deque>
+
+namespace fuzzer {
+namespace {
+
+// Forward declaration.
+class Remote;
+Remote *R = nullptr;
+
+// |fuzzer::Remote| assumes that |__sanitizer_cov_inline_8bit_counters_init| and
+// |__sanitizer_cov_pc_tables_init| are called sequentially, i.e. by the module
+// constructor. Before |main| is called, the calls are made from a single
+// thread. After |main| is called, they may be called as a result of concurrent
+// calls to |dlopen| on separate threads.
+//
+// Since |AddCounters| and |AddPCs| may be called before any specific module is
+// loaded, this simple, singly-linked list avoids using any library functions or
+// types.
+struct PendingModule {
+  uint8_t *CountersStart = nullptr;
+  uint8_t *CountersStop = nullptr;
+  const uintptr_t *PCsStart = nullptr;
+  const uintptr_t *PCsStop = nullptr;
+  PendingModule *Next = nullptr;
+};
+thread_local PendingModule *FirstPendingModule = nullptr;
+thread_local PendingModule *LastPendingCounters = nullptr;
+thread_local PendingModule *LastPendingPCs = nullptr;
+
+// This class keeps track of coverage initialization and fuzzer loop
+// execution. This methods of this class may be invoked by two separate threads:
+// the thread loading modules (via __sanitizer_cov_*_init methods), and the IPC
+// library thread handling (via the FuzzerRemote* method).
+//
+// Note that this class does not use a usual RAII wrapper such as
+// std::lock_guard around its mutex. Instead, the mutex is acquired in
+// |FinishExecution| and released in |StartExecution|. This prevents counters
+// and PCs from being added during coverage collection.
+class Remote final {
+public:
+  // See |fuzzer::RemoteSingleton| below. This constructor will be called
+  // exactly once with a init priority to ensure it runs after the module
+  // constructors.
+  Remote() : PID(GetPid()) {
+    // Only one |Remote| and |ExternalFunctions| per process.
+    assert(!R);
+    assert(!EF);
+    EF = new ExternalFunctions();
+    CheckRemoteFunctions();
+
+    // See the note on |fuzzer::RemoteInstance|. This constructor is guaranteed
+    // to run after all other module constructors.
+    ProxiedOptions Proxied;
+    FuzzerProxyConnect(PID, &Proxied, sizeof(Proxied));
+    Proxied.Unpack(&Options);
+
+    // Check for LSan. If present, perform the shutdown check now to force it to
+    // be skipped later. There is no good way to get the shutdown check result
+    // back to the proxy when this process exits; for that leak check, use
+    // |FuzzerRemoteDetectLeaksAtExit| instead.
+    if (EF->__lsan_do_leak_check)
+      EF->__lsan_do_leak_check();
+
+    // Start monitoring error conditions.
+    if (!Options.IgnoreRemoteExits)
+      std::atexit([]() { FuzzerExitCallback(GetPid()); });
+    if (EF->__sanitizer_set_death_callback)
+      EF->__sanitizer_set_death_callback(FuzzerDeathCallback);
+    SetSignalHandler(Options);
+    StartRssThread(Options.RssLimitMb);
+    AllocMonitorConfigure(Options);
+
+    // Remote processes are started during a fuzzing iteration; trace mallocs as
+    // if |StartExecution| had been called..
+    if (EF->__msan_scoped_enable_interceptor_checks)
+      EF->__msan_scoped_enable_interceptor_checks();
+    AllocMonitorStartTracing(Options.TraceMalloc);
+
+    R = this;
+    Remote::ProcessPendingModules();
+  }
+
+  ~Remote() = default;
+
+  // Initialize a set of inline, 8-bit counters. This will be called before the
+  // singleton is constructed. It may also be called before any specific module
+  // is loaded. |AddPCs| must be called before this method can be called again.
+  static void AddCounters(uint8_t *Start, uint8_t *Stop) {
+    if (!Start || !Stop || Stop <= Start)
+      return;
+    auto *M = AddPendingModule(&LastPendingCounters);
+    M->CountersStart = Start;
+    M->CountersStop = Stop;
+    ProcessPendingModules();
+  }
+
+  // Initialize a set of inline, 8-bit counters. This will be called before the
+  // singleton is constructed. It may also be called before any specific module
+  // is loaded. |AddCounters| must be called before this method can be called
+  // again.
+  static void AddPCs(const uintptr_t *Start, const uintptr_t *Stop) {
+    if (!Start || !Stop || Stop <= Start)
+      return;
+    auto *M = AddPendingModule(&LastPendingPCs);
+    M->PCsStart = Start;
+    M->PCsStop = Stop;
+    ProcessPendingModules();
+  }
+
+  static PendingModule *AddPendingModule(PendingModule **LastPending) {
+    PendingModule *M;
+    if (*LastPending) {
+      M = new PendingModule();
+      (*LastPending)->Next = M;
+
+    } else if (FirstPendingModule) {
+      M = FirstPendingModule;
+    } else {
+      M = new PendingModule();
+      FirstPendingModule = M;
+    }
+    *LastPending = M;
+    return M;
+  }
+
+  static void ProcessPendingModules() {
+    if (!R)
+      return;
+    auto *M = FirstPendingModule;
+    while (M && R->AddCoverage(M->CountersStart, M->CountersStop, M->PCsStart,
+                               M->PCsStop)) {
+      FirstPendingModule = M->Next;
+      if (M == LastPendingCounters)
+        LastPendingCounters = nullptr;
+      if (M == LastPendingPCs)
+        LastPendingPCs = nullptr;
+      delete M;
+      M = FirstPendingModule;
+    }
+  }
+
+  // Share the memory containing a module's coverage with the |FuzzerProxy| in
+  // the fuzzing engine process.
+  bool AddCoverage(uint8_t *CountersStart, uint8_t *CountersStop,
+                   const uintptr_t *PCsStart, const uintptr_t *PCsStop)
+      EXCLUDES(Mutex) {
+    if (!CountersStart || !CountersStop || !PCsStart || !PCsStop)
+      return false;
+    assert(CountersStart < CountersStop);
+    assert(PCsStart < PCsStop);
+    auto NumCounters = static_cast<size_t>(CountersStop - CountersStart);
+    assert(NumCounters == static_cast<size_t>(PCsStop - PCsStart) / 2);
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      Modules.push_back({CountersStart, NumCounters});
+    }
+    FuzzerProxyAddCoverage(PID, CountersStart, CountersStop, PCsStart, PCsStop);
+    return true;
+  }
+
+  // Start a fuzzing iteration. At this point, the remote process has sole
+  // control of the shared counter memory. |Flags| can specify details about the
+  // fuzzing iteration, e.g. whether to perform additional leak detection.
+  static void StartExecution(uint32_t Flags) RELEASE(Mutex) {
+    assert(R);
+    R->StartExecutionImpl(Flags);
+  }
+
+  void StartExecutionImpl(uint32_t Flags) RELEASE(Mutex) {
+    if (FirstIteration) {
+      AllocMonitorConfigure(Options);
+      FirstIteration = false;
+    }
+    // See |Fuzzer::TryDetectingAMemoryLeak|. Counterintuitively, LSan is
+    // disabled when trying to detect a leak ro avoid reporting it twice.
+    // See |FinishExecutionImpl| for where the actual LSan pass is done.
+    DetectingLeaks = Flags & kLeakDetection;
+    if (DetectingLeaks && EF->__lsan_disable)
+      EF->__lsan_disable();
+    if (EF->__msan_scoped_enable_interceptor_checks)
+      EF->__msan_scoped_enable_interceptor_checks();
+    AllocMonitorStartTracing(Options.TraceMalloc);
+    for (auto &M : Modules) {
+      memset(M.Counters, 0, M.CountersLen);
+    }
+    FuzzerProxyExecutionStarted(PID);
+    Mutex.unlock();
+  }
+
+  // Complete a fuzzing iteration. If leak detection was requested in
+  // |StartExecution| and a leak was detected, this will trigger a
+  // |FuzzerLeakCallback|. Otherwise, it will notify the |FuzzerProxy| in the
+  // fuzzing engine process that it may read and/or modify the shared counter
+  // memory.
+  static void FinishExecution() {
+    assert(R);
+    R->FinishExecutionImpl();
+  }
+
+  void FinishExecutionImpl() ACQUIRE(Mutex) {
+    Mutex.lock();
+    int HasMoreMallocsThanFrees = AllocMonitorStopTracing() ? 1 : 0;
+    if (EF->__msan_scoped_disable_interceptor_checks)
+      EF->__msan_scoped_disable_interceptor_checks();
+    bool LeakDetected = false;
+    if (DetectingLeaks) {
+      if (EF->__lsan_enable)
+        EF->__lsan_enable();
+      // See |Fuzzer::TryDetectingAMemoryLeak|. This is the actual LSan pass.
+      if (HasMoreMallocsThanFrees && EF->__lsan_do_recoverable_leak_check)
+        LeakDetected = EF->__lsan_do_recoverable_leak_check();
+      DetectingLeaks = false;
+    }
+    if (LeakDetected) {
+      FuzzerLeakCallback(PID);
+    } else {
+      AllocMonitorPurge();
+      FuzzerProxyExecutionFinished(PID, HasMoreMallocsThanFrees);
+    }
+  }
+
+  // Perform end-of-process leak detection. If a leak is detected, it will
+  // trigger a |FuzzerLeakCallback|.
+  static void DetectLeaksAtExit() {
+    assert(R);
+    R->DetectLeaksAtExitImpl();
+  }
+
+  void DetectLeaksAtExitImpl() {
+    if (EF->__lsan_enable)
+      EF->__lsan_enable();
+    if (EF->__lsan_do_recoverable_leak_check &&
+        EF->__lsan_do_recoverable_leak_check())
+      FuzzerLeakCallback(PID);
+  }
+
+private:
+  // General parameters for the remote process.
+  unsigned long PID;
+  FuzzingOptions Options;
+
+  // Per-fuzzing iteration options.
+  bool FirstIteration = true;
+  bool DetectingLeaks = false;
+
+  // Describes the mutable portion of a module's coverage.
+  std::mutex Mutex;
+  struct Module {
+    uint8_t *Counters;
+    size_t CountersLen;
+  };
+  std::vector<Module> Modules GUARDED_BY(Mutex);
+
+  FUZZER_NO_COPY_OR_MOVE(Remote);
+};
+
+// |ExternalFunctions| stipulates that it should not be constructed before
+// calling |main|, in order to guarantee all of the module constructors execute
+// first. This is straightforward when using libFuzzer's |main|.  When fuzzing
+// remote processes, however, it is desirable to require no additional
+// modifications beyond adding the sanitizer-coverage instrumentation and
+// linking against the fuzzer-remote runtime. To achieve this, the |Remote|
+// constructor uses a priority attribute to ensure it is run last.
+[[gnu::init_priority(0xffff)]] Remote RemoteSingleton;
+
+} // namespace
+
+// Storage for global ExternalFunctions object.
+ExternalFunctions *EF = nullptr;
+
+} // namespace fuzzer
+
+// libFuzzerRemote's implementation of the FuzzerRemote* interface.
+//
+// Note that the process ID parameter is ignored in the remote process. As
+// described in FuzzerRemoteInterface.h, when either a FuzzerRemote in a remote
+// process calls |FuzzerProxy*| methods, or the FuzzerProxy in the engine
+// process calls |FuzzerRemote*| methods, an OS-specific IPC transport library
+// transparently forwards the IPC. The PID is necessary for routing the IPC in
+// the engine process and is superfluous here. It is kept as a parameter here to
+// keep a single interface definition.
+
+extern "C" {
+
+ATTRIBUTE_INTERFACE
+void __sanitizer_cov_8bit_counters_init(uint8_t *Start, uint8_t *Stop) {
+  fuzzer::Remote::AddCounters(Start, Stop);
+}
+
+ATTRIBUTE_INTERFACE
+void __sanitizer_cov_pcs_init(const uintptr_t *Start, const uintptr_t *Stop) {
+  fuzzer::Remote::AddPCs(Start, Stop);
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteStartExecution(unsigned long PID,
+                                                    uint32_t Flags) {
+  fuzzer::Remote::StartExecution(Flags);
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteFinishExecution(unsigned long PID) {
+  fuzzer::Remote::FinishExecution();
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintPC(unsigned long PID,
+                                             const char *SymbolizedFMT,
+                                             const char *FallbackFMT,
+                                             uintptr_t PC) {
+  fuzzer::PrintPC(SymbolizedFMT, FallbackFMT, PC);
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDescribePC(unsigned long PID,
+                                                const char *SymbolizedFMT,
+                                                uintptr_t PC, char *Desc,
+                                                size_t DescLen) {
+  std::string Description = fuzzer::DescribePC(SymbolizedFMT, PC);
+  snprintf(Desc, DescLen, "%s", Description.c_str());
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintStackTrace(unsigned long PID) {
+  fuzzer::PrintStackTrace();
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintMemoryProfile(unsigned long PID) {
+  fuzzer::PrintMemoryProfile();
+}
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDetectLeaksAtExit(unsigned long PID) {
+  fuzzer::Remote::DetectLeaksAtExit();
+}
+
+// These callbacks should never be triggered in a remote process.
+ATTRIBUTE_INTERFACE void FuzzerAlarmCallback() { _Exit(1); }
+
+ATTRIBUTE_INTERFACE void FuzzerFileSizeExceedCallback() { _Exit(1); }
+
+ATTRIBUTE_INTERFACE void FuzzerGracefulExitCallback() { _Exit(1); }
+
+ATTRIBUTE_INTERFACE void FuzzerInterruptCallback() { _Exit(1); }
+
+} // extern "C"

compiler-rt/lib/fuzzer/FuzzerRemoteInterface.h

+//===- FuzzerRemoteInterface.h ----------------------------------*- C++ -* ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// Defines the interface between libFuzzer and a remote process being tested.
+//===----------------------------------------------------------------------===//
+
+// When Options.Remote=1, libFuzzer acts as a fuzzing "proxy" that can be used
+// to fuzz multiple processes ("remotes") communicating via IPC.
+
+// The interface methods below define method calls in both directions: libFuzzer
+// (or more specifically, the "fuzzer" runtime) can call FuzzerRemote* methods
+// on remote processes, and the remote processes (or more specifically, the
+// "fuzzer_remote" runtime) can call FuzzerProxy* methods on libFuzzer, i.e.:
+//
+//  +-- -----+ -R-> +-----------+       +------------+ -R-> +---------------+
+//  | fuzzer |      | ipc_proxy | <-I-> | ipc_remote |      | fuzzer_remote |
+//  +--------+ <-P- +-----------+       +------------+ <-P- +---------------+
+//
+//   (P) are calls to FuzzerProxy* interface methods.
+//   (R) are calls to FuzzerRemote* interface methods.
+//   (I) is platform-specific IPC.
+
+// The "ipc_proxy" and "ipc_remote" libraries above are platform-provided
+// libraries that should use platform-specific IPC mechanisms to implement these
+// same interfaces.
+
+// WARNING: keep the interfaces in C (external interfaces need a C ABI).
+
+#ifndef LLVM_FUZZER_REMOTE_INTERFACE_H
+#define LLVM_FUZZER_REMOTE_INTERFACE_H
+
+#include "FuzzerPlatform.h"
+#include <stddef.h>
+#include <stdint.h>
+
+#ifdef __cplusplus
+#include "FuzzerOptions.h"
+#include <limits>
+
+extern "C" {
+#endif // __cplusplus
+
+// Initializes the proxy side of the IPC layer to accept incoming connections
+// from remote prcoesses.
+ATTRIBUTE_INTERFACE void FuzzerAcceptRemotes();
+
+// Disconnect all remotes and tear down the IPC layer. This MUST be called
+// before the FuzzerProxy destructor is invoked.
+ATTRIBUTE_INTERFACE void FuzzerShutdownRemotes();
+
+// FuzzerProxy* functions are implemented twice:
+//  1. The platform-specific IPC front-end must provide an implementation that
+//     can be called by libFuzzerRemote. PIDs can be ignored.
+//  2. libFuzzer provides an implementation that can be called by the platform-
+//     specific IPC back-end. PIDs are used for identification.
+// Functions are synchronous unless otherwise noted; i.e. it can be safely
+// assumed that the remote method has returned when the function returns.
+
+// Connects a new remote process. Processes are only connected during the user
+// callback, i.e. as part of a call to |LLVMFuzzerTestOneInput|. Requests to
+// connect received outside this window will be deferred to the next iteration.
+ATTRIBUTE_INTERFACE void FuzzerProxyConnect(unsigned long PID, void *Options,
+                                            size_t OptionsLen);
+
+// Adds coverage memory regions and returns an index that only depends on the
+// relative offsets of the PCs, i.e. adding another set of PCs that match a
+// previous set except for a fixed offset will return the same index. If adding
+// coverage fails, it returns |fuzzer::kInvalidIdx| instead.
+//
+// NOTE: The counter memory is added to the TracePC and must remain mapped until
+// the fuzzer exits. It may be replaced by an equivalent memory region, but it
+// cannot be unmapped. The PC memory may be safely released after this call
+// completes. The returned index can be used by the IPC layer to track the
+// counter memory. The FuzzerRemote should ignore the index.
+ATTRIBUTE_INTERFACE uintptr_t FuzzerProxyAddCoverage(unsigned long PID,
+                                                     uint8_t *CountersBegin,
+                                                     uint8_t *CountersEnd,
+                                                     const uintptr_t *PCsBegin,
+                                                     const uintptr_t *PCsEnd);
+
+// See also FuzzerRemoteStartExecution, below.
+ATTRIBUTE_INTERFACE void FuzzerProxyExecutionStarted(unsigned long PID);
+
+// See also FuzzerRemoteFinishExecution, below.
+ATTRIBUTE_INTERFACE void
+FuzzerProxyExecutionFinished(unsigned long PID, int HasMoreMallocsThanFrees);
+
+ATTRIBUTE_INTERFACE void FuzzerProxyDisconnect(unsigned long PID);
+
+// FuzzerRemote* functions are implemented twice:
+//  1. The platform-specific IPC back-end must provide an implementation that
+//     can be called by libFuzzer. PIDs can be used for routing.
+//  2. libFuzzerRemote provides an implementation that can be called by the
+//     platform-specific IPC front-end. PIDs are ignored.
+// Functions are synchronous unless otherwise noted; i.e. it can be safely
+// assumed that the proxy method has returned when the function returns.
+
+// Asynchronous; callers should wait for a corresponding call to
+// FuzzerProxyExecutionStarted.
+ATTRIBUTE_INTERFACE void FuzzerRemoteStartExecution(unsigned long PID,
+                                                    uint32_t Flags);
+
+// Asynchronous; callers should wait for a corresponding call to
+// FuzzerProxyExecutionFinished.
+ATTRIBUTE_INTERFACE void FuzzerRemoteFinishExecution(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintPC(unsigned long PID,
+                                             const char *SymbolizedFMT,
+                                             const char *FallbackFMT,
+                                             uintptr_t PC);
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDescribePC(unsigned long PID,
+                                                const char *SymbolizedFMT,
+                                                uintptr_t PC, char *Desc,
+                                                size_t DescLen);
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintStackTrace(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemotePrintMemoryProfile(unsigned long PID);
+
+ATTRIBUTE_INTERFACE void FuzzerRemoteDetectLeaksAtExit(unsigned long PID);
+
+#ifdef __cplusplus
+} // extern "C"
+
+namespace fuzzer {
+
+// Subset of fuzzer::FuzzingOptions that is forwarded from the FuzzerProxy to
+// the FuzzerRemotes upon connection.
+struct ProxiedOptions final {
+public:
+  void Pack(const FuzzingOptions &Options);
+  void Unpack(FuzzingOptions *Options) const;
+
+private:
+  void Swap(FuzzingOptions *Options);
+
+  int Verbosity;
+  int OOMExitCode;
+  int InterruptExitCode;
+  int ErrorExitCode;
+  int RssLimitMb;
+  int MallocLimitMb;
+  int PurgeAllocatorIntervalSec;
+  int UseValueProfile;
+  int TraceMalloc;
+  uint32_t BitFlags;
+};
+
+// Index indicating a call to FuzzerProxyAddCoverage failed.
+constexpr uintptr_t kInvalidIdx = std::numeric_limits<uintptr_t>::max();
+
+// Flags that can be sent as part of FuzzerRemoteStartExecution.
+constexpr uint32_t kLeakDetection = 1 << 0;
+
+} // namespace fuzzer
+
+#endif // __cplusplus
+#endif // LLVM_FUZZER_REMOTE_INTERFACE_H

compiler-rt/lib/fuzzer/build.sh

ar r libFuzzer.a \
   FuzzerDeprecated.o \
   FuzzerExtFunctionsDlsym.o \
   FuzzerExtFunctionsWeak.o \
   FuzzerExtFunctionsWindows.o \
   FuzzerIO.o \
   FuzzerIOPosix.o \
   FuzzerIOWindows.o \
   FuzzerMonitor.o \
+  FuzzerProxiedOptions.o \
   FuzzerUtil.o \
   FuzzerUtilDarwin.o \
   FuzzerUtilFuchsia.o \
   FuzzerUtilLinux.o \
   FuzzerUtilPosix.o \
   FuzzerUtilWindows.o \
   FuzzerCrossOver.o \
   FuzzerDataFlowTrace.o \
   FuzzerDriver.o \
   FuzzerExtraCounters.o \
   FuzzerFork.o \
   FuzzerLoop.o \
   FuzzerMain.o \
   FuzzerMerge.o \
   FuzzerModuleRelative.o \
   FuzzerMonitor.o \
   FuzzerMutate.o \
   FuzzerSHA1.o \
   FuzzerTracePC.o
 
+rm -f libFuzzerRemote.a
+ar r libFuzzerRemote.a \
+  FuzzerDeprecated.o \
+  FuzzerExtFunctionsDlsym.o \
+  FuzzerExtFunctionsWeak.o \
+  FuzzerExtFunctionsWindows.o \
+  FuzzerIO.o \
+  FuzzerIOPosix.o \
+  FuzzerIOWindows.o \
+  FuzzerMonitor.o \
+  FuzzerProxiedOptions.o \
+  FuzzerUtil.o \
+  FuzzerUtilDarwin.o \
+  FuzzerUtilFuchsia.o \
+  FuzzerUtilLinux.o \
+  FuzzerUtilPosix.o \
+  FuzzerUtilWindows.o \
+  FuzzerRemote.o
+
 rm -f Fuzzer*.o

compiler-rt/lib/fuzzer/tests/CMakeLists.txt

OBJECT_LIBS RTfuzzer_base
                 RTfuzzer)
 
   generate_libfuzzer_unittests(FuzzedDataProviderUnitTests ${arch}
     NAME FuzzerUtils-${arch}-Test
     SOURCES FuzzedDataProviderUnittest.cpp
     OBJECT_LIBS RTfuzzer_base
                 RTfuzzer
     DEPS ${COMPILER_RT_SOURCE_DIR}/include/fuzzer/FuzzedDataProvider.h)
+
+  generate_libfuzzer_unittests(FuzzerRemoteUnitTests ${arch}
+    SOURCES FuzzerRemoteUnittest.cpp
+    OBJECT_LIBS RTfuzzer_base
+                RTfuzzer_remote)
 endif()

compiler-rt/lib/fuzzer/tests/FuzzerRemoteUnittest.cpp

+//===- FuzzerRemoteUnittest.cpp -------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+// Unit tests of FuzzerRemote.
+//
+// This is kept separate from the other unit tests as it must link against the
+// fuzzer_remote runtime, which has symbols that conflict with the fuzzer
+// runtime, e.g. __sanitizer_cov_8bit_counters_init.
+//===----------------------------------------------------------------------===//
+
+// #include "FuzzerExtFunctions.h"
+#include "FuzzerMonitor.h"
+#include "FuzzerRemoteInterface.h"
+// #include "FuzzerTracePC.h"
+// #include "FuzzerUtil.h"
+#include "tests/FuzzerTestUtil.h"
+#include "gtest/gtest.h"
+// #include <cstddef>
+// #include <cstdint>
+
+#include "FuzzerPlatform.h"
+
+namespace fuzzer {
+
+// Keep this small so real allocations succeed.
+static const size_t kMallocLimitMb = 1;
+
+// A collection of variables used to record the parameters to the interface
+// functions below.
+static struct {
+  unsigned long PID;
+  uint8_t *CountersBegin;
+  uint8_t *CountersEnd;
+  const uintptr_t *PCsBegin;
+  const uintptr_t *PCsEnd;
+  int HasMoreMallocsThanFrees;
+  size_t LSanEnables;
+  size_t LSanChecks;
+  size_t LSanDisables;
+  size_t LeakCallbacks;
+  size_t MallocLimitCallbackSize;
+
+  bool HasChanged() {
+    return PID != 0 || CountersBegin != nullptr || CountersEnd != nullptr ||
+           PCsBegin != nullptr || PCsEnd != nullptr ||
+           HasMoreMallocsThanFrees != -1 || LeakCallbacks != 0;
+  }
+  void Reset() {
+    PID = 0;
+    CountersBegin = nullptr;
+    CountersEnd = nullptr;
+    PCsBegin = nullptr;
+    PCsEnd = nullptr;
+    HasMoreMallocsThanFrees = -1;
+    LSanEnables = 0;
+    LSanChecks = 0;
+    LSanDisables = 0;
+    LeakCallbacks = 0;
+    MallocLimitCallbackSize = 0;
+  }
+
+} Last;
+
+} // namespace fuzzer
+
+// The remote library expects to be able to call the IPC layer's implementation
+// of these interface functions.
+extern "C" {
+
+void FuzzerProxyConnect(unsigned long PID, void *Options, size_t OptionsLen) {
+  fuzzer::FuzzingOptions Defaults; // default values.
+  fuzzer::ProxiedOptions Proxied;
+  Defaults.MallocLimitMb = fuzzer::kMallocLimitMb;
+  Proxied.Pack(Defaults);
+  ASSERT_EQ(OptionsLen, sizeof(Proxied));
+  memcpy(Options, &Proxied, OptionsLen);
+}
+
+uintptr_t FuzzerProxyAddCoverage(unsigned long PID, uint8_t *CountersBegin,
+                                 uint8_t *CountersEnd,
+                                 const uintptr_t *PCsBegin,
+                                 const uintptr_t *PCsEnd) {
+  fuzzer::Last.PID = PID;
+  fuzzer::Last.CountersBegin = CountersBegin;
+  fuzzer::Last.CountersEnd = CountersEnd;
+  fuzzer::Last.PCsBegin = PCsBegin;
+  fuzzer::Last.PCsEnd = PCsEnd;
+  return fuzzer::kInvalidIdx;
+}
+
+void FuzzerProxyExecutionStarted(unsigned long PID) { fuzzer::Last.PID = PID; }
+
+void FuzzerProxyExecutionFinished(unsigned long PID,
+                                  int HasMoreMallocsThanFrees) {
+  fuzzer::Last.PID = PID;
+  fuzzer::Last.HasMoreMallocsThanFrees = HasMoreMallocsThanFrees;
+}
+
+// The monitoring callbacks should not be called as part of the unit tests...
+void FuzzerCrashSignalCallback(unsigned long PID) { FAIL(); }
+void FuzzerDeathCallback() { FAIL(); }
+void FuzzerRssLimitCallback(unsigned long PID) { FAIL(); }
+
+// ...with the exceptions of:
+// * The exit callback, invoked on normal exit.
+// * The leak callback, invoked by LSan.
+// * The malloc limit callback, invoked by the allocation monitor.
+void FuzzerExitCallback(unsigned long PID) {}
+void FuzzerLeakCallback(unsigned long PID) { fuzzer::Last.LeakCallbacks++; }
+void FuzzerMallocLimitCallback(unsigned long PID, size_t Size) {
+  fuzzer::Last.MallocLimitCallbackSize = Size;
+}
+
+// Defined by the remote library, but not declared in any headers.
+void __sanitizer_cov_8bit_counters_init(uint8_t *Start, uint8_t *Stop);
+void __sanitizer_cov_pcs_init(const uintptr_t *Start, const uintptr_t *Stop);
+
+} // extern "C"
+
+namespace fuzzer {
+
+// Test fixtures
+
+class RemoteTest : public ::testing::Test {
+protected:
+  void SetUp() override {
+    PID = GetPid();
+    Last.Reset();
+  }
+
+  unsigned long PID;
+};
+
+// Instances of this class replace LSan, if present, with simple counters to
+// track calls. LSan function pointers are restored upon destruction.
+class FakeLSan {
+public:
+  explicit FakeLSan(bool Available)
+      : RealEnable(EF->__lsan_enable),
+        RealCheck(EF->__lsan_do_recoverable_leak_check),
+        RealDisable(EF->__lsan_disable),
+        RealHookInstaller(EF->__sanitizer_install_malloc_and_free_hooks),
+        RealPurgeAllocator(EF->__sanitizer_purge_allocator) {
+    assert(!Current);
+    Current = this;
+    SetAvailable(Available);
+    EF->__sanitizer_purge_allocator = nullptr;
+    HookMallocAndFree();
+  }
+
+  void HookMallocAndFree() {
+    // Reset the allocation monitor and check if it can detect malloc/free
+    // mismatches.
+    AllocMonitorStopTracing();
+    AllocMonitorStartTracing(0);
+    auto *Tmp = malloc(1);
+    bool HasMoreMallocsThanFrees = AllocMonitorStopTracing();
+    free(Tmp);
+
+    // If so, nothing more needs to be done.
+    if (HasMoreMallocsThanFrees)
+      return;
+
+    // Otherwise, the hooks aren't being called. This could be because there is
+    // no hook installer at all (e.g. no "-fsanitize=..."), or because the test
+    // has sanitizer instrumentation but no sanitizer (e.g.
+    // "-fsanitize=fuzzer-no-link" without "-fsanitize=address"). Either way,
+    // replace the hook installer and reconfigure the allocation monitor to get
+    // hooks this object can call directly.
+    EF->__sanitizer_install_malloc_and_free_hooks = InstallHooks;
+    ShouldCallHooks = true;
+    FuzzingOptions Defaults;
+    Defaults.MallocLimitMb = kMallocLimitMb;
+    AllocMonitorConfigure(Defaults);
+  }
+
+  ~FakeLSan() {
+    assert(Current == this);
+    EF->__lsan_enable = RealEnable;
+    EF->__lsan_do_recoverable_leak_check = RealCheck;
+    EF->__lsan_disable = RealDisable;
+    EF->__sanitizer_install_malloc_and_free_hooks = RealHookInstaller;
+    EF->__sanitizer_purge_allocator = RealPurgeAllocator;
+    Current = nullptr;
+  }
+
+  void SetAvailable(bool Available) {
+    if (!Available) {
+      EF->__lsan_enable = nullptr;
+      EF->__lsan_do_recoverable_leak_check = nullptr;
+      EF->__lsan_disable = nullptr;
+    } else {
+      EF->__lsan_enable = Enable;
+      EF->__lsan_do_recoverable_leak_check = Check;
+      EF->__lsan_disable = Disable;
+    }
+  }
+
+  static int InstallHooks(void (*MallocHook)(const volatile void *, size_t),
+                          void (*FreeHook)(const volatile void *)) {
+    assert(Current);
+    Current->OnMalloc = MallocHook;
+    Current->OnFree = FreeHook;
+    return 0;
+  }
+
+  void *Malloc(size_t Size) {
+    void *Ptr = malloc(Size);
+    if (ShouldCallHooks)
+      OnMalloc(Ptr, Size);
+    return Ptr;
+  }
+
+  void Free(void *Ptr) {
+    if (ShouldCallHooks)
+      OnFree(Ptr);
+    free(Ptr);
+  }
+
+  static void Enable() { Last.LSanEnables++; }
+
+  void SetLeakDetected(bool Detected) { LeakDetected = Detected; }
+
+  static int Check() {
+    assert(Current);
+    Last.LSanChecks++;
+    return Current->LeakDetected;
+  }
+
+  static void Disable() { Last.LSanDisables++; }
+
+private:
+  static FakeLSan *Current;
+
+  void (*RealEnable)();
+  int (*RealCheck)();
+  void (*RealDisable)();
+  int (*RealHookInstaller)(void (*MallocHook)(const volatile void *, size_t),
+                           void (*FreeHook)(const volatile void *));
+  void (*RealPurgeAllocator)();
+  void (*OnMalloc)(const volatile void *ptr, size_t size);
+  void (*OnFree)(const volatile void *ptr);
+
+  bool ShouldCallHooks = false;
+  bool LeakDetected = false;
+};
+
+FakeLSan *FakeLSan::Current = nullptr;
+
+// Unit tests for the remote interface.
+
+TEST_F(RemoteTest, SanitizerCovInit) {
+  FakeModule<0x100> M1;
+  FakeModule<0x200> M2;
+  FakeModule<0x300> M3;
+
+  // Invalid arguments are ignored.
+  Last.Reset();
+  __sanitizer_cov_8bit_counters_init(nullptr, M1.CountersEnd());
+  __sanitizer_cov_pcs_init(nullptr, M1.PCsEnd());
+  EXPECT_FALSE(Last.HasChanged());
+
+  Last.Reset();
+  __sanitizer_cov_8bit_counters_init(M1.Counters, nullptr);
+  __sanitizer_cov_pcs_init(M1.PCs, nullptr);
+  EXPECT_FALSE(Last.HasChanged());
+
+  Last.Reset();
+  __sanitizer_cov_8bit_counters_init(M1.Counters, M1.Counters);
+  __sanitizer_cov_pcs_init(M1.PCs, M1.PCs);
+  EXPECT_FALSE(Last.HasChanged());
+
+  Last.Reset();
+  __sanitizer_cov_8bit_counters_init(M1.CountersEnd(), M1.Counters);
+  __sanitizer_cov_pcs_init(M1.PCsEnd(), M1.PCs);
+  EXPECT_FALSE(Last.HasChanged());
+
+  // Counters and PCs are added together.
+  // Send two counter regions; nothing should happen without PC regions.
+  Last.Reset();
+  __sanitizer_cov_8bit_counters_init(M1.Counters, M1.CountersEnd());
+  __sanitizer_cov_8bit_counters_init(M2.Counters, M2.CountersEnd());
+  EXPECT_FALSE(Last.HasChanged());
+
+  // Should match the first counter region.
+  __sanitizer_cov_pcs_init(M1.PCs, M1.PCsEnd());
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.CountersBegin, M1.Counters);
+  EXPECT_EQ(Last.CountersEnd, M1.CountersEnd());
+  EXPECT_EQ(Last.PCsBegin, M1.PCs);
+  EXPECT_EQ(Last.PCsEnd, M1.PCsEnd());
+
+  // Should match the second counter region.
+  Last.Reset();
+  __sanitizer_cov_pcs_init(M2.PCs, M2.PCsEnd());
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.CountersBegin, M2.Counters);
+  EXPECT_EQ(Last.CountersEnd, M2.CountersEnd());
+  EXPECT_EQ(Last.PCsBegin, M2.PCs);
+  EXPECT_EQ(Last.PCsEnd, M2.PCsEnd());
+
+  // Nothing should happen without another counter region.
+  Last.Reset();
+  __sanitizer_cov_pcs_init(M3.PCs, M3.PCsEnd());
+  EXPECT_FALSE(Last.HasChanged());
+
+  // Should match the third PC region.
+  __sanitizer_cov_8bit_counters_init(M3.Counters, M3.CountersEnd());
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.CountersBegin, M3.Counters);
+  EXPECT_EQ(Last.CountersEnd, M3.CountersEnd());
+  EXPECT_EQ(Last.PCsBegin, M3.PCs);
+  EXPECT_EQ(Last.PCsEnd, M3.PCsEnd());
+}
+
+TEST_F(RemoteTest, StartAndFinishExecution) {
+  FakeLSan LSan(/* Available= */ true);
+  FuzzerRemoteFinishExecution(PID);
+
+  // No leaks or detection. This should call |FuzzerProxyExecutionFinished| with
+  // |HasMoreMallocsThanFrees == 0|.
+  Last.Reset();
+  FuzzerRemoteStartExecution(PID, 0);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.LSanDisables, 0U);
+
+  Last.Reset();
+  FuzzerRemoteFinishExecution(PID);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.HasMoreMallocsThanFrees, 0);
+  EXPECT_EQ(Last.LSanEnables, 0U);
+  EXPECT_EQ(Last.LSanChecks, 0U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // Wuth leak detection, but no leaks. This should call
+  // |FuzzerProxyExecutionFinished| with |HasMoreMallocsThanFrees == 0|.
+  Last.Reset();
+  FuzzerRemoteStartExecution(PID, kLeakDetection);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.LSanDisables, 1U);
+
+  Last.Reset();
+  FuzzerRemoteFinishExecution(PID);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.HasMoreMallocsThanFrees, 0);
+  EXPECT_EQ(Last.LSanEnables, 1U);
+  EXPECT_EQ(Last.LSanChecks, 0U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // With leak, but no leak detection. LSan is available, so this should call
+  // |FuzzerProxyExecutionFinished| with |HasMoreMallocsThanFrees != 0|.
+  // The "leak" isn't really a leak, but it's enough for the allocation monitor
+  // to report more mallocs and frees.
+  Last.Reset();
+  FuzzerRemoteStartExecution(PID, 0);
+  EXPECT_EQ(Last.PID, PID);
+  auto *ExtraMalloc = LSan.Malloc(1);
+  LSan.SetLeakDetected(true);
+  EXPECT_EQ(Last.LSanDisables, 0U);
+
+  Last.Reset();
+  FuzzerRemoteFinishExecution(PID);
+  LSan.Free(ExtraMalloc);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.HasMoreMallocsThanFrees, 1);
+  EXPECT_EQ(Last.LSanEnables, 0U);
+  EXPECT_EQ(Last.LSanChecks, 0U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // With leak detection and more mallocs than frees, but LSan determines that
+  // there isn't any leaks. This should call |FuzzerProxyExecutionFinished|
+  // with |HasMoreMallocsThanFrees != 0|.
+  Last.Reset();
+  FuzzerRemoteStartExecution(PID, kLeakDetection);
+  EXPECT_EQ(Last.PID, PID);
+  ExtraMalloc = LSan.Malloc(1);
+  LSan.SetLeakDetected(false);
+  EXPECT_EQ(Last.LSanDisables, 1U);
+
+  Last.Reset();
+  FuzzerRemoteFinishExecution(PID);
+  LSan.Free(ExtraMalloc);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.HasMoreMallocsThanFrees, 1);
+  EXPECT_EQ(Last.LSanEnables, 1U);
+  EXPECT_EQ(Last.LSanChecks, 1U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // With leak and detection. LSan is NOT available, so this should call
+  // |FuzzerProxyExecutionFinished| with |HasMoreMallocsThanFrees != 0|.
+  Last.Reset();
+  LSan.SetAvailable(false);
+  FuzzerRemoteStartExecution(PID, 0);
+  EXPECT_EQ(Last.PID, PID);
+  ExtraMalloc = LSan.Malloc(1);
+  LSan.SetLeakDetected(true);
+  EXPECT_EQ(Last.LSanDisables, 0U);
+
+  Last.Reset();
+  FuzzerRemoteFinishExecution(PID);
+  LSan.Free(ExtraMalloc);
+  EXPECT_EQ(Last.PID, PID);
+  EXPECT_EQ(Last.HasMoreMallocsThanFrees, 1);
+  EXPECT_EQ(Last.LSanEnables, 0U);
+  EXPECT_EQ(Last.LSanChecks, 0U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // With an oversized malloc. This should work with the allocation monitor
+  // even if LSan isn't available.
+  size_t Size = (kMallocLimitMb << 20) + 1;
+  Last.Reset();
+  FuzzerRemoteStartExecution(PID, 0);
+  EXPECT_EQ(Last.PID, PID);
+  ExtraMalloc = LSan.Malloc(Size);
+  LSan.Free(ExtraMalloc);
+  EXPECT_EQ(Last.LSanDisables, 0U);
+  EXPECT_EQ(Last.MallocLimitCallbackSize, Size);
+}
+
+TEST_F(RemoteTest, DescribePC) {
+  // "Placeholders" from sanitizer_common/sanitizer_stacktrace_printer.h
+  char SymbolizedFMT[3] = {'%', '_', '\0'};
+  constexpr const char FormatChars[] = "npmofqslcFSLM";
+
+  for (size_t i = 0; i < strlen(FormatChars); ++i) {
+    SCOPED_TRACE(FormatChars[i]);
+    SymbolizedFMT[1] = FormatChars[i];
+    uintptr_t PC = 0xdeadbeef;
+
+    std::string Expected = DescribePC(SymbolizedFMT, PC);
+    char Actual[256];
+    FuzzerRemoteDescribePC(PID, SymbolizedFMT, PC, Actual, sizeof(Actual));
+
+    EXPECT_STREQ(Expected.c_str(), Actual);
+  }
+}
+
+TEST_F(RemoteTest, DetectLeaksAtExit) {
+  // No LSan
+  FakeLSan LSan(/* Available*/ false);
+  Last.Reset();
+  FuzzerRemoteDetectLeaksAtExit(PID);
+  EXPECT_EQ(Last.LSanChecks, 0U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // LSan, but no leaks
+  LSan.SetAvailable(true);
+  Last.Reset();
+  FuzzerRemoteDetectLeaksAtExit(1);
+  EXPECT_EQ(Last.LSanChecks, 1U);
+  EXPECT_EQ(Last.LeakCallbacks, 0U);
+
+  // With LSan and leaks
+  LSan.SetLeakDetected(true);
+  Last.Reset();
+  FuzzerRemoteDetectLeaksAtExit(PID);
+  EXPECT_EQ(Last.LSanChecks, 1U);
+  EXPECT_EQ(Last.LeakCallbacks, 1U);
+}
+
+} // namespace fuzzer
+
+int main(int argc, char **argv) {
+  testing::InitGoogleTest(&argc, argv);
+  testing::AddGlobalTestEnvironment(new fuzzer::TestEnvironment());
+  return RUN_ALL_TESTS();
+}

compiler-rt/test/fuzzer/CMakeLists.txt

 set(FUZZER_TEST_ARCH ${FUZZER_SUPPORTED_ARCH})
 if (APPLE)
   darwin_filter_host_archs(FUZZER_SUPPORTED_ARCH FUZZER_TEST_ARCH)
 endif()
 
 if(COMPILER_RT_INCLUDE_TESTS)
   list(APPEND LIBFUZZER_TEST_DEPS FuzzerUnitTests)
   list(APPEND LIBFUZZER_TEST_DEPS FuzzedDataProviderUnitTests)
+  list(APPEND LIBFUZZER_TEST_DEPS FuzzerRemoteUnitTests)
 endif()
 
 add_custom_target(check-fuzzer)
 
 if(COMPILER_RT_INCLUDE_TESTS)
   # libFuzzer unit tests.
   configure_lit_site_cfg(
     ${CMAKE_CURRENT_SOURCE_DIR}/unit/lit.site.cfg.py.in