[fuzzer][fuchsia] Close exception channel before exiting.

[aarongreen]

Summary

On Fuchsia, killing or exiting a process that has a thread listening to its own process's debugger exception channel can hang. Zircon may kill all the threads, send a synthetic exceptions to debugger, and wait for the debugger to have received them. This means the thread listening to the debug exception channel may be killed even as Zircon is waiting for that thread to drain the exception channel, and the process can become stuck in a half-dead state.

This situation is "weird" as it only arises when a process is trying to debug itself. Unfortunately, this is exactly the scenario for libFuzzer on Fuchsia: FuzzerUtilFuchsia spawns a crash-handling thread that acts like a debugger in order to be able to rewrite the crashed threads stack and resume them into libFuzzer's usual POSIX signal handlers. In practice, approximately 25% of fuzzers appear to hang on exit, after generating output and artifacts. These processes hang around until the platform is torn done, which is typically a ClusterFuzz VM. Thus, real-world impact has been somewhat mitigated. The issue should still be resolved for local users, though.

This change improves the behavior of exit() in libFuzzer by adding an atexit handler which closes an event shared with the crash handling thread. This signals to the crash handler that it should close the exception channel and be joined before the process actually exits.

compiler-rt/lib/fuzzer/FuzzerUtilFuchsia.cpp

 // Forward declaration of assembly trampoline needed to resume crashed threads.
 // This appears to have external linkage to  C++, which is why it's not in the
 // anonymous namespace.  The assembly definition inside MakeTrampoline()
 // actually defines the symbol with internal linkage only.
 void CrashTrampolineAsm() __asm__("CrashTrampolineAsm");
 
 namespace {
 
+// The signal handler thread uses Zircon exceptions to resume crashed threads
+// into libFuzzer's POSIX signal handlers. The associated event is used to
+// signal when the thread is running, and when it should stop.
+std::thread SignalHandler;
+zx_handle_t SignalHandlerEvent = ZX_HANDLE_INVALID;
+
 // Helper function to handle Zircon syscall failures.
 void ExitOnErr(zx_status_t Status, const char *Syscall) {
   if (Status != ZX_OK) {
     Printf("libFuzzer: %s failed: %s\n", Syscall,
            _zx_status_get_string(Status));
     exit(1);
   }
 }
".cfi_startproc\n"
       : // No outputs
       : FOREACH_REGISTER(ASM_OPERAND_REG, ASM_OPERAND_NUM)
 #if defined(__aarch64__)
         ASM_OPERAND_REG(pc) ASM_OPERAND_REG(lr)
 #endif
         [StaticCrashHandler] "i"(StaticCrashHandler));
 }
 
-void CrashHandler(zx_handle_t *Event) {
+void CrashHandler() {
+  assert(SignalHandlerEvent != ZX_HANDLE_INVALID);
+
   // This structure is used to ensure we close handles to objects we create in
   // this handler.
   struct ScopedHandle {
     ~ScopedHandle() { _zx_handle_close(Handle); }
     zx_handle_t Handle = ZX_HANDLE_INVALID;
   };
 
   // Create the exception channel.  We need to claim to be a "debugger" so the
   // kernel will allow us to modify and resume dying threads (see below). Once
   // the channel is set, we can signal the main thread to continue and wait
   // for the exception to arrive.
   ScopedHandle Channel;
   zx_handle_t Self = _zx_process_self();
   ExitOnErr(_zx_task_create_exception_channel(
                 Self, ZX_EXCEPTION_CHANNEL_DEBUGGER, &Channel.Handle),
             "_zx_task_create_exception_channel");
 
-  ExitOnErr(_zx_object_signal(*Event, 0, ZX_USER_SIGNAL_0),
+  ExitOnErr(_zx_object_signal(SignalHandlerEvent, 0, ZX_USER_SIGNAL_0),
             "_zx_object_signal");
 
   // This thread lives as long as the process in order to keep handling
   // crashes.  In practice, the first crashed thread to reach the end of the
   // StaticCrashHandler will end the process.
   while (true) {
-    ExitOnErr(_zx_object_wait_one(Channel.Handle, ZX_CHANNEL_READABLE,
-                                  ZX_TIME_INFINITE, nullptr),
-              "_zx_object_wait_one");
+    zx_wait_item_t WaitItems[] = {
+        {
+            .handle = SignalHandlerEvent,
+            .waitfor = ZX_SIGNAL_HANDLE_CLOSED,
+            .pending = 0,
+        },
+        {
+            .handle = Channel.Handle,
+            .waitfor = ZX_CHANNEL_READABLE | ZX_CHANNEL_PEER_CLOSED,
+            .pending = 0,
+        },
+    };
+    auto Status = _zx_object_wait_many(
+        WaitItems, sizeof(WaitItems) / sizeof(WaitItems[0]), ZX_TIME_INFINITE);
+    if (Status != ZX_OK || (WaitItems[1].pending & ZX_CHANNEL_READABLE) == 0) {
+      break;
+    }
 
     zx_exception_info_t ExceptionInfo;
     ScopedHandle Exception;
     ExitOnErr(_zx_channel_read(Channel.Handle, 0, &ExceptionInfo,
                                &Exception.Handle, sizeof(ExceptionInfo), 1,
                                nullptr, nullptr),
               "_zx_channel_read");
 
#endif
     // Set the exception to HANDLED so it resumes the thread on close.
     uint32_t ExceptionState = ZX_EXCEPTION_STATE_HANDLED;
     ExitOnErr(_zx_object_set_property(Exception.Handle, ZX_PROP_EXCEPTION_STATE,
                                       &ExceptionState, sizeof(ExceptionState)),
               "zx_object_set_property");
   }
 }
 
+void StopSignalHandler() {
+  _zx_handle_close(SignalHandlerEvent);
+  if (SignalHandler.joinable()) {
+    SignalHandler.join();
+  }
+}
+
 } // namespace
 
 // Platform specific functions.
 void SetSignalHandler(const FuzzingOptions &Options) {
   // Make sure information from libFuzzer and the sanitizers are easy to
   // reassemble. `__sanitizer_log_write` has the added benefit of ensuring the
   // DSO map is always available for the symbolizer.
   // A uint64_t fits in 20 chars, so 64 is plenty.
void SetSignalHandler(const FuzzingOptions &Options) {
   // Options.HandleInt and Options.HandleTerm are not supported on Fuchsia
 
   // Early exit if no crash handler needed.
   if (!Options.HandleSegv && !Options.HandleBus && !Options.HandleIll &&
       !Options.HandleFpe && !Options.HandleAbrt)
     return;
 
   // Set up the crash handler and wait until it is ready before proceeding.
-  zx_handle_t Event;
-  ExitOnErr(_zx_event_create(0, &Event), "_zx_event_create");
+  ExitOnErr(_zx_event_create(0, &SignalHandlerEvent), "_zx_event_create");
 
-  std::thread T(CrashHandler, &Event);
-  zx_status_t Status =
-      _zx_object_wait_one(Event, ZX_USER_SIGNAL_0, ZX_TIME_INFINITE, nullptr);
-  _zx_handle_close(Event);
+  SignalHandler = std::thread(CrashHandler);
+  zx_status_t Status = _zx_object_wait_one(SignalHandlerEvent, ZX_USER_SIGNAL_0,
+                                           ZX_TIME_INFINITE, nullptr);
   ExitOnErr(Status, "_zx_object_wait_one");
 
-  T.detach();
+  std::atexit(StopSignalHandler);
 }
 
 void SleepSeconds(int Seconds) {
   _zx_nanosleep(_zx_deadline_after(ZX_SEC(Seconds)));
 }
 
 unsigned long GetPid() {
   zx_status_t rc;