[libFuzzer] Properly handle exceptions with UnhandledExceptionFilter

[mpividori]

Summary

I realized we should use SetUnhandledExceptionFilter instead of AddVectoredExceptionHandler.
After reading the documentation again on Structured Exception Handling, I have a better understanding on the exception handling. It is a bit confusing because all the documentation is not in the same place. (from: https://msdn.microsoft.com/en-us/library/windows/desktop/ms679327(v=vs.85).aspx, https://msdn.microsoft.com/en-us/library/windows/desktop/ms681420(v=vs.85).aspx and https://msdn.microsoft.com/en-us/library/windows/desktop/ms680634(v=vs.85).aspx)

This is the order for the Exception Dispatching:

• If the process is being debugged, the system notifies the debugger.
• The Vectored Exception Handler is called. ("Vectored handlers are called in the order that they were added, after the debugger gets a first chance notification, but before the system begins unwinding the stack.")
• The system attempts to locate a frame-based exception handler by searching the stack frames of the thread in which the exception occurred.
• If no frame-based handler can be found, the UnhandledExceptionFilter filter is called.
• Default handling based on the exception type.

So, similar to what we do for asan, we should use SetUnhandledExceptionFilter instead of AddVectoredExceptionHandler, so the user's code that is been fuzzed can execute frame-based exception handlers before we catch them . I mean, we want to catch unhandled exceptions, not all the exceptions.

[rnk]

lgtm Yes, we should only be using VEH for shadow memory page faults.

llvm/trunk/lib/Fuzzer/FuzzerUtilWindows.cpp

 #include <sys/types.h>
 #include <windows.h>
 #include <Psapi.h>
 
 namespace fuzzer {
 
 static const FuzzingOptions* HandlerOpt = nullptr;
 
-LONG CALLBACK ExceptionHandler(PEXCEPTION_POINTERS ExceptionInfo) {
+static LONG CALLBACK ExceptionHandler(PEXCEPTION_POINTERS ExceptionInfo) {
   switch (ExceptionInfo->ExceptionRecord->ExceptionCode) {
     case EXCEPTION_ACCESS_VIOLATION:
     case EXCEPTION_ARRAY_BOUNDS_EXCEEDED:
     case EXCEPTION_STACK_OVERFLOW:
       if (HandlerOpt->HandleSegv)
         Fuzzer::StaticCrashSignalCallback();
       break;
     case EXCEPTION_DATATYPE_MISALIGNMENT:
if (!SetConsoleCtrlHandler(CtrlHandler, TRUE)) {
       DWORD LastError = GetLastError();
       Printf("libFuzzer: SetConsoleCtrlHandler failed (Error code: %lu).\n",
         LastError);
       exit(1);
     }
 
   if (Options.HandleSegv || Options.HandleBus || Options.HandleIll ||
       Options.HandleFpe)
-    if (!AddVectoredExceptionHandler(1, ExceptionHandler)) {
-      Printf("libFuzzer: AddVectoredExceptionHandler failed.\n");
-      exit(1);
-    }
+    SetUnhandledExceptionFilter(ExceptionHandler);
 
   if (Options.HandleAbrt)
     if (SIG_ERR == signal(SIGABRT, CrashHandler)) {
       Printf("libFuzzer: signal failed with %d\n", errno);
       exit(1);
     }
 }