[Support] Emulate SIGPIPE handling in raw_fd_ostream write for Windows

[andrewng]

Summary

Prevent errors and crash dumps for broken pipes on Windows.

Fixes: https://github.com/llvm/llvm-project/issues/48672

[andrewng]

Ping.

[MaskRay]

I don't know much about Windows. Best to wait for @mstorsjo

llvm/lib/Support/Windows/Signals.inc

210

ATOMIC_VAR_INIT is deprecated. Just use nullptr?

[mstorsjo]

I think this looks fine, but I'm not really familiar with this area of the code. Do @aganea or @thakis or @hans want to comment on it? Otherwise I think it's probably fine...

[aganea]

You'd probably need to remove this #define: https://github.com/llvm/llvm-project/blob/main/clang/lib/Driver/Driver.cpp#L1876

[mstorsjo]

Also, while I kinda see what this does in a code level, can you describe what this changes from a user point of view - does some usecase behave differently?

Half relatedly - I remember seeing some other discussion (I don’t have a reference handy right now) about changes to sigpipe handling; right now, if doing e.g. clang -E | less, you get a rather ugly crash for the clang process, if exiting less before the end of the clang output. (I remember seeing wishes to not treat this like a regular crash.)

[aganea]

Sorry for the spam. Test coverage would be welcome too :-) The following should exercise the crash: RUN: llvm-readobj --help | head -c1

[andrewng]

In D142224#4097372, @aganea wrote:

You'd probably need to remove this #define: https://github.com/llvm/llvm-project/blob/main/clang/lib/Driver/Driver.cpp#L1876

Thanks for spotting this.

In D142224#4097437, @aganea wrote:

Sorry for the spam. Test coverage would be welcome too :-) The following should exercise the crash: RUN: llvm-readobj --help | head -c1

Yes, I did consider adding something like this as a simple test but it does kind of assume that llvm-readobj --help is using raw_ostream. But if you think it has value then I'll add it.

[andrewng]

In D142224#4097415, @mstorsjo wrote:

Also, while I kinda see what this does in a code level, can you describe what this changes from a user point of view - does some usecase behave differently?

Half relatedly - I remember seeing some other discussion (I don’t have a reference handy right now) about changes to sigpipe handling; right now, if doing e.g. clang -E | less, you get a rather ugly crash for the clang process, if exiting less before the end of the clang output. (I remember seeing wishes to not treat this like a regular crash.)

Yes, this patch addresses the issue you've described which is also mentioned in issue https://github.com/llvm/llvm-project/issues/48672 (although there it is clang --help).

[andrewng]

Updated for review suggestions.

In D142224#4097437, @aganea wrote:

Sorry for the spam. Test coverage would be welcome too :-) The following should exercise the crash: RUN: llvm-readobj --help | head -c1

Yes, I did consider adding something like this as a simple test but it does kind of assume that llvm-readobj --help is using raw_ostream. But if you think it has value then I'll add it.

I tried adding this "simple" test but unfortunately it doesn't reproduce the issue in lit. I think it has something to do with how lit handles the piping. Would it be OK without a test?

[mstorsjo]

In D142224#4099148, @andrewng wrote:

In D142224#4097415, @mstorsjo wrote:

Also, while I kinda see what this does in a code level, can you describe what this changes from a user point of view - does some usecase behave differently?

Half relatedly - I remember seeing some other discussion (I don’t have a reference handy right now) about changes to sigpipe handling; right now, if doing e.g. clang -E | less, you get a rather ugly crash for the clang process, if exiting less before the end of the clang output. (I remember seeing wishes to not treat this like a regular crash.)

Yes, this patch addresses the issue you've described which is also mentioned in issue https://github.com/llvm/llvm-project/issues/48672 (although there it is clang --help).

Oh, sorry that I entirely missed the link in the review description. Then the purpose of the patch is much clearer!

This patch does indeed fix the issue for clang --help, but doesn't make any difference for clang -E. Do you happen to know how hard it would be to fix the same issue for clang -E too? (I've noticed that clang -E | less did show errors in clang 15.0 on unix too, but not in 14.0 and not in current git main.)

[mstorsjo]

I did a bit more digging on the unix side, and https://github.com/llvm/llvm-project/issues/59037 and https://reviews.llvm.org/D138244 fixed the issue for clang -E on the unix side. Is there something that we could do about the default handler here, not specific to raw_ostream?

[mstorsjo]

In D142224#4099809, @andrewng wrote:

I tried adding this "simple" test but unfortunately it doesn't reproduce the issue in lit. I think it has something to do with how lit handles the piping. Would it be OK without a test?

These kinds of things are usually kinda tricky to test reliably, so FWIW, I'd be ok with landing this without a testcase for it.

[andrewng]

In D142224#4101799, @mstorsjo wrote:

I did a bit more digging on the unix side, and https://github.com/llvm/llvm-project/issues/59037 and https://reviews.llvm.org/D138244 fixed the issue for clang -E on the unix side. Is there something that we could do about the default handler here, not specific to raw_ostream?

This patch now also fixes the clang -E related issue on Windows. Was more complicated than expected...

[mstorsjo]

In D142224#4102406, @andrewng wrote:

In D142224#4101799, @mstorsjo wrote:

I did a bit more digging on the unix side, and https://github.com/llvm/llvm-project/issues/59037 and https://reviews.llvm.org/D138244 fixed the issue for clang -E on the unix side. Is there something that we could do about the default handler here, not specific to raw_ostream?

This patch now also fixes the clang -E related issue on Windows. Was more complicated than expected...

Thanks - this now seems to work great! Sorry for adding the extra detour, but it was much appreciated!

llvm/lib/Support/Windows/Signals.inc

831

Do you have any reference for/comment about what 0xE0000000 is here?

[andrewng]

llvm/lib/Support/Windows/Signals.inc

831

The constant is used in CrashRecoveryContext::HandleExit() to raise an exception on Windows. I'll add a comment.

[aganea]

llvm/lib/Support/Windows/Signals.inc

831

That is not super-well documented in the Windows APIs. The 0xE prefix is the recommended way to throw user exceptions, it's actually a NTSTATUS code, with bit 29 set (that is, customer) and bits 30, 31 set (that is, STATUS_SEVERITY_ERROR): https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/87fba13e-bf06-450e-83b1-9241dc81e781
This prefix code is also used for C++ exceptions in the MSVC ABI.

[andrewng]

Updated to add comment regarding the strange looking use of 0xE0000000.

[mstorsjo]

In D142224#4113245, @andrewng wrote:

Updated to add comment regarding the strange looking use of 0xE0000000.

Thanks, LGTM!

llvm/lib/Support/Windows/Signals.inc

831

Thanks @aganea - perhaps we should explain this as reference in CrashRecoveryContext::HandleExit()? (Within this particular case, it's IMO enough to just reference CrashRecoveryContext::HandleExit().)

[erichkeane]

llvm/lib/Support/Windows/Signals.inc

834

This patch seems to cause a self-build Werror regression. The mask here is large enough to cause the RHS of this to be unsigned, so the comparison hits -Wsign-compare. See the example here https://godbolt.org/z/fa5q889jh

[aganea]

llvm/lib/Support/Windows/Signals.inc

834

Please see rG1d0a5f11c04e6ac4dab578b81d02eabb83b31428

[erichkeane]

llvm/lib/Support/Windows/Signals.inc

834

Wonderful, thanks! Looks like our build system didn't get to that yet.

[mstorsjo]

llvm/lib/Support/Windows/Signals.inc

834

Thanks @aganea for the fix! This has been backported to 16.x, so I filed a request to backport your warning fix too, see https://github.com/llvm/llvm-project/issues/61012.

clang/lib/Driver/Driver.cpp

if (!isSaveTempsEnabled()) {
       const JobAction *JA = cast<JobAction>(&FailingCommand->getSource());
       C.CleanupFileMap(C.getResultFiles(), JA, true);
 
       // Failure result files are valid unless we crashed.
       if (CommandRes < 0)
         C.CleanupFileMap(C.getFailureResultFiles(), JA, true);
     }
 
-#if LLVM_ON_UNIX
-    // llvm/lib/Support/Unix/Signals.inc will exit with a special return code
+    // llvm/lib/Support/*/Signals.inc will exit with a special return code
     // for SIGPIPE. Do not print diagnostics for this case.
     if (CommandRes == EX_IOERR) {
       Res = CommandRes;
       continue;
     }
-#endif
 
     // Print extra information about abnormal failures, if possible.
     //
     // This is ad-hoc, but we don't want to be excessively noisy. If the result
     // status was 1, assume the command failed normally. In particular, if it
     // was the compiler then assume it gave a reasonable error code. Failures
     // in other tools are less common, and they generally have worse
     // diagnostics, so always print the diagnostic there.

llvm/include/llvm/Support/ExitCodes.h

 
 #ifndef LLVM_SUPPORT_EXITCODES_H
 #define LLVM_SUPPORT_EXITCODES_H
 
 #include "llvm/Config/llvm-config.h"
 
 #if HAVE_SYSEXITS_H
 #include <sysexits.h>
-#elif __MVS__
-// <sysexits.h> does not exist on z/OS. The only value used in LLVM is
-// EX_IOERR, which is used to signal a special error condition (broken pipe).
+#elif __MVS__ || defined(_WIN32)
+// <sysexits.h> does not exist on z/OS and Windows. The only value used in LLVM
+// is EX_IOERR, which is used to signal a special error condition (broken pipe).
 // Define the macro with its usual value from BSD systems, which is chosen to
 // not clash with more standard exit codes like 1.
 #define EX_IOERR 74
 #elif LLVM_ON_UNIX
 #error Exit code EX_IOERR not available
 #endif
 
 #endif

llvm/include/llvm/Support/Signals.h

namespace sys {
   /// DefaultOneShotPipeSignalHandler). This handler must be provided before
   /// any other LLVM signal handlers are installed: the \ref InitLLVM
   /// constructor has a flag that can simplify this setup.
   ///
   /// Note that the handler is not allowed to call any non-reentrant
   /// functions.  A null handler pointer disables the current installed
   /// function.  Note also that the handler may be executed on a
   /// different thread on some platforms.
-  ///
-  /// This is a no-op on Windows.
   void SetOneShotPipeSignalFunction(void (*Handler)());
 
-  /// On Unix systems, this function exits with an "IO error" exit code.
-  /// This is a no-op on Windows.
+  /// On Unix systems and Windows, this function exits with an "IO error" exit
+  /// code.
   void DefaultOneShotPipeSignalHandler();
 
+#ifdef _WIN32
+  /// Windows does not support signals and this handler must be called manually.
+  void CallOneShotPipeSignalHandler();
+#endif
+
   /// This function does the following:
   /// - clean up any temporary files registered with RemoveFileOnSignal()
   /// - dump the callstack from the exception context
   /// - call any relevant interrupt/signal handlers
   /// - create a core/mini dump of the exception context whenever possible
   /// Context is a system-specific failure context: it is the signal type on
   /// Unix; the ExceptionContext on Windows.
   void CleanupOnSignal(uintptr_t Context);
 
   void unregisterHandlers();
 } // End sys namespace
 } // End llvm namespace
 
 #endif

llvm/lib/Support/Windows/Signals.inc

 //===- Win32/Signals.cpp - Win32 Signals Implementation ---------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file provides the Win32 specific implementation of the Signals class.
 //
 //===----------------------------------------------------------------------===//
 #include "llvm/Support/ConvertUTF.h"
+#include "llvm/Support/ExitCodes.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Process.h"
 #include "llvm/Support/WindowsError.h"
 #include <algorithm>
 #include <io.h>
 #include <signal.h>
 #include <stdio.h>

 // The function to call if ctrl-c is pressed.
 static void (*InterruptFunction)() = 0;
 
 static std::vector<std::string> *FilesToRemove = NULL;
 static bool RegisteredUnhandledExceptionFilter = false;
 static bool CleanupExecuted = false;
 static PTOP_LEVEL_EXCEPTION_FILTER OldFilter = NULL;
 
+/// The function to call on "SIGPIPE" (one-time use only).
+static std::atomic<void (*)()> OneShotPipeSignalFunction(nullptr);
+
 // Windows creates a new thread to execute the console handler when an event
 // (such as CTRL/C) occurs.  This causes concurrency issues with the above
 // globals which this critical section addresses.
 static CRITICAL_SECTION CriticalSection;
 static bool CriticalSectionInitialized = false;
 
 static StringRef Argv0;
 
void llvm::sys::SetInterruptFunction(void (*IF)()) {
   LeaveCriticalSection(&CriticalSection);
 }
 
 void llvm::sys::SetInfoSignalFunction(void (*Handler)()) {
   // Unimplemented.
 }
 
 void llvm::sys::SetOneShotPipeSignalFunction(void (*Handler)()) {
-  // Unimplemented.
+  OneShotPipeSignalFunction.exchange(Handler);
 }
 
 void llvm::sys::DefaultOneShotPipeSignalHandler() {
-  // Unimplemented.
+  llvm::sys::Process::Exit(EX_IOERR, /*NoCleanup=*/true);
+}
+
+void llvm::sys::CallOneShotPipeSignalHandler() {
+  if (auto OldOneShotPipeFunction = OneShotPipeSignalFunction.exchange(nullptr))
+    OldOneShotPipeFunction();
 }
 
 /// Add a function to be called when a signal is delivered to the process. The
 /// handler can have a cookie passed to it to identify what instance of the
 /// handler it is.
 void llvm::sys::AddSignalHandler(sys::SignalHandlerCallback FnPtr,
                                  void *Cookie) {
   insertSignalHandler(FnPtr, Cookie);
if (!fMiniDumpWriteDump(::GetCurrentProcess(), ::GetCurrentProcessId(),
                           FileHandle, DumpType, ExceptionInfo, NULL, NULL))
     return mapWindowsError(::GetLastError());
 
   llvm::errs() << "Wrote crash dump file \"" << DumpPath << "\"\n";
   return std::error_code();
 }
 
 void sys::CleanupOnSignal(uintptr_t Context) {
-  LLVMUnhandledExceptionFilter((LPEXCEPTION_POINTERS)Context);
+  LPEXCEPTION_POINTERS EP = (LPEXCEPTION_POINTERS)Context;
+  // Broken pipe is not a crash.
+  //
+  // 0xE0000000 is combined with the return code in the exception raised in
+  // CrashRecoveryContext::HandleExit().
+  int RetCode = (int)EP->ExceptionRecord->ExceptionCode;
+  if (RetCode == (0xE0000000 | EX_IOERR))
+    return;
+  LLVMUnhandledExceptionFilter(EP);
 }
 
 static LONG WINAPI LLVMUnhandledExceptionFilter(LPEXCEPTION_POINTERS ep) {
   Cleanup(true);
 
   // Write out the exception code.
   if (ep && ep->ExceptionRecord)
     llvm::errs() << format("Exception Code: 0x%08X",

llvm/lib/Support/raw_ostream.cpp

 #endif
 #ifndef STDERR_FILENO
 # define STDERR_FILENO 2
 #endif
 #endif
 
 #ifdef _WIN32
 #include "llvm/Support/ConvertUTF.h"
+#include "llvm/Support/Signals.h"
 #include "llvm/Support/Windows/WindowsSupport.h"
 #endif
 
 using namespace llvm;
 
 constexpr raw_ostream::Colors raw_ostream::BLACK;
 constexpr raw_ostream::Colors raw_ostream::RED;
 constexpr raw_ostream::Colors raw_ostream::GREEN;
if (ret < 0) {
       // don't use O_NONBLOCK file descriptors with raw_ostream.
       if (errno == EINTR || errno == EAGAIN
 #ifdef EWOULDBLOCK
           || errno == EWOULDBLOCK
 #endif
           )
         continue;
 
+#ifdef _WIN32
+      // Windows equivalents of SIGPIPE/EPIPE.
+      DWORD WinLastError = GetLastError();
+      if (WinLastError == ERROR_BROKEN_PIPE ||
+          (WinLastError == ERROR_NO_DATA && errno == EINVAL)) {
+        llvm::sys::CallOneShotPipeSignalHandler();
+        errno = EPIPE;
+      }
+#endif
       // Otherwise it's a non-recoverable error. Note it and quit.
       error_detected(std::error_code(errno, std::generic_category()));
       break;
     }
 
     // The write may have written some or all of the data. Update the
     // size and buffer pointer to reflect the remainder that needs
     // to be written. If there are no bytes left, we're done.