[llvm] Do not use Console API if output isn't a console device

[kasper93]

Summary

This fixes -fcolor-diagnostics to properly enable color output when
not outputting to console devices. Example would be ninja.

This affects only Windows.

Signed-off-by: Kacper Michajłow <[e-mail]>

[kasper93]

Ok, so I recognize that this is global state and it is not very good to flip it like that.

Frankly this Console API usage in llvm is little bit messy it only sets color on STD_OUTPUT_HANDLE, what about STD_ERROR_HANDLE? Anyhow Microsoft documentation strongly suggests to use virtual terminal sequences. And I believe they are supported on all Windows versions that are still not EOL.

So how about removing this static bool UseANSI = false; and related code? I can do that, but since I already this review, I might as well ask you for opinion.

see https://learn.microsoft.com/en-us/windows/console/setconsoletextattribute

llvm/lib/Support/raw_ostream.cpp

raw_fd_ostream::raw_fd_ostream(int fd, bool shouldClose, bool unbuffered,
   // users must simply be aware that mixed output and remarks is a possibility.
   if (FD <= STDERR_FILENO)
     ShouldClose = false;
 
 #ifdef _WIN32
   // Check if this is a console device. This is not equivalent to isatty.
   IsWindowsConsole =
       ::GetFileType((HANDLE)::_get_osfhandle(fd)) == FILE_TYPE_CHAR;
+
+  // If this isn't a Windows console device, don't try to use API to set color.
+  // Switch to ANSI escape codes instead.
+  if (!IsWindowsConsole)
+    llvm::sys::Process::UseANSIEscapeCodes(true);
 #endif
 
   // Get the starting position.
   off_t loc = ::lseek(FD, 0, SEEK_CUR);
   sys::fs::file_status Status;
   std::error_code EC = status(FD, Status);
   IsRegularFile = Status.type() == sys::fs::file_type::regular_file;
 #ifdef _WIN32