Allow some test suite options for dealing with crashing on Windows

[zturner]

Summary

When running the test suite on Windows, we need a way to:

a) Not show a new console window each time LLDB launches an inferior. Without this there is new window spam as new windows get rapidly created and destroyed while the test suite runs.

b) Not display the Windows system crash dialog when LLDB crashes. Without this, the test suite hangs whenever LLDB crashes.

[clayborg]

Looks good.

[zturner]

+ Scott for comments on the Crt reporting stuff. I haven't used these APIs much, so there may be some nuances I've overlooked.

[scottmg]

Otherwise, lgtm.

source/lldb.cpp

149

Normally, preserve existing flags (they're returned). Chrome does something this:

UINT new_flags = SEM_FAILCRITICALERRORS |
                 SEM_NOGPFAULTERRORBOX |
                 SEM_NOOPENFILEERRORBOX;
UINT existing_flags = SetErrorMode(new_flags);
SetErrorMode(existing_flags | new_flags);

153

I don't know if you link with /MT or /MD, but this will only work in this binary if /MT.

[zturner]

source/lldb.cpp

153

Ideally I need to be able to suppress the dialog regardless of how we're linking. Is there a similar mechanism for /MD?

[scottmg]

source/lldb.cpp

153

Sorry, I was unclear -- it will work across all exe and dlls if you're using /MD (because there's only one CRT that way), but only in this binary if /MT. I believe the only fix is to call that from every binary's main/DllMain/whatever. (Maybe this isn't really a problem if there's not actually multiple binaries involved here?)

You also may get link errors in /MT and /MD (i.e. not /MTd or /MDd), so confirm a release build works. (I can never remember, it may be fine as-is).

Instead of _CrtSetReportHook, below might be what you want (so that it prints an error/exception rather than just silently exiting).

_CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
_CrtSetReportMode(_CRT_WARN, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
_CrtSetReportMode(_CRT_ERROR, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
_CrtSetReportFile(_CRT_ASSERT, _CRTDBG_FILE_STDERR);
_CrtSetReportFile(_CRT_WARN, _CRTDBG_FILE_STDERR);
_CrtSetReportFile(_CRT_ERROR, _CRTDBG_FILE_STDERR);

[zturner]

source/lldb.cpp

153

I'll give that a try. I had tried that before, but in my quest of trying different things I guess I got rid of it and never went back to it. Certainly simpler though.

[zturner]

BTW, can you look at what I did in ProcessLauncherWindows? What are the implications of launching with a hidden window, as I did, and launching with CREATE_NO_WINDOW or DETACHED_PROCESS? The latter 2 seemed to be preferable, if they worked correctly, but I wasn't sure if it would result in strange behavior since the program might still be using stdout, stdin, and stderr.

[scottmg]

source/Host/windows/ProcessLauncherWindows.cpp

45

I think DETACHED_PROCESS is more like what you want, but I'm not that familiar with all the intricacies of how gui->console and console->gui are affected. (Carlos might know better.)

.gitignore

 *.orig
 # Byte compiled python modules.
 *.pyc
+*.pyproj
+*.sln
+*.suo
 # vim swap files
 .*.swp
 .sw?

source/Host/windows/ProcessLauncherWindows.cpp

     startupinfo.hStdInput = stdin_handle;
     startupinfo.hStdOutput = stdout_handle;
 
+    const char *hide_console_var = getenv("LLDB_LAUNCH_INFERIORS_WITHOUT_CONSOLE");
+    if (hide_console_var && llvm::StringRef(hide_console_var).equals_lower("true"))
+    {
+        startupinfo.dwFlags |= STARTF_USESHOWWINDOW;
+        startupinfo.wShowWindow = SW_HIDE;
+    }
+
     DWORD flags = CREATE_NEW_CONSOLE;
     if (launch_info.GetFlags().Test(eLaunchFlagDebug))
         flags |= DEBUG_ONLY_THIS_PROCESS;

source/lldb.cpp

 #endif
 
 #if defined (_WIN32)
+#include "lldb/Host/windows/windows.h"
 #include "Plugins/Process/Windows/DynamicLoaderWindows.h"
 #include "Plugins/Process/Windows/ProcessWindows.h"
 #endif

 using namespace lldb;
 using namespace lldb_private;
 
+namespace
+{
+#if defined(_MSC_VER)
+// Emulates the effect of the user pressing the "retry" button when the abort/retry/ignore
+// assertion failure dialog pops up.
+static int AvoidMessageBoxHook(int ReportType, char *Message, int *Return)
+{
+    // FIXME: Do we need to raise a python exception here?  Otherwise, if this is being
+    // run from Python, Python will just exit.  It might be possible to allow it to continue
+    // running other tests.
+    static const int kRetryButton = 1;
+
+    if (Return)
+        *Return = kRetryButton;
+    return TRUE;
+}
+#endif
+}
+
 static void fatal_error_handler(void *user_data, const std::string& reason,
                                 bool gen_crash_diag) {
     Host::SetCrashDescription(reason.c_str());

     if (!g_inited)
     {
         g_inited = true;
+
+#if defined(_MSC_VER)
+        const char *disable_crash_dialog_var = getenv("LLDB_DISABLE_CRASH_DIALOG");
+        if (disable_crash_dialog_var && llvm::StringRef(disable_crash_dialog_var).equals_lower("true"))
+        {
+            // This will prevent Windows from displaying a dialog box requiring user interaction when
+            // LLDB crashes.  This is mostly useful when automating LLDB, for example via the test
+            // suite, so that a crash in LLDB does not prevent completion of the test suite.
+            ::SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX);
+
+            // This will prevent Windows from displaying an abort/retry/ignore dialog box requiring
+            // user interaction when LLDB asserts.
+            _CrtSetReportHook(AvoidMessageBoxHook);
+        }
+#endif
+
         Log::Initialize();
         HostInfo::Initialize();
         Timer::Initialize ();

test/CMakeLists.txt

   -u CFLAGS
   )
 
+if ( CMAKE_SYSTEM_NAME MATCHES "Windows" )
+  set(LLDB_TEST_DEBUG_TEST_CRASHES
+    0
+    CACHE BOOL "(Windows only) Enables debugging of tests in the test suite by showing the crash dialog when lldb crashes")
+
+  set(LLDB_TEST_HIDE_CONSOLE_WINDOWS
+    1
+    CACHE BOOL "(Windows only) Hides the console window for an inferior when it is launched through the test suite")
+
+  if (LLDB_TEST_DEBUG_TEST_CRASHES)
+    set(LLDB_TEST_COMMON_ARGS ${LLDB_TEST_COMMON_ARGS} --enable-crash-dialog)
+  endif()
+
+  if (NOT LLDB_TEST_HIDE_CONSOLE_WINDOWS)
+    set(LLDB_TEST_COMMON_ARGS ${LLDB_TEST_COMMON_ARGS} --show-inferior-console)
+  endif()
+endif()
+
 add_python_test_target(check-lldb-single
   ${LLDB_SOURCE_DIR}/test/dotest.py
   "${LLDB_TEST_COMMON_ARGS};${LLDB_TEST_USER_ARGS}"

 set(LLDB_DOSEP_ARGS -o;\"-q;${LLDB_TEST_COMMON_ARGS};${LLDB_TEST_USER_ARGS}\")
 
 # If tests crash cause LLDB to crash, or things are otherwise unstable, or if machine-parsable
-# output is desired (i.e. in continuous integration contexts) check-lldb-sep is a better target.
+# output is desired (i.e. in continuous integration contexts) check-lldb-single is a better target.
 add_python_test_target(check-lldb
   ${LLDB_SOURCE_DIR}/test/dosep.py
   "${LLDB_DOSEP_ARGS}"

test/dotest.py

     X('-v', 'Do verbose mode of unittest framework (print out each test case invocation)')
     X('-w', 'Insert some wait time (currently 0.5 sec) between consecutive test cases')
     X('-T', 'Obtain and dump svn information for this checkout of LLDB (off by default)')
+    group.add_argument('--enable-crash-dialog', dest='disable_crash_dialog', action='store_false', help='(Windows only) When LLDB crashes, display the Windows crash dialog.')
+    group.add_argument('--show-inferior-console', dest='hide_inferior_console', action='store_false', help='(Windows only) When launching an inferior, dont hide its console window.')
+    group.set_defaults(disable_crash_dialog=True)
+    group.set_defaults(hide_inferior_console=True)
 
     # Remove the reference to our helper function
     del X

     if args.sharp:
         count = args.sharp
 
+    if sys.platform.startswith('win32'):
+        os.environ['LLDB_DISABLE_CRASH_DIALOG'] = str(args.disable_crash_dialog)
+        os.environ['LLDB_LAUNCH_INFERIORS_WITHOUT_CONSOLE'] = str(args.hide_inferior_console)
+
     if do_help == True:
         usage(parser)