Respect exitcode sanitizer option in UBSan

[fjricci]

Summary

This patch ensures that UBSan returns the exit code specified
by sanitizer_flags if it detects an error during execution.

[rsmith]

Sure, it makes sense for UBSan to behave like other sanitizers in this regard.

lib/ubsan/ubsan_init.cc

44

Is this the right place to do this? If UBSan is hosted inside another sanitizer, does that sanitizer already set up such a callback?

[fjricci]

lib/ubsan/ubsan_init.cc

44

That's a good question. I know that all of the tests pass for me on Linux and Darwin (both for UBSan hosted inside other sanitizers and for the other sanitizers themselves), but that's been the extent of my testing. Should that be sufficient, or should I investigate further?

[vsk]

lib/ubsan/ubsan_init.cc

44

Asan implements exitcode support by calling Die after the first error. Tsan has an interceptor for exit, and implements exitcode support in that interceptor.

I only took a quick look but I don't think any of the sanitizers call exit() in an atexit() handler, which seems a bit suspicious to me. Couldn't that cause an infinite loop?

If calling exit() in an atexit() handler has safe/defined behavior, I'm fine with adding the handler here (though we should only add it if common_flags()->exitcode is set).

test/ubsan/TestCases/Float/cast-overflow.cpp

3

This test doesn't use the exitcode flag, so why does this test need to change? I have the same question about the other test case changes here.

[fjricci]

lib/ubsan/ubsan_init.cc

44

LSan uses Atext(DoLeakCheck), and DoLeakCheck() calls Die(), so it should be safe.

test/ubsan/TestCases/Float/cast-overflow.cpp

3

The exitcode flag defaults to returning failure on error (lib/sanitizer_common/sanitizer_flags.inc), which I think makes sense.

[vsk]

Could you explain how you intend to make use of these changes? As it stands I'm not convinced this patch is headed in the right direction, but I'd like to give some more constructive feedback.

lib/ubsan/ubsan_init.cc

44

I'm still not sure that calling exit in an atexit handler is a good idea. Example: there are two instrumented shared libraries loaded into a program. Both would get atexit handlers. Calling exit() in one handler would prevent the other handlers from being run. This would break -fprofile-instr-generate, and user-provided atexit handlers.

CC'ing @eugenis for any comment since he reviewed D12120, which revamped exitcode handling.

test/ubsan/TestCases/Float/cast-overflow.cpp

3

I don't think that enabling a sanitizer should, by default, change the way a program's exit codes work. Users may intend to quit with specific exit codes in scenarios which trigger sanitizer diagnostics.

[fjricci]

It might be possible to change the default exitcode for UBSan, but I still think it should be possible for the user to make UBSan errors to return failure. Two primary reasons:

• Consistency: As far as I'm aware, all of the other sanitizers return a failing exit code when they detect an error in a program, and I'm not sure why UBSan should be different in that respect.
• Utility: A common use case I could imagine would be the case where a user wants to prevent UB regressions in their codebase by running a unit test suite with UBSan enabled. If it's not possible to return a failing exit code (either by default or otherwise), the test suite will continue passing just fine when UB is introduced.

A workaround that can be used currently to force a failing exit code on failure would be to use halt_on_error=1. However, it's not always desirable to limit error output to only one error (for example, brining up a unit test suite in a new codebase, where there may be many UB errors). In addition, ASan currently returns a failing exit code at the end of program execution even when halt_on_error=0 and it doesn't immediately abort.

[fjricci]

test/ubsan/TestCases/Float/cast-overflow.cpp

3

Failing after observing an error is the default in the other sanitizers already, and I'm not sure it makes sense to special-case UBSan here.

[vsk]

In D35085#804183, @fjricci wrote:

It might be possible to change the default exitcode for UBSan, but I still think it should be possible for the user to make UBSan errors to return failure. Two primary reasons:

• Consistency: As far as I'm aware, all of the other sanitizers return a failing exit code when they detect an error in a program, and I'm not sure why UBSan should be different in that respect.
• Utility: A common use case I could imagine would be the case where a user wants to prevent UB regressions in their codebase by running a unit test suite with UBSan enabled. If it's not possible to return a failing exit code (either by default or otherwise), the test suite will continue passing just fine when UB is introduced.

Makes sense to me.

A workaround that can be used currently to force a failing exit code on failure would be to use halt_on_error=1. However, it's not always desirable to limit error output to only one error (for example, brining up a unit test suite in a new codebase, where there may be many UB errors).

Also fair.

Reading the atexit man page, I'm more convinced that calling exit() in the atexit handler is not OK, since it also breaks dlclose():
"If the provided function is located in a library that has been dynamically loaded (e.g. by dlopen()), it will be called when the library is unloaded (due to a call to dlclose())"

One possible solution is to add an interceptor for exit() to ubsan, just like the one tsan has. But this breaks (a|t)san+ubsan compatibility.

Another possible solution is to make the "HasReportedError" bit shared in sanitizer_common. That way, if ubsan reports an error, _and_ is in use with (a|t)san, the exitcode support would work properly. This is just a partial solution, however, since it doesn't improve the situation for standalone ubsan. In practice people tend to run asan+ubsan together, so it would still be a win.

[eugenis]

ASan has a mode where it does not exit after the first error, -fsanitize-recover=address and ASAN_OPTIONS=halt_on_error=0. It appears that the errorcode flag is ignored in that mode.

I think TSan does the right thing here. How about this:

• intercept exit() in asan and ubsan
• put HasReportedError somewhere in sanitizer_common, and use it all sanitizers.

[fjricci]

The only issue I have come across is that standalone UBSan currently doesn't link against interception. If we want to intercept _exit, we'll need to add interception as a dependency. Is that something we're okay with?

[fjricci]

There's a further complication with intercepting in UBSan - there doesn't seem to be a way to tell whether you're building standalone at compile time (although I could add a cmake define for that), so the UBSan interceptor will conflict with the TSan/ASan interceptor.

[eugenis]

Right, there are no interceptors in ubsan. I confused it with LSan, which has its own interceptors that are used in standalone mode only.

I'm not sure if adding interception to UBSan is a good idea. I think the current library is OK to link into shared libraries (i.e. does not have real global state). Or is it not the case any longer? I don't think we use this property anywhere, and clang does not link the ubsan runtime library to shared libraries. So maybe this is fine. Thoughts?

In D35085#805845, @fjricci wrote:

There's a further complication with intercepting in UBSan - there doesn't seem to be a way to tell whether you're building standalone at compile time (although I could add a cmake define for that), so the UBSan interceptor will conflict with the TSan/ASan interceptor.

Just put interceptors in UBSAN_STANDALONE_SOURCES.

[vsk]

IMO making ubsan require interceptor support would make using it much less convenient. On macOS, we'd need to change Xcode to get the DYLD_INSERT_LIBRARIES bit right. There are also lots of people who don't use Xcode, and they would need to update their projects as well. I'm in favor of the partial solution where exitcode handling is provided by sanitizer runtimes which already install interceptors (e.g it would work in asan+ubsan mode, or tsan+ubsan mode).

[fjricci]

I'll put up the partial solution where we respect exitcode in all sanitizers that already use interceptors when halt_on_error=0. I won't add interception to UBSan. It's unfortunate for me personally, since my codebases only use standalone UBSan, but I can understand why we wouldn't want to add interception to UBSan just for this one feature.

[fjricci]

Only adding this for non-standalone builds makes testing hard, since you don't have consistent behavior between standalone and non-standalone ubsan builds. I'm debating leaving UBSan be, and just fixing this for the other sanitizers (ie making sure ASan returns failure when halt_on_error=0).

[fjricci]

Hmmm, it actually appears that even TSan doesn't successfully intercept _exit on Linux, only on Darwin (a CHECK(0) statement in OnExit() is never reached on Linux). If we can't successfully intercept _exit, and we can't exit from atexit, I'm not sure that there's a good way to do this.

[fjricci]

Looking further into it, _exit family functions don't get called on linux when return from main is used (GIexit in libc calls the exit syscall directly), so interception won't work. TSan and LSan get around this by calling exit() from their atexit() handlers, but we've established that we don't want to do this because of dlclose() issues. I think we're stuck with the current behavior, at least on Linux.

lib/ubsan/ubsan_diag.h

class ScopedReport {
   Location SummaryLoc;
   ErrorType Type;
 
 public:
   ScopedReport(ReportOptions Opts, Location SummaryLoc, ErrorType Type);
   ~ScopedReport();
 };
 
+bool HasReportedErrors();
 void InitializeSuppressions();
 bool IsVptrCheckSuppressed(const char *TypeName);
 // Sometimes UBSan runtime can know filename from handlers arguments, even if
 // debug info is missing.
 bool IsPCSuppressed(ErrorType ET, uptr PC, const char *Filename);
 
 } // namespace __ubsan
 
 #endif // UBSAN_DIAG_H

lib/ubsan/ubsan_diag.cc

Diag::~Diag() {
 
   Buffer.append("%s\n", Decor.Default());
   Printf("%s", Buffer.data());
 
   if (Loc.isMemoryLocation())
     PrintMemorySnippet(Decor, Loc.getMemoryLocation(), Ranges, NumRanges, Args);
 }
 
+static bool ReportedErrors = false;
+
+bool __ubsan::HasReportedErrors() { return ReportedErrors; }
+
 ScopedReport::ScopedReport(ReportOptions Opts, Location SummaryLoc,
                            ErrorType Type)
     : Opts(Opts), SummaryLoc(SummaryLoc), Type(Type) {
   InitAsStandaloneIfNecessary();
   CommonSanitizerReportMutex.Lock();
+  ReportedErrors = true;
 }
 
 ScopedReport::~ScopedReport() {
   MaybePrintStackTrace(Opts.pc, Opts.bp);
   MaybeReportErrorSummary(SummaryLoc, Type);
   CommonSanitizerReportMutex.Unlock();
   if (flags()->halt_on_error)
     Die();

lib/ubsan/ubsan_init.cc

 
 static enum {
   UBSAN_MODE_UNKNOWN = 0,
   UBSAN_MODE_STANDALONE,
   UBSAN_MODE_PLUGIN
 } ubsan_mode;
 static StaticSpinMutex ubsan_init_mu;
 
+static void ReportFailure() {
+  if (HasReportedErrors() && common_flags()->exitcode)
+    Die();
+}
+
 static void CommonInit() {
   InitializeSuppressions();
+  Atexit(ReportFailure);
 }
 
 static void CommonStandaloneInit() {
   SanitizerToolName = GetSanititizerToolName();
   InitializeFlags();
   CacheBinaryName();
   __sanitizer_set_report_path(common_flags()->log_path);
   AndroidLogInit();

test/ubsan/TestCases/Float/cast-overflow.cpp

 // RUN: %clangxx -fsanitize=float-cast-overflow %s -o %t
 // RUN: %run %t _
-// RUN: %env_ubsan_opts=print_summary=1:report_error_type=1 %run %t 0 2>&1 | FileCheck %s --check-prefix=CHECK-0
-// RUN: %run %t 1 2>&1 | FileCheck %s --check-prefix=CHECK-1
-// RUN: %run %t 2 2>&1 | FileCheck %s --check-prefix=CHECK-2
-// RUN: %run %t 3 2>&1 | FileCheck %s --check-prefix=CHECK-3
-// RUN: %run %t 4 2>&1 | FileCheck %s --check-prefix=CHECK-4
-// RUN: %run %t 5 2>&1 | FileCheck %s --check-prefix=CHECK-5
-// RUN: %run %t 6 2>&1 | FileCheck %s --check-prefix=CHECK-6
+// RUN: %env_ubsan_opts=print_summary=1:report_error_type=1 not %run %t 0 2>&1 | FileCheck %s --check-prefix=CHECK-0
+// RUN: not %run %t 1 2>&1 | FileCheck %s --check-prefix=CHECK-1
+// RUN: not %run %t 2 2>&1 | FileCheck %s --check-prefix=CHECK-2
+// RUN: not %run %t 3 2>&1 | FileCheck %s --check-prefix=CHECK-3
+// RUN: not %run %t 4 2>&1 | FileCheck %s --check-prefix=CHECK-4
+// RUN: not %run %t 5 2>&1 | FileCheck %s --check-prefix=CHECK-5
+// RUN: not %run %t 6 2>&1 | FileCheck %s --check-prefix=CHECK-6
 // FIXME: %run %t 7 2>&1 | FileCheck %s --check-prefix=CHECK-7
 // FIXME: not %run %t 8 2>&1 | FileCheck %s --check-prefix=CHECK-8
 // RUN: not %run %t 9 2>&1 | FileCheck %s --check-prefix=CHECK-9
 
 // This test assumes float and double are IEEE-754 single- and double-precision.
 
 #if defined(__APPLE__)
 # include <machine/endian.h>
case '6': {
     // CHECK-6: cast-overflow.cpp:[[@LINE+2]]:{{34: runtime error: 0xffffff00000000000000000000000001 is outside the range of representable values of type 'float'| __int128 not supported}}
 #if defined(__SIZEOF_INT128__) && !defined(_WIN32)
     static int test_int = (float)(FloatMaxAsUInt128 + 1);
     return 0;
 #else
     // Print the same line as the check above. That way the test is robust to
     // line changes around it
     printf("%s:%d: __int128 not supported", __FILE__, __LINE__ - 5);
-    return 0;
+    return 1;
 #endif
   }
   // FIXME: The backend cannot lower __fp16 operations on x86 yet.
   //case '7':
   //  (__fp16)65504; // ok
   //  // CHECK-7: runtime error: 65505 is outside the range of representable values of type '__fp16'
   //  return (__fp16)65505;
 

test/ubsan/TestCases/Integer/add-overflow.cpp

-// RUN: %clangxx -DADD_I32 -fsanitize=signed-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I32
-// RUN: %clangxx -DADD_I64 -fsanitize=signed-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I64
-// RUN: %clangxx -DADD_I128 -fsanitize=signed-integer-overflow %s -o %t3 && %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I128
+// RUN: %clangxx -DADD_I32 -fsanitize=signed-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I32
+// RUN: %clangxx -DADD_I64 -fsanitize=signed-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I64
+// RUN: %clangxx -DADD_I128 -fsanitize=signed-integer-overflow %s -o %t3 && not %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I128
 
 #include <stdint.h>
 #include <stdio.h>
 
 int main() {
   // These promote to 'int'.
   (void)(int8_t(0x7f) + int8_t(0x7f));
   (void)(int16_t(0x3fff) + int16_t(0x4000));
#ifdef ADD_I64
   // CHECK-ADD_I64: 8000000000000000000 + 2000000000000000000 cannot be represented in type '{{long( long)?}}'
 #endif
 
 #ifdef ADD_I128
 # if defined(__SIZEOF_INT128__) && !defined(_WIN32)
   (void)((__int128_t(1) << 126) + (__int128_t(1) << 126));
 # else
   puts("__int128 not supported");
+  return 1;
 # endif
   // CHECK-ADD_I128: {{0x40000000000000000000000000000000 \+ 0x40000000000000000000000000000000 cannot be represented in type '__int128'|__int128 not supported}}
 #endif
 }

test/ubsan/TestCases/Integer/div-overflow.cpp

-// RUN: %clangxx -fsanitize=signed-integer-overflow %s -o %t && %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=signed-integer-overflow %s -o %t && not %run %t 2>&1 | FileCheck %s
 
 #include <stdint.h>
 
 int main() {
   unsigned(0x80000000) / -1;
 
   // CHECK: div-overflow.cpp:9:23: runtime error: division of -2147483648 by -1 cannot be represented in type 'int'
   int32_t(0x80000000) / -1;
 }

test/ubsan/TestCases/Integer/div-zero.cpp

-// RUN: %clangxx -fsanitize=integer-divide-by-zero -DDIVIDEND=0 %s -o %t && %run %t 2>&1 | FileCheck %s
-// RUN: %clangxx -fsanitize=integer-divide-by-zero -DDIVIDEND=1U %s -o %t && %run %t 2>&1 | FileCheck %s
-// RUN: %clangxx -fsanitize=float-divide-by-zero -DDIVIDEND=1.5 %s -o %t && %run %t 2>&1 | FileCheck %s
-// RUN: %clangxx -fsanitize=integer-divide-by-zero -DDIVIDEND='intmax(123)' %s -o %t && %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=integer-divide-by-zero -DDIVIDEND=0 %s -o %t && not %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=integer-divide-by-zero -DDIVIDEND=1U %s -o %t && not %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=float-divide-by-zero -DDIVIDEND=1.5 %s -o %t && not %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=integer-divide-by-zero -DDIVIDEND='intmax(123)' %s -o %t && not %run %t 2>&1 | FileCheck %s
 
 #if defined(__SIZEOF_INT128__) && !defined(_WIN32)
 typedef __int128 intmax;
 #else
 typedef long long intmax;
 #endif
 
 int main() {
   // CHECK: div-zero.cpp:[[@LINE+1]]:12: runtime error: division by zero
   DIVIDEND / 0;
 }

test/ubsan/TestCases/Integer/incdec-overflow.cpp

-// RUN: %clangxx -DOP=n++ -fsanitize=signed-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck %s --check-prefix=PLUS
-// RUN: %clangxx -DOP=++n -fsanitize=signed-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck %s --check-prefix=PLUS
-// RUN: %clangxx -DOP=m-- -fsanitize=signed-integer-overflow %s -o %t3 && %run %t3 2>&1 | FileCheck %s --check-prefix=MINUS
-// RUN: %clangxx -DOP=--m -fsanitize=signed-integer-overflow %s -o %t4 && %run %t4 2>&1 | FileCheck %s --check-prefix=MINUS
+// RUN: %clangxx -DOP=n++ -fsanitize=signed-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck %s --check-prefix=PLUS
+// RUN: %clangxx -DOP=++n -fsanitize=signed-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck %s --check-prefix=PLUS
+// RUN: %clangxx -DOP=m-- -fsanitize=signed-integer-overflow %s -o %t3 && not %run %t3 2>&1 | FileCheck %s --check-prefix=MINUS
+// RUN: %clangxx -DOP=--m -fsanitize=signed-integer-overflow %s -o %t4 && not %run %t4 2>&1 | FileCheck %s --check-prefix=MINUS
 
 #include <stdint.h>
 
 int main() {
   int n = 0x7ffffffd;
   n++;
   n++;
   int m = -n - 1;
   OP;
   // PLUS: incdec-overflow.cpp:[[@LINE-1]]:3: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
   // MINUS: incdec-overflow.cpp:[[@LINE-2]]:3: runtime error: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int'
 }

test/ubsan/TestCases/Integer/mul-overflow.cpp

-// RUN: %clangxx -fsanitize=signed-integer-overflow %s -o %t && %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=signed-integer-overflow %s -o %t && not %run %t 2>&1 | FileCheck %s
 
 #include <stdint.h>
 
 int main() {
   // These promote to 'int'.
   (void)(int8_t(-2) * int8_t(0x7f));
   (void)(int16_t(0x7fff) * int16_t(0x7fff));
   (void)(uint16_t(0xffff) * int16_t(0x7fff));
   (void)(uint16_t(0xffff) * uint16_t(0x8000));
 
   // CHECK: mul-overflow.cpp:13:27: runtime error: signed integer overflow: 65535 * 32769 cannot be represented in type 'int'
   (void)(uint16_t(0xffff) * uint16_t(0x8001));
 }

test/ubsan/TestCases/Integer/negate-overflow.cpp

-// RUN: %clangxx -fsanitize=signed-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck %s --check-prefix=CHECKS
-// RUN: %clangxx -fsanitize=unsigned-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck %s --check-prefix=CHECKU
+// RUN: %clangxx -fsanitize=signed-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck %s --check-prefix=CHECKS
+// RUN: %clangxx -fsanitize=unsigned-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck %s --check-prefix=CHECKU
 
 int main() {
   // CHECKS-NOT: runtime error
   // CHECKU: negate-overflow.cpp:[[@LINE+2]]:3: runtime error: negation of 2147483648 cannot be represented in type 'unsigned int'
   // CHECKU-NOT: cast to an unsigned
   -unsigned(-0x7fffffff - 1); // ok
   // CHECKS: negate-overflow.cpp:[[@LINE+2]]:3: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
   // CHECKU-NOT: runtime error
   -(-0x7fffffff - 1);
 
   return 0;
 }

test/ubsan/TestCases/Integer/no-recover.cpp

-// RUN: %clangxx -fsanitize=unsigned-integer-overflow %s -o %t && %run %t 2>&1 | FileCheck %s --check-prefix=RECOVER
-// RUN: %clangxx -fsanitize=unsigned-integer-overflow -fno-sanitize-recover=all -fsanitize-recover=unsigned-integer-overflow %s -o %t && %run %t 2>&1 | FileCheck %s --check-prefix=RECOVER
+// RUN: %clangxx -fsanitize=unsigned-integer-overflow %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=RECOVER
+// RUN: %clangxx -fsanitize=unsigned-integer-overflow -fno-sanitize-recover=all -fsanitize-recover=unsigned-integer-overflow %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=RECOVER
 // RUN: %clangxx -fsanitize=unsigned-integer-overflow -fno-sanitize-recover=unsigned-integer-overflow %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=ABORT
 
 #include <stdint.h>
 
 int main() {
   // These promote to 'int'.
   (void)(uint8_t(0xff) + uint8_t(0xff));
   (void)(uint16_t(0xf0fff) + uint16_t(0x0fff));

test/ubsan/TestCases/Integer/sub-overflow.cpp

-// RUN: %clangxx -DSUB_I32 -fsanitize=signed-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I32
-// RUN: %clangxx -DSUB_I64 -fsanitize=signed-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I64
-// RUN: %clangxx -DSUB_I128 -fsanitize=signed-integer-overflow %s -o %t3 && %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I128
+// RUN: %clangxx -DSUB_I32 -fsanitize=signed-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I32
+// RUN: %clangxx -DSUB_I64 -fsanitize=signed-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I64
+// RUN: %clangxx -DSUB_I128 -fsanitize=signed-integer-overflow %s -o %t3 && not %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I128
 
 #include <stdint.h>
 #include <stdio.h>
 
 int main() {
   // These promote to 'int'.
   (void)(int8_t(-2) - int8_t(0x7f));
   (void)(int16_t(-2) - int16_t(0x7fff));
 
 #ifdef SUB_I32
   (void)(int32_t(-2) - int32_t(0x7fffffff));
   // CHECK-SUB_I32: sub-overflow.cpp:[[@LINE-1]]:22: runtime error: signed integer overflow: -2 - 2147483647 cannot be represented in type 'int'
 #endif
 
 #ifdef SUB_I64
   (void)(int64_t(-8000000000000000000ll) - int64_t(2000000000000000000ll));
   // CHECK-SUB_I64: -8000000000000000000 - 2000000000000000000 cannot be represented in type '{{long( long)?}}'
 #endif
 
 #ifdef SUB_I128
 # if defined(__SIZEOF_INT128__) && !defined(_WIN32)
   (void)(-(__int128_t(1) << 126) - (__int128_t(1) << 126) - 1);
 # else
   puts("__int128 not supported");
+  return 1;
 # endif
   // CHECK-SUB_I128: {{0x80000000000000000000000000000000 - 1 cannot be represented in type '__int128'|__int128 not supported}}
 #endif
 }

test/ubsan/TestCases/Integer/summary.cpp

 // RUN: %clangxx -fsanitize=integer %s -o %t
-// RUN: %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-NOTYPE
-// RUN: %env_ubsan_opts=report_error_type=1 %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-TYPE
+// RUN: not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-NOTYPE
+// RUN: %env_ubsan_opts=report_error_type=1 not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-TYPE
 // REQUIRES: ubsan-asan
 
 #include <stdint.h>
 
 int main() {
   (void)(uint64_t(10000000000000000000ull) + uint64_t(9000000000000000000ull));
   // CHECK-NOTYPE: SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior {{.*}}summary.cpp:[[@LINE-1]]:44
   // CHECK-TYPE: SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow {{.*}}summary.cpp:[[@LINE-2]]:44
   return 0;
 }

test/ubsan/TestCases/Integer/uadd-overflow.cpp

-// RUN: %clangxx -DADD_I32 -fsanitize=unsigned-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I32
-// RUN: %clangxx -DADD_I64 -fsanitize=unsigned-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I64
-// RUN: %clangxx -DADD_I128 -fsanitize=unsigned-integer-overflow %s -o %t3 && %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I128
+// RUN: %clangxx -DADD_I32 -fsanitize=unsigned-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I32
+// RUN: %clangxx -DADD_I64 -fsanitize=unsigned-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I64
+// RUN: %clangxx -DADD_I128 -fsanitize=unsigned-integer-overflow %s -o %t3 && not %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-ADD_I128
 
 #include <stdint.h>
 #include <stdio.h>
 
 int main() {
   // These promote to 'int'.
   (void)(uint8_t(0xff) + uint8_t(0xff));
   (void)(uint16_t(0xf0fff) + uint16_t(0x0fff));
#ifdef ADD_I64
   // CHECK-ADD_I64: 10000000000000000000 + 9000000000000000000 cannot be represented in type 'unsigned {{long( long)?}}'
 #endif
 
 #ifdef ADD_I128
 # if defined(__SIZEOF_INT128__) && !defined(_WIN32)
   (void)((__uint128_t(1) << 127) + (__uint128_t(1) << 127));
 # else
   puts("__int128 not supported");
+  return 1;
 # endif
   // CHECK-ADD_I128: {{0x80000000000000000000000000000000 \+ 0x80000000000000000000000000000000 cannot be represented in type 'unsigned __int128'|__int128 not supported}}
 #endif
 }

test/ubsan/TestCases/Integer/uincdec-overflow.cpp

-// RUN: %clangxx -DOP=n++ -fsanitize=unsigned-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck --check-prefix=CHECK-INC %s
-// RUN: %clangxx -DOP=++n -fsanitize=unsigned-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck --check-prefix=CHECK-INC %s
-// RUN: %clangxx -DOP=m-- -fsanitize=unsigned-integer-overflow %s -o %t3 && %run %t3 2>&1 | FileCheck --check-prefix=CHECK-DEC %s
-// RUN: %clangxx -DOP=--m -fsanitize=unsigned-integer-overflow %s -o %t4 && %run %t4 2>&1 | FileCheck --check-prefix=CHECK-DEC %s
+// RUN: %clangxx -DOP=n++ -fsanitize=unsigned-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck --check-prefix=CHECK-INC %s
+// RUN: %clangxx -DOP=++n -fsanitize=unsigned-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck --check-prefix=CHECK-INC %s
+// RUN: %clangxx -DOP=m-- -fsanitize=unsigned-integer-overflow %s -o %t3 && not %run %t3 2>&1 | FileCheck --check-prefix=CHECK-DEC %s
+// RUN: %clangxx -DOP=--m -fsanitize=unsigned-integer-overflow %s -o %t4 && not %run %t4 2>&1 | FileCheck --check-prefix=CHECK-DEC %s
 
 #include <stdint.h>
 
 int main() {
   unsigned n = 0xfffffffd;
   n++;
   n++;
   unsigned m = 0;
   // CHECK-INC: uincdec-overflow.cpp:15:3: runtime error: unsigned integer overflow: 4294967295 + 1 cannot be represented in type 'unsigned int'
   // CHECK-DEC: uincdec-overflow.cpp:15:3: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'
   OP;
 }

test/ubsan/TestCases/Integer/umul-overflow.cpp

-// RUN: %clangxx -fsanitize=unsigned-integer-overflow %s -o %t && %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=unsigned-integer-overflow %s -o %t && not %run %t 2>&1 | FileCheck %s
 
 #include <stdint.h>
 
 int main() {
   // These promote to 'int'.
   (void)(int8_t(-2) * int8_t(0x7f));
   (void)(int16_t(0x7fff) * int16_t(0x7fff));
   (void)(uint16_t(0xffff) * int16_t(0x7fff));

test/ubsan/TestCases/Integer/usub-overflow.cpp

-// RUN: %clangxx -DSUB_I32 -fsanitize=unsigned-integer-overflow %s -o %t1 && %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I32
-// RUN: %clangxx -DSUB_I64 -fsanitize=unsigned-integer-overflow %s -o %t2 && %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I64
-// RUN: %clangxx -DSUB_I128 -fsanitize=unsigned-integer-overflow %s -o %t3 && %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I128
+// RUN: %clangxx -DSUB_I32 -fsanitize=unsigned-integer-overflow %s -o %t1 && not %run %t1 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I32
+// RUN: %clangxx -DSUB_I64 -fsanitize=unsigned-integer-overflow %s -o %t2 && not %run %t2 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I64
+// RUN: %clangxx -DSUB_I128 -fsanitize=unsigned-integer-overflow %s -o %t3 && not %run %t3 2>&1 | FileCheck %s --check-prefix=CHECK-SUB_I128
 
 #include <stdint.h>
 #include <stdio.h>
 
 int main() {
   // These promote to 'int'.
   (void)(uint8_t(0) - uint8_t(0x7f));
   (void)(uint16_t(0) - uint16_t(0x7fff));
 
 #ifdef SUB_I32
   (void)(uint32_t(1) - uint32_t(2));
   // CHECK-SUB_I32: usub-overflow.cpp:[[@LINE-1]]:22: runtime error: unsigned integer overflow: 1 - 2 cannot be represented in type 'unsigned int'
 #endif
 
 #ifdef SUB_I64
   (void)(uint64_t(8000000000000000000ll) - uint64_t(9000000000000000000ll));
   // CHECK-SUB_I64: 8000000000000000000 - 9000000000000000000 cannot be represented in type 'unsigned {{long( long)?}}'
 #endif
 
 #ifdef SUB_I128
 # if defined(__SIZEOF_INT128__) && !defined(_WIN32)
   (void)((__uint128_t(1) << 126) - (__uint128_t(1) << 127));
 # else
   puts("__int128 not supported\n");
+  return 1;
 # endif
   // CHECK-SUB_I128: {{0x40000000000000000000000000000000 - 0x80000000000000000000000000000000 cannot be represented in type 'unsigned __int128'|__int128 not supported}}
 #endif
 }

test/ubsan/TestCases/Misc/bounds.cpp

 // RUN: %clangxx -fsanitize=bounds %s -O3 -o %t
 // RUN: %run %t 0 0 0
 // RUN: %run %t 1 2 3
 // RUN: %expect_crash %run %t 2 0 0 2>&1 | FileCheck %s --check-prefix=CHECK-A-2
-// RUN: %run %t 0 3 0 2>&1 | FileCheck %s --check-prefix=CHECK-B-3
-// RUN: %run %t 0 0 4 2>&1 | FileCheck %s --check-prefix=CHECK-C-4
+// RUN: not %run %t 0 3 0 2>&1 | FileCheck %s --check-prefix=CHECK-B-3
+// RUN: not %run %t 0 0 4 2>&1 | FileCheck %s --check-prefix=CHECK-C-4
 
 int main(int argc, char **argv) {
   int arr[2][3][4] = {};
 
   return arr[argv[1][0] - '0'][argv[2][0] - '0'][argv[3][0] - '0'];
   // CHECK-A-2: bounds.cpp:[[@LINE-1]]:10: runtime error: index 2 out of bounds for type 'int [2][3][4]'
   // CHECK-B-3: bounds.cpp:[[@LINE-2]]:10: runtime error: index 3 out of bounds for type 'int [3][4]'
   // CHECK-C-4: bounds.cpp:[[@LINE-3]]:10: runtime error: index 4 out of bounds for type 'int [4]'
 }

test/ubsan/TestCases/Misc/coverage-levels.cc

 // Test various levels of coverage
 //
 // FIXME: Port the environment variable logic below for the lit shell.
 // REQUIRES: shell
 //
 // RUN: rm -rf %T/coverage-levels && mkdir %T/coverage-levels
 // RUN: %clangxx -fsanitize=shift                        -DGOOD_SHIFT=1 -O1 -fsanitize-coverage=func  %s -o %t
 // RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 --check-prefix=CHECK_NOWARN
 // RUN: %clangxx -fsanitize=undefined                    -DGOOD_SHIFT=1 -O1 -fsanitize-coverage=func  %s -o %t
 // RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 --check-prefix=CHECK_NOWARN
 
 // Also works without any sanitizer.
 // RUN: %clangxx                                         -DGOOD_SHIFT=1 -O1 -fsanitize-coverage=func  %s -o %t
 // RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 --check-prefix=CHECK_NOWARN
 
 // RUN: %clangxx -fsanitize=shift -O1 -fsanitize-coverage=func  %s -o %t
-// RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 --check-prefix=CHECK_WARN
+// RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK1 --check-prefix=CHECK_WARN
 // RUN: %clangxx -fsanitize=shift -O1 -fsanitize-coverage=bb  %s -o %t
-// RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 --check-prefix=CHECK_WARN
+// RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK2 --check-prefix=CHECK_WARN
 // RUN: %clangxx -fsanitize=shift -O1 -fsanitize-coverage=edge  %s -o %t
-// RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' %run %t 2>&1 | FileCheck %s --check-prefix=CHECK3 --check-prefix=CHECK_WARN
+// RUN: %env_ubsan_opts=coverage=1:verbosity=1:coverage_dir='"%T/coverage-levels"' not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK3 --check-prefix=CHECK_WARN
 
 // Coverage is not yet implemented in TSan.
 // XFAIL: ubsan-tsan
 
 volatile int sink;
 int main(int argc, char **argv) {
   int shift = argc * 32;
 #if GOOD_SHIFT

test/ubsan/TestCases/Misc/deduplication.cpp

-// RUN: %clangxx -fsanitize=undefined %s -o %t && %run %t 2>&1 | FileCheck %s
+// RUN: %clangxx -fsanitize=undefined %s -o %t && not %run %t 2>&1 | FileCheck %s
 // Verify deduplication works by ensuring only one diag is emitted.
 #include <limits.h>
 #include <stdio.h>
 
 void overflow() {
   int i = INT_MIN;
   --i;
 }

test/ubsan/TestCases/Misc/enum.cpp

-// RUN: %clangxx -fsanitize=enum %s -O3 -o %t && %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-PLAIN
+// RUN: %clangxx -fsanitize=enum %s -O3 -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-PLAIN
 // RUN: %clangxx -fsanitize=enum -std=c++11 -DE="class E" %s -O3 -o %t && %run %t
 // RUN: %clangxx -fsanitize=enum -std=c++11 -DE="class E : bool" %s -O3 -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-BOOL
 
 // FIXME: UBSan fails to add the correct instrumentation code for some reason on
 // Windows.
 // XFAIL: win32
 
 enum E { a = 1 } e;

test/ubsan/TestCases/Misc/exitcode.cc

+// RUN: %clangxx -fsanitize=undefined -DDIVISOR=0 %s -o %t-ZERO
+// RUN: %clangxx -fsanitize=undefined -DDIVISOR=1 %s -o %t-ONE
+// RUN: not %run %t-ZERO 2>&1 | FileCheck %s
+// RUN: %env_ubsan_opts=exitcode=15 not %run %t-ZERO 2>&1 | FileCheck %s
+// RUN: %env_ubsan_opts=exitcode=0 %run %t-ZERO 2>&1 | FileCheck %s
+// RUN: %run %t-ONE 2>&1 | count 0
+
+int main() {
+  int a = 1 / DIVISOR;
+  return 0;
+}
+
+// CHECK: runtime error: division by zero

test/ubsan/TestCases/Misc/log-path_test.cc

 // FIXME: https://code.google.com/p/address-sanitizer/issues/detail?id=316
 // XFAIL: android
 
 // The globs below do not work in the lit shell.
 // REQUIRES: shell
 
 // RUN: %clangxx -fsanitize=undefined %s -O1 -o %t
 
 // Regular run.
-// RUN: %run %t -4 2> %t.out
+// RUN: not %run %t -4 2> %t.out
 // RUN: FileCheck %s --check-prefix=CHECK-ERROR < %t.out
 
 // Good log_path.
 // RUN: rm -f %t.log.*
-// RUN: %env_ubsan_opts=log_path='"%t.log"' %run %t -4 2> %t.out
+// RUN: %env_ubsan_opts=log_path='"%t.log"' not %run %t -4 2> %t.out
 // RUN: FileCheck %s --check-prefix=CHECK-ERROR < %t.log.*
 
 // Run w/o errors should not produce any log.
 // RUN: rm -f %t.log.*
 // RUN: %env_ubsan_opts=log_path='"%t.log"'  %run %t 4
 // RUN: not cat %t.log.*
 
 // FIXME: log_path is not supported on Windows yet.

test/ubsan/TestCases/Misc/nonnull.cpp

 // RUN: %clangxx -fsanitize=returns-nonnull-attribute -w %s -O3 -o %t
 // RUN: %run %t foo 2>&1 | count 0
-// RUN: %run %t 2>&1 | FileCheck %s
+// RUN: not %run %t 2>&1 | FileCheck %s
 // RUN: %clangxx -fsanitize=returns-nonnull-attribute -fno-sanitize-recover=returns-nonnull-attribute -w %s -O3 -o %t.abort
 // RUN: not %run %t.abort &> /dev/null
 
 __attribute__((returns_nonnull)) char *foo(char *a);
 
 char *foo(char *a) {
   // CHECK: nonnull.cpp:[[@LINE+2]]:3: runtime error: null pointer returned from function declared to never return null
   // CHECK-NEXT: nonnull.cpp:[[@LINE-4]]:16: note: returns_nonnull attribute specified here

test/ubsan/TestCases/Misc/nullability.c

 // RUN: %clang -w -fsanitize=nullability-arg,nullability-assign,nullability-return %s -O3 -o %t
 // RUN: %run %t foo 2>&1 | count 0
-// RUN: %run %t 2>&1 | FileCheck %s
+// RUN: not %run %t 2>&1 | FileCheck %s
 
 // CHECK: nullability.c:[[@LINE+2]]:41: runtime error: null pointer returned from function declared to never return null
 // CHECK-NEXT: nullability.c:[[@LINE+1]]:6: note: _Nonnull return type annotation specified here
 int *_Nonnull nonnull_retval1(int *p) { return p; }
 
 // CHECK: nullability.c:1001:22: runtime error: null pointer passed as argument 2, which is declared to never be null
 // CHECK-NEXT: nullability.c:[[@LINE+1]]:56: note: _Nonnull type annotation specified here
 int *_Nonnull nonnull_retval2(int *_Nonnull arg1, int *_Nonnull arg2,

test/ubsan/TestCases/Misc/vla.c

 // RUN: %clang -fsanitize=vla-bound %s -O3 -o %t
-// RUN: %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-MINUS-ONE
-// RUN: %run %t a 2>&1 | FileCheck %s --check-prefix=CHECK-ZERO
+// RUN: not %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-MINUS-ONE
+// RUN: not %run %t a 2>&1 | FileCheck %s --check-prefix=CHECK-ZERO
 // RUN: %run %t a b
 
 int main(int argc, char **argv) {
   // CHECK-MINUS-ONE: vla.c:[[@LINE+2]]:11: runtime error: variable length array bound evaluates to non-positive value -1
   // CHECK-ZERO: vla.c:[[@LINE+1]]:11: runtime error: variable length array bound evaluates to non-positive value 0
   int arr[argc - 2];
   return 0;
 }

test/ubsan/TestCases/Pointer/index-overflow.cpp

 // RUN: %clangxx -fsanitize=pointer-overflow %s -o %t
-// RUN: %t 1 2>&1 | FileCheck %s --check-prefix=ERR
+// RUN: not %t 1 2>&1 | FileCheck %s --check-prefix=ERR
 // RUN: %t 0 2>&1 | FileCheck %s --check-prefix=SAFE
 // RUN: %t -1 2>&1 | FileCheck %s --check-prefix=SAFE
 
 #include <stdio.h>
 #include <stdint.h>
 #include <stdlib.h>
 
 int main(int argc, char *argv[]) {

test/ubsan/TestCases/Pointer/unsigned-index-expression.cpp

 // RUN: %clangxx -fsanitize=pointer-overflow %s -o %t
-// RUN: %t 2>&1 | FileCheck %s
+// RUN: not %t 2>&1 | FileCheck %s
 
 int main(int argc, char *argv[]) {
   char c;
   char *p = &c;
   unsigned long long offset = -1;
 
   // CHECK: unsigned-index-expression.cpp:[[@LINE+1]]:15: runtime error: unsigned pointer index expression result is 0x{{.*}}, preceding its base 0x{{.*}}
   char *q = p + offset;
 
   return 0;
 }

test/ubsan/TestCases/TypeCheck/Function/function.cpp

 // RUN: %clangxx -fsanitize=function %s -O3 -g -o %t
-// RUN: %run %t 2>&1 | FileCheck %s
+// RUN: not %run %t 2>&1 | FileCheck %s
 // Verify that we can disable symbolization if needed:
-// RUN: %env_ubsan_opts=symbolize=0 %run %t 2>&1 | FileCheck %s --check-prefix=NOSYM
+// RUN: %env_ubsan_opts=symbolize=0 not %run %t 2>&1 | FileCheck %s --check-prefix=NOSYM
 
 // -fsanitize=function is unsupported on Darwin yet.
 // XFAIL: darwin
 
 #include <stdint.h>
 
 void f() {}
 

test/ubsan/TestCases/TypeCheck/Linux/PR33221.cpp

 // RUN: %clangxx -std=c++11 -frtti -fsanitize=vptr -g %s -O3 -o %t
-// RUN: %run %t &> %t.log
+// RUN: not %run %t &> %t.log
 // RUN: cat %t.log | not count 0 && FileCheck --input-file %t.log %s || cat %t.log | count 0
 
 // REQUIRES: cxxabi
 
 #include <sys/mman.h>
 #include <unistd.h>
 
 class Base {
 public:
   int i;
   virtual void print() {}
 };
 
 class Derived : public Base {
 public:
   void print() {}
 };
 
 
 int main() {
   int page_size = getpagesize();
 
   void *non_accessible = mmap(nullptr, page_size, PROT_NONE,
                               MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
-  
   if (non_accessible == MAP_FAILED)
     return 0;
 
   void *accessible = mmap((char*)non_accessible + page_size, page_size,
                           PROT_READ | PROT_WRITE,
                           MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
   if (accessible == MAP_FAILED)
     return 0;

test/ubsan/TestCases/TypeCheck/PR33221.cpp

 // RUN: %clangxx -frtti -fsanitize=vptr -g %s -O3 -o %t
-// RUN: %run %t 2>&1 | FileCheck %s
+// RUN: not %run %t 2>&1 | FileCheck %s
 
 // REQUIRES: cxxabi
 
 #include <string.h>
 
 class Base {
 public:
   int i;

test/ubsan/TestCases/TypeCheck/misaligned.cpp

 // RUN: %clangxx %gmlt -fsanitize=alignment %s -O3 -o %t
 // RUN: %run %t l0 && %run %t s0 && %run %t r0 && %run %t m0 && %run %t f0 && %run %t n0 && %run %t u0
-// RUN: %run %t l1 2>&1 | FileCheck %s --check-prefix=CHECK-LOAD --strict-whitespace
-// RUN: %run %t s1 2>&1 | FileCheck %s --check-prefix=CHECK-STORE
-// RUN: %run %t r1 2>&1 | FileCheck %s --check-prefix=CHECK-REFERENCE
-// RUN: %run %t m1 2>&1 | FileCheck %s --check-prefix=CHECK-MEMBER
-// RUN: %run %t f1 2>&1 | FileCheck %s --check-prefix=CHECK-MEMFUN
-// RUN: %run %t n1 2>&1 | FileCheck %s --check-prefix=CHECK-NEW
-// RUN: %run %t u1 2>&1 | FileCheck %s --check-prefix=CHECK-UPCAST
-// RUN: %env_ubsan_opts=print_stacktrace=1 %run %t l1 2>&1 | FileCheck %s --check-prefix=CHECK-LOAD --check-prefix=CHECK-STACK-LOAD
+// RUN: not %run %t l1 2>&1 | FileCheck %s --check-prefix=CHECK-LOAD --strict-whitespace
+// RUN: not %run %t s1 2>&1 | FileCheck %s --check-prefix=CHECK-STORE
+// RUN: not %run %t r1 2>&1 | FileCheck %s --check-prefix=CHECK-REFERENCE
+// RUN: not %run %t m1 2>&1 | FileCheck %s --check-prefix=CHECK-MEMBER
+// RUN: not %run %t f1 2>&1 | FileCheck %s --check-prefix=CHECK-MEMFUN
+// RUN: not %run %t n1 2>&1 | FileCheck %s --check-prefix=CHECK-NEW
+// RUN: not %run %t u1 2>&1 | FileCheck %s --check-prefix=CHECK-UPCAST
+// RUN: %env_ubsan_opts=print_stacktrace=1 not %run %t l1 2>&1 | FileCheck %s --check-prefix=CHECK-LOAD --check-prefix=CHECK-STACK-LOAD
 
 // RUN: %clangxx -fsanitize=alignment -fno-sanitize-recover=alignment %s -O3 -o %t
 // RUN: not %run %t w1 2>&1 | FileCheck %s --check-prefix=CHECK-WILD
 
 #include <new>
 
 struct S {
   S() {}