[ASan] Enable optional ASan recovery

[ygribov]

Summary

This patch adds flags to clang and libasan to continue execution after detection of first error (in hope that more errors can be detected during single run). Such mode allows for a faster run-report-fix loop on embedded system where fixing a bug and preparing new firmware may take a significant time. To enable, user should build his code with -fsanitize-recover=address and then run it under ASAN_OPTIONS=keep_going=1.

I know kcc's generally negative attitude towards keep_going mode but I want the patch to at least be available publicly so that poor souls dealing with complex systems may access it easily.

[eugenis]

include/clang/Frontend/CodeGenOptions.def

118

Could you make it not specific to ASan? Smth like SanitizeRecover so that it could affect MSan as well in the future.

lib/CodeGen/BackendUtil.cpp

343

Why do you need separate "noabort" functions for this? See addMemorySanitizerPass which inspects CodeGenOptions in the function body.

lib/Transforms/Instrumentation/AddressSanitizer.cpp

125

Lets stick to existing terminology (we already have "recover" and "keep-going" names for the same concept).

lib/asan/asan_flags.inc

147

call it max_errors, similarly to the compiler flag?

lib/asan/asan_poisoning.cc

221

why "false"? Do you need to check the keep_going flag?

[kcc]

lib/asan/asan_report.cc

652

OMG

[ygribov]

include/clang/Frontend/CodeGenOptions.def

118

Right.

lib/CodeGen/BackendUtil.cpp

343

Ok.

lib/Transforms/Instrumentation/AddressSanitizer.cpp

125

Ok, I'll use "recover" then. Most of this was done in pre-recover times.

lib/asan/asan_flags.inc

147

Sure.

lib/asan/asan_poisoning.cc

221

Hm, probably yes.

lib/asan/asan_report.cc

652

This section could be removed if we are ok with ignoring bugs that run in parallel.

[ygribov]

lib/asan/asan_report.cc

652

Could you suggest a cleaner way to do this?

[ygribov]

Updated based on initial feedback. It seems we have way too many different names for recovery feature (UBSan's trap-on-error, MSan's keep-going, ASan's recover, noabort, etc.).

[samsonov]

include/clang/Basic/Sanitizers.h

82

Do you actually use it anywhere?

lib/CodeGen/BackendUtil.cpp

216

Is recovery supposed to work for -fsanitize=kernel-address? Should you mark it as Unrecoverable?

lib/asan/asan_flags.inc

144

This is really sad, but we tend to use halt_on_error runtime flag for that. This name is already used in TSan, MSan and UBSan.

lib/asan/asan_interface_internal.h

134

This function is also declared in public header: include/sanitizer/asan_interface.h (which is kind of stupid), so you have to fix it in both places. Looks like it's not an ABI break, though.

lib/asan/asan_internal.h

129

We have atomic_fetch_add

[samsonov]

In D12318#233103, @ygribov wrote:

Updated based on initial feedback. It seems we have way too many different names for recovery feature (UBSan's trap-on-error, MSan's keep-going, ASan's recover, noabort, etc.).

Right, naming is hard :( Speaking of user-visible flags, we should use -fsanitize-recover=address flag (UBSan's trap-on-error corresponds to -fsanitize-trap= and is slightly different).

[ygribov]

include/clang/Basic/Sanitizers.h

82

No, that's remains of some removed code.

lib/CodeGen/BackendUtil.cpp

216

KAsan is actually recoverable by default. Perhaps I should do explicit Recover for CompileKernel as well.

lib/asan/asan_internal.h

129

Yeah but it may be slower.

[filcab]

What about the file test/asan/TestCases/Darwin/interface_symbols_darwin.c? Shouldn't it be changed too?

lib/asan/asan_report.cc

695–700

Comment on line 632 would apply here too.

697

The usual "I see no benefit in using LIKELY here".

lib/asan/scripts/asan_symbolize.py

429

Why? I might have missed the reason for this change.

test/Driver/fsanitize.c

168

Shouldn't address be added here too?

[filcab]

lib/asan/asan_internal.h

125

Technically it might be unlikely, but this is noise in the code.
If we're not recovering, we will blow up anyway. Removing UNLIKELY would make the code easier to read with 0 practical difference in performance for any case.
In fact, it we're not silencing, *then* we want to be as fast as possible. (I'd go with the simple if, though)

129

Slower is better than undefined behavior, no? ;-)

lib/asan/asan_report.cc

632–644

cmpxchg should work, no?

646–664

Again, no benefit in tagging these as UNLIKELY.

652

At the very least, you can create a helper function with the code executed when you get the lock, no?

659

This comment should be changed, too. It's not true after this diff gets applied.

661

Why did you remove the first part of the comment?
It's still the reason why we have the sleep call here.

[ygribov]

Thanks a lot for detailed review!

What about the file test/asan/TestCases/Darwin/interface_symbols_darwin.c? Shouldn't it be changed too?

Yup.

lib/asan/asan_internal.h

125

Sorry, could you give more details? I indeed favor the default (non-recovery) case here. As for the UNLIKELY, we've seen cases where it significantly improved performance.

129

Again, I was probably too scary to slow down the default path. I'll switch to atomics in next version then.

lib/asan/asan_poisoning.cc

221

BTW what are these sanitizer_unaligned functions for? I don't think compiler emits then but they are also missing in public headers.

lib/asan/asan_report.cc

632–644

Yup.

652

Ah, so it's all about this tiny goto (not about hacky trial-and-error mutex acquisition). Ok, I'll put the code to a separate function.

659

Right.

661

I moved the first part of the comment above because it's applicable to both branches.

lib/asan/scripts/asan_symbolize.py

429

Hm, I don't remember doing this hunk at all. I'll double check when at work.

test/Driver/fsanitize.c

168

Ok.

[ygribov]

Updated based on reviews.

[filcab]

Still missing the Darwin interface symbols file.

Thanks for working on this.

Filipe

lib/asan/asan_interceptors.cc

73

Why false? Shouldn't it be the flag for halt_on_error?

lib/asan/asan_internal.h

125

So, I see two things that can happen:

halt_on_error is true:
We go through the if once (maybe more times, if we find several errors in several threads).
QuickCheckSilence returns false
LIKELY Isn't really doing much. Tweaking a jump that is executed once doesn't really matter.

halt_on_error is false:
We go through the if roughly max_errors times (let's say around 100, which seems to me is already a too big number for max_errors (if you actually manage to skip 100 errors and still execute code, that's a huge amount of luck!)

From these two, I would actually say that, at most, you should have an UNLIKELY, and not a LIKELY. *If* we execute that if more than once, then I should always expect it to be false.

Still, even in the case where we execute it "a bunch of times" we're still "almost never" executing it. QuickCheckSilence is called only when reporting errors, which means that:
If halt_on_error, then the program is terminating right after this report. We're not winning anything.
If halt_on_error, then the LIKELY is reversed. And we're still not winning much, we're not expecting this code to be executed more than max_errors, which, I'm assuming, will always be a small number.

My opinion is that it's not making the code faster, and it's making it less clear.

You mentioned performance increases. What tests were you running, and how?

131

You can change the previous atomic_load to be an atomic_fetch_add of 0 or 1, no?

test/Driver/fsanitize.c

168

Why didn't you update the number too? Does the test pass?

test/asan/TestCases/Linux/halt_on_error-torture.cc

1 ↗

(On Diff #33321)

Why is this test linux only?

[kcc]

lib/asan/asan_flags.inc

147

Shouldn't this go to sanitizer_common?

lib/asan/asan_interceptors.cc

52

why do we need it here?

lib/asan/asan_internal.h

124

Does it have to be static inline?
Also, I don't like the name. But it depends on another question later.

lib/asan/asan_report.cc

29

Don't define this macro, just use the C++ line as is

697

Agree.

1014

I think we should revert the name and the meaning of the flag:
make it "int recover" instead of "int fatal" to match the flag.
No?

Also, we need to make the parameter more clear.
One way is to declare a enum {ABORT_ON_ERROR, RECOVER_ON_ERROR} and pass it as the parameter.
Another is to
define two more internal functions
asan_report_error_and_abort() &
asan_report_error_and_maybe_recover()
and instead of calling __asan_report_error with a constant in the last parameter call one of these.

At the very least, write /*recover=*/false

test/asan/TestCases/Linux/halt_on_error-torture.cc

29 ↗

(On Diff #33321)

don't we want to check that multiple errors were reported?

check how the tests set ASAN_OPTIONS nowadays.
There were some massive changes...

44 ↗

(On Diff #33321)

use C++ new/delete

[ygribov]

Still missing the Darwin interface symbols file.

Right, I iterated Phab's comments and completely forgot about this item. I can add a test but I don't have Mac at hand so I'll have to XFAIL it if problem arises after commit.

lib/asan/asan_flags.inc

147

Hm, perhaps we should implement it for UBSan/MSan and then move to sanitizer_common?

lib/asan/asan_interceptors.cc

52

This enables fast path once error reporting limit is reached. Otherwise an error in a busy loop may cause significant (100x) performance degradation. One might argue that we should not be interested in continuing execution at all after we can't report more errors but in fully sanitized distribution it may be beneficial to continue execution to provoke errors in dependent processes.

73

This is covered by quick check above.

lib/asan/asan_internal.h

124

It doesn't but I thought this would be more efficient.

125

halt_on_error is true:
We go through the if once (maybe more times, if we find several errors in several threads).

I don't quite see it like this. Quick check may be called in non-fatal context (e.g. to enable fast path in ACCESS_MEMORY_RANGE). Information about likeliness may inform compiler to e.g. allocate registers in favor of preferred code or align it in a more appropriate way (e.g. to the same cache line).

You mentioned performance increases. What tests were you running, and how?

Check https://www.mail-archive.com/[e-mail]/msg00403.html . Adding unlikely hint to noreturning if improved code by some 40%.

131

Well yeah but this would add a risk of overflow.

lib/asan/asan_report.cc

29

Ok.

1014

Frankly I don't like the idea of adding fatal/recover parameter to public interface at all. Fatality should be controlled by ASAN_OPTIONS and be transparent for user code. Thoughts?

test/Driver/fsanitize.c

168

Yup, it did. I'll reexamine it.

test/asan/TestCases/Linux/halt_on_error-torture.cc

1 ↗

(On Diff #33321)

Mainly because I'm afraid that Mac/BSD maintainers will shout at me when I break their bots)

29 ↗

(On Diff #33321)

Ok.

44 ↗

(On Diff #33321)

Ok.

[ygribov]

Updated based on review. I haven't yet made torture test generic because I'm yet to test this patch on Mac (I plan to do this for next iteration).

[ygribov]

test/Driver/fsanitize.c

168

This should be ok - recovery is not enabled by default for ASan so resulting number didn't change.

[ygribov]

Folks, any feedback on this?

[kcc]

The patch is already too long and has too many comments.
Some parts are less controversial, some are more.
I suggest you to split it at this point and start reviewing/submitting it in parts.
Probably start from the driver/flags changes.

lib/Transforms/Instrumentation/AddressSanitizer.cpp

1081–1086

I think we use {} in such cases (if followed by an else with {})

lib/asan/asan_flags.inc

147

No, just add it to sanitizer_common and write something like ("currently supported by asan") in the description.

If we add things that are potentially useful to other sanitizers to one of them (not to sanitizer_common)
we then end up having duplicates in all sanitizers.

lib/asan/asan_internal.h

124

I don't like the name,
I don't like the fact that it has a parameter,
I don't like that it's static inline,
I don't like its behavior.

We have two situations:

1. We are reporting an error and don't want to do it (is_error=true).

There is no harm in making this function non-inline in this case.
bool WantToReportMoreError();

2. We don't know whether there will be an error, but we are shutting down the checking functionality

just in case (or to win a bit of performance).
I don't think we need to do this at all.
The win is not clear, but the added complexity is.

lib/asan/asan_report.cc

633

This is all exceptionally horrible.
I've spent quite a bit of time and failed to understand how it works,
also I don't understand how to test it.
Why not just grab a lock for the entire duration of error reporting?

The current logic was there because we never reported more than one bug,
now that we do -- simply grab a lock!

703

class members have the _ suffix.

lib/asan/asan_rtl.cc

131

please leave __asan_report_error here. It is used as a target for break points in debugger.

You may then check halt_on_error inside __asan_report_error

[kcc]

One more interesting question that came to my mind: is this going to be async-signal-safe?
If you think it will, please add tests.
If not, the whole idea is kind of questionable.

[ygribov]

Some parts are less controversial, some are more.

What's your general impression on the feature though?

I suggest you to split it at this point and start reviewing/submitting it in parts.
Probably start from the driver/flags changes.

Hm but I don't see any functionality which can be moved to independent commit (except the flags part which is really minor). We could split to separate patches for rt, clang and llvm but I don't see how this is going to simplify review.

lib/asan/asan_flags.inc

147

If we add things that are potentially useful to other sanitizers to one of them (not to sanitizer_common)
we then end up having duplicates in all sanitizers.

That makes sense, we already have too many semi-duplicate flags.

lib/asan/asan_internal.h

124

Problem is that lack of this check would cause program to crawl in some cases. But ok, this may live in a private branch.

lib/asan/asan_report.cc

633

also I don't understand how to test it.

Did you ever understand how to test original code :) ? The behavior for fatal errors didn't change by the way.

Why not just grab a lock for the entire duration of error reporting?

Because we may run into second error while reporting the first one in the same thread. Grabbing a lock would cause a deadlock in this case.

703

I'm not sure that's true for statics, I see numerous places where they aren't undered.

[ygribov]

One more interesting question that came to my mind: is this going to be async-signal-safe?
If not, the whole idea is kind of questionable.

I don't see why it shouldn't - the only problematic place is acquiring the mutex which will either wait (in case reporting_tid is different from current_tid) or fail with informative message after timeout.

If you think it will, please add tests.

I could generate errors in endless loop in a thread and bombard it with signals but that's probably not very reliable.

[kcc]

In D12318#240258, @ygribov wrote:

One more interesting question that came to my mind: is this going to be async-signal-safe?
If not, the whole idea is kind of questionable.

I don't see why it shouldn't - the only problematic place is acquiring the mutex which will either wait (in case reporting_tid is different from current_tid) or fail with informative message after timeout.

If you think it will, please add tests.

I could generate errors in endless loop in a thread and bombard it with signals but that's probably not very reliable.

The more interesting would be to test the case where we have asan-ish reports in signal handlers (and at the same time in regular code).

[kcc]

What's your general impression on the feature though?

Same as before.
It is an anti-feature for the majority of users and it will bring much additional complexity to the code
and extra maintenance cost.

I am willing to go forward with it because for some (still obscure) reasons you need this feature
and because your contribution is valuable.

[kcc]

Hm but I don't see any functionality which can be moved to independent commit (except the flags part which is really minor). We could split to separate patches for rt, clang and llvm but I don't see how this is going to simplify review.

Review is an O(N^2) process. The smaller are the patches the quicker you get through.

[kcc]

lib/asan/asan_report.cc

633

Did you ever understand how to test original code :) ?

That's not an argument. We don't want to make things worse just because they are not good at the moment.

Because we may run into second error while reporting the first one in the same thread. Grabbing a lock would cause a deadlock in this case.

Yea. Async signal safety...

[ygribov]

Folks,

In case anyone wants this feature it's probably good time to speak up. If there's no interest, I'll probably abandon this revision in a few days.

-Y

[eugenis]

I think this is a useful feature but I'm a bit worried about the implementation complexity.

We've been running entire Android devices with ASan, and having them crash at boot due to a single bug is moderately annoying.

For the reference, https://source.android.com/devices/tech/debug/asan.html#sanitize_target.

[ygribov]

I think this is a useful feature but I'm a bit worried about the implementation complexity.

You mean the report lock part? I agree, although I'm not sure how to simplify it. Perhaps always abort in case of conflict?

We've been running entire Android devices with ASan, and having them crash at boot due to a single bug is moderately annoying.

Yup, that's my use-case as well.

[kcc]

Let's try to finish it.
I won't be able to participate in the review before 28-th, but feel free to
continue/finish w/o me.

[eugenis]

lib/asan/asan_report.cc

671

I think this timeout complicates things without any benefit. Why not wait indefinitely?

I think the logic should be like this:

1. if the current thread is already reporting, die immediately
2. if (recover) {lock()} else { if (!try_lock()) sleep(inf) }
3. report
4. unlock()

1014–1015

I agree, if the outside caller thinks the error is fatal, it can just abort() or something.

[samsonov]

lib/asan/asan_internal.h

124

The name is really confusing. It should be smth. like IgnoreError. Also, instead of passing is_error as a parameter, and doing conditional atomic_fetch_add, can you just split this function into two: one would check if need to ignore the error (halt_on_error is true, or number of reported errors is too large already), and another one would increment the number of detected errors. (could you then call the latter one from ScopedInErrorReport constructor)?

Can you make it ALWAYS_INLINE?

lib/asan/asan_report.cc

703

There are no trailing _ in simple structs, but there should be in classes.

[ygribov]

Sorry, I got distracted by other projects. I'm mostly done with all remarks with just one issue (can't commonize halt_on_error).

lib/asan/asan_flags.inc

147

Here's the problem - we need two halt_on_error's for ASan and integrated UBSan. Another problem is that msan does some local magic to alias current halt_on_error to keep_going flag.

lib/asan/asan_report.cc

633

Funny enough, even current ASan isn't asynch signal stable. If you have repeated error in nodeferred signals which are being send to a thread, it may deadlock during error reporting in ScopedInErrorLock. The sequence is roughly:

• kernel delivers signal
• trigger error
• lock report file
• call internal_write
• enter kernel
• kernel delivers next nodeferred signal
• trigger the same error
• etc.

[ygribov]

I've addressed the remarks from previous review except the one for halt_on_error (we want separate halt_on_error for ASan and UBSan). The llvm and clang parts of the patch will be submitted for review separately per Kostya's request.

[ygribov]

BTW I haven't yet ported tests to Darwin, this is something tbd.

[ygribov]

lib/asan/asan_report.cc

651

This could be much simpler if we had recursive locks.

[eugenis]

LGTM
Thanks for working on this!

[ygribov]

BTW I haven't yet ported tests to Darwin, this is something tbd.

Filipe, Kuba, the feature has been merged to upstream and seems to work robustly. But I don't think I'll have time to test it on OS X in near future (I don't have easy access to hardware and my customer does not need it). So I was wondering whether you could help? I don't expect any errors, just maybe some really simple portability fixes.

[kubamracek]

In D12318#290215, @ygribov wrote:

BTW I haven't yet ported tests to Darwin, this is something tbd.

Filipe, Kuba, the feature has been merged to upstream and seems to work robustly. But I don't think I'll have time to test it on OS X in near future (I don't have easy access to hardware and my customer does not need it). So I was wondering whether you could help? I don't expect any errors, just maybe some really simple portability fixes.

Hi, are you talking about the halt_on_error-signals and halt_on_error-torture testcases? On my OS X machine, they both pass, so you might just want to move them out of the Linux/ directory. Let me know if the OS X buildbot fails and I'll take a look at it.

[ygribov]

On my OS X machine, they both pass, so you might just want to move them out of the Linux/ directory

Great, thanks!

include/clang/Basic/Sanitizers.h

struct SanitizerSet {
   /// \brief Bitmask of enabled sanitizers.
   SanitizerMask Mask;
 };
 
 /// Parse a single value from a -fsanitize= or -fno-sanitize= value list.
 /// Returns a non-zero SanitizerMask, or \c 0 if \p Value is not known.
 SanitizerMask parseSanitizerValue(StringRef Value, bool AllowGroups);
 
+/// Parse a single value from a -fsanitize= or -fno-sanitize= value list.
+/// Returns a non-zero SanitizerMask, or \c 0 if \p Value is not known.
+const char *getSanitizerName(SanitizerKind::SanitizerOrdinal SO);
+
 /// For each sanitizer group bit set in \p Kinds, set the bits for sanitizers
 /// this group enables.
 SanitizerMask expandSanitizerGroups(SanitizerMask Kinds);
 
 }  // end namespace clang
 
 #endif

include/clang/Frontend/CodeGenOptions.def

 ENUM_CODEGENOPT(StructReturnConvention, StructReturnConventionKind, 2, SRCK_Default)
 
 CODEGENOPT(RelaxAll          , 1, 0) ///< Relax all machine code instructions.
 CODEGENOPT(RelaxedAliasing   , 1, 0) ///< Set when -fno-strict-aliasing is enabled.
 CODEGENOPT(StructPathTBAA    , 1, 0) ///< Whether or not to use struct-path TBAA.
 CODEGENOPT(SaveTempLabels    , 1, 0) ///< Save temporary labels.
 CODEGENOPT(SanitizeAddressZeroBaseShadow , 1, 0) ///< Map shadow memory at zero
                                                  ///< offset in AddressSanitizer.
+CODEGENOPT(SanitizeAddressNoAbort, 1, 0) ///< Enable continue-after-error mode
+                                         ///< in AddressSanitizer.
 CODEGENOPT(SanitizeMemoryTrackOrigins, 2, 0) ///< Enable tracking origins in
                                              ///< MemorySanitizer
 CODEGENOPT(SanitizeMemoryUseAfterDtor, 1, 0) ///< Enable use-after-delete detection
                                              ///< in MemorySanitizer
 CODEGENOPT(SanitizeCoverageType, 2, 0) ///< Type of sanitizer coverage
                                        ///< instrumentation.
 CODEGENOPT(SanitizeCoverageIndirectCalls, 1, 0) ///< Enable sanitizer coverage
                                                 ///< for indirect calls.

include/llvm/Transforms/Instrumentation.h

struct InstrProfOptions {
   std::string InstrProfileOutput;
 };
 
 /// Insert frontend instrumentation based profiling.
 ModulePass *createInstrProfilingPass(
     const InstrProfOptions &Options = InstrProfOptions());
 
 // Insert AddressSanitizer (address sanity checking) instrumentation
-FunctionPass *createAddressSanitizerFunctionPass(bool CompileKernel = false);
-ModulePass *createAddressSanitizerModulePass(bool CompileKernel = false);
+FunctionPass *createAddressSanitizerFunctionPass(bool CompileKernel = false,
+                                                 bool NoAbort = false);
+ModulePass *createAddressSanitizerModulePass(bool CompileKernel = false,
+                                             bool NoAbort = false);
 
 // Insert MemorySanitizer instrumentation (detection of uninitialized reads)
 FunctionPass *createMemorySanitizerPass(int TrackOrigins = 0);
 
 // Insert ThreadSanitizer (race detection) instrumentation
 FunctionPass *createThreadSanitizerPass();
 
 // Insert DataFlowSanitizer (dynamic data flow analysis) instrumentation

lib/CodeGen/BackendUtil.cpp

 }
 
 static void addAddressSanitizerPasses(const PassManagerBuilder &Builder,
                                       legacy::PassManagerBase &PM) {
   PM.add(createAddressSanitizerFunctionPass(/*CompileKernel*/false));
   PM.add(createAddressSanitizerModulePass(/*CompileKernel*/false));
 }
 
+static void addAddressSanitizerNoAbortPasses(const PassManagerBuilder &Builder,
+                                      legacy::PassManagerBase &PM) {
+  PM.add(createAddressSanitizerFunctionPass(/*CompileKernel*/false, /*NoAbort*/true));
+  PM.add(createAddressSanitizerModulePass(/*CompileKernel*/false, /*NoAbort*/true));
+}
+
 static void addKernelAddressSanitizerPasses(const PassManagerBuilder &Builder,
                                             legacy::PassManagerBase &PM) {
   PM.add(createAddressSanitizerFunctionPass(/*CompileKernel*/true));
   PM.add(createAddressSanitizerModulePass(/*CompileKernel*/true));
 }
 
 static void addMemorySanitizerPass(const PassManagerBuilder &Builder,
                                    legacy::PassManagerBase &PM) {
   const PassManagerBuilderWrapper &BuilderWrapper =
       static_cast<const PassManagerBuilderWrapper&>(Builder);
   const CodeGenOptions &CGOpts = BuilderWrapper.getCGOpts();
if (CodeGenOpts.SanitizeCoverageType ||
       CodeGenOpts.SanitizeCoverageTraceCmp) {
     PMBuilder.addExtension(PassManagerBuilder::EP_OptimizerLast,
                            addSanitizerCoveragePass);
     PMBuilder.addExtension(PassManagerBuilder::EP_EnabledOnOptLevel0,
                            addSanitizerCoveragePass);
   }
 
   if (LangOpts.Sanitize.has(SanitizerKind::Address)) {
+    if (CodeGenOpts.SanitizeAddressNoAbort) {
+      PMBuilder.addExtension(PassManagerBuilder::EP_OptimizerLast,
+                             addAddressSanitizerNoAbortPasses);
+      PMBuilder.addExtension(PassManagerBuilder::EP_EnabledOnOptLevel0,
+                             addAddressSanitizerNoAbortPasses);
+    } else {
       PMBuilder.addExtension(PassManagerBuilder::EP_OptimizerLast,
                              addAddressSanitizerPasses);
       PMBuilder.addExtension(PassManagerBuilder::EP_EnabledOnOptLevel0,
                              addAddressSanitizerPasses);
     }
+  }
 
   if (LangOpts.Sanitize.has(SanitizerKind::KernelAddress)) {
     PMBuilder.addExtension(PassManagerBuilder::EP_OptimizerLast,
                            addKernelAddressSanitizerPasses);
     PMBuilder.addExtension(PassManagerBuilder::EP_EnabledOnOptLevel0,
                            addKernelAddressSanitizerPasses);
   }
 

lib/Driver/SanitizerArgs.cpp

 enum : SanitizerMask {
   NeedsUbsanRt = Undefined | Integer | CFI,
   NeedsUbsanCxxRt = Vptr | CFI,
   NotAllowedWithTrap = Vptr,
   RequiresPIE = Memory | DataFlow,
   NeedsUnwindTables = Address | Thread | Memory | DataFlow,
   SupportsCoverage = Address | Memory | Leak | Undefined | Integer | DataFlow,
   RecoverableByDefault = Undefined | Integer,
-  Unrecoverable = Address | Unreachable | Return,
+  Unrecoverable = Unreachable | Return,
   LegacyFsanitizeRecoverMask = Undefined | Integer,
   NeedsLTO = CFI,
   TrappingSupported =
       (Undefined & ~Vptr) | UnsignedIntegerOverflow | LocalBounds | CFI,
   TrappingDefault = CFI,
 };
 
 enum CoverageFeature {

lib/Frontend/CompilerInvocation.cpp

static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
   // Parse -fsanitize-recover= arguments.
   // FIXME: Report unrecoverable sanitizers incorrectly specified here.
   parseSanitizerKinds("-fsanitize-recover=",
                       Args.getAllArgValues(OPT_fsanitize_recover_EQ), Diags,
                       Opts.SanitizeRecover);
   parseSanitizerKinds("-fsanitize-trap=",
                       Args.getAllArgValues(OPT_fsanitize_trap_EQ), Diags,
                       Opts.SanitizeTrap);
+  Opts.SanitizeAddressNoAbort = Opts.SanitizeRecover.has(SanitizerKind::Address);
+
 
   Opts.CudaGpuBinaryFileNames =
       Args.getAllArgValues(OPT_fcuda_include_gpubinary);
 
   return Success;
 }
 
 static void ParseDependencyOutputArgs(DependencyOutputOptions &Opts,

lib/Transforms/Instrumentation/AddressSanitizer.cpp

 static const size_t kNumberOfAccessSizes = 5;
 
 static const unsigned kAllocaRzSize = 32;
 
 // Command-line flags.
 static cl::opt<bool> ClEnableKasan(
     "asan-kernel", cl::desc("Enable KernelAddressSanitizer instrumentation"),
     cl::Hidden, cl::init(false));
+static cl::opt<bool> ClEnableNoAbort(
+    "asan-noabort",
+    cl::desc("Enable continue-after-error mode (warning: not supported)"),
+    cl::Hidden, cl::init(false));
 
 // This flag may need to be replaced with -f[no-]asan-reads.
 static cl::opt<bool> ClInstrumentReads("asan-instrument-reads",
                                        cl::desc("instrument read instructions"),
                                        cl::Hidden, cl::init(true));
 static cl::opt<bool> ClInstrumentWrites(
     "asan-instrument-writes", cl::desc("instrument write instructions"),
     cl::Hidden, cl::init(true));

 static size_t RedzoneSizeForScale(int MappingScale) {
   // Redzone used for stack and globals is at least 32 bytes.
   // For scales 6 and 7, the redzone has to be 64 and 128 bytes respectively.
   return std::max(32U, 1U << MappingScale);
 }
 
 /// AddressSanitizer: instrument the code in module to find memory bugs.
 struct AddressSanitizer : public FunctionPass {
-  explicit AddressSanitizer(bool CompileKernel = false)
-      : FunctionPass(ID), CompileKernel(CompileKernel || ClEnableKasan) {
+  explicit AddressSanitizer(bool CompileKernel = false, bool NoAbort = false)
+      : FunctionPass(ID), CompileKernel(CompileKernel || ClEnableKasan),
+        NoAbort(NoAbort || ClEnableNoAbort) {
     initializeAddressSanitizerPass(*PassRegistry::getPassRegistry());
   }
   const char *getPassName() const override {
     return "AddressSanitizerFunctionPass";
   }
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<DominatorTreeWrapperPass>();
     AU.addRequired<TargetLibraryInfoWrapperPass>();
struct FunctionStateRAII {
     }
     ~FunctionStateRAII() { Pass->ProcessedAllocas.clear(); }
   };
 
   LLVMContext *C;
   Triple TargetTriple;
   int LongSize;
   bool CompileKernel;
+  bool NoAbort;
   Type *IntptrTy;
   ShadowMapping Mapping;
   DominatorTree *DT;
   Function *AsanCtorFunction = nullptr;
   Function *AsanInitFunction = nullptr;
   Function *AsanHandleNoReturnFunc;
   Function *AsanPtrCmpFunction, *AsanPtrSubFunction;
   // This array is indexed by AccessIsWrite, Experiment and log2(AccessSize).
   Function *AsanErrorCallback[2][2][kNumberOfAccessSizes];
   Function *AsanMemoryAccessCallback[2][2][kNumberOfAccessSizes];
   // This array is indexed by AccessIsWrite and Experiment.
   Function *AsanErrorCallbackSized[2][2];
   Function *AsanMemoryAccessCallbackSized[2][2];
   Function *AsanMemmove, *AsanMemcpy, *AsanMemset;
   InlineAsm *EmptyAsm;
   GlobalsMetadata GlobalsMD;
   DenseMap<AllocaInst *, bool> ProcessedAllocas;
 
   friend struct FunctionStackPoisoner;
 };
 
 class AddressSanitizerModule : public ModulePass {
  public:
-  explicit AddressSanitizerModule(bool CompileKernel = false)
-      : ModulePass(ID), CompileKernel(CompileKernel || ClEnableKasan) {}
+  explicit AddressSanitizerModule(bool CompileKernel = false,
+                                  bool NoAbort = false)
+      : ModulePass(ID), CompileKernel(CompileKernel || ClEnableKasan),
+        NoAbort(NoAbort || ClEnableNoAbort) {}
   bool runOnModule(Module &M) override;
   static char ID;  // Pass identification, replacement for typeid
   const char *getPassName() const override { return "AddressSanitizerModule"; }
 
  private:
   void initializeCallbacks(Module &M);
 
   bool InstrumentGlobals(IRBuilder<> &IRB, Module &M);
   bool ShouldInstrumentGlobal(GlobalVariable *G);
   void poisonOneInitializer(Function &GlobalInit, GlobalValue *ModuleName);
   void createInitializerPoisonCalls(Module &M, GlobalValue *ModuleName);
   size_t MinRedzoneSizeForGlobal() const {
     return RedzoneSizeForScale(Mapping.Scale);
   }
 
   GlobalsMetadata GlobalsMD;
   bool CompileKernel;
+  bool NoAbort;
   Type *IntptrTy;
   LLVMContext *C;
   Triple TargetTriple;
   ShadowMapping Mapping;
   Function *AsanPoisonGlobals;
   Function *AsanUnpoisonGlobals;
   Function *AsanRegisterGlobals;
   Function *AsanUnregisterGlobals;
INITIALIZE_PASS_BEGIN(
     AddressSanitizer, "asan",
     "AddressSanitizer: detects use-after-free and out-of-bounds bugs.", false,
     false)
 INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
 INITIALIZE_PASS_END(
     AddressSanitizer, "asan",
     "AddressSanitizer: detects use-after-free and out-of-bounds bugs.", false,
     false)
-FunctionPass *llvm::createAddressSanitizerFunctionPass(bool CompileKernel) {
-  return new AddressSanitizer(CompileKernel);
+FunctionPass *llvm::createAddressSanitizerFunctionPass(bool CompileKernel,
+                                                       bool NoAbort) {
+  return new AddressSanitizer(CompileKernel, NoAbort);
 }
 
 char AddressSanitizerModule::ID = 0;
 INITIALIZE_PASS(
     AddressSanitizerModule, "asan-module",
     "AddressSanitizer: detects use-after-free and out-of-bounds bugs."
     "ModulePass",
     false, false)
-ModulePass *llvm::createAddressSanitizerModulePass(bool CompileKernel) {
-  return new AddressSanitizerModule(CompileKernel);
+ModulePass *llvm::createAddressSanitizerModulePass(bool CompileKernel,
+                                                   bool NoAbort) {
+  return new AddressSanitizerModule(CompileKernel, NoAbort);
 }
 
 static size_t TypeSizeToSizeIndex(uint32_t TypeSize) {
   size_t Res = countTrailingZeros(TypeSize / 8);
   assert(Res < kNumberOfAccessSizes);
   return Res;
 }
 
if (ClAlwaysSlowPath || (TypeSize < 8 * Granularity)) {
     // We use branch weights for the slow path check, to indicate that the slow
     // path is rarely taken. This seems to be the case for SPEC benchmarks.
     TerminatorInst *CheckTerm = SplitBlockAndInsertIfThen(
         Cmp, InsertBefore, false, MDBuilder(*C).createBranchWeights(1, 100000));
     assert(cast<BranchInst>(CheckTerm)->isUnconditional());
     BasicBlock *NextBB = CheckTerm->getSuccessor(0);
     IRB.SetInsertPoint(CheckTerm);
     Value *Cmp2 = createSlowPathCmp(IRB, AddrLong, ShadowValue, TypeSize);
+    if (NoAbort) {
+      // Errors are rare
+      CrashTerm = SplitBlockAndInsertIfThen(
+          Cmp2, CheckTerm, false, MDBuilder(*C).createBranchWeights(1, 100000));
+    } else {
       BasicBlock *CrashBlock =
         BasicBlock::Create(*C, "", NextBB->getParent(), NextBB);
       CrashTerm = new UnreachableInst(*C, CrashBlock);
       BranchInst *NewTerm = BranchInst::Create(CrashBlock, NextBB, Cmp2);
       ReplaceInstWithInst(CheckTerm, NewTerm);
+    }
   } else {
-    CrashTerm = SplitBlockAndInsertIfThen(Cmp, InsertBefore, true);
+    CrashTerm = SplitBlockAndInsertIfThen(Cmp, InsertBefore, !NoAbort);
   }
 
   Instruction *Crash = generateCrashCode(CrashTerm, AddrLong, IsWrite,
                                          AccessSizeIndex, SizeArgument, Exp);
   Crash->setDebugLoc(OrigIns->getDebugLoc());
 }
 
 // Instrument unusual size or unusual alignment.
void AddressSanitizer::initializeCallbacks(Module &M) {
   IRBuilder<> IRB(*C);
   // Create __asan_report* callbacks.
   // IsWrite, TypeSize and Exp are encoded in the function name.
   for (int Exp = 0; Exp < 2; Exp++) {
     for (size_t AccessIsWrite = 0; AccessIsWrite <= 1; AccessIsWrite++) {
       const std::string TypeStr = AccessIsWrite ? "store" : "load";
       const std::string ExpStr = Exp ? "exp_" : "";
       const std::string SuffixStr = CompileKernel ? "N" : "_n";
-      const std::string EndingStr = CompileKernel ? "_noabort" : "";
+      const std::string EndingStr =
+        (CompileKernel || NoAbort) ? "_noabort" : "";
       Type *ExpType = Exp ? Type::getInt32Ty(*C) : nullptr;
-      // TODO(glider): for KASan builds add _noabort to error reporting
-      // functions and make them actually noabort (remove the UnreachableInst).
       AsanErrorCallbackSized[AccessIsWrite][Exp] =
           checkSanitizerInterfaceFunction(M.getOrInsertFunction(
-              kAsanReportErrorTemplate + ExpStr + TypeStr + SuffixStr,
+              kAsanReportErrorTemplate + ExpStr + TypeStr + SuffixStr + EndingStr,
               IRB.getVoidTy(), IntptrTy, IntptrTy, ExpType, nullptr));
       AsanMemoryAccessCallbackSized[AccessIsWrite][Exp] =
           checkSanitizerInterfaceFunction(M.getOrInsertFunction(
               ClMemoryAccessCallbackPrefix + ExpStr + TypeStr + "N" + EndingStr,
               IRB.getVoidTy(), IntptrTy, IntptrTy, ExpType, nullptr));
       for (size_t AccessSizeIndex = 0; AccessSizeIndex < kNumberOfAccessSizes;
            AccessSizeIndex++) {
         const std::string Suffix = TypeStr + itostr(1 << AccessSizeIndex);
         AsanErrorCallback[AccessIsWrite][Exp][AccessSizeIndex] =
             checkSanitizerInterfaceFunction(M.getOrInsertFunction(
-                kAsanReportErrorTemplate + ExpStr + Suffix,
+                kAsanReportErrorTemplate + ExpStr + Suffix + EndingStr,
                 IRB.getVoidTy(), IntptrTy, ExpType, nullptr));
         AsanMemoryAccessCallback[AccessIsWrite][Exp][AccessSizeIndex] =
             checkSanitizerInterfaceFunction(M.getOrInsertFunction(
                 ClMemoryAccessCallbackPrefix + ExpStr + Suffix + EndingStr,
                 IRB.getVoidTy(), IntptrTy, ExpType, nullptr));
       }
     }
   }

lib/asan/asan_flags.inc

ASAN_FLAG(
     "See https://code.google.com/p/address-sanitizer/wiki/ContainerOverflow")
 ASAN_FLAG(int, detect_odr_violation, 2,
           "If >=2, detect violation of One-Definition-Rule (ODR); "
           "If ==1, detect ODR-violation only if the two variables "
           "have different sizes")
 ASAN_FLAG(bool, dump_instruction_bytes, false,
           "If true, dump 16 bytes starting at the instruction that caused SEGV")
 ASAN_FLAG(const char *, suppressions, "", "Suppressions file name.")
+
+ASAN_FLAG(bool, keep_going, false,
+          "If true, continue execution after error"
+          "(warning: use at your own risk!)")
+ASAN_FLAG(int, num_reported_errors, 10,
+          "Maximum number of errors reported if keep_going mode is enabled")
+

lib/asan/asan_interceptors.cc

 };
 
 // We implement ACCESS_MEMORY_RANGE, ASAN_READ_RANGE,
 // and ASAN_WRITE_RANGE as macro instead of function so
 // that no extra frames are created, and stack trace contains
 // relevant information only.
 // We check all shadow bytes.
 #define ACCESS_MEMORY_RANGE(ctx, offset, size, isWrite) do {            \
+    if (QuickCheckSilence(false)) break;                                \
     uptr __offset = (uptr)(offset);                                     \
     uptr __size = (uptr)(size);                                         \
     uptr __bad = 0;                                                     \
     if (__offset > __offset + __size) {                                 \
       GET_STACK_TRACE_FATAL_HERE;                                       \
       ReportStringFunctionSizeOverflow(__offset, __size, &stack);       \
     }                                                                   \
     if (!QuickCheckForUnpoisonedRegion(__offset, __size) &&             \
         (__bad = __asan_region_is_poisoned(__offset, __size))) {        \
       AsanInterceptorContext *_ctx = (AsanInterceptorContext *)ctx;     \
       bool suppressed = false;                                          \
       if (_ctx) {                                                       \
         suppressed = IsInterceptorSuppressed(_ctx->interceptor_name);   \
         if (!suppressed && HaveStackTraceBasedSuppressions()) {         \
           GET_STACK_TRACE_FATAL_HERE;                                   \
           suppressed = IsStackTraceSuppressed(&stack);                  \
         }                                                               \
       }                                                                 \
       if (!suppressed) {                                                \
         GET_CURRENT_PC_BP_SP;                                           \
-        __asan_report_error(pc, bp, sp, __bad, isWrite, __size, 0);     \
+        __asan_report_error(pc, bp, sp, __bad, isWrite, __size, 0, false);\
       }                                                                 \
     }                                                                   \
   } while (0)
 
 #define ASAN_READ_RANGE(ctx, offset, size) \
   ACCESS_MEMORY_RANGE(ctx, offset, size, false)
 #define ASAN_WRITE_RANGE(ctx, offset, size) \
   ACCESS_MEMORY_RANGE(ctx, offset, size, true)

lib/asan/asan_interface_internal.h

extern "C" {
   SANITIZER_INTERFACE_ATTRIBUTE
   uptr __asan_get_free_stack(uptr addr, uptr *trace, uptr size,
                              u32 *thread_id);
 
   SANITIZER_INTERFACE_ATTRIBUTE
   void __asan_get_shadow_mapping(uptr *shadow_scale, uptr *shadow_offset);
 
   SANITIZER_INTERFACE_ATTRIBUTE
   void __asan_report_error(uptr pc, uptr bp, uptr sp,
-                           uptr addr, int is_write, uptr access_size, u32 exp);
+                           uptr addr, int is_write, uptr access_size, u32 exp,
+                           int fatal);
 
   SANITIZER_INTERFACE_ATTRIBUTE
   int __asan_set_error_exit_code(int exit_code);
   SANITIZER_INTERFACE_ATTRIBUTE
   void __asan_set_death_callback(void (*callback)(void));
   SANITIZER_INTERFACE_ATTRIBUTE
   void __asan_set_error_report_callback(void (*callback)(const char*));
 
extern "C" {
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_store1(uptr p);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_store2(uptr p);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_store4(uptr p);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_store8(uptr p);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_store16(uptr p);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_loadN(uptr p, uptr size);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_storeN(uptr p, uptr size);
 
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_load1_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_load2_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_load4_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_load8_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_load16_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_store1_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_store2_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_store4_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_store8_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_store16_noabort(uptr p);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_loadN_noabort(uptr p, uptr size);
+  SANITIZER_INTERFACE_ATTRIBUTE void __asan_storeN_noabort(uptr p, uptr size);
+
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_load1(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_load2(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_load4(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_load8(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_load16(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_store1(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_store2(uptr p, u32 exp);
   SANITIZER_INTERFACE_ATTRIBUTE void __asan_exp_store4(uptr p, u32 exp);

lib/asan/asan_internal.h

#define ASAN_FREE_HOOK(ptr) \
   if (&__sanitizer_free_hook) __sanitizer_free_hook(ptr)
 #define ASAN_ON_ERROR() \
   if (&__asan_on_error) __asan_on_error()
 
 extern int asan_inited;
 // Used to avoid infinite recursion in __asan_init().
 extern bool asan_init_is_running;
 extern void (*death_callback)(void);
+extern int num_reported_errors;
+
+// This function performs a quick check of whether we need to bother reporting
+// error. The intent is to stop further processing quickly if it's
+// unnecessary.
+static inline bool QuickCheckSilence(bool is_error) {
+  if (UNLIKELY(flags()->keep_going)) {
+    if (num_reported_errors >= flags()->num_reported_errors)
+      return true;
+    // This is racy but we don't need 100% correctness.
+    if (is_error) ++num_reported_errors;
+  }
+  return false;
+}
 
 // These magic values are written to shadow for better error reporting.
 const int kAsanHeapLeftRedzoneMagic = 0xfa;
 const int kAsanHeapRightRedzoneMagic = 0xfb;
 const int kAsanHeapFreeMagic = 0xfd;
 const int kAsanStackLeftRedzoneMagic = 0xf1;
 const int kAsanStackMidRedzoneMagic = 0xf2;
 const int kAsanStackRightRedzoneMagic = 0xf3;

lib/asan/asan_poisoning.cc

 #define CHECK_SMALL_REGION(p, size, isWrite)                  \
   do {                                                        \
     uptr __p = reinterpret_cast<uptr>(p);                     \
     uptr __size = size;                                       \
     if (UNLIKELY(__asan::AddressIsPoisoned(__p) ||            \
         __asan::AddressIsPoisoned(__p + __size - 1))) {       \
       GET_CURRENT_PC_BP_SP;                                   \
       uptr __bad = __asan_region_is_poisoned(__p, __size);    \
-      __asan_report_error(pc, bp, sp, __bad, isWrite, __size, 0);\
+      __asan_report_error(pc, bp, sp, __bad, isWrite, __size, 0, false); \
     }                                                         \
   } while (false);                                            \
 
 
 extern "C" SANITIZER_INTERFACE_ATTRIBUTE
 u16 __sanitizer_unaligned_load16(const uu16 *p) {
   CHECK_SMALL_REGION(p, sizeof(*p), false);
   return *p;

lib/asan/asan_report.h

 bool DescribeAddressIfShadow(uptr addr, AddressDescription *descr = nullptr,
                              bool print = true);
 bool ParseFrameDescription(const char *frame_descr,
                            InternalMmapVector<StackVarDescr> *vars);
 bool DescribeAddressIfStack(uptr addr, uptr access_size);
 void DescribeThread(AsanThreadContext *context);
 
 // Different kinds of error reports.
-void NORETURN ReportStackOverflow(const SignalContext &sig);
-void NORETURN ReportDeadlySignal(const char* description,
-                                 const SignalContext &sig);
-void NORETURN ReportNewDeleteSizeMismatch(uptr addr, uptr delete_size,
-                                          BufferedStackTrace *free_stack);
-void NORETURN ReportDoubleFree(uptr addr, BufferedStackTrace *free_stack);
-void NORETURN ReportFreeNotMalloced(uptr addr, BufferedStackTrace *free_stack);
-void NORETURN ReportAllocTypeMismatch(uptr addr, BufferedStackTrace *free_stack,
-                                      AllocType alloc_type,
-                                      AllocType dealloc_type);
-void NORETURN
-    ReportMallocUsableSizeNotOwned(uptr addr, BufferedStackTrace *stack);
-void NORETURN
-    ReportSanitizerGetAllocatedSizeNotOwned(uptr addr,
+void ReportStackOverflow(const SignalContext &sig);
+void ReportDeadlySignal(const char *description, const SignalContext &sig);
+void ReportNewDeleteSizeMismatch(uptr addr, uptr delete_size,
+void ReportDoubleFree(uptr addr, BufferedStackTrace *free_stack);
+void ReportFreeNotMalloced(uptr addr, BufferedStackTrace *free_stack);
+void ReportAllocTypeMismatch(uptr addr, BufferedStackTrace *free_stack,
+void ReportMallocUsableSizeNotOwned(uptr addr, BufferedStackTrace *stack);
+void ReportSanitizerGetAllocatedSizeNotOwned(uptr addr,
                                              BufferedStackTrace *stack);
-void NORETURN
-    ReportStringFunctionMemoryRangesOverlap(const char *function,
+void ReportStringFunctionMemoryRangesOverlap(const char *function,
                                              const char *offset1, uptr length1,
                                              const char *offset2, uptr length2,
                                              BufferedStackTrace *stack);
-void NORETURN ReportStringFunctionSizeOverflow(uptr offset, uptr size,
-                                               BufferedStackTrace *stack);
-void NORETURN
-    ReportBadParamsToAnnotateContiguousContainer(uptr beg, uptr end,
+void ReportStringFunctionSizeOverflow(uptr offset, uptr size,
+void ReportBadParamsToAnnotateContiguousContainer(uptr beg, uptr end,
                                                   uptr old_mid, uptr new_mid,
                                                   BufferedStackTrace *stack);
 
-void NORETURN
-ReportODRViolation(const __asan_global *g1, u32 stack_id1,
+void ReportODRViolation(const __asan_global *g1, u32 stack_id1,
                         const __asan_global *g2, u32 stack_id2);
 
 // Mac-specific errors and warnings.
 void WarnMacFreeUnallocated(uptr addr, uptr zone_ptr, const char *zone_name,
                             BufferedStackTrace *stack);
-void NORETURN ReportMacMzReallocUnknown(uptr addr, uptr zone_ptr,
-                                        const char *zone_name,
-                                        BufferedStackTrace *stack);
-void NORETURN ReportMacCfReallocUnknown(uptr addr, uptr zone_ptr,
-                                        const char *zone_name,
-                                        BufferedStackTrace *stack);
+void ReportMacMzReallocUnknown(uptr addr, uptr zone_ptr,
+void ReportMacCfReallocUnknown(uptr addr, uptr zone_ptr,
 
 }  // namespace __asan

lib/asan/asan_report.cc

 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_flags.h"
 #include "sanitizer_common/sanitizer_report_decorator.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
 #include "sanitizer_common/sanitizer_symbolizer.h"
 
 namespace __asan {
 
+#define QUICK_CHECK_SILENCE \
+  if (QuickCheckSilence(true)) return
+
 // -------------------- User-specified callbacks ----------------- {{{1
 static void (*error_report_callback)(const char*);
 static char *error_message_buffer = 0;
 static uptr error_message_buffer_pos = 0;
 static uptr error_message_buffer_size = 0;
 
 struct ReportData {
   uptr pc;
void DescribeThread(AsanThreadContext *context) {
   }
 }
 
 // -------------------- Different kinds of reports ----------------- {{{1
 
 // Use ScopedInErrorReport to run common actions just before and
 // immediately after printing error report.
 class ScopedInErrorReport {
- public:
-  explicit ScopedInErrorReport(ReportData *report = nullptr) {
-    static atomic_uint32_t num_calls;
-    static u32 reporting_thread_tid;
+  bool keep_going;
+
+  bool TryLockReporting() {
     if (atomic_fetch_add(&num_calls, 1, memory_order_relaxed) != 0) {
+      atomic_fetch_add(&num_calls, -1, memory_order_relaxed);
+      return false;
+    }
+    return true;
+  }
+
+ public:
+  explicit ScopedInErrorReport(ReportData *report = nullptr,
+                               bool fatal = false) {
+    keep_going = !fatal && flags()->keep_going;
+    if (!TryLockReporting()) {
+      // ASan found two bugs in different threads simultaneously.
+      Report("AddressSanitizer: while reporting a bug found another one.");
+      u32 current_tid = GetCurrentTidOrInvalid();
+      if (UNLIKELY(keep_going)) {
+        if (current_tid != reporting_thread_tid) {
+          // Try to wait for other thread to finish reporting.
+          for (int retry = 0; retry < 5; ++retry) {
+            SleepForSeconds(5);
+            if (TryLockReporting())
+              goto locked;
+          }
+        }
+      } else {
+        Report("Ignoring.\n");
         // Do not print more than one report, otherwise they will mix up.
         // Error reporting functions shouldn't return at this situation, as
         // they are defined as no-return.
-      Report("AddressSanitizer: while reporting a bug found another one. "
-                 "Ignoring.\n");
-      u32 current_tid = GetCurrentTidOrInvalid();
         if (current_tid != reporting_thread_tid) {
-        // ASan found two bugs in different threads simultaneously. Sleep
-        // long enough to make sure that the thread which started to print
-        // an error report will finish doing it.
+          // Sleep long enough to make sure that the thread which started
+          // to print an error report will finish doing it.
           SleepForSeconds(Max(100, flags()->sleep_before_dying + 1));
         }
+      }
       // If we're still not dead for some reason, use raw _exit() instead of
       // Die() to bypass any additional checks.
       internal__exit(flags()->exitcode);
     }
+ locked:
     if (report) report_data = *report;
     report_happened = true;
     ASAN_ON_ERROR();
     // Make sure the registry and sanitizer report mutexes are locked while
     // we're printing an error report.
     // We can lock them only here to avoid self-deadlock in case of
     // recursive reports.
     asanThreadRegistry().Lock();
     CommonSanitizerReportMutex.Lock();
     reporting_thread_tid = GetCurrentTidOrInvalid();
     Printf("===================================================="
            "=============\n");
   }
-  // Destructor is NORETURN, as functions that report errors are.
-  NORETURN ~ScopedInErrorReport() {
+  ~ScopedInErrorReport() {
     // Make sure the current thread is announced.
     DescribeThread(GetCurrentThread());
     // We may want to grab this lock again when printing stats.
     asanThreadRegistry().Unlock();
     // Print memory stats.
     if (flags()->print_stats)
       __asan_print_accumulated_stats();
     if (error_report_callback) {
       error_report_callback(error_message_buffer);
     }
+    atomic_fetch_add(&num_calls, -1, memory_order_relaxed);
+    CommonSanitizerReportMutex.Unlock();
+    if (LIKELY(!keep_going)) {
       Report("ABORTING\n");
       Die();
     }
+  }
 };
 
+atomic_uint32_t ScopedInErrorReport::num_calls;
+u32 ScopedInErrorReport::reporting_thread_tid;
+
 void ReportStackOverflow(const SignalContext &sig) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   Report(
       "ERROR: AddressSanitizer: stack-overflow on address %p"
       " (pc %p bp %p sp %p T%d)\n",
       (void *)sig.addr, (void *)sig.pc, (void *)sig.bp, (void *)sig.sp,
       GetCurrentTidOrInvalid());
   Printf("%s", d.EndWarning());
   GET_STACK_TRACE_SIGNAL(sig);
   stack.Print();
   ReportErrorSummary("stack-overflow", &stack);
 }
 
 void ReportDeadlySignal(const char *description, const SignalContext &sig) {
-  ScopedInErrorReport in_report;
+  ScopedInErrorReport in_report(/*report*/nullptr, /*fatal*/true);
   Decorator d;
   Printf("%s", d.Warning());
   Report(
       "ERROR: AddressSanitizer: %s on unknown address %p"
       " (pc %p bp %p sp %p T%d)\n",
       description, (void *)sig.addr, (void *)sig.pc, (void *)sig.bp,
       (void *)sig.sp, GetCurrentTidOrInvalid());
   if (sig.pc < GetPageSizeCached()) {
     Report("Hint: pc points to the zero page.\n");
   }
   Printf("%s", d.EndWarning());
   GET_STACK_TRACE_SIGNAL(sig);
   stack.Print();
   MaybeDumpInstructionBytes(sig.pc);
   Printf("AddressSanitizer can not provide additional info.\n");
   ReportErrorSummary(description, &stack);
 }
 
 void ReportDoubleFree(uptr addr, BufferedStackTrace *free_stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   char tname[128];
   u32 curr_tid = GetCurrentTidOrInvalid();
   Report("ERROR: AddressSanitizer: attempting double-free on %p in "
          "thread T%d%s:\n",
          addr, curr_tid,
          ThreadNameWithParenthesis(curr_tid, tname, sizeof(tname)));
   Printf("%s", d.EndWarning());
   CHECK_GT(free_stack->size, 0);
   GET_STACK_TRACE_FATAL(free_stack->trace[0], free_stack->top_frame_bp);
   stack.Print();
   DescribeHeapAddress(addr, 1);
   ReportErrorSummary("double-free", &stack);
 }
 
 void ReportNewDeleteSizeMismatch(uptr addr, uptr delete_size,
                                  BufferedStackTrace *free_stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   char tname[128];
   u32 curr_tid = GetCurrentTidOrInvalid();
   Report("ERROR: AddressSanitizer: new-delete-type-mismatch on %p in "
          "thread T%d%s:\n",
          addr, curr_tid,
          ThreadNameWithParenthesis(curr_tid, tname, sizeof(tname)));
   Printf("%s  object passed to delete has wrong type:\n", d.EndWarning());
   Printf("  size of the allocated type:   %zd bytes;\n"
          "  size of the deallocated type: %zd bytes.\n",
          asan_mz_size(reinterpret_cast<void*>(addr)), delete_size);
   CHECK_GT(free_stack->size, 0);
   GET_STACK_TRACE_FATAL(free_stack->trace[0], free_stack->top_frame_bp);
   stack.Print();
   DescribeHeapAddress(addr, 1);
   ReportErrorSummary("new-delete-type-mismatch", &stack);
   Report("HINT: if you don't care about these warnings you may set "
          "ASAN_OPTIONS=new_delete_type_mismatch=0\n");
 }
 
 void ReportFreeNotMalloced(uptr addr, BufferedStackTrace *free_stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   char tname[128];
   u32 curr_tid = GetCurrentTidOrInvalid();
   Report("ERROR: AddressSanitizer: attempting free on address "
              "which was not malloc()-ed: %p in thread T%d%s\n", addr,
          curr_tid, ThreadNameWithParenthesis(curr_tid, tname, sizeof(tname)));
   Printf("%s", d.EndWarning());
   CHECK_GT(free_stack->size, 0);
   GET_STACK_TRACE_FATAL(free_stack->trace[0], free_stack->top_frame_bp);
   stack.Print();
   DescribeHeapAddress(addr, 1);
   ReportErrorSummary("bad-free", &stack);
 }
 
 void ReportAllocTypeMismatch(uptr addr, BufferedStackTrace *free_stack,
                              AllocType alloc_type,
                              AllocType dealloc_type) {
+  QUICK_CHECK_SILENCE;
   static const char *alloc_names[] =
     {"INVALID", "malloc", "operator new", "operator new []"};
   static const char *dealloc_names[] =
     {"INVALID", "free", "operator delete", "operator delete []"};
   CHECK_NE(alloc_type, dealloc_type);
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: alloc-dealloc-mismatch (%s vs %s) on %p\n",
         alloc_names[alloc_type], dealloc_names[dealloc_type], addr);
   Printf("%s", d.EndWarning());
   CHECK_GT(free_stack->size, 0);
   GET_STACK_TRACE_FATAL(free_stack->trace[0], free_stack->top_frame_bp);
   stack.Print();
   DescribeHeapAddress(addr, 1);
   ReportErrorSummary("alloc-dealloc-mismatch", &stack);
   Report("HINT: if you don't care about these warnings you may set "
          "ASAN_OPTIONS=alloc_dealloc_mismatch=0\n");
 }
 
 void ReportMallocUsableSizeNotOwned(uptr addr, BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: attempting to call "
              "malloc_usable_size() for pointer which is "
              "not owned: %p\n", addr);
   Printf("%s", d.EndWarning());
   stack->Print();
   DescribeHeapAddress(addr, 1);
   ReportErrorSummary("bad-malloc_usable_size", stack);
 }
 
 void ReportSanitizerGetAllocatedSizeNotOwned(uptr addr,
                                              BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: attempting to call "
              "__sanitizer_get_allocated_size() for pointer which is "
              "not owned: %p\n", addr);
   Printf("%s", d.EndWarning());
   stack->Print();
   DescribeHeapAddress(addr, 1);
   ReportErrorSummary("bad-__sanitizer_get_allocated_size", stack);
 }
 
 void ReportStringFunctionMemoryRangesOverlap(const char *function,
                                              const char *offset1, uptr length1,
                                              const char *offset2, uptr length2,
                                              BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   char bug_type[100];
   internal_snprintf(bug_type, sizeof(bug_type), "%s-param-overlap", function);
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: %s: "
              "memory ranges [%p,%p) and [%p, %p) overlap\n", \
              bug_type, offset1, offset1 + length1, offset2, offset2 + length2);
   Printf("%s", d.EndWarning());
   stack->Print();
   DescribeAddress((uptr)offset1, length1, bug_type);
   DescribeAddress((uptr)offset2, length2, bug_type);
   ReportErrorSummary(bug_type, stack);
 }
 
 void ReportStringFunctionSizeOverflow(uptr offset, uptr size,
                                       BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   const char *bug_type = "negative-size-param";
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: %s: (size=%zd)\n", bug_type, size);
   Printf("%s", d.EndWarning());
   stack->Print();
   DescribeAddress(offset, size, bug_type);
   ReportErrorSummary(bug_type, stack);
 }
 
 void ReportBadParamsToAnnotateContiguousContainer(uptr beg, uptr end,
                                                   uptr old_mid, uptr new_mid,
                                                   BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Report("ERROR: AddressSanitizer: bad parameters to "
          "__sanitizer_annotate_contiguous_container:\n"
          "      beg     : %p\n"
          "      end     : %p\n"
          "      old_mid : %p\n"
          "      new_mid : %p\n",
          beg, end, old_mid, new_mid);
   uptr granularity = SHADOW_GRANULARITY;
   if (!IsAligned(beg, granularity))
     Report("ERROR: beg is not aligned by %d\n", granularity);
   stack->Print();
   ReportErrorSummary("bad-__sanitizer_annotate_contiguous_container", stack);
 }
 
 void ReportODRViolation(const __asan_global *g1, u32 stack_id1,
                         const __asan_global *g2, u32 stack_id2) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Decorator d;
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: odr-violation (%p):\n", g1->beg);
   Printf("%s", d.EndWarning());
   InternalScopedString g1_loc(256), g2_loc(256);
   PrintGlobalLocation(&g1_loc, *g1);
   PrintGlobalLocation(&g2_loc, *g2);
void ReportODRViolation(const __asan_global *g1, u32 stack_id1,
   error_msg.append("odr-violation: global '%s' at %s",
                    MaybeDemangleGlobalName(g1->name), g1_loc.data());
   ReportErrorSummary(error_msg.data());
 }
 
 // ----------------------- CheckForInvalidPointerPair ----------- {{{1
 static NOINLINE void
 ReportInvalidPointerPair(uptr pc, uptr bp, uptr sp, uptr a1, uptr a2) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   const char *bug_type = "invalid-pointer-pair";
   Decorator d;
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: invalid-pointer-pair: %p %p\n", a1, a2);
   Printf("%s", d.EndWarning());
   GET_STACK_TRACE_FATAL(pc, bp);
   stack.Print();
Printf("free_common(%p) -- attempting to free unallocated memory.\n"
              addr);
   PrintZoneForPointer(addr, zone_ptr, zone_name);
   stack->Print();
   DescribeHeapAddress(addr, 1);
 }
 
 void ReportMacMzReallocUnknown(uptr addr, uptr zone_ptr, const char *zone_name,
                                BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Printf("mz_realloc(%p) -- attempting to realloc unallocated memory.\n"
              "This is an unrecoverable problem, exiting now.\n",
              addr);
   PrintZoneForPointer(addr, zone_ptr, zone_name);
   stack->Print();
   DescribeHeapAddress(addr, 1);
 }
 
 void ReportMacCfReallocUnknown(uptr addr, uptr zone_ptr, const char *zone_name,
                                BufferedStackTrace *stack) {
+  QUICK_CHECK_SILENCE;
   ScopedInErrorReport in_report;
   Printf("cf_realloc(%p) -- attempting to realloc unallocated memory.\n"
              "This is an unrecoverable problem, exiting now.\n",
              addr);
   PrintZoneForPointer(addr, zone_ptr, zone_name);
   stack->Print();
   DescribeHeapAddress(addr, 1);
 }
 
 }  // namespace __asan
 
 // --------------------------- Interface --------------------- {{{1
 using namespace __asan;  // NOLINT
 
 void __asan_report_error(uptr pc, uptr bp, uptr sp, uptr addr, int is_write,
-                         uptr access_size, u32 exp) {
+                         uptr access_size, u32 exp, int fatal) {
   ENABLE_FRAME_POINTER;
 
+  if (!fatal)
+    QUICK_CHECK_SILENCE;
+
   // Optimization experiments.
   // The experiments can be used to evaluate potential optimizations that remove
   // instrumentation (assess false negatives). Instead of completely removing
   // some instrumentation, compiler can emit special calls into runtime
   // (e.g. __asan_report_exp_load1 instead of __asan_report_load1) and pass
   // mask of experiments (exp).
   // The reaction to a non-zero value of exp is to be defined.
   (void)exp;
switch (*shadow_addr) {
       case kAsanAllocaRightMagic:
         bug_descr = "dynamic-stack-buffer-overflow";
         break;
     }
   }
 
   ReportData report = { pc, sp, bp, addr, (bool)is_write, access_size,
                         bug_descr };
-  ScopedInErrorReport in_report(&report);
+  ScopedInErrorReport in_report(&report, fatal);
 
   Decorator d;
   Printf("%s", d.Warning());
   Report("ERROR: AddressSanitizer: %s on address "
              "%p at pc %p bp %p sp %p\n",
              bug_descr, (void*)addr, pc, bp, sp);
   Printf("%s", d.EndWarning());
 

lib/asan/asan_rtl.cc

static void AsanCheckFailed(const char *file, int line, const char *cond,
   // FIXME: check for infinite recursion without a thread-local counter here.
   PRINT_CURRENT_STACK_CHECK();
   Die();
 }
 
 // -------------------------- Globals --------------------- {{{1
 int asan_inited;
 bool asan_init_is_running;
+int num_reported_errors;
 
 #if !ASAN_FIXED_MAPPING
 uptr kHighMemEnd, kMidMemBeg, kMidMemEnd;
 #endif
 
 // -------------------------- Misc ---------------- {{{1
 void ShowStatsAndAbort() {
   __asan_print_accumulated_stats();

 }
 
 // -------------------------- Run-time entry ------------------- {{{1
 // exported functions
 #define ASAN_REPORT_ERROR(type, is_write, size)                     \
 extern "C" NOINLINE INTERFACE_ATTRIBUTE                             \
 void __asan_report_ ## type ## size(uptr addr) {                    \
   GET_CALLER_PC_BP_SP;                                              \
-  __asan_report_error(pc, bp, sp, addr, is_write, size, 0);         \
+  __asan_report_error(pc, bp, sp, addr, is_write, size, 0, true);   \
 }                                                                   \
 extern "C" NOINLINE INTERFACE_ATTRIBUTE                             \
 void __asan_report_exp_ ## type ## size(uptr addr, u32 exp) {       \
   GET_CALLER_PC_BP_SP;                                              \
-  __asan_report_error(pc, bp, sp, addr, is_write, size, exp);       \
-}
+  __asan_report_error(pc, bp, sp, addr, is_write, size, exp, true); \
+}                                                                   \
+extern "C" NOINLINE INTERFACE_ATTRIBUTE                             \
+void __asan_report_ ## type ## size ## _noabort(uptr addr) {        \
+  GET_CALLER_PC_BP_SP;                                              \
+  __asan_report_error(pc, bp, sp, addr, is_write, size, 0, false);  \
+}                                                                   \
 
 ASAN_REPORT_ERROR(load, false, 1)
 ASAN_REPORT_ERROR(load, false, 2)
 ASAN_REPORT_ERROR(load, false, 4)
 ASAN_REPORT_ERROR(load, false, 8)
 ASAN_REPORT_ERROR(load, false, 16)
 ASAN_REPORT_ERROR(store, true, 1)
 ASAN_REPORT_ERROR(store, true, 2)
 ASAN_REPORT_ERROR(store, true, 4)
 ASAN_REPORT_ERROR(store, true, 8)
 ASAN_REPORT_ERROR(store, true, 16)
 
-#define ASAN_REPORT_ERROR_N(type, is_write)                    \
-extern "C" NOINLINE INTERFACE_ATTRIBUTE                        \
-void __asan_report_ ## type ## _n(uptr addr, uptr size) {      \
-  GET_CALLER_PC_BP_SP;                                         \
-  __asan_report_error(pc, bp, sp, addr, is_write, size, 0);    \
-}                                                              \
-extern "C" NOINLINE INTERFACE_ATTRIBUTE                        \
+#define ASAN_REPORT_ERROR_N(type, is_write)                                 \
+extern "C" NOINLINE INTERFACE_ATTRIBUTE                                     \
+void __asan_report_ ## type ## _n(uptr addr, uptr size) {                   \
+  GET_CALLER_PC_BP_SP;                                                      \
+  __asan_report_error(pc, bp, sp, addr, is_write, size, 0, true);           \
+}                                                                           \
+extern "C" NOINLINE INTERFACE_ATTRIBUTE                                     \
 void __asan_report_exp_ ## type ## _n(uptr addr, uptr size, u32 exp) {      \
   GET_CALLER_PC_BP_SP;                                                      \
-  __asan_report_error(pc, bp, sp, addr, is_write, size, exp);               \
-}
+  __asan_report_error(pc, bp, sp, addr, is_write, size, exp, true);         \
+}                                                                           \
+extern "C" NOINLINE INTERFACE_ATTRIBUTE                                     \
+void __asan_report_ ## type ## _n_noabort(uptr addr, uptr size) {           \
+  GET_CALLER_PC_BP_SP;                                                      \
+  __asan_report_error(pc, bp, sp, addr, is_write, size, 0, false);          \
+}                                                                           \
 
 ASAN_REPORT_ERROR_N(load, false)
 ASAN_REPORT_ERROR_N(store, true)
 
-#define ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, exp_arg)        \
+#define ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, exp_arg, fatal) \
     uptr sp = MEM_TO_SHADOW(addr);                                             \
     uptr s = size <= SHADOW_GRANULARITY ? *reinterpret_cast<u8 *>(sp)          \
                                         : *reinterpret_cast<u16 *>(sp);        \
     if (UNLIKELY(s)) {                                                         \
       if (UNLIKELY(size >= SHADOW_GRANULARITY ||                               \
                    ((s8)((addr & (SHADOW_GRANULARITY - 1)) + size - 1)) >=     \
                        (s8)s)) {                                               \
         if (__asan_test_only_reported_buggy_pointer) {                         \
           *__asan_test_only_reported_buggy_pointer = addr;                     \
         } else {                                                               \
           GET_CALLER_PC_BP_SP;                                                 \
-          __asan_report_error(pc, bp, sp, addr, is_write, size, exp_arg);      \
+          __asan_report_error(pc, bp, sp, addr, is_write, size, exp_arg,       \
+                              fatal);                                          \
         }                                                                      \
       }                                                                        \
     }
 
 #define ASAN_MEMORY_ACCESS_CALLBACK(type, is_write, size)                      \
   extern "C" NOINLINE INTERFACE_ATTRIBUTE                                      \
   void __asan_##type##size(uptr addr) {                                        \
-    ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, 0)                  \
+    ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, 0, true)            \
   }                                                                            \
   extern "C" NOINLINE INTERFACE_ATTRIBUTE                                      \
   void __asan_exp_##type##size(uptr addr, u32 exp) {                           \
-    ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, exp)                \
-  }
+    ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, exp, true)          \
+  }                                                                            \
+  extern "C" NOINLINE INTERFACE_ATTRIBUTE                                      \
+  void __asan_##type##size ## _noabort(uptr addr) {                            \
+    ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, 0, false)           \
+  }                                                                            \
 
 ASAN_MEMORY_ACCESS_CALLBACK(load, false, 1)
 ASAN_MEMORY_ACCESS_CALLBACK(load, false, 2)
 ASAN_MEMORY_ACCESS_CALLBACK(load, false, 4)
 ASAN_MEMORY_ACCESS_CALLBACK(load, false, 8)
 ASAN_MEMORY_ACCESS_CALLBACK(load, false, 16)
 ASAN_MEMORY_ACCESS_CALLBACK(store, true, 1)
 ASAN_MEMORY_ACCESS_CALLBACK(store, true, 2)
 ASAN_MEMORY_ACCESS_CALLBACK(store, true, 4)
 ASAN_MEMORY_ACCESS_CALLBACK(store, true, 8)
 ASAN_MEMORY_ACCESS_CALLBACK(store, true, 16)
 
 extern "C"
 NOINLINE INTERFACE_ATTRIBUTE
 void __asan_loadN(uptr addr, uptr size) {
   if (__asan_region_is_poisoned(addr, size)) {
     GET_CALLER_PC_BP_SP;
-    __asan_report_error(pc, bp, sp, addr, false, size, 0);
+    __asan_report_error(pc, bp, sp, addr, false, size, 0, true);
   }
 }
 
 extern "C"
 NOINLINE INTERFACE_ATTRIBUTE
 void __asan_exp_loadN(uptr addr, uptr size, u32 exp) {
   if (__asan_region_is_poisoned(addr, size)) {
     GET_CALLER_PC_BP_SP;
-    __asan_report_error(pc, bp, sp, addr, false, size, exp);
+    __asan_report_error(pc, bp, sp, addr, false, size, exp, true);
+  }
+}
+
+extern "C"
+NOINLINE INTERFACE_ATTRIBUTE
+void __asan_loadN_noabort(uptr addr, uptr size) {
+  if (__asan_region_is_poisoned(addr, size)) {
+    GET_CALLER_PC_BP_SP;
+    __asan_report_error(pc, bp, sp, addr, false, size, 0, false);
   }
 }
 
 extern "C"
 NOINLINE INTERFACE_ATTRIBUTE
 void __asan_storeN(uptr addr, uptr size) {
   if (__asan_region_is_poisoned(addr, size)) {
     GET_CALLER_PC_BP_SP;
-    __asan_report_error(pc, bp, sp, addr, true, size, 0);
+    __asan_report_error(pc, bp, sp, addr, true, size, 0, true);
   }
 }
 
 extern "C"
 NOINLINE INTERFACE_ATTRIBUTE
 void __asan_exp_storeN(uptr addr, uptr size, u32 exp) {
   if (__asan_region_is_poisoned(addr, size)) {
     GET_CALLER_PC_BP_SP;
-    __asan_report_error(pc, bp, sp, addr, true, size, exp);
+    __asan_report_error(pc, bp, sp, addr, true, size, exp, true);
+  }
+}
+
+extern "C"
+NOINLINE INTERFACE_ATTRIBUTE
+void __asan_storeN_noabort(uptr addr, uptr size) {
+  if (__asan_region_is_poisoned(addr, size)) {
+    GET_CALLER_PC_BP_SP;
+    __asan_report_error(pc, bp, sp, addr, true, size, 0, false);
   }
 }
 
 // Force the linker to keep the symbols for various ASan interface functions.
 // We want to keep those in the executable in order to let the instrumented
 // dynamic libraries access the symbol even if it is not used by the executable
 // itself. This should help if the build system is removing dead code at link
 // time.

lib/asan/scripts/asan_symbolize.py

class SymbolizationLoop(object):
 
   def process_line_echo(self, line):
     return [line.rstrip()]
 
   def process_line_posix(self, line):
     self.current_line = line.rstrip()
     #0 0x7f6e35cf2e45  (/blah/foo.so+0x11fe45)
     stack_trace_line_format = (
-        '^( *#([0-9]+) *)(0x[0-9a-f]+) *\((.*)\+(0x[0-9a-f]+)\)')
+        '^(?:[A-Z]\/[a-z ]*\([0-9 ]+\):)?( *#([0-9]+) *)(0x[0-9a-f]+) *\((.*)\+(0x[0-9a-f]+)\)')
     match = re.match(stack_trace_line_format, line)
     if not match:
       return [self.current_line]
     if DEBUG:
       print line
     _, frameno_str, addr, binary, offset = match.groups()
     if frameno_str == '0':
       # Assume that frame #0 is the first frame of new stack trace.

test/Driver/fsanitize.c

 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fsanitize-recover -### 2>&1 | FileCheck %s --check-prefix=CHECK-RECOVER
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fsanitize-recover=all -### 2>&1 | FileCheck %s --check-prefix=CHECK-RECOVER
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fno-sanitize-recover=undefined -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-RECOVER
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fno-sanitize-recover=all -fsanitize-recover=thread -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-RECOVER
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fno-sanitize-recover -fsanitize-recover=undefined -### 2>&1 | FileCheck %s --check-prefix=CHECK-RECOVER
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fsanitize-recover=all -fno-sanitize-recover=undefined -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-RECOVER
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fno-sanitize-recover=all -fsanitize-recover=object-size,shift-base -### 2>&1 | FileCheck %s --check-prefix=CHECK-PARTIAL-RECOVER
 
 // CHECK-RECOVER: "-fsanitize-recover={{((signed-integer-overflow|integer-divide-by-zero|float-divide-by-zero|function|shift-base|shift-exponent|vla-bound|alignment|null|vptr|object-size|float-cast-overflow|array-bounds|enum|bool|returns-nonnull-attribute|nonnull-attribute),?){17}"}}
 // CHECK-NO-RECOVER-NOT: sanitize-recover
 // CHECK-PARTIAL-RECOVER: "-fsanitize-recover={{((object-size|shift-base),?){2}"}}
 
-// RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fsanitize-recover=address,foobar,object-size,unreachable -### 2>&1 | FileCheck %s --check-prefix=CHECK-DIAG-RECOVER
+// RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fsanitize-recover=foobar,object-size,unreachable -### 2>&1 | FileCheck %s --check-prefix=CHECK-DIAG-RECOVER
 // CHECK-DIAG-RECOVER: unsupported argument 'foobar' to option 'fsanitize-recover='
-// CHECK-DIAG-RECOVER: unsupported argument 'address,unreachable' to option 'fsanitize-recover='
+// CHECK-DIAG-RECOVER: unsupported argument 'unreachable' to option 'fsanitize-recover='
 
 // RUN: %clang -target x86_64-linux-gnu %s -fsanitize=undefined -fsanitize-recover -fno-sanitize-recover -### 2>&1 | FileCheck %s --check-prefix=CHECK-DEPRECATED-RECOVER
 // CHECK-DEPRECATED-RECOVER: argument '-fsanitize-recover' is deprecated, use '-fsanitize-recover=undefined,integer' instead
 // CHECK-DEPRECATED-RECOVER: argument '-fno-sanitize-recover' is deprecated, use '-fno-sanitize-recover=undefined,integer' instead
 // CHECK-DEPRECATED-RECOVER-NOT: is deprecated
 
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=leak %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-SANL
 // CHECK-SANL: "-fsanitize=leak"

test/asan/TestCases/Linux/interface_symbols_linux.c

 // RUN: echo __asan_report_load16 >> %t.interface
 // RUN: echo __asan_report_store1 >> %t.interface
 // RUN: echo __asan_report_store2 >> %t.interface
 // RUN: echo __asan_report_store4 >> %t.interface
 // RUN: echo __asan_report_store8 >> %t.interface
 // RUN: echo __asan_report_store16 >> %t.interface
 // RUN: echo __asan_report_load_n >> %t.interface
 // RUN: echo __asan_report_store_n >> %t.interface
+// RUN: echo __asan_report_load1_noabort >> %t.interface
+// RUN: echo __asan_report_load2_noabort >> %t.interface
+// RUN: echo __asan_report_load4_noabort >> %t.interface
+// RUN: echo __asan_report_load8_noabort >> %t.interface
+// RUN: echo __asan_report_load16_noabort >> %t.interface
+// RUN: echo __asan_report_store1_noabort >> %t.interface
+// RUN: echo __asan_report_store2_noabort >> %t.interface
+// RUN: echo __asan_report_store4_noabort >> %t.interface
+// RUN: echo __asan_report_store8_noabort >> %t.interface
+// RUN: echo __asan_report_store16_noabort >> %t.interface
+// RUN: echo __asan_report_load_n_noabort >> %t.interface
+// RUN: echo __asan_report_store_n_noabort >> %t.interface
 // RUN: echo __asan_report_exp_load1 >> %t.interface
 // RUN: echo __asan_report_exp_load2 >> %t.interface
 // RUN: echo __asan_report_exp_load4 >> %t.interface
 // RUN: echo __asan_report_exp_load8 >> %t.interface
 // RUN: echo __asan_report_exp_load16 >> %t.interface
 // RUN: echo __asan_report_exp_store1 >> %t.interface
 // RUN: echo __asan_report_exp_store2 >> %t.interface
 // RUN: echo __asan_report_exp_store4 >> %t.interface

test/asan/TestCases/Linux/keep_going_torture.cc

+// Stress test keep_going with many threads.
+//
+// RUN: %clangxx_asan -fsanitize-recover=address %s -o %t
+// RUN: export ASAN_OPTIONS=keep_going=1:num_reported_errors=1000
+// RUN: %run %t 1 10 2>&1 | FileCheck %s
+// RUN: %run %t 10 20 2>&1 | FileCheck %s
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <pthread.h>
+#include <time.h>
+
+#include <sanitizer/asan_interface.h>
+
+size_t nthreads = 10;
+size_t niter = 10;
+
+void *run(void *arg) {
+  unsigned seed = (unsigned)(size_t)arg;
+
+  volatile char tmp[2];
+  __asan_poison_memory_region(&tmp, sizeof(tmp)); 
+
+  for (size_t i = 0; i < niter; ++i) {
+    struct timespec delay = { 0, rand_r(&seed) * 1000000 };
+    nanosleep(&delay, 0);
+
+    // Expect error collisions here
+    // CHECK: AddressSanitizer: use-after-poison
+    volatile int idx = 0;
+    tmp[idx] = 0;
+  }
+
+  return 0;
+}
+
+int main(int argc, char **argv) {
+  if (argc > 1)
+    nthreads = (size_t)strtoul(argv[1], 0, 0);
+
+  if (argc > 2)
+    niter = (size_t)strtoul(argv[2], 0, 0);
+
+  pthread_t *tids = (pthread_t *)malloc(nthreads * sizeof(pthread_t));
+
+  for (size_t i = 0; i < nthreads; ++i) {
+    if (0 != pthread_create(&tids[i], 0, run, (void *)i)) {
+      fprintf(stderr, "Failed to create thread\n");
+      exit(1);
+    }
+  }
+
+  for (size_t i = 0; i < nthreads; ++i) {
+    if (0 != pthread_join(tids[i], 0)) {
+      fprintf(stderr, "Failed to join thread\n");
+      exit(1);
+    }
+  }
+
+  // CHECK: All threads terminated
+  printf("All threads terminated\n");
+
+  free(tids);
+
+  return 0;
+}

test/asan/TestCases/keep-going.c

+// Test keep_going option.
+//
+// RUN: %clang_asan -fsanitize-recover=address %s -o %t
+// RUN: env not %run %t 2>&1 | FileCheck %s
+// RUN: env ASAN_OPTIONS=$ASAN_OPTIONS:keep_going=1 %run %t 2>&1 | FileCheck %s --check-prefix CHECK-KEEP-GOING
+// RUN: env ASAN_OPTIONS=$ASAN_OPTIONS:keep_going=1:num_reported_errors=2 %run %t 2>&1 | FileCheck %s --check-prefix CHECK-NUM-ERRORS
+
+#include <string.h>
+
+volatile int ten = 10;
+
+int main() {
+  char x[10];
+  // CHECK: WRITE of size 11
+  // CHECK-KEEP-GOING: WRITE of size 11
+  // CHECK-NUM-ERRORS: WRITE of size 11
+  memset(x, 0, 11);
+  // CHECK-NOT: READ of size 1
+  // CHECK-KEEP-GOING: READ of size 1
+  // CHECK-NUM-ERRORS: READ of size 1
+  volatile int res = x[ten];
+  // CHECK-NOT: WRITE of size 1
+  // CHECK-KEEP-GOINF: WRITE of size 1
+  // CHECK-NUM-ERRORS-NOT: WRITE of size 1
+  x[ten] = res + 3;
+  // CHECK-NOT: READ of size 1
+  // CHECK-KEEP-GOING: READ of size 1
+  // CHECK-NUM-ERRORS-NOT: READ of size 1
+  res = x[ten];
+  return  0;
+}
+