[analyzer] Turn suppress-c++-stdlib on by default

[zaks.anna]

Summary

We have several reports of false positives coming from libc++. For example, there are reports of false positives in std::regex, std::wcout, and also a bunch of issues are reported in https://reviews.llvm.org/D30593. In many cases, the analyzer trips over the complex libc++ code invariants. Let's turn off the reports coming from these headers until we can re-evalate the support.

We can turn this back on once we individually suppress all known false positives and perform deeper evaluation on large codebases that use libc++. We'd also need to commit to doing these evaluations regularly as libc++ headers change.

[dcoughlin]

LGTM,

[zaks.anna]

I've committed the change, but would very much appreciate community feedback here if if there is any!

[xazax.hun]

In D30798#697115, @zaks.anna wrote:

I've committed the change, but would very much appreciate community feedback here if if there is any!

I agree with the change. Users are usually not interested in the results from the standard library, and since the standard library supposed to be well tested, it is very unlikely that the analyzer could find a true positive there. One side effect would be the suppression of results that materialize due to the misuse of those APIs.

[a.sidorin]

I have no any objection on this change.

[NoQ]

I'd like to clarify that in case of path-sensitive analysis there are actually three warning classes to consider.

1. Warnings that reside completely in system headers and indicate bugs in system headers, most likely falsely.
2. Warnings that originate from the main file and indicate misuse of system header APIs, likely correctly.
3. Warnings that originate from the main file, but still indicate bugs in system headers, because the part of the path that stays in the main file is irrelevant to the warning.

This change suppresses 3, and as a side effect we suppress 1, because discriminating between 1 and 3 is hard* . We never needed to suppress 2, because they're never created to begin with: AnalysisConsumer doesn't consider header-only functions for top-level analysis.
__
// * I think that actually such discrimination is curious and probably worth investigating. We could try to see which symbols (originating from where) participate in the equation that expresses the invariant violation of which causes the warning. We could potentially improve our suppressions greatly (maybe even make them non-hackish) and probably avoid the common problems with inlining this way. Just brainstorming.

[zaks.anna]

Correct, this will suppress some valid warnings that occur due to user errors and valid warnings coming from the standard library.

However, I believe this is the right choice right now since we know that the analyzer is not currently effective in understanding invariants coming from libc++ headers and users have no ability to help the analyzer (ex: by inserting assertions). This results in a wave of false positives every time the headers change. Furthermore, we do not perform sufficient evaluation and testing for every new version of the headers.

cfe/trunk/lib/StaticAnalyzer/Core/AnalyzerOptions.cpp

bool AnalyzerOptions::shouldSuppressInlinedDefensiveChecks() {
   return getBooleanOption(SuppressInlinedDefensiveChecks,
                           "suppress-inlined-defensive-checks",
                           /* Default = */ true);
 }
 
 bool AnalyzerOptions::shouldSuppressFromCXXStandardLibrary() {
   return getBooleanOption(SuppressFromCXXStandardLibrary,
                           "suppress-c++-stdlib",
-                          /* Default = */ false);
+                          /* Default = */ true);
 }
 
 bool AnalyzerOptions::shouldReportIssuesInMainSourceFile() {
   return getBooleanOption(ReportIssuesInMainSourceFile,
                           "report-in-main-source-file",
                           /* Default = */ false);
 }
 

cfe/trunk/test/Analysis/diagnostics/explicit-suppression.cpp

 // RUN: %clang_analyze_cc1 -analyzer-checker=core,debug.ExprInspection -analyzer-config suppress-c++-stdlib=false -verify %s
 // RUN: %clang_analyze_cc1 -analyzer-checker=core,debug.ExprInspection -analyzer-config suppress-c++-stdlib=true -DSUPPRESSED=1 -verify %s
+// RUN: %clang_analyze_cc1 -analyzer-checker=core,debug.ExprInspection -DSUPPRESSED=1 -verify %s
 
 #ifdef SUPPRESSED
 // expected-no-diagnostics
 #endif
 
 #include "../Inputs/system-header-simulator-cxx.h"
 
 void clang_analyzer_eval(bool);