[analyzer] Add a new frontend flag to display all checker options

[Szelethus]

Summary

Title says it all, here is how it looks like locally:

clang -cc1 -analyzer-checker-option-help

OVERVIEW: Clang Static Analyzer Checker and Package Option List

USAGE: clang -cc1 [CLANG_OPTIONS] -analyzer-config <OPTION1=VALUE,OPTION2=VALUE,...>

       clang -cc1 [CLANG_OPTIONS] -analyzer-config OPTION1=VALUE, -analyzer-config OPTION2=VALUE, ...

       clang [CLANG_OPTIONS] -Xclang -analyzer-config -Xclang<OPTION1=VALUE,OPTION2=VALUE,...>

       clang [CLANG_OPTIONS] -Xclang -analyzer-config -Xclang OPTION1=VALUE, -Xclang -analyzer-config -Xclang OPTION2=VALUE, ...

OPTIONS:

  alpha.clone.CloneChecker:MinimumCloneComplexity   (int) Ensures that every clone has at least
                                                    the given complexity. Complexity is here
                                                    defined as the total amount of children
                                                    of a statement. This constraint assumes
                                                    the first statement in the group is representative
                                                    for all other statements in the group in
                                                    terms of complexity. (default: 50)

  alpha.clone.CloneChecker:ReportNormalClones       (bool) Report all clones, even less suspicious
                                                    ones. (default: true)

  alpha.clone.CloneChecker:IgnoredFilesPattern      (string) If supplied, the checker wont analyze
                                                    files with a filename that matches the given
                                                    pattern. (default: "")

  alpha.cplusplus.UninitializedObject:Pedantic      (bool) If set to false, the checker won't
                                                    emit warnings for objects that don't have
                                                    at least one initialized field. (default: false)

  alpha.cplusplus.UninitializedObject:NotesAsWarnings
                                                    (bool) If set to true, the checker will
                                                    emit a warning for each uninitalized field,
                                                    as opposed to emitting one warning per constructor
                                                    call, and listing the uninitialized fields
                                                    that belongs to it in notes. (default: false)

  alpha.cplusplus.UninitializedObject:CheckPointeeInitialization
                                                    (bool) If set to false, the checker will
                                                    not analyze the pointee of pointer/reference
                                                    fields, and will only check whether the
                                                    object itself is initialized. (default: false)

  alpha.cplusplus.UninitializedObject:IgnoreRecordsWithField
                                                    (string) If supplied, the checker will not
                                                    analyze structures that have a field with
                                                    a name or type name that matches the given
                                                    pattern. (default: "")

  alpha.cplusplus.UninitializedObject:IgnoreGuardedFields
                                                    (bool) If set to true, the checker will
                                                    analyze _syntactically_ whether the found
                                                    uninitialized object is used without a preceding
                                                    assert call. Defaults to false. (default: false)

  alpha.security.MmapWriteExec:MmapProtExec         (int) Specifies the value of PROT_EXEC 
                                                    (default: 0x04)

  alpha.security.MmapWriteExec:MmapProtRead         (int) Specifies the value of PROT_READ 
                                                    (default: 0x01)

  cplusplus.Move:WarnOn                             (string) In non-aggressive mode, only warn
                                                    on use-after-move of local variables (or
                                                    local rvalue references) and of STL objects.
                                                    The former is possible because local variables
                                                    (or local rvalue references) are not tempting
                                                    their user to re-use the storage. The latter
                                                    is possible because STL objects are known
                                                    to end up in a valid but unspecified state
                                                    after the move and their state-reset methods
                                                    are also known, which allows us to predict
                                                    precisely when use-after-move is invalid.
                                                    Some STL objects are known to conform to
                                                    additional contracts after move, so they
                                                    are not tracked. However, smart pointers
                                                    specifically are tracked because we can
                                                    perform extra checking over them. In aggressive
                                                    mode, warn on any use-after-move because
                                                    the user has intentionally asked us to completely
                                                    eliminate use-after-move in his code. Values:
                                                    "KnownsOnly", "KnownsAndLocals", "All". 
                                                    (default: KnownsAndLocals)

  debug.AnalysisOrder:PreStmtCastExpr               (bool) (default: false)

  debug.AnalysisOrder:PostStmtCastExpr              (bool) (default: false)

  debug.AnalysisOrder:PreStmtArraySubscriptExpr     (bool) (default: false)

  debug.AnalysisOrder:PostStmtArraySubscriptExpr    (bool) (default: false)

  debug.AnalysisOrder:PreStmtCXXNewExpr             (bool) (default: false)

  debug.AnalysisOrder:PostStmtCXXNewExpr            (bool) (default: false)

  debug.AnalysisOrder:PreStmtOffsetOfExpr           (bool) (default: false)

  debug.AnalysisOrder:PostStmtOffsetOfExpr          (bool) (default: false)

  debug.AnalysisOrder:PreCall                       (bool) (default: false)

  debug.AnalysisOrder:PostCall                      (bool) (default: false)

  debug.AnalysisOrder:EndFunction                   (bool) (default: false)

  debug.AnalysisOrder:NewAllocator                  (bool) (default: false)

  debug.AnalysisOrder:Bind                          (bool) (default: false)

  debug.AnalysisOrder:LiveSymbols                   (bool) (default: false)

  debug.AnalysisOrder:RegionChanges                 (bool) (default: false)

  debug.AnalysisOrder:*                             (bool) Enables all callbacks. (default: false)

  nullability:NoDiagnoseCallsToSystemHeaders        (bool) (default: false)

  optin.cplusplus.VirtualCall:PureOnly              (bool) Whether to only report calls to pure
                                                    virtual methods. (default: false)

  optin.osx.cocoa.localizability.NonLocalizedStringChecker:AggressiveReport
                                                    (bool) Marks a string being returned by
                                                    any call as localized if it is in LocStringFunctions
                                                    (LSF) or the function is annotated. Otherwise,
                                                    we mark it as NonLocalized (Aggressive)
                                                    or NonLocalized only if it is not backed
                                                    by a SymRegion (Non-Aggressive), basically
                                                    leaving only string literals as NonLocalized. 
                                                    (default: false)

  optin.performance.Padding:AllowedPad              (int) Reports are only generated if the
                                                    excessive padding exceeds 'AllowedPad' in
                                                    bytes. (default: 24)

  osx.NumberObjectConversion:Pedantic               (bool) Enables detection of more conversion
                                                    patterns (which are most likely more harmless,
                                                    and therefore are more likely to produce
                                                    false positives). (default: false)

  osx.cocoa.RetainCount:leak-diagnostics-reference-allocation
                                                    (bool) (default: false)

  osx.cocoa.RetainCount:CheckOSObject               (bool) (default: true)

  osx.cocoa.RetainCount:TrackNSCFStartParam         (bool) (default: false)

  unix.DynamicMemoryModeling:Optimistic             (bool) If set to true, the checker assumes
                                                    that all the allocating and deallocating
                                                    functions are annotated with ownership_holds,
                                                    ownership_takes and ownership_returns. 
                                                    (default: false)

[NoQ]

I think this is awesome!

[Szelethus]

Also, don't forget package options :^)

[Szelethus]

Add a test case for then checker plugin.

[dcoughlin]

I'm pretty worried about exposing this flag to end users.

• Almost none of the options you've listed are user facing. Many represent options intended for use by static analyzer developers: debugging options, feature flags, and checkers that were never finished. Others represent mechanisms for build systems to control the behavior of the analyzer. Even these are not meant for end users to interact with but rather for implementers of build systems and IDEs. I don't think end users should have to understand these options to use the analyzer.
• The help text refers to analyzer implementation details (such as "SymRegion") that users won't have the context or knowledge to understand.
• The help text also recommends invoking -cc1 directly or through the driver with -Xclang. Neither of these are supported end-user interfaces to the analyzer. Instead, users should use scan-build or another tool (such as CodeChecker) that was designed to be used by humans.

[Szelethus]

This patch isn't intended for end users, which is why -analyzer-checker-option-help is a frontend flag -- besides, some of the plain frontend flags aren't either. It is purely for development purposes, though exposing an end user friendly subset of these through a driver flag would be cool.

[dkrupp]

@dcoughlin I don't necessarily agree with you.
Let me explain why we think this feature is important.

We should give the users the possibility to list all possibly configurable checker options and their meaning.

Many of these options should be possible to be set by the end user to be able to fine tune the checker behaviour to match the analyzed code style.
Such examples are:
alpha.clone.CloneChecker:MinimumCloneComplexity
alpha.cplusplus.UninitializedObject:CheckPointeeInitialization
alpha.clone.CloneChecker:ReportNormalClones
alpha.cplusplus.UninitializedObject:IgnoreRecordsWithField
etc.
Actually except for the debug checker options and unix.DynamicMemoryModeling:Optimistic all these options are meaningful end-user options.

Of course there are the debug checker options which are not interesting for the end user, but debug checkers should anyway be ignored by the end-users.

We always wanted to add checker option listing to CodeChecker, however did not want to duplicate the option list and option documentation in the CodeChecker codebase.
It belongs to the analyzer and actually to the checker implementation.
So from CodeChecker we would like to invoke the "clang -cc1 -analyzer-checker-option-help" to be able to list these options to the end users.

The same feature is available already in clang-tidy: clang-tidy -dump-config

I think it is the responsibility of the end-user to decide what option he may want to configure.

I understand you would like to differentiate between "developer" and "end-user" options.
What we could do maybe is to indicate in the option explanation that the given option is "analyzer-internal", "experimental" or "developer only".

What do you think about that?

In D57858#1432714, @dcoughlin wrote:

I'm pretty worried about exposing this flag to end users.

• Almost none of the options you've listed are user facing. Many represent options intended for use by static analyzer developers: debugging options, feature flags, and checkers that were never finished. Others represent mechanisms for build systems to control the behavior of the analyzer. Even these are not meant for end users to interact with but rather for implementers of build systems and IDEs. I don't think end users should have to understand these options to use the analyzer.
• The help text refers to analyzer implementation details (such as "SymRegion") that users won't have the context or knowledge to understand.
• The help text also recommends invoking -cc1 directly or through the driver with -Xclang. Neither of these are supported end-user interfaces to the analyzer. Instead, users should use scan-build or another tool (such as CodeChecker) that was designed to be used by humans.

[Szelethus]

Ping.

[Szelethus]

Ping^2

[Szelethus]

Ping^3

[whisperity]

@dcoughlin How would removing the USAGE part of the dump and keeping only the list of options and their formatted help sound? That way, this option will not invite the user to directly call the analyzer.

In D57858#1432714, @dcoughlin wrote:

• The help text also recommends invoking -cc1 directly or through the driver with -Xclang. Neither of these are supported end-user interfaces to the analyzer.

Calling this option itself, at least based on the original commit's first line, is through -cc1, and thus using a "checker/SA developer interface". This seems more purely as a tooling help, which as expressed by @dkrupp earlier, is helpful for "wrangler tools".

@Szelethus From a CodeChecker guy's perspective, I am a bit scared about the size this dump can get assuming every option is given a description/help text nicely, but all in all I like the direction of this patch.

lib/StaticAnalyzer/Frontend/CheckerRegistry.cpp

584–592 ↗

(On Diff #190997)

(I mean killing these lines.)

[Szelethus]

Similarly to D60925, I plan to make a patch that will hide most the implementation options.

[Szelethus]

Ping^4

[NoQ]

Let's land this together with @Szelethus's proposed solution of hiding hidden options (like checkers in D60925). I'm still in favor of hiding alpha checkers (as they are for development only, much like debug flags; i'd recommend hiding them in the CodeChecker UI as well) and we should probably automatically hide options of checker that are hidden. Let's also, indeed, remove the USAGE thingy as it doesn't add much.

So, like, the global picture is as follows. In our case the Driver (i.e., --analyze) is not much more user facing than frontend flags. It's still fairly unusable directly, as our Static Analyzer is generally not a command-line tool. The real user-facing stuff is the GUIs such as scan-build or CodeChecker. These GUIs decide themselves on what options they want to expose. For instance, you have a right to decide that CodeChecker shouldn't support the aggressive mode of the move-checker and don't expose it as an option in your GUI. In this sense it's not really useful to provide a centralized -help of all user-facing options.

But it sounds as if you want to change this situation and provide a single source of truth on what are the user-facing options. Particular GUIs can still ignore them, but you don't want to hardcode flags in CodeChecker, but instead you want to rely on clang to provide the list of supported options for you and, as a side effect, for scan-build users (if you also add a scan-build help flag). I'm totally in favor of crystallizing such list of user-facing flags, and this patch sounds like a good step in that direction, assuming that non-user-facing options are hidden.

Now, why do we care about frontend/driver flags when they're unusable by definition? That's because we have a mental trauma after seeing a few powerusers actively explore those flags, observe that they don't work, and then tell everybody that the Analyzer is broken. So there's a threshold, based on a tiny but painful bit of practical experience, that says that documentation of developer-only features on llvm.org or in code comments is fine, but documentation printed by the released binary itself is not fine.

[NoQ]

Also, for CodeChecker purposes wouldn't you rather have a machine-readable dump?

[Szelethus]

• Make the educated observation that the code for formatted output is duplicated for -analyzer-checker-help, -analyzer-config-help and this patch, reuse it via the new static function AnalyzerOptions::printFormattedEntry.
• Remove the USAGE that suggested the use of the driver (did you mean to remove it entirely? Really, without reading test files, you'd never know how to use it)
• Hide certain options by default, implemented in a followup patch.

[Szelethus]

In D57858#1498651, @NoQ wrote:

Also, for CodeChecker purposes wouldn't you rather have a machine-readable dump?

My main purpose was readability in the terminal, but I will need to think about that eventually. The current diff allows such formatting changes very easily, should CodeChecker have any issues, and I do like the output format as it is currently (as far as I know, clang-tidy check options are formatted similarly).

[Szelethus]

In D57858#1498640, @NoQ wrote:

So, like, the global picture is as follows. In our case the Driver (i.e., --analyze) is not much more user facing than frontend flags. It's still fairly unusable directly, as our Static Analyzer is generally not a command-line tool. The real user-facing stuff is the GUIs such as scan-build or CodeChecker. These GUIs decide themselves on what options they want to expose. For instance, you have a right to decide that CodeChecker shouldn't support the aggressive mode of the move-checker and don't expose it as an option in your GUI. In this sense it's not really useful to provide a centralized -help of all user-facing options.

But it sounds as if you want to change this situation and provide a single source of truth on what are the user-facing options. Particular GUIs can still ignore them, but you don't want to hardcode flags in CodeChecker, but instead you want to rely on clang to provide the list of supported options for you and, as a side effect, for scan-build users (if you also add a scan-build help flag). I'm totally in favor of crystallizing such list of user-facing flags, and this patch sounds like a good step in that direction, assuming that non-user-facing options are hidden.

That describes my intention quite well :)

I'm still in favor of hiding alpha checkers (as they are for development only, much like debug flags; i'd recommend hiding them in the CodeChecker UI as well)

Now, why do we care about frontend/driver flags when they're unusable by definition? That's because we have a mental trauma after seeing a few powerusers actively explore those flags, observe that they don't work, and then tell everybody that the Analyzer is broken. So there's a threshold, based on a tiny but painful bit of practical experience, that says that documentation of developer-only features on llvm.org or in code comments is fine, but documentation printed by the released binary itself is not fine.

What you say sounds very reasonable. Still, I am kind of hesitant about hiding all alpha checkers: I initially intended to hide only are developer-only checkers (modeling, debug). I guess if we define alpha checkers (as you stated numerous times) as incomplete, under development, are missing half their limbs and crash if you look at them the wrong way, sure, they belong in the developer-only category. But checkers such as mine (UninitializedObjectChecker), for the longest time were very stable, have been enabled by default for our internal projects, despite only recently moving out of alpha.

Then agaaain, if we're that stubborn about alpha checkers, we could might as well dig them out of -analyzer-checker-help-hidden, and leave the rest there. Untangling what alpha checkers depend on one another could be solved by making yet another frontend flag that would display checker dependencies, which would be super easy since D54438, or create an -analyzer-checker-help-alpha flag that would display alpha, but not developer-only checkers. @dkrupp @o.gyorgy Do you have any feelings on this?

and we should probably automatically hide options of checker that are hidden.

Checker options are a different kind of animal entirely. I think we should definitely let the user fine-tune some modeling checkers, for instance, unix.DynamicMemoryModeling:Optimistic, despite us not really wanting to let anyone (even developers really) mess around whether unix.DynamicMemoryModeling should be enabled. While that specific option is, to put it nicely, a little esoteric, making some decisions the analyzer makes less conservative, or limiting state splits to help performance may be desirable in the future.

Let's move the rest of the discussion directly related to hiding checker options to D61839!

[dkrupp]

In D57858#1499414, @Szelethus wrote:

In D57858#1498640, @NoQ wrote:

So, like, the global picture is as follows. In our case the Driver (i.e., --analyze) is not much more user facing than frontend flags. It's still fairly unusable directly, as our Static Analyzer is generally not a command-line tool. The real user-facing stuff is the GUIs such as scan-build or CodeChecker. These GUIs decide themselves on what options they want to expose. For instance, you have a right to decide that CodeChecker shouldn't support the aggressive mode of the move-checker and don't expose it as an option in your GUI. In this sense it's not really useful to provide a centralized -help of all user-facing options.

But it sounds as if you want to change this situation and provide a single source of truth on what are the user-facing options. Particular GUIs can still ignore them, but you don't want to hardcode flags in CodeChecker, but instead you want to rely on clang to provide the list of supported options for you and, as a side effect, for scan-build users (if you also add a scan-build help flag). I'm totally in favor of crystallizing such list of user-facing flags, and this patch sounds like a good step in that direction, assuming that non-user-facing options are hidden.

That describes my intention quite well :)

I'm still in favor of hiding alpha checkers (as they are for development only, much like debug flags; i'd recommend hiding them in the CodeChecker UI as well)

Now, why do we care about frontend/driver flags when they're unusable by definition? That's because we have a mental trauma after seeing a few powerusers actively explore those flags, observe that they don't work, and then tell everybody that the Analyzer is broken. So there's a threshold, based on a tiny but painful bit of practical experience, that says that documentation of developer-only features on llvm.org or in code comments is fine, but documentation printed by the released binary itself is not fine.

What you say sounds very reasonable. Still, I am kind of hesitant about hiding all alpha checkers: I initially intended to hide only are developer-only checkers (modeling, debug). I guess if we define alpha checkers (as you stated numerous times) as incomplete, under development, are missing half their limbs and crash if you look at them the wrong way, sure, they belong in the developer-only category. But checkers such as mine (UninitializedObjectChecker), for the longest time were very stable, have been enabled by default for our internal projects, despite only recently moving out of alpha.

Then agaaain, if we're that stubborn about alpha checkers, we could might as well dig them out of -analyzer-checker-help-hidden, and leave the rest there. Untangling what alpha checkers depend on one another could be solved by making yet another frontend flag that would display checker dependencies, which would be super easy since D54438, or create an -analyzer-checker-help-alpha flag that would display alpha, but not developer-only checkers. @dkrupp @o.gyorgy Do you have any feelings on this?

and we should probably automatically hide options of checker that are hidden.

Checker options are a different kind of animal entirely. I think we should definitely let the user fine-tune some modeling checkers, for instance, unix.DynamicMemoryModeling:Optimistic, despite us not really wanting to let anyone (even developers really) mess around whether unix.DynamicMemoryModeling should be enabled. While that specific option is, to put it nicely, a little esoteric, making some decisions the analyzer makes less conservative, or limiting state splits to help performance may be desirable in the future.

Let's move the rest of the discussion directly related to hiding checker options to D61839!

Yes, it would be great if the clang static analyzer would be the ultimate source of information with respect to the checkers and checker options. Then we would not need to split this info between the front-end (scanbuild, codechecker) and the analyzer.

Some alpha checkers are considerably more mature than others and are quite usable. In our experience, there are some users who are keen to run these checkers on their code and report back any false positives to us. So in this sense these are not "developer only" checkers. So I think we should let the users list them, read their descriptions and try them out. Some of them will come back with useful feedback as to how to improve them further. Some users would not care if the checker gives some more false positives than the "mature" checkers if they can catch some true positives with them.

@Szelethus has a good point in supporting three checker and option categories
-stable: checkers that are stable and which report "small number of false positives". (-analyzer-checker-help)
-experimental: checkers in alpha stage, with more false positives than usual, but which still should not crash. (-analyzer-checker-help-experimental or -analyzer-checker-help-alpha or -analyzer-checker-help-hidden ). We can add a disclaimer for this option.
-developer: modeling checkers, debug checkers that is used by checker/analyzer developer (-analyzer-checker-help-devel)

we could create the same help categories for checker options and non-checker configuration.

[NoQ]

In D57858#1499996, @dkrupp wrote:

Some alpha checkers are considerably more mature than others and are quite usable. In our experience, there are some users who are keen to run these checkers on their code and report back any false positives to us. So in this sense these are not "developer only" checkers. So I think we should let the users list them, read their descriptions and try them out. Some of them will come back with useful feedback as to how to improve them further.

What are such checkers currently? Like, the ones that aren't clearly "missing limbs" and that have somebody happy to address feedback sent against them?

Do you have a chance to call out to your users for testing the checker and actively request feedback, as @Szelethus did on the mailing list?

I feel that we could do some sort of "early access checkers" programme, but i believe this would require a more careful PR than just dumping a list of alpha checkers on our users' heads.

In D57858#1499996, @dkrupp wrote:

Some users would not care if the checker gives some more false positives than the "mature" checkers if they can catch some true positives with them.

Yeah, and these are pretty much the users we're trying to protect from themselves :)

[Szelethus]

In D57858#1500635, @NoQ wrote:

In D57858#1499996, @dkrupp wrote:

Some alpha checkers are considerably more mature than others and are quite usable. In our experience, there are some users who are keen to run these checkers on their code and report back any false positives to us. So in this sense these are not "developer only" checkers. So I think we should let the users list them, read their descriptions and try them out. Some of them will come back with useful feedback as to how to improve them further.

What are such checkers currently? Like, the ones that aren't clearly "missing limbs" and that have somebody happy to address feedback sent against them?

IteratorChecker is a prime example that still suffers from not ideal FP/TP ratio, but users at Ericsson value it plenty enough not to be bothered by it. Many of the changes here and in private directly address user bug reports (That's just one, but I do remember having others around too!).

Do you have a chance to call out to your users for testing the checker and actively request feedback, as @Szelethus did on the mailing list?

Actually yes! Experimental features, even if not checkers, such as the macro expansion thingy I wrote was very much feedback driven.

I feel that we could do some sort of "early access checkers" programme, but i believe this would require a more careful PR than just dumping a list of alpha checkers on our users' heads.

How about making the -analyzer-checker-help-alpha flag only emit alpha checkers, surrounded by giant disclaimers, and adding some scary text to each checkers description?

*** DISCLAIMER: Alpha checkers might be incomplet and inkorrekt, unstable and emit high amount of false positives! Use it as your own discretion! ***

  alpha.cplusplus.ReinterpretCastIsBad  **EXPERIMENTAL, UNDER DEVELOPMENT** A checker
                                        that shames you for using reinterpret_cast.

  alpha.optin.optout.Random             **EXPERIMENTAL, UNDER DEVELOPMENT** This
                                        checker will cause a crash when enabled. Really.

  alpha.pointless.Congrats              **EXPERIMENTAL, UNDER DEVELOPMENT** A checker
                                        that emits a report for each correct void* to
                                        function pointer C-style cast. Nice!

*** DISCLAIMER: Alpha checkers might be incomplet and inkorrekt, unstable and emit high amount of false positives! Use it as your own discretion! ***

[dkrupp]

In D57858#1500635, @NoQ wrote:

In D57858#1499996, @dkrupp wrote:

Some alpha checkers are considerably more mature than others and are quite usable. In our experience, there are some users who are keen to run these checkers on their code and report back any false positives to us. So in this sense these are not "developer only" checkers. So I think we should let the users list them, read their descriptions and try them out. Some of them will come back with useful feedback as to how to improve them further.

What are such checkers currently? Like, the ones that aren't clearly "missing limbs" and that have somebody happy to address feedback sent against them?

Do you have a chance to call out to your users for testing the checker and actively request feedback, as @Szelethus did on the mailing list?

I feel that we could do some sort of "early access checkers" programme, but i believe this would require a more careful PR than just dumping a list of alpha checkers on our users' heads.

In D57858#1499996, @dkrupp wrote:

Some users would not care if the checker gives some more false positives than the "mature" checkers if they can catch some true positives with them.

Yeah, and these are pretty much the users we're trying to protect from themselves :)

These are the alpha checkers that we are testing in Ericsson:
alpha.core.BoolAssignment
alpha.core.CastSize
alpha.core.Conversion
alpha.core.DynamicTypeChecker
alpha.core.SizeofPtr
alpha.core.TestAfterDivZero
alpha.cplusplus.DeleteWithNonVirtualDtor
alpha.cplusplus.MisusedMovedObject
alpha.cplusplus.UninitializedObject
alpha.security.MallocOverflow
alpha.security.MmapWriteExec
alpha.security.ReturnPtrRange
alpha.security.taint.TaintPropagation
alpha.unix.BlockInCriticalSection
alpha.unix.Chroot
alpha.unix.PthreadLock
alpha.unix.SimpleStream
alpha.unix.Stream
alpha.unix.cstring.NotNullTerminated
alpha.unix.cstring.OutOfBounds

These 2 have just been moved out of alpha lately:
alpha.cplusplus.MisusedMovedObject
alpha.cplusplus.UninitializedObject

According to our tests these checkers do not crash and do not give a large number of reports (<~50) even on large code base.
Probably these are the closest to come out from alpha.
So we can check for false positives in them one by one on large open source projects and move them out from alpha when we are confident enough in the quality.

[NoQ]

In D57858#1501065, @dkrupp wrote:

These are the alpha checkers that we are testing in Ericsson:

Hmm, if you don't mind i'll take this to cfe-dev, as it's an interesting topic :)

[NoQ]

http://lists.llvm.org/pipermail/cfe-dev/2019-May/062298.html

[Szelethus]

In D57858#1502052, @NoQ wrote:

In D57858#1501065, @dkrupp wrote:

These are the alpha checkers that we are testing in Ericsson:

Hmm, if you don't mind i'll take this to cfe-dev, as it's an interesting topic :)

Not at all, I started to feel like we're deviating from the patch discussion quite badly anyways :)

cfe/trunk/include/clang/Driver/CC1Options.td

def analyzer_config_help : Flag<["-"], "analyzer-config-help">,
   HelpText<"Display the list of -analyzer-config options">;
 
 def analyzer_list_enabled_checkers : Flag<["-"], "analyzer-list-enabled-checkers">,
   HelpText<"Display the list of enabled analyzer checkers">;
 
 def analyzer_config : Separate<["-"], "analyzer-config">,
   HelpText<"Choose analyzer options to enable">;
 
+def analyzer_checker_option_help : Flag<["-"], "analyzer-checker-option-help">,
+  HelpText<"Display the list of checker and package options">;
+
 def analyzer_config_compatibility_mode : Separate<["-"], "analyzer-config-compatibility-mode">,
   HelpText<"Don't emit errors on invalid analyzer-config inputs">;
 
 def analyzer_config_compatibility_mode_EQ : Joined<["-"], "analyzer-config-compatibility-mode=">,
   Alias<analyzer_config_compatibility_mode>;
 
 //===----------------------------------------------------------------------===//
 // Migrator Options

cfe/trunk/include/clang/StaticAnalyzer/Core/AnalyzerOptions.h

 /// make sense for things that do not affect the actual analysis.
 class AnalyzerOptions : public RefCountedBase<AnalyzerOptions> {
 public:
   using ConfigTable = llvm::StringMap<std::string>;
 
   static std::vector<StringRef>
   getRegisteredCheckers(bool IncludeExperimental = false);
 
+  /// Convenience function for printing options or checkers and their
+  /// description in a formatted manner. If \p MinLineWidth is set to 0, no line
+  /// breaks are introduced for the description.
+  ///
+  /// Format, depending whether the option name's length is less then
+  /// \p OptionWidth:
+  ///
+  ///   <padding>EntryName<padding>Description
+  ///   <---------padding--------->Description
+  ///   <---------padding--------->Description
+  ///
+  ///   <padding>VeryVeryLongOptionName
+  ///   <---------padding--------->Description
+  ///   <---------padding--------->Description
+  ///   ^~~~~~~~ InitialPad
+  ///   ^~~~~~~~~~~~~~~~~~~~~~~~~~ EntryWidth
+  ///   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~MinLineWidth
+  static void printFormattedEntry(
+      llvm::raw_ostream &Out,
+      std::pair<StringRef, StringRef> EntryDescPair,
+      size_t EntryWidth, size_t InitialPad, size_t MinLineWidth = 0);
+
+
   /// Pair of checker name and enable/disable.
   std::vector<std::pair<std::string, bool>> CheckersControlList;
 
   /// A key-value table of use-specified configuration values.
   // TODO: This shouldn't be public.
   ConfigTable Config;
   AnalysisStores AnalysisStoreOpt = RegionStoreModel;
   AnalysisConstraints AnalysisConstraintsOpt = RangeConstraintsModel;
public:
   /// This flag allows one to disable analyzer checks on the code processed by
   /// the given analysis consumer. Note, the code will get parsed and the
   /// command-line options will get checked.
   unsigned DisableAllChecks : 1;
 
   unsigned ShowCheckerHelp : 1;
   unsigned ShowCheckerHelpHidden : 1;
   unsigned ShowEnabledCheckerList : 1;
+  unsigned ShowCheckerOptionList : 1;
   unsigned ShowConfigOptionsList : 1;
   unsigned ShouldEmitErrorsOnInvalidConfigValue : 1;
   unsigned AnalyzeAll : 1;
   unsigned AnalyzerDisplayProgress : 1;
   unsigned AnalyzeNestedBlocks : 1;
 
   unsigned eagerlyAssumeBinOpBifurcation : 1;
 
bool isUnknownAnalyzerConfig(StringRef Name) const {
 
     return !std::binary_search(AnalyzerConfigCmdFlags.begin(),
                                AnalyzerConfigCmdFlags.end(), Name);
   }
 
   AnalyzerOptions()
       : DisableAllChecks(false), ShowCheckerHelp(false),
         ShowCheckerHelpHidden(false), ShowEnabledCheckerList(false),
-        ShowConfigOptionsList(false), AnalyzeAll(false),
-        AnalyzerDisplayProgress(false), AnalyzeNestedBlocks(false),
-        eagerlyAssumeBinOpBifurcation(false), TrimGraph(false),
-        visualizeExplodedGraphWithGraphViz(false), UnoptimizedCFG(false),
-        PrintStats(false), NoRetryExhausted(false) {
+        ShowCheckerOptionList(false), ShowConfigOptionsList(false),
+        AnalyzeAll(false), AnalyzerDisplayProgress(false),
+        AnalyzeNestedBlocks(false), eagerlyAssumeBinOpBifurcation(false),
+        TrimGraph(false), visualizeExplodedGraphWithGraphViz(false),
+        UnoptimizedCFG(false), PrintStats(false), NoRetryExhausted(false) {
     llvm::sort(AnalyzerConfigCmdFlags);
   }
 
   /// Interprets an option's string value as a boolean. The "true" string is
   /// interpreted as true and the "false" string is interpreted as false.
   ///
   /// If an option value is not provided, returns the given \p DefaultVal.
   /// @param [in] CheckerName The *full name* of the checker. One may retrieve

cfe/trunk/include/clang/StaticAnalyzer/Frontend/CheckerRegistry.h

public:
   /// Check if every option corresponds to a specific checker or package.
   void validateCheckerOptions() const;
 
   /// Prints the name and description of all checkers in this registry.
   /// This output is not intended to be machine-parseable.
   void printCheckerWithDescList(raw_ostream &Out,
                                 size_t MaxNameChars = 30) const;
   void printEnabledCheckerList(raw_ostream &Out) const;
+  void printCheckerOptionList(raw_ostream &Out) const;
 
 private:
   /// Collect all enabled checkers. The returned container preserves the order
   /// of insertion, as dependencies have to be enabled before the checkers that
   /// depend on them.
   CheckerInfoSet getEnabledCheckers() const;
 
   /// Return an iterator range of mutable CheckerInfos \p CmdLineArg applies to.

cfe/trunk/include/clang/StaticAnalyzer/Frontend/FrontendActions.h

void printCheckerHelp(raw_ostream &OS,
                       AnalyzerOptions &opts,
                       DiagnosticsEngine &diags,
                       const LangOptions &LangOpts);
 void printEnabledCheckerList(raw_ostream &OS, ArrayRef<std::string> plugins,
                              AnalyzerOptions &opts,
                              DiagnosticsEngine &diags,
                              const LangOptions &LangOpts);
 void printAnalyzerConfigList(raw_ostream &OS);
+void printCheckerConfigList(raw_ostream &OS, ArrayRef<std::string> plugins,
+                            AnalyzerOptions &opts,
+                            DiagnosticsEngine &diags,
+                            const LangOptions &LangOpts);
 
 } // end GR namespace
 
 } // end namespace clang
 
 #endif

cfe/trunk/lib/Frontend/CompilerInvocation.cpp

if (Value == NumInliningModes) {
       Success = false;
     } else {
       Opts.InliningMode = Value;
     }
   }
 
   Opts.ShowCheckerHelp = Args.hasArg(OPT_analyzer_checker_help);
   Opts.ShowCheckerHelpHidden = Args.hasArg(OPT_analyzer_checker_help_hidden);
+  Opts.ShowCheckerOptionList = Args.hasArg(OPT_analyzer_checker_option_help);
   Opts.ShowConfigOptionsList = Args.hasArg(OPT_analyzer_config_help);
   Opts.ShowEnabledCheckerList = Args.hasArg(OPT_analyzer_list_enabled_checkers);
   Opts.ShouldEmitErrorsOnInvalidConfigValue =
       /* negated */!llvm::StringSwitch<bool>(
                    Args.getLastArgValue(OPT_analyzer_config_compatibility_mode))
         .Case("true", true)
         .Case("false", false)
         .Default(false);

cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp

if (AnOpts.ShowCheckerHelp || AnOpts.ShowCheckerHelpHidden) {
     ento::printCheckerHelp(llvm::outs(),
                            Clang->getFrontendOpts().Plugins,
                            AnOpts,
                            Clang->getDiagnostics(),
                            Clang->getLangOpts());
     return true;
   }
 
+  // Honor -analyzer-checker-option-help.
+  if (Clang->getAnalyzerOpts()->ShowCheckerOptionList) {
+    ento::printCheckerConfigList(llvm::outs(),
+                                 Clang->getFrontendOpts().Plugins,
+                                 *Clang->getAnalyzerOpts(),
+                                 Clang->getDiagnostics(),
+                                 Clang->getLangOpts());
+    return true;
+  }
+
   // Honor -analyzer-list-enabled-checkers.
   if (AnOpts.ShowEnabledCheckerList) {
     ento::printEnabledCheckerList(llvm::outs(),
                                   Clang->getFrontendOpts().Plugins,
                                   AnOpts,
                                   Clang->getDiagnostics(),
                                   Clang->getLangOpts());
   }

cfe/trunk/lib/StaticAnalyzer/Core/AnalyzerOptions.cpp

 #include "clang/StaticAnalyzer/Core/AnalyzerOptions.h"
 #include "clang/StaticAnalyzer/Core/Checker.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cassert>
 #include <cstddef>
 #include <utility>
 #include <vector>
 
 using namespace clang;
 using namespace ento;
#undef GET_CHECKERS
   for (StringRef CheckName : StaticAnalyzerChecks) {
     if (!CheckName.startswith("debug.") &&
         (IncludeExperimental || !CheckName.startswith("alpha.")))
       Result.push_back(CheckName);
   }
   return Result;
 }
 
+void AnalyzerOptions::printFormattedEntry(
+    llvm::raw_ostream &Out,
+    std::pair<StringRef, StringRef> EntryDescPair,
+    size_t InitialPad, size_t EntryWidth, size_t MinLineWidth) {
+
+  llvm::formatted_raw_ostream FOut(Out);
+
+  const size_t PadForDesc = InitialPad + EntryWidth;
+
+  FOut.PadToColumn(InitialPad) << EntryDescPair.first;
+  // If the buffer's length is greater then PadForDesc, print a newline.
+  if (FOut.getColumn() > PadForDesc)
+    FOut << '\n';
+
+  FOut.PadToColumn(PadForDesc);
+
+  if (MinLineWidth == 0) {
+    FOut << EntryDescPair.second;
+    return;
+  }
+
+  for (char C : EntryDescPair.second) {
+    if (FOut.getColumn() > MinLineWidth && C == ' ') {
+      FOut << '\n';
+      FOut.PadToColumn(PadForDesc);
+      continue;
+    }
+    FOut << C;
+  }
+}
+
 ExplorationStrategyKind
 AnalyzerOptions::getExplorationStrategy() const {
   auto K =
     llvm::StringSwitch<llvm::Optional<ExplorationStrategyKind>>(
                                                             ExplorationStrategy)
           .Case("dfs", ExplorationStrategyKind::DFS)
           .Case("bfs", ExplorationStrategyKind::BFS)
           .Case("unexplored_first",

cfe/trunk/lib/StaticAnalyzer/Frontend/CheckerRegistration.cpp

 #include "clang/StaticAnalyzer/Frontend/CheckerRegistration.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Frontend/FrontendDiagnostic.h"
 #include "clang/StaticAnalyzer/Core/AnalyzerOptions.h"
 #include "clang/StaticAnalyzer/Core/CheckerManager.h"
 #include "clang/StaticAnalyzer/Frontend/CheckerRegistry.h"
 #include "clang/StaticAnalyzer/Frontend/FrontendActions.h"
 #include "llvm/ADT/SmallVector.h"
-#include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/raw_ostream.h"
 #include <memory>
 
 using namespace clang;
 using namespace ento;
 
 std::unique_ptr<CheckerManager> ento::createCheckerManager(
     ASTContext &context,
void ento::printEnabledCheckerList(raw_ostream &out,
                                    DiagnosticsEngine &diags,
                                    const LangOptions &langOpts) {
   out << "OVERVIEW: Clang Static Analyzer Enabled Checkers List\n\n";
 
   CheckerRegistry(plugins, diags, anopts, langOpts)
       .printEnabledCheckerList(out);
 }
 
+void ento::printCheckerConfigList(raw_ostream &OS,
+                                  ArrayRef<std::string> plugins,
+                                  AnalyzerOptions &opts,
+                                  DiagnosticsEngine &diags,
+                                  const LangOptions &LangOpts) {
+  CheckerRegistry(plugins, diags, opts, LangOpts)
+      .printCheckerOptionList(OS);
+}
+
 void ento::printAnalyzerConfigList(raw_ostream &out) {
   out << "OVERVIEW: Clang Static Analyzer -analyzer-config Option List\n\n";
-  out << "USAGE: clang -cc1 [CLANG_OPTIONS] -analyzer-config "
-                                        "<OPTION1=VALUE,OPTION2=VALUE,...>\n\n";
-  out << "       clang -cc1 [CLANG_OPTIONS] -analyzer-config OPTION1=VALUE, "
-                                      "-analyzer-config OPTION2=VALUE, ...\n\n";
-  out << "       clang [CLANG_OPTIONS] -Xclang -analyzer-config -Xclang"
-                                        "<OPTION1=VALUE,OPTION2=VALUE,...>\n\n";
-  out << "       clang [CLANG_OPTIONS] -Xclang -analyzer-config -Xclang "
-                              "OPTION1=VALUE, -Xclang -analyzer-config -Xclang "
-                              "OPTION2=VALUE, ...\n\n";
+  out << "USAGE: -analyzer-config <OPTION1=VALUE,OPTION2=VALUE,...>\n\n";
+  out << "       -analyzer-config OPTION1=VALUE, -analyzer-config "
   out << "OPTIONS:\n\n";
 
   using OptionAndDescriptionTy = std::pair<StringRef, std::string>;
   OptionAndDescriptionTy PrintableOptions[] = {
 #define ANALYZER_OPTION(TYPE, NAME, CMDFLAG, DESC, DEFAULT_VAL)                \
     {                                                                          \
       CMDFLAG,                                                                 \
       llvm::Twine(llvm::Twine() + "(" +                                        \

 #undef ANALYZER_OPTION_DEPENDS_ON_USER_MODE
   };
 
   llvm::sort(PrintableOptions, [](const OptionAndDescriptionTy &LHS,
                                   const OptionAndDescriptionTy &RHS) {
     return LHS.first < RHS.first;
   });
 
-  constexpr size_t MinLineWidth = 70;
-  constexpr size_t PadForOpt = 2;
-  constexpr size_t OptionWidth = 30;
-  constexpr size_t PadForDesc = PadForOpt + OptionWidth;
-  static_assert(MinLineWidth > PadForDesc, "MinLineWidth must be greater!");
-
-  llvm::formatted_raw_ostream FOut(out);
-
   for (const auto &Pair : PrintableOptions) {
-    FOut.PadToColumn(PadForOpt) << Pair.first;
-
-    // If the buffer's length is greater then PadForDesc, print a newline.
-    if (FOut.getColumn() > PadForDesc)
-      FOut << '\n';
-
-    FOut.PadToColumn(PadForDesc);
-
-    for (char C : Pair.second) {
-      if (FOut.getColumn() > MinLineWidth && C == ' ') {
-        FOut << '\n';
-        FOut.PadToColumn(PadForDesc);
-        continue;
-      }
-      FOut << C;
-    }
-    FOut << "\n\n";
+    AnalyzerOptions::printFormattedEntry(out, Pair, /*InitialPad*/ 2,
+                                         /*EntryWidth*/ 30,
+                                         /*MinLineWidth*/ 70);
+    out << "\n\n";
   }
 }

cfe/trunk/lib/StaticAnalyzer/Frontend/CheckerRegistry.cpp

if (NameLength <= MaxNameChars)
       OptionFieldWidth = std::max(OptionFieldWidth, NameLength);
   }
 
   const size_t InitialPad = 2;
   for (const auto &Checker : Checkers) {
     if (!AnOpts.ShowCheckerHelpHidden && Checker.IsHidden)
       continue;
 
-    Out.indent(InitialPad) << Checker.FullName;
-
-    int Pad = OptionFieldWidth - Checker.FullName.size();
-
-    // Break on long option names.
-    if (Pad < 0) {
-      Out << '\n';
-      Pad = OptionFieldWidth + InitialPad;
-    }
-    Out.indent(Pad + 2) << Checker.Desc;
-
+    AnalyzerOptions::printFormattedEntry(Out, {Checker.FullName, Checker.Desc},
+                                         InitialPad, OptionFieldWidth);
     Out << '\n';
   }
 }
 
 void CheckerRegistry::printEnabledCheckerList(raw_ostream &Out) const {
   // Collect checkers enabled by the options.
   CheckerInfoSet EnabledCheckers = getEnabledCheckers();
 
   for (const auto *i : EnabledCheckers)
     Out << i->FullName << '\n';
 }
+
+void CheckerRegistry::printCheckerOptionList(raw_ostream &Out) const {
+  Out << "OVERVIEW: Clang Static Analyzer Checker and Package Option List\n\n";
+  Out << "USAGE: -analyzer-config <OPTION1=VALUE,OPTION2=VALUE,...>\n\n";
+  Out << "       -analyzer-config OPTION1=VALUE, -analyzer-config "
+         "OPTION2=VALUE, ...\n\n";
+  Out << "OPTIONS:\n\n";
+
+  std::multimap<StringRef, const CmdLineOption &> OptionMap;
+
+  for (const CheckerInfo &Checker : Checkers) {
+    for (const CmdLineOption &Option : Checker.CmdLineOptions) {
+      OptionMap.insert({Checker.FullName, Option});
+    }
+  }
+
+  for (const PackageInfo &Package : Packages) {
+    for (const CmdLineOption &Option : Package.CmdLineOptions) {
+      OptionMap.insert({Package.FullName, Option});
+    }
+  }
+
+  for (const std::pair<StringRef, const CmdLineOption &> &Entry : OptionMap) {
+    const CmdLineOption &Option = Entry.second;
+    std::string FullOption = (Entry.first + ":" + Option.OptionName).str();
+
+    std::string Desc =
+        ("(" + Option.OptionType + ") " + Option.Description + " (default: " +
+         (Option.DefaultValStr.empty() ? "\"\"" : Option.DefaultValStr) + ")")
+            .str();
+
+    AnalyzerOptions::printFormattedEntry(Out, {FullOption, Desc},
+                                         /*InitialPad*/ 2,
+                                         /*EntryWidth*/ 50,
+                                         /*MinLineWidth*/ 90);
+    Out << "\n\n";
+  }
+}

cfe/trunk/test/Analysis/analyzer-checker-option-help.c

+// RUN: %clang_cc1 -analyzer-checker-option-help 2>&1 | FileCheck %s
+
+// CHECK: OVERVIEW: Clang Static Analyzer Checker and Package Option List
+//
+// CHECK: USAGE: -analyzer-config <OPTION1=VALUE,OPTION2=VALUE,...>
+//
+// CHECK:        -analyzer-config OPTION1=VALUE, -analyzer-config
+// CHECK-SAME:   OPTION2=VALUE, ...
+//
+// CHECK: OPTIONS:
+//
+// CHECK:   alpha.clone.CloneChecker:MinimumCloneComplexity
+// CHECK-SAME:   (int) Ensures that every clone has at least
+// CHECK:        the given complexity. Complexity is here
+// CHECK:        defined as the total amount of children
+// CHECK:        of a statement. This constraint assumes
+// CHECK:        the first statement in the group is representative
+// CHECK:        for all other statements in the group in
+// CHECK:        terms of complexity. (default: 50)

cfe/trunk/test/Analysis/analyzer-list-configs.c

 // RUN: %clang_cc1 -analyzer-config-help 2>&1 | FileCheck %s
+
 // CHECK: OVERVIEW: Clang Static Analyzer -analyzer-config Option List
 //
-// CHECK: USAGE: clang -cc1 [CLANG_OPTIONS] -analyzer-config <OPTION1=VALUE,OPTION2=VALUE,...>
-//
-// CHECK:      clang -cc1 [CLANG_OPTIONS] -analyzer-config OPTION1=VALUE, -analyzer-config OPTION2=VALUE, ...
-//
-// CHECK:      clang [CLANG_OPTIONS] -Xclang -analyzer-config -Xclang<OPTION1=VALUE,OPTION2=VALUE,...>
-//
-// CHECK:      clang [CLANG_OPTIONS] -Xclang -analyzer-config -Xclang OPTION1=VALUE, -Xclang -analyzer-config -Xclang OPTION2=VALUE, ...
+// CHECK: USAGE: -analyzer-config <OPTION1=VALUE,OPTION2=VALUE,...>
 //
+// CHECK:        -analyzer-config OPTION1=VALUE, -analyzer-config
+// CHECK-SAME:   OPTION2=VALUE, ...
 //
 // CHECK: OPTIONS:
 //
 // CHECK: aggressive-binary-operation-simplification
 // CHECK:                   (bool) Whether SValBuilder should rearrange
 // CHECK:                   comparisons and additive operations of symbolic
 // CHECK:                   expressions which consist of a sum of a
 // CHECK:                   symbol and a concrete integer into the format

cfe/trunk/test/Analysis/checker-plugins.c

 // RUN:   -load %llvmshlibdir/CheckerOptionHandlingAnalyzerPlugin%pluginext\
 // RUN:   -analyzer-checker=example.MyChecker \
 // RUN:   -analyzer-checker=debug.ConfigDumper \
 // RUN:   -analyzer-config-compatibility-mode=true \
 // RUN:   -analyzer-config example.MyChecker:ExampleOption=example \
 // RUN:   2>&1 | FileCheck %s -check-prefix=CHECK-CORRECTED-BOOL-VALUE
 
 // CHECK-CORRECTED-BOOL-VALUE: example.MyChecker:ExampleOption = false
+
+// RUN: %clang_analyze_cc1 %s \
+// RUN:   -load %llvmshlibdir/CheckerOptionHandlingAnalyzerPlugin%pluginext\
+// RUN:   -analyzer-checker=example.MyChecker \
+// RUN:   -analyzer-checker-option-help \
+// RUN:   2>&1 | FileCheck %s -check-prefix=CHECK-CHECKER-OPTION-HELP
+
+// CHECK-CHECKER-OPTION-HELP: example.MyChecker:ExampleOption  (bool) This is an
+// CHECK-CHECKER-OPTION-HELP-SAME: example checker opt. (default: false)