Page MenuHomePhabricator

[analyzer] Don't display implementation checkers under -analyzer-checker-help, but do under the new flag -analyzer-checker-help-hidden
ClosedPublic

Authored by Szelethus on Apr 19 2019, 3:36 PM.

Details

Summary

During my work on analyzer dependencies, I created a great amount of new checkers that emitted no diagnostics at all, and were purely modeling some function or another.

However, the user shouldn't really disable/enable these by hand, so I think it would be great to hide them by default, along with other, older modeling-only checkers and debug checkers. I intentionally chose not to hide alpha checkers, because they have a scary enough name, in my opinion, to cause no surprise when they emit false positives or cause crashes.

CodeChecker also works by individually enabling and disabling each and every checker, and the dependency patches would create a great amount of unnecessary work on that side. But this is yet another reason, not the main motivation behind this patch :)

The patch introduces the Hidden bit into the TableGen files (you may remember it before I removed it in D53995), and checkers that are either marked as hidden, or are in a package that is marked hidden won't be displayed under -analyzer-checker-help. -analyzer-checker-help-hidden, a new flag meant for developers only, displays the full list.

Some dumps I made while coding (not a part of this patch):

CHECKER HIDDEN: debug.AnalysisOrder
CHECKER HIDDEN: debug.Stats
CHECKER NOT HIDDEN: alpha.security.ArrayBound
CHECKER NOT HIDDEN: alpha.security.ArrayBoundV2
CHECKER NOT HIDDEN: osx.cocoa.AutoreleaseWrite
CHECKER NOT HIDDEN: alpha.unix.BlockInCriticalSection
CHECKER NOT HIDDEN: alpha.core.BoolAssignment
CHECKER HIDDEN: core.builtin.BuiltinFunctions
CHECKER NOT HIDDEN: osx.coreFoundation.CFError
CHECKER HIDDEN: debug.DumpCFG
CHECKER HIDDEN: debug.ViewCFG
CHECKER NOT HIDDEN: osx.coreFoundation.CFNumber
CHECKER NOT HIDDEN: osx.coreFoundation.CFRetainRelease
CHECKER NOT HIDDEN: alpha.unix.cstring.BufferOverlap
CHECKER HIDDEN: unix.cstring.CStringModeling
CHECKER NOT HIDDEN: alpha.unix.cstring.NotNullTerminated
CHECKER NOT HIDDEN: unix.cstring.NullArg
CHECKER NOT HIDDEN: alpha.unix.cstring.OutOfBounds
CHECKER NOT HIDDEN: unix.cstring.BadSizeArg
CHECKER HIDDEN: cplusplus.SelfAssignment
CHECKER NOT HIDDEN: core.CallAndMessage
CHECKER NOT HIDDEN: alpha.core.CallAndMessageUnInitRefArg
CHECKER HIDDEN: debug.DumpCalls
CHECKER HIDDEN: debug.DumpCallGraph
CHECKER HIDDEN: debug.ViewCallGraph
CHECKER NOT HIDDEN: alpha.core.CastSize
CHECKER NOT HIDDEN: alpha.core.CastToStruct
CHECKER NOT HIDDEN: alpha.unix.Chroot
CHECKER NOT HIDDEN: osx.cocoa.ClassRelease
CHECKER NOT HIDDEN: alpha.clone.CloneChecker
CHECKER HIDDEN: debug.ConfigDumper
CHECKER NOT HIDDEN: alpha.core.Conversion
CHECKER NOT HIDDEN: valist.CopyToSelf
CHECKER NOT HIDDEN: deadcode.DeadStores
CHECKER NOT HIDDEN: alpha.cplusplus.DeleteWithNonVirtualDtor
CHECKER NOT HIDDEN: security.insecureAPI.DeprecatedOrUnsafeBufferHandling
CHECKER NOT HIDDEN: core.NullDereference
CHECKER NOT HIDDEN: alpha.osx.cocoa.DirectIvarAssignment
CHECKER NOT HIDDEN: alpha.osx.cocoa.DirectIvarAssignmentForAnnotatedFunctions
CHECKER NOT HIDDEN: core.DivideZero
CHECKER HIDDEN: debug.DumpDominators
CHECKER HIDDEN: unix.DynamicMemoryModeling
CHECKER NOT HIDDEN: alpha.core.DynamicTypeChecker
CHECKER NOT HIDDEN: core.DynamicTypePropagation
CHECKER NOT HIDDEN: optin.osx.cocoa.localizability.EmptyLocalizationContextChecker
CHECKER NOT HIDDEN: alpha.cplusplus.EnumCastOutOfRange
CHECKER HIDDEN: debug.ViewExplodedGraph
CHECKER HIDDEN: debug.ExprInspection
CHECKER NOT HIDDEN: alpha.core.FixedAddr
CHECKER NOT HIDDEN: security.FloatLoopCounter
CHECKER NOT HIDDEN: optin.performance.GCDAntipattern
CHECKER HIDDEN: apiModeling.google.GTest
CHECKER NOT HIDDEN: alpha.security.taint.TaintPropagation
CHECKER NOT HIDDEN: alpha.core.IdenticalExpr
CHECKER NOT HIDDEN: cplusplus.InnerPointer
CHECKER NOT HIDDEN: alpha.osx.cocoa.InstanceVariableInvalidation
CHECKER NOT HIDDEN: alpha.cplusplus.InvalidatedIterator
CHECKER HIDDEN: alpha.cplusplus.IteratorModeling
CHECKER NOT HIDDEN: alpha.cplusplus.IteratorRange
CHECKER HIDDEN: alpha.osx.cocoa.IvarInvalidationModeling
CHECKER NOT HIDDEN: alpha.llvm.Conventions
CHECKER HIDDEN: debug.DumpLiveStmts
CHECKER HIDDEN: debug.DumpLiveVars
CHECKER NOT HIDDEN: osx.MIG
CHECKER NOT HIDDEN: optin.mpi.MPI-Checker
CHECKER NOT HIDDEN: osx.SecKeychainAPI
CHECKER NOT HIDDEN: osx.API
CHECKER NOT HIDDEN: unix.Malloc
CHECKER NOT HIDDEN: alpha.security.MallocOverflow
CHECKER NOT HIDDEN: unix.MallocSizeof
CHECKER NOT HIDDEN: unix.MismatchedDeallocator
CHECKER NOT HIDDEN: alpha.cplusplus.MismatchedIterator
CHECKER NOT HIDDEN: alpha.osx.cocoa.MissingInvalidationMethod
CHECKER NOT HIDDEN: alpha.security.MmapWriteExec
CHECKER NOT HIDDEN: cplusplus.Move
CHECKER NOT HIDDEN: osx.cocoa.NSAutoreleasePool
CHECKER NOT HIDDEN: osx.cocoa.NSError
CHECKER HIDDEN: osx.NSOrCFErrorDerefChecker
CHECKER NOT HIDDEN: cplusplus.NewDelete
CHECKER NOT HIDDEN: cplusplus.NewDeleteLeaks
CHECKER NOT HIDDEN: osx.cocoa.NilArg
CHECKER HIDDEN: core.builtin.NoReturnFunctions
CHECKER NOT HIDDEN: optin.osx.cocoa.localizability.NonLocalizedStringChecker
CHECKER NOT HIDDEN: core.NonNullParamChecker
CHECKER HIDDEN: core.NonnilStringConstants
CHECKER NOT HIDDEN: nullability.NullPassedToNonnull
CHECKER NOT HIDDEN: nullability.NullReturnedFromNonnull
CHECKER HIDDEN: nullability.NullabilityBase
CHECKER NOT HIDDEN: nullability.NullableDereferenced
CHECKER NOT HIDDEN: nullability.NullablePassedToNonnull
CHECKER NOT HIDDEN: nullability.NullableReturnedFromNonnull
CHECKER NOT HIDDEN: osx.NumberObjectConversion
CHECKER NOT HIDDEN: optin.osx.OSObjectCStyleCast
CHECKER NOT HIDDEN: osx.OSObjectRetainCount
CHECKER NOT HIDDEN: osx.cocoa.AtSync
CHECKER NOT HIDDEN: osx.coreFoundation.containers.PointerSizedValues
CHECKER NOT HIDDEN: osx.coreFoundation.containers.OutOfBounds
CHECKER NOT HIDDEN: osx.cocoa.Dealloc
CHECKER NOT HIDDEN: osx.cocoa.ObjCGenerics
CHECKER NOT HIDDEN: osx.cocoa.Loops
CHECKER NOT HIDDEN: osx.cocoa.IncompatibleMethodTypes
CHECKER NOT HIDDEN: osx.cocoa.NonNilReturnValue
CHECKER NOT HIDDEN: osx.ObjCProperty
CHECKER NOT HIDDEN: osx.cocoa.SelfInit
CHECKER NOT HIDDEN: osx.cocoa.MissingSuperCall
CHECKER NOT HIDDEN: osx.cocoa.SuperDealloc
CHECKER NOT HIDDEN: osx.cocoa.UnusedIvars
CHECKER NOT HIDDEN: optin.performance.Padding
CHECKER NOT HIDDEN: alpha.osx.cocoa.localizability.PluralMisuseChecker
CHECKER NOT HIDDEN: alpha.core.PointerArithm
CHECKER NOT HIDDEN: alpha.nondeterminism.PointerSorting
CHECKER NOT HIDDEN: alpha.core.PointerSub
CHECKER NOT HIDDEN: alpha.unix.PthreadLock
CHECKER HIDDEN: debug.ReportStmts
CHECKER HIDDEN: osx.cocoa.RetainCountBase
CHECKER NOT HIDDEN: osx.cocoa.RetainCount
CHECKER NOT HIDDEN: alpha.security.ReturnPtrRange
CHECKER NOT HIDDEN: core.uninitialized.UndefReturn
CHECKER NOT HIDDEN: osx.cocoa.RunLoopAutoreleaseLeak
CHECKER HIDDEN: security.insecureAPI.SecuritySyntaxChecker
CHECKER NOT HIDDEN: alpha.unix.SimpleStream
CHECKER NOT HIDDEN: alpha.core.SizeofPtr
CHECKER NOT HIDDEN: alpha.core.StackAddressAsyncEscape
CHECKER HIDDEN: core.StackAddrEscapeBase
CHECKER NOT HIDDEN: core.StackAddressEscape
CHECKER HIDDEN: apiModeling.StdCLibraryFunctions
CHECKER NOT HIDDEN: alpha.unix.Stream
CHECKER HIDDEN: debug.TaintTest
CHECKER NOT HIDDEN: alpha.core.TestAfterDivZero
CHECKER HIDDEN: debug.DumpTraversal
CHECKER HIDDEN: apiModeling.TrustNonnull
CHECKER NOT HIDDEN: security.insecureAPI.UncheckedReturn
CHECKER NOT HIDDEN: core.uninitialized.Branch
CHECKER NOT HIDDEN: core.uninitialized.CapturedBlockVariable
CHECKER NOT HIDDEN: core.UndefinedBinaryOperatorResult
CHECKER NOT HIDDEN: core.uninitialized.ArraySubscript
CHECKER NOT HIDDEN: core.uninitialized.Assign
CHECKER NOT HIDDEN: valist.Uninitialized
CHECKER NOT HIDDEN: alpha.cplusplus.UninitializedObject
CHECKER NOT HIDDEN: unix.API
CHECKER NOT HIDDEN: optin.portability.UnixAPI
CHECKER NOT HIDDEN: alpha.deadcode.UnreachableCode
CHECKER NOT HIDDEN: valist.Unterminated
CHECKER NOT HIDDEN: core.VLASize
CHECKER HIDDEN: valist.ValistBase
CHECKER NOT HIDDEN: osx.cocoa.VariadicMethodTypes
CHECKER NOT HIDDEN: unix.Vfork
CHECKER NOT HIDDEN: optin.cplusplus.VirtualCall
CHECKER NOT HIDDEN: security.insecureAPI.bcmp
CHECKER NOT HIDDEN: security.insecureAPI.bcopy
CHECKER NOT HIDDEN: security.insecureAPI.bzero
CHECKER NOT HIDDEN: security.insecureAPI.getpw
CHECKER NOT HIDDEN: security.insecureAPI.gets
CHECKER NOT HIDDEN: security.insecureAPI.mkstemp
CHECKER NOT HIDDEN: security.insecureAPI.mktemp
CHECKER NOT HIDDEN: security.insecureAPI.rand
CHECKER NOT HIDDEN: security.insecureAPI.strcpy
CHECKER NOT HIDDEN: security.insecureAPI.vfork

Diff Detail

Repository
rL LLVM

Event Timeline

Szelethus created this revision.Apr 19 2019, 3:36 PM
NoQ added a comment.Apr 22 2019, 5:17 PM

Yup, that sounds useful!

I intentionally chose not to hide alpha checkers, because they have a scary enough name, in my opinion, to cause no surprise when they emit false positives or cause crashes.

I've seen people yelling really loudly, like "look, your clang is crap, it crashes all over the place", after enabling all alpha checkers at once. It's fairly hard to explain to the wider audience that LLVM doesn't have feature branches, so we have to put all unfinished features into the released product with a kind notice of "please don't use them, they're not ready yet". Our development policy is fairly unusual in this regard. It's like selling a washing machine with 30 extra buttons and by pressing any of them you get a slight chance that it washes a bit better and also a slight chance that it explodes and covers your house with white soapy goop and pieces of underwear, and all it says about these buttons is that they're "alpha".

NoQ accepted this revision.Apr 22 2019, 5:17 PM
This revision is now accepted and ready to land.Apr 22 2019, 5:17 PM

I plan to do the same for non-checker and checker options too.

I'll leave this up for just a little while to give people time to object.

Szelethus updated this revision to Diff 197609.May 1 2019, 12:28 PM
Szelethus edited the summary of this revision. (Show Details)

Hide security.insecureAPI.SecuritySyntaxChecker by default.

This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptMay 1 2019, 12:54 PM

Great, at least the users will not enable the debug checkers by accident!
We will check on the CodeChecker side if any configuration needs to be updated.

LGTM

Great, at least the users will not enable the debug checkers by accident!
We will check on the CodeChecker side if any configuration needs to be updated.

LGTM

I committed after checking whether this works with CodeChecker. I listed the enabled-by-default list with CodeChecker checkers --only-enabled, and made sure that the list didn't change after this patch. I also made sure that CodeChecker checkers --only-enabled is consistent with clang [blahblah] -analyzer-list-enabled-checkers, meaning that what CodeChecker requested was actually enabled, and nothing else (except modeling checkers, of course).

I guess adding a strictly developer only option to list the hidden checkers as well might be desirable, if we need to disable certain modeling checkers to get rid of some crashes.