[Proposal] [Analyzer] Individual options for checkers

[a.sidorin]

Summary

This patch allows to set up options for checkers. These option may include verbosity, search strategy and others. As I understand, the only way to pass options to checker is to create new checker kinds (for example, MallocPessimistic, MallocOptimistic). This patch allows passing options to checker via '-analyzer-checker-option' argument.
Example: -analyzer-checker-option Malloc:Strategy:Optimistic
Checkers can retrieve options with AnalyzerOptions::getCheckerOption(...) method.
What is your opinion?

[jordan_rose]

Definitely needs a look from Ted or Anna as well. In theory the existing infrastructure can already be used for this, so adding a new flag seems questionable to me:

-analyzer-option unix.Malloc:Strategy=optimistic

Having a helper function that constructs the full name from the checker name and option name seems fine, but doesn't seem like more than a convenience. The real convenience would be adding a helper method on CheckerContext to access options (taking either the combined name or a name that gets built up from the checker and the option).

I'm more interested in coming up with a naming schema here, and specifically I think we should be using the full name of the checker (including the package part) in prefixing options, not just the short name. This also makes more sense if we eventually have options that apply to several checkers—the option prefix could be their common package.

[a.sidorin]

Use analyzer-config command line option instead of a separate option to retrieve checker options.

[krememek]

Hi Aleksei,

Can you explain at a high level what functionality change this patch is implementing? I'd like to understand the motivation a bit more.

Thanks,
Ted

[a.sidorin]

Hello Ted,

This patch (its current version) implements a helper function to retrieve individual options for checkers. Currently the only way to set options for checkers is a registration of a special checker. I tried to implement a more convenient way to do this. For example,

checker->setCheckEnabled(Expr::SET_VolatileRead, getBooleanOption("EmitOnVolatileRead", true, checker));

instead of registration of a new checker (Malloc and CString checkers have even macros to do this). It is only a small convenience and doesn't break any existing functionality.
This should also work for groups of checkers. For example, ... -analyzer-config unix=some_option may be specified to set behaviour for all checkers in unix group.

[krememek]

include/clang/StaticAnalyzer/Core/AnalyzerOptions.h

251

Please add a doxygen comment for this method.

It's not clear if 'CheckerName' needs to be completely "qualified" with the package name.

lib/StaticAnalyzer/Core/AnalyzerOptions.cpp

104–105

Is there a reason to use StringRef&? Why not just use StringRef? This makes it look like we need to modify the StringRef value itself. StringRef is a very lightweight value that we can pass around. There's no need to use '&' or 'const'.

[krememek]

Thanks Aleksei. That explanation helps quite a bit. I've responded in Phabricator with some comments.

[a.sidorin]

Add doxygen comments, replace pass method for StringRefs.

[zaks.anna]

Aleksei,

Thanks for working on this!

I've added a small comment above.

The main problem with this patch is that it is lacking test cases. Is it possible to use it for the cases in CString checker and/or MallocChecker you've mentioned above?

Anna.

include/clang/StaticAnalyzer/Core/AnalyzerOptions.h

274

Please, add a comment about CheckerBase parameter in all the changed public methods.

[a.sidorin]

This is how MallocChecker looks with options used.

[zaks.anna]

Hi Aleksei,

We have discussed the patch and summarized the comments (from Jordan, Ted, and me) on this thread: "[cfe-dev] Static Analyzer Checker Options Proposal". This patch is very close, but does not address some of the points raised in the proposal. Would you be interested in discussing the proposal further?

[a.sidorin]

Hello Anna,

I leaved a comment in that thread (I don't know why they aren't showing
in the main thread).
Please see
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-October/039572.html for
this comment and Gabor Kozar's reply.

14.11.2014 02:38, Anna Zaks пишет:

Hi Aleksei,

We have discussed the patch and summarized the comments (from Jordan, Ted, and me) on this thread: "[cfe-dev] Static Analyzer Checker Options Proposal". This patch is very close, but does not address some of the points raised in the proposal. Would you be interested in discussing the proposal further?

http://reviews.llvm.org/D3967

[zaks.anna]

Thanks Alexei! I've replied.

[xazax.hun]

Hi Aleksei!

What is the state of this patch? Do you still working on it? I am very interested in this feature.

One minor note though: there are some hard limit on some platforms on the length of the command line (as far as I know it's about 8k characters on windows. source: http://support.microsoft.com/kb/830473). Command line also contains include paths and several other valuable information. At Ericsson we can have command lines that are several thousand characters long when invoking the static analyzer. Adding configuration options to the command line might result in hitting the character limit. In the long run I think it would be better to support configuration files.

[a.sidorin]

Hello Gábor,

I was thinking that a discussion is dead. It seems like I have fixed all
the issues that were pointed with this review, but there were no
replies. So, I was thinking that nobody needs this functionality.
You can try this patch and leave your opinion about it if you have any
suggestions.
About long command lines... AFAIR command line is a common way to pass
parameters for all parts of Clang, including CSA. Anyway, there are only
few checkers that may support this option now so it cannot add a lot to
the command line currently.

Hi Aleksei!

What is the state of this patch? Do you still working on it? I am very interested in this feature.

One minor note though: there are some hard limit on some platforms on the length of the command line (as far as I know it's about 8k characters on windows. source: http://support.microsoft.com/kb/830473). Command line also contains include paths and several other valuable information. At Ericsson we can have command lines that are several thousand characters long when invoking the static analyzer. Adding configuration options to the command line might result in hitting the character limit. In the long run I think it would be better to support configuration files.

http://reviews.llvm.org/D3967

EMAIL PREFERENCES

http://reviews.llvm.org/settings/panel/emailpreferences/

[zaks.anna]

Aleksei,

There must have been some miscommunication. I've replied the following on the thread discussing the "Static Analyzer Checker Options Proposal". As far as I can tell, your implementation differs to what was proposed. I have not received any reply to that. We are interested in this functionality, but do not agree with some aspects of this patch.

"
On Nov 14, 2014, at 11:24 AM, Anna Zaks <[e-mail]> wrote:

Hi Aleksei,

I believe that with your implementation, it's assumed that the checker always wants to inherit the options of the parent package (it’s not opt in).

Also, since you don’t support query by checker/package name, there is no ability for the checker to query the package options.

Anna.
"

[xazax.hun]

Hi Aleksei,

According to Anna's comment, there are only two things left:

• Adding opt-in for inheritance and turning it off by default.
• Make it possible to query options for the package. There is one nontrivial design decision though. Should it be possible to query the options of another package (e.g.: should the Malloc checker be able to query the options of the osx package)?

Are you going to do these changes? If you need any help feel free to ping me. If you do not want to finish this patch I am volunteering to do it.

lib/StaticAnalyzer/Core/AnalyzerOptions.cpp

114

nit: Local variables should start with upper case letters.

[a.sidorin]

Abandon this change since the patch based on this is committed in http://reviews.llvm.org/D7905

include/clang/StaticAnalyzer/Core/AnalyzerOptions.h

 class DiagnosticsEngine;
 class Preprocessor;
 class LangOptions;
+namespace ento {
+class CheckerBase;
+}
 
 /// Analysis - Set of available source code analyses.
 enum Analyses {

   AnalysisPurgeMode AnalysisPurgeOpt;
   
   std::string AnalyzeSpecificFunction;
-  
   /// \brief The maximum number of times the analyzer visits a block.
   unsigned maxBlockVisitOnPath;
   

   /// \sa getMaxNodesPerTopLevelFunction
   Optional<unsigned> MaxNodesPerTopLevelFunction;
 
+  /// A helper function that retrieves option for a given full-qualified
+  /// checker name.
+  /// Options for checkers can be specified via 'analyzer-config' command-line
+  /// option.
+  /// Example:
+  /// @code-analyzer-config unix.Malloc:OptionName=CheckerOptionValue @endcode
+  /// or @code-analyzer-config unix:OptionName=GroupOptionValue @endcode
+  /// for groups of checkers
+  /// @param [in] CheckerName  Full-qualified checker name, like
+  /// alpha.unix.StreamChecker
+  /// @param [in] OptionName  Name of the option to get
+  /// @param [in] Default  Default value if no option is specified
+  /// @retval CheckerOptionValue  An option for a checker if it was specified
+  /// @retval GroupOptionValue  An option for group if it was specified and no
+  /// checker-specific options were found. The closer group to checker,
+  /// the more priority it has. For example, @c coregroup.subgroup has more
+  /// priority than @c coregroup for @c coregroup.subgroup.CheckerName checker
+  /// @retval Default  If nor checker option, nor group option was found
+  StringRef getCheckerOption(StringRef CheckerName, StringRef OptionName,
+                             StringRef Default) const;
+
 public:
   /// Interprets an option's string value as a boolean.
   ///
   /// Accepts the strings "true" and "false".
   /// If an option value is not provided, returns the given \p DefaultVal.
-  bool getBooleanOption(StringRef Name, bool DefaultVal);
+  /// @param [in] Name Name for option to retrieve
+  /// @param [in] DefaultVal Default value returned if no such option was
+  /// specified
+  /// @param [in] C Optional parameter that may be used to retrieve
+  /// checker-related option for a given checker
+  bool getBooleanOption(StringRef Name, bool DefaultVal,
+                        const ento::CheckerBase *C = nullptr);
 
   /// Variant that accepts a Optional value to cache the result.
-  bool getBooleanOption(Optional<bool> &V, StringRef Name, bool DefaultVal);
+  ///
+  /// @param [in,out] V Return value storage, returned if parameter contains
+  /// an existing valid option, else it is used to store a return value
+  /// @param [in] Name Name for option to retrieve
+  /// @param [in] DefaultVal Default value returned if no such option was
+  /// specified
+  /// @param [in] C Optional parameter that may be used to retrieve
+  /// checker-related option for a given checker
+  bool getBooleanOption(Optional<bool> &V, StringRef Name, bool DefaultVal,
+                        const ento::CheckerBase *C = nullptr);
 
   /// Interprets an option's string value as an integer value.
-  int getOptionAsInteger(StringRef Name, int DefaultVal);
+  ///
+  /// If an option value is not provided, returns the given \p DefaultVal.
+  /// @param [in] Name Name for option to retrieve
+  /// @param [in] DefaultVal Default value returned if no such option was
+  /// specified
+  /// @param [in] C Optional parameter that may be used to retrieve
+  /// checker-related option for a given checker
+  int getOptionAsInteger(StringRef Name, int DefaultVal,
+                         const ento::CheckerBase *C = nullptr);
 
   /// \brief Retrieves and sets the UserMode. This is a high-level option,
   /// which is used to set other low-level options. It is not accessible

lib/StaticAnalyzer/Checkers/Checkers.td

   HelpText<"Check calls to various UNIX/Posix functions">,
   DescFile<"UnixAPIChecker.cpp">;
 
-def MallocPessimistic : Checker<"Malloc">,
+def MallocChecker: Checker<"Malloc">,
   HelpText<"Check for memory leaks, double free, and use-after-free problems. Traces memory managed by malloc()/free().">,
   DescFile<"MallocChecker.cpp">;
   

   HelpText<"Check improper use of chroot">,
   DescFile<"ChrootChecker.cpp">;
 
-def MallocOptimistic : Checker<"MallocWithAnnotations">,
-  HelpText<"Check for memory leaks, double free, and use-after-free problems. Traces memory managed by malloc()/free(). Assumes that all user-defined functions which might free a pointer are annotated.">,
-  DescFile<"MallocChecker.cpp">;
-
 def PthreadLockChecker : Checker<"PthreadLock">,
   HelpText<"Simple lock -> unlock checker">,
   DescFile<"PthreadLockChecker.cpp">;

lib/StaticAnalyzer/Checkers/MallocChecker.cpp

   /// In pessimistic mode, the checker assumes that it does not know which
   /// functions might free the memory.
   enum CheckKind {
-    CK_MallocPessimistic,
-    CK_MallocOptimistic,
+    CK_MallocChecker,
     CK_NewDeleteChecker,
     CK_NewDeleteLeaksChecker,
     CK_MismatchedDeallocatorChecker,
     CK_NumCheckKinds
   };
 
+  DefaultBool IsOptimistic;
+
   DefaultBool ChecksEnabled[CK_NumCheckKinds];
   CheckName CheckNames[CK_NumCheckKinds];
 

       return true;
   }
 
-  if (ChecksEnabled[CK_MallocOptimistic] && FD->hasAttrs())
+  if (IsOptimistic && FD->hasAttrs())
     for (const auto *I : FD->specific_attrs<OwnershipAttr>())
       if (I->getOwnKind() == OwnershipAttr::Returns)
         return true;

       return true;
   }
 
-  if (ChecksEnabled[CK_MallocOptimistic] && FD->hasAttrs())
+  if (IsOptimistic && FD->hasAttrs())
     for (const auto *I : FD->specific_attrs<OwnershipAttr>())
       if (I->getOwnKind() == OwnershipAttr::Takes ||
           I->getOwnKind() == OwnershipAttr::Holds)

     }
   }
 
-  if (ChecksEnabled[CK_MallocOptimistic] ||
-      ChecksEnabled[CK_MismatchedDeallocatorChecker]) {
+  if (IsOptimistic || ChecksEnabled[CK_MismatchedDeallocatorChecker]) {
     // Check all the attributes, if there are any.
     // There can be multiple of these attributes.
     if (FD->hasAttrs())

 MallocChecker::getCheckIfTracked(AllocationFamily Family) const {
   switch (Family) {
   case AF_Malloc: {
-    if (ChecksEnabled[CK_MallocOptimistic]) {
-      return CK_MallocOptimistic;
-    } else if (ChecksEnabled[CK_MallocPessimistic]) {
-      return CK_MallocPessimistic;
-    }
+    if (ChecksEnabled[CK_MallocChecker])
+      return CK_MallocChecker;
     return Optional<MallocChecker::CheckKind>();
   }
   case AF_CXXNew:

                                   SourceRange Range, 
                                   const Expr *DeallocExpr) const {
 
-  if (!ChecksEnabled[CK_MallocOptimistic] &&
-      !ChecksEnabled[CK_MallocPessimistic] &&
-      !ChecksEnabled[CK_NewDeleteChecker])
+  if (!ChecksEnabled[CK_MallocChecker] && !ChecksEnabled[CK_NewDeleteChecker])
     return;
 
   Optional<MallocChecker::CheckKind> CheckKind =

                                      SourceRange Range, const Expr *DeallocExpr,
                                      const Expr *AllocExpr) const {
 
-  if (!ChecksEnabled[CK_MallocOptimistic] &&
-      !ChecksEnabled[CK_MallocPessimistic] &&
-      !ChecksEnabled[CK_NewDeleteChecker])
+  if (!ChecksEnabled[CK_MallocChecker] && !ChecksEnabled[CK_NewDeleteChecker])
     return;
 
   Optional<MallocChecker::CheckKind> CheckKind =

 void MallocChecker::ReportUseAfterFree(CheckerContext &C, SourceRange Range,
                                        SymbolRef Sym) const {
 
-  if (!ChecksEnabled[CK_MallocOptimistic] &&
-      !ChecksEnabled[CK_MallocPessimistic] &&
-      !ChecksEnabled[CK_NewDeleteChecker])
+  if (!ChecksEnabled[CK_MallocChecker] && !ChecksEnabled[CK_NewDeleteChecker])
     return;
 
   Optional<MallocChecker::CheckKind> CheckKind = getCheckIfTracked(C, Sym);

                                      bool Released, SymbolRef Sym, 
                                      SymbolRef PrevSym) const {
 
-  if (!ChecksEnabled[CK_MallocOptimistic] &&
-      !ChecksEnabled[CK_MallocPessimistic] &&
-      !ChecksEnabled[CK_NewDeleteChecker])
+  if (!ChecksEnabled[CK_MallocChecker] && !ChecksEnabled[CK_NewDeleteChecker])
     return;
 
   Optional<MallocChecker::CheckKind> CheckKind = getCheckIfTracked(C, Sym);

 void MallocChecker::reportLeak(SymbolRef Sym, ExplodedNode *N,
                                CheckerContext &C) const {
 
-  if (!ChecksEnabled[CK_MallocOptimistic] &&
-      !ChecksEnabled[CK_MallocPessimistic] &&
+  if (!ChecksEnabled[CK_MallocChecker] &&
       !ChecksEnabled[CK_NewDeleteLeaksChecker])
     return;
 

     if (!FD)
       return;
 
-    if ((ChecksEnabled[CK_MallocOptimistic] ||
-         ChecksEnabled[CK_MallocPessimistic]) &&
+    if (ChecksEnabled[CK_MallocChecker] &&
         isFreeFunction(FD, C.getASTContext()))
       return;
 

 void ento::registerNewDeleteLeaksChecker(CheckerManager &mgr) {
   registerCStringCheckerBasic(mgr);
   MallocChecker *checker = mgr.registerChecker<MallocChecker>();
+  checker->IsOptimistic = mgr.getAnalyzerOptions().getBooleanOption(
+        "Optimistic", false, checker);
   checker->ChecksEnabled[MallocChecker::CK_NewDeleteLeaksChecker] = true;
   checker->CheckNames[MallocChecker::CK_NewDeleteLeaksChecker] =
       mgr.getCurrentCheckName();

   void ento::register##name(CheckerManager &mgr) {                             \
     registerCStringCheckerBasic(mgr);                                          \
     MallocChecker *checker = mgr.registerChecker<MallocChecker>();             \
+    checker->IsOptimistic = mgr.getAnalyzerOptions().getBooleanOption(         \
+          "Optimistic", false, checker);                                       \
     checker->ChecksEnabled[MallocChecker::CK_##name] = true;                   \
     checker->CheckNames[MallocChecker::CK_##name] = mgr.getCurrentCheckName(); \
   }
 
-REGISTER_CHECKER(MallocPessimistic)
-REGISTER_CHECKER(MallocOptimistic)
+REGISTER_CHECKER(MallocChecker)
 REGISTER_CHECKER(NewDeleteChecker)
 REGISTER_CHECKER(MismatchedDeallocatorChecker)

lib/StaticAnalyzer/Core/AnalyzerOptions.cpp

 //===----------------------------------------------------------------------===//
 
 #include "clang/StaticAnalyzer/Core/AnalyzerOptions.h"
+#include "clang/StaticAnalyzer/Core/Checker.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace clang;
+using namespace ento;
 using namespace llvm;
 
 AnalyzerOptions::UserModeKind AnalyzerOptions::getUserMode() {

 
 static StringRef toString(bool b) { return b ? "true" : "false"; }
 
-bool AnalyzerOptions::getBooleanOption(StringRef Name, bool DefaultVal) {
+StringRef AnalyzerOptions::getCheckerOption(StringRef CheckerName,
+                                            StringRef OptionName,
+                                            StringRef Default) const {
+  std::string Name = CheckerName;
+  std::string Suffix = ":" + OptionName.str();
+  // Search for a category option if option for checker is not specified
+  ConfigTable::const_iterator e = Config.end();
+  while (!Name.empty()) {
+    ConfigTable::const_iterator i = Config.find(Name + Suffix);
+    if (i != e)
+      return StringRef(i->getValue());
+    size_t pos = Name.rfind('.');
+    if (pos == std::string::npos)
+      return Default;
+    Name = Name.substr(0, pos);
+  }
+  return Default;
+}
+
+bool AnalyzerOptions::getBooleanOption(StringRef Name, bool DefaultVal,
+                                       const CheckerBase *C) {
   // FIXME: We should emit a warning here if the value is something other than
   // "true", "false", or the empty string (meaning the default value),
   // but the AnalyzerOptions doesn't have access to a diagnostic engine.
-  StringRef V(Config.GetOrCreateValue(Name, toString(DefaultVal)).getValue());
+  StringRef Default = toString(DefaultVal);
+  StringRef V = C ? getCheckerOption(C->getTagDescription(), Name, Default)
+                  : StringRef(Config.GetOrCreateValue(Name,Default).getValue());
   return llvm::StringSwitch<bool>(V)
       .Case("true", true)
       .Case("false", false)

 }
 
 bool AnalyzerOptions::getBooleanOption(Optional<bool> &V, StringRef Name,
-                                       bool DefaultVal) {
+                                       bool DefaultVal, const CheckerBase *C) {
   if (!V.hasValue())
-    V = getBooleanOption(Name, DefaultVal);
+    V = getBooleanOption(Name, DefaultVal, C);
   return V.getValue();
 }
 

                           /* Default = */ false);
 }
 
-int AnalyzerOptions::getOptionAsInteger(StringRef Name, int DefaultVal) {
+int AnalyzerOptions::getOptionAsInteger(StringRef Name, int DefaultVal,
+                                        const CheckerBase *C) {
   SmallString<10> StrBuf;
   llvm::raw_svector_ostream OS(StrBuf);
   OS << DefaultVal;
   
-  StringRef V(Config.GetOrCreateValue(Name, OS.str()).getValue());
+  StringRef V = C ? getCheckerOption(C->getTagDescription(), Name, OS.str())
+                  : StringRef(Config.GetOrCreateValue(Name,OS.str()).getValue());
   int Res = DefaultVal;
   bool b = V.getAsInteger(10, Res);
   assert(!b && "analyzer-config option should be numeric");

test/Analysis/free.c

 // RUN: %clang_cc1 -analyze -analyzer-store=region -analyzer-checker=core,unix.Malloc -fblocks -verify %s
-// RUN: %clang_cc1 -analyze -analyzer-store=region -analyzer-checker=core,alpha.unix.MallocWithAnnotations -fblocks -verify %s
+// RUN: %clang_cc1 -analyze -analyzer-store=region -analyzer-checker=core,unix.Malloc -fblocks -verify -analyzer-config unix.Malloc:Optimistic=true %s
 void free(void *);
 
 void t1 () {

test/Analysis/malloc-annotations.c

-// RUN: %clang_cc1 -analyze -analyzer-checker=core,alpha.deadcode.UnreachableCode,alpha.core.CastSize,alpha.unix.MallocWithAnnotations -analyzer-store=region -verify %s
+// RUN: %clang_cc1 -analyze -analyzer-checker=core,alpha.deadcode.UnreachableCode,alpha.core.CastSize,unix.Malloc -analyzer-store=region -verify -analyzer-config unix:Optimistic=true %s
+// RUN: %clang_cc1 -analyze -analyzer-checker=core,alpha.deadcode.UnreachableCode,alpha.core.CastSize,unix.Malloc -analyzer-store=region -verify -analyzer-config unix.Malloc:Optimistic=true %s
 typedef __typeof(sizeof(int)) size_t;
 void *malloc(size_t);
 void free(void *);

test/Analysis/outofbound.c

-// RUN: %clang_cc1 -Wno-array-bounds -analyze -analyzer-checker=core,alpha.unix,alpha.security.ArrayBound -analyzer-store=region -verify %s
+// RUN: %clang_cc1 -Wno-array-bounds -analyze -analyzer-checker=core,unix,alpha.security.ArrayBound -analyzer-store=region -verify -analyzer-config unix:Optimistic=true %s
 
 typedef __typeof(sizeof(int)) size_t;
 void *malloc(size_t);

test/Analysis/undef-buffers.c

-// RUN: %clang_cc1 -analyze -analyzer-checker=core,alpha.unix,core.uninitialized -analyzer-store=region -verify %s
+// RUN: %clang_cc1 -analyze -analyzer-checker=core,unix,core.uninitialized -analyzer-store=region -verify -analyzer-config unix:Optimistic=true %s
 typedef __typeof(sizeof(int)) size_t;
 void *malloc(size_t);
 void free(void *);