[analyzer] fix accessing GDM data from shared libraries

[jranieri-grammatech]

Summary

The GDMIndex functions return a pointer that's used as a key for looking up data, but addresses of local statics defined in header files aren't the same across shared library boundaries and the result is that analyzer plugins can't access this data.

[NoQ]

Hmmm, interesting. A checker doesn't usually need to access these specific static locals, at least not directly. These are usually accessed through functions in .cpp files that are supposed to be compiled with a pointer to the correct instance of the static local, and it doesn't seem to be necessary to expose them to plugins, simply because i don't immediately see why would a plugin want to use them. In this sense, i believe that the entire definition of these traits should be moved to .cpp files and be made private, accessed only via public methods of respective classes. But i guess it's more difficult and it's a separate chunk of work, so i totally approve this patch.

Also, i guess that's what they meant when they were saying that REGISTER_TRAIT_WITH_PROGRAMSTATE [and similar] macros shouldn't be used in headers (https://reviews.llvm.org/D51388?id=162968#inline-455495).

Did you use any sort of tool to find those? I.e., are there more such bugs, and can we prevent this from regressing and breaking your workflow later?

You didn't add reviewers. Does it mean that you are still planning to work on this patch further, or do you want this patch to be committed in its current shape? Static Analyzer patches are usually prefixed with [analyzer] (a few people auto-subscribe to those) and please feel free to add me and @george.karpenkov as reviewers, and the code owner is @dcoughlin.

[jranieri-grammatech]

In D52905#1257040, @NoQ wrote:

Hmmm, interesting. A checker doesn't usually need to access these specific static locals, at least not directly. These are usually accessed through functions in .cpp files that are supposed to be compiled with a pointer to the correct instance of the static local, and it doesn't seem to be necessary to expose them to plugins, simply because i don't immediately see why would a plugin want to use them. In this sense, i believe that the entire definition of these traits should be moved to .cpp files and be made private, accessed only via public methods of respective classes. But i guess it's more difficult and it's a separate chunk of work, so i totally approve this patch.

My current goal is to examine all of the information the analyzer has available at callsites and extract the portions that the project I'm working on might be interested in. I wouldn't disagree with your assessment in general, but it's definitely not something I have time allocated towards doing.

Also, i guess that's what they meant when they were saying that REGISTER_TRAIT_WITH_PROGRAMSTATE [and similar] macros shouldn't be used in headers (https://reviews.llvm.org/D51388?id=162968#inline-455495).

Yeah, I think so.

Did you use any sort of tool to find those? I.e., are there more such bugs, and can we prevent this from regressing and breaking your workflow later?

I wrote a very quick and dirty clang plugin that has this AST matcher:

returnStmt(hasReturnValue(ignoringImplicit(unaryOperator(
               hasOperatorName("&"),
               hasUnaryOperand(declRefExpr(
                   to(varDecl(isStaticLocal()))))))),
           isExpansionInFileMatching("/include/.*\\.h"))

where isStaticLocal is defined as:

AST_MATCHER(VarDecl, isStaticLocal) {
  return Node.isStaticLocal();
}

I've since adapted it to a clang-tidy checker under the llvm 'module', which I'll aim at getting approval to open source as well. Do you know if there's a way to run clang-tidy over all of clang+llvm automatically? My plugin approach had the advantage of just needing to fiddle with CMAKE_CXX_FLAGS to run against the whole codebase...

You didn't add reviewers. Does it mean that you are still planning to work on this patch further, or do you want this patch to be committed in its current shape? Static Analyzer patches are usually prefixed with [analyzer] (a few people auto-subscribe to those) and please feel free to add me and @george.karpenkov as reviewers, and the code owner is @dcoughlin.

This is just my inexperience with the Phabricator patch submission process showing through; I've traditionally emailed patches to the various -dev lists.

[Szelethus]

Please reupload with full context (-U99999).

[jranieri-grammatech]

Added more context.

[NoQ]

Cool, thanks! I'll commit.

[NoQ]

where isStaticLocal is defined as:

You can send this one as well if you like! It's weird that we don't already have it.

[NoQ]

In D52905#1259249, @NoQ wrote:

where isStaticLocal is defined as:

You can send this one as well if you like! It's weird that we don't already have it.

Or actually maybe it can be implemented as allOf(hasStaticStorageDuration(), unless(hasGlobalStorage())).

[george.karpenkov]

there's a way to run clang-tidy over all of clang+llvm automatically

cmake should generate compile_commands.json by default, and then you could just point clang-tidy at that.

include/clang/StaticAnalyzer/Core/PathSensitive/DynamicTypeMap.h

 template <>
 struct ProgramStateTrait<DynamicTypeMap>
     : public ProgramStatePartialTrait<DynamicTypeMapImpl> {
-  static void *GDMIndex() {
-    static int index = 0;
-    return &index;
-  }
+  static void *GDMIndex();
 };
 
 /// Get dynamic type information for a region.

include/clang/StaticAnalyzer/Core/PathSensitive/ExprEngine.h

 template <>
 struct ProgramStateTrait<ReplayWithoutInlining> :
   public ProgramStatePartialTrait<const void*> {
-  static void *GDMIndex() { static int index = 0; return &index; }
+  static void *GDMIndex();
 };
 
 } // namespace ento

include/clang/StaticAnalyzer/Core/PathSensitive/RangedConstraintManager.h

 template <>
 struct ProgramStateTrait<ConstraintRange>
   : public ProgramStatePartialTrait<ConstraintRangeTy> {
-  static void *GDMIndex() { static int Index; return &Index; }
+  static void *GDMIndex();
 };
 
 

include/clang/StaticAnalyzer/Core/PathSensitive/TaintManager.h

 
 template<> struct ProgramStateTrait<TaintMap>
     :  public ProgramStatePartialTrait<TaintMapImpl> {
-  static void *GDMIndex() {
-    static int index = 0;
-    return &index;
-  }
+  static void *GDMIndex();
 };
 
 /// The GDM component mapping derived symbols' parent symbols to their

 
 template<> struct ProgramStateTrait<DerivedSymTaint>
     :  public ProgramStatePartialTrait<DerivedSymTaintImpl> {
-  static void *GDMIndex() {
-    static int index;
-    return &index;
-  }
+  static void *GDMIndex();
 };
 
 class TaintManager {

lib/StaticAnalyzer/Core/CMakeLists.txt

   Store.cpp
   SubEngine.cpp
   SymbolManager.cpp
+  TaintManager.cpp
   WorkList.cpp
   Z3ConstraintManager.cpp
 

lib/StaticAnalyzer/Core/DynamicTypeMap.cpp

   }
 }
 
+void *ProgramStateTrait<DynamicTypeMap>::GDMIndex() {
+  static int index = 0;
+  return &index;
+}
+
 } // namespace ento
 } // namespace clang

lib/StaticAnalyzer/Core/ExprEngine.cpp

   llvm::errs() << "Warning: dumping graph requires assertions" << "\n";
   return "";
 }
+
+void *ProgramStateTrait<ReplayWithoutInlining>::GDMIndex() {
+  static int index = 0;
+  return &index;
+}

lib/StaticAnalyzer/Core/RangedConstraintManager.cpp

   }
 }
 
+void *ProgramStateTrait<ConstraintRange>::GDMIndex() {
+  static int Index;
+  return &Index;
+}
+
 } // end of namespace ento
 
 } // end of namespace clang

lib/StaticAnalyzer/Core/TaintManager.cpp

+//== TaintManager.cpp ------------------------------------------ -*- C++ -*--=//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/StaticAnalyzer/Core/PathSensitive/TaintManager.h"
+
+using namespace clang;
+using namespace ento;
+
+void *ProgramStateTrait<TaintMap>::GDMIndex() {
+  static int index = 0;
+  return &index;
+}
+
+void *ProgramStateTrait<DerivedSymTaint>::GDMIndex() {
+  static int index;
+  return &index;
+}