[analyzer] Model some library functions

[NoQ]

Summary

I've put together a simple checker that throws no warnings, but models some library functions, which has already helped us to suppress some false positives in other checkers in our runs.

For pure functions, i chose the old evalCall() approach instead of the body-farm approach because i wanted to produce less state splits. For example, this checker produce a single exploded graph branch for ispunct()'s non-zero branch, when its argument is in range:

['!', '/'] U [':', '@'] U ['[', '\`'] U ['{', '~']

I'm not sure if there's a way to write this out with if's and produce less than 4 branches. (Do we have any plans on merging branches more aggressively during analysis?) Because these functions are pure, we'd hardly ever want to catch them with`evalCall()` again in another checker.

Additionally, this checker's brace-initializers for function specifications are quite short - of course they're limited to very simple cases - the list of these cases can be expanded though.

The checker doesn't seem to be noticeably degrading performance. Here's an example of a false positve squashed:

report-fdc422.html135 KBDownload

Here line is taken to be "", the line++ statement is executed at least once (by looking at the exploded graph; there's lack of "entering loop body" diagnostic piece because loop condition has complicated CFG, which is why it fails to highlight - a separate issue), and the analyzer fails to realize that isspace('\0') is false.

[xazax.hun]

It is great to model more widely used functions! However I think the LibraryFunctionsChecker name might be a bit to broad, wouldn't something like StdCLibraryFunctions be more informative?

[NoQ]

Yeah, good point, the "Std" part should definitely appear in the name, not sure about the "C" thing, as we could probably expand this checker to model some simple C++ functions as well (and then we'd make a separate checker section to move from unix. to cplusplus. or something like that, not sure maybe we'd need to reside in core. anyway).

[zaks.anna]

Thanks for the patch! Here are the comments, most of which are nits.

Could you add the high level description of what we are doing somewhere or maybe just describe the meaning of FunctionSpec since it encodes how functions are modeled.

Also, we should explain why we are not using BodyFarm somewhere in the comment.

lib/StaticAnalyzer/Checkers/LibraryFunctionsChecker.cpp

10

Please, list the functions.

27

Naming looks odd: maybe "OutOfRange" and "WithinRange"?

33

nit: "is Unsigned" -> "as Unsigned"
Please, use a typedef for the type as you are using it below in getArgType.

83

Are the types in FunctionSpec already canonical? If so, please, add a comment.

178

nit: If you could factor these out into separate helper functions, it would be easier to read. Lot's of nesting..

219

What happens when NewState == State? I guess addTransition would just not do anything, but maybe we should just make the intent explicit and not call it at all.

245

Replace "Normal" with a more descriptive name.

273

Looks like you might want to have the checking code as a member on FunctionSpec.

322

Let's explain what we are doing next. For example, "Let's initialize the FunctionSpec for the functions we are modeling."

Remove "NOTE:"

326

not clear where this is used or if it is used in the initialization at all.

337

Is every item in the range set used to bifurcate the state?

[NoQ]

Renamed the checker as xazax.hun suggested, added a lot more comments, done with inline comments :)

[zaks.anna]

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

191 ↗

(On Diff #65248)

Do we need to talk about crashes when describing what this does?
Also, please, use oxygen throughout.

205 ↗

(On Diff #65248)

We could either provide these APIs in CallEvent or at least have variants that return canonical types. Maybe we already do some of that?

445 ↗

(On Diff #65248)

When can this go wrong? Are we checking if there is a mismatch between the function declaration and the call expression? It is strange that findFunctionSpec takes both of those. Couldn't you always get FunctionDecl out of CallExpr?

508 ↗

(On Diff #65248)

you could also use /*NameOfTheField*/ convention to name the arguments if that would make this map more readable.

test/Analysis/std-library-functions.c

3 ↗

(On Diff #65248)

Why are you not testing all of the functions?

[NoQ]

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

191 ↗

(On Diff #65248)

Added more comments below.

205 ↗

(On Diff #65248)

Maybe a separate commit? There are quite a few checkers from which the .getArgExpr(N)->getType() pattern could be de-duplicated, but i don't think many of them are interested in canonical types.

445 ↗

(On Diff #65248)

Callee decl is path-sensitive information because functions can be passed around by pointers, as mentioned in the comment at the beginning of the function. Expanded the comment, added a test.

508 ↗

(On Diff #65248)

I think compactness is worth it here, and specs are pretty easy to remember, imho. Added an example to the first spec to see how it looks and make it easier for the reader to adapt and remember, but i'm not quite convinced that verbosity is worth it here.

test/Analysis/std-library-functions.c

3 ↗

(On Diff #65248)

I was too bored to generate tests for all branches of all functions (and if i auto-generate such tests, it defeats the purpose), but i added some of the more creative tests and covered at least some branches of all functions with them.

[NoQ]

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

192 ↗

(On Diff #65369)

Even though there are some doxygen-style comments in the checkers, i’ve never seen doxygen actually generate any docs for checker classes. Are they useful for IDE quick-hints only?

[NoQ]

Answering myself: Hmm, so the only reason why MPI checker class appears in doxygen (http://clang.llvm.org/doxygen/classclang_1_1ento_1_1mpi_1_1MPIChecker.html) is because this class is not in anonymous namespace (as far as i understand, they needed to be multi-file for some reason). CheckerDocumentation says that every checker must be wrapped in anonymous namespace, except CheckerDocumentationChecker :)

I don’t really see a good reason for the library functions checker to be moved out of anonymous namespace or deserve a doxygen page - after all, it’s all in one file, and the docs are right in front of the reader’s eyes anyway. But maybe if this checker expands enough, we could expose its data structures into public use, and then they'd be worth documenting :)

[dcoughlin]

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

509 ↗

(On Diff #65369)

I disagree about compactness being valuable here. I think it is more important to intrinsically document the spec. These will be written once and read frequently. When they are written, they will copied from a previous example -- probably by someone who is not familiar with the code or the spec format.

Another possibility (not sure if it is the right one here) is to use macro tricks to define a simple DSL like Kulpreet did in the LocalizationChecker.cpp.

[NoQ]

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

509 ↗

(On Diff #65369)

These will be written once and read frequently.

If only it was so :))

Hmm. What do you think of the following format? Macros mostly expand to empty or (argument), but it should be more readable than the /*...*/ noise.

SPEC {
  FOR_FUNCTION("isalnum"),
  SPEC_DATA {
    ARGUMENT_TYPES { IntTy },
    RETURN_TYPE(IntTy),
    INVALIDATION_APPROACH(EvalCallAsPure),
    BRANCHES {
      BRANCH { // Boils down to isupper() or islower() or isdigit()
        RANGE {
          ARG_NO(0), RANGE_KIND(WithinRange),
          SET { SEG('0', '9') U SEG('A', 'Z') U SEG('a', 'z') }
        },
        RANGE {
          RET_VAL, RANGE_KIND(OutOfRange),
          SET { SEG(0, 0) }
        }
      },
      BRANCH { // The locale-specific branch.
        RANGE {
          ARG_NO(0), RANGE_KIND(WithinRange),
          SET { SEG(128, 255) }
        }
      },
      BRANCH { // Other.
        RANGE {
          ARG_NO(0), RANGE_KIND(OutOfRange),
          SET { SEG('0', '9') U SEG('A', 'Z')
                              U SEG('a', 'z') U SEG(128, 255)}
        },
        RANGE {
          RET_VAL, RANGE_KIND(WithinRange),
          SET { SEG(0, 0) }
        }
      }
    }
  }
},

[zaks.anna]

Even though there are some doxygen-style comments in the checkers, i’ve never seen doxygen actually generate any docs for checker classes.
Are they useful for IDE quick-hints only?

I think it's useful to have consistent documentation format.

In D20811#497585, @NoQ wrote:

Answering myself: Hmm, so the only reason why MPI checker class appears in doxygen (http://clang.llvm.org/doxygen/classclang_1_1ento_1_1mpi_1_1MPIChecker.html) is because this class is not in anonymous namespace (as far as i understand, they needed to be multi-file for some reason). CheckerDocumentation says that every checker must be wrapped in anonymous namespace, except CheckerDocumentationChecker :)

I don’t really see a good reason for the library functions checker to be moved out of anonymous namespace or deserve a doxygen page - after all, it’s all in one file, and the docs are right in front of the reader’s eyes anyway. But maybe if this checker expands enough, we could expose its data structures into public use, and then they'd be worth documenting :)

It has been originally written as a large set of files. If you feel strongly about it, we could merge it into a single file. That makes sense to me. @Alexander_Droste, what do you think?

[zaks.anna]

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

206 ↗

(On Diff #65369)

Separate commit is fine. I'd provide both APIs in CallEvent.

test/Analysis/std-library-functions.c

4 ↗

(On Diff #65369)

Ok.

[Alexander_Droste]

It has been originally written as a large set of files. If you feel strongly about it, we could merge it into a single file. That makes sense to me. @Alexander_Droste, what do you think?

Hi,
I would still strongly prefer to keep them in separate files if possible. One of the headers (MPIFunctionClassifier.hpp)
also got moved to include/clang/StaticAnalyzer/Checkers, as it is needed by some MPI clang-tidy checks.
Is it really a problem if the checker comments are part of the Doxygen documentation? Further, I think that
the separation of concerns in form of distinct files might be valuable for people being new to the Clang Static Analyzer
framework, as the grouping of functionality is visible on a higher level of abstraction. Regardless, I would of course
accept if you prefer to merge the files into a single one, excluding the MPIFunctionClassifier.hpp header.

[NoQ]

Is it really a problem if the checker comments are part of the Doxygen documentation?

Of course not :) I've been mostly thinking about the benefits of the anonymous namespace itself (cleaner global scope, no name collisions, but even these benefits are extremely minor).

[NoQ]

Added a huge amount of macros in order to improve readability of function specs.
Other inline comments should have been addressed before.

[dcoughlin]

Thanks for adding the macros. I've provided some feedback inline.

I think a good rule of thumb for readability is: suppose you are a maintainer and need to add a summary for a new function. Can you copy the the summary for an existing function and figure out what each component means so you can change it for the new function?

include/clang/StaticAnalyzer/Checkers/Checkers.td

383

I know you and Gábor already discussed this -- but shouldn't this be CStdLibraryFunctionsChecker or 'StdCLibraryFunctionsChecker'? Or is is your intent that both C and C++ standard libraries would be modeled by this checker?

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

11 ↗

(On Diff #71510)

"throw" --> "generate"

534 ↗

(On Diff #71510)

Is "specification" the right term here? Or is this really a "summary"?

536 ↗

(On Diff #71510)

'SPEC_DATA' doesn't seem to add much in terms of readability. Is it needed?

537 ↗

(On Diff #71510)

The argument and return types seem like more a property of the function than than the summary. Why are they here and not with the function name?

540 ↗

(On Diff #71510)

"Cases" seems more appropriate than "branches" (branching is an implementation detail).

542 ↗

(On Diff #71510)

If I understand correctly, the first "argument" to branch describes the constraints on the function arguments and the second (if present) describes the resulting constraint on the return value when the argument constraint holds. Is there a way to make this apparent in the spelling of the summary?

As a straw proposal, what about renaming the first 'RANGE' to 'ARGUMENT_CONSTRAINT' and the second the 'RETURN_VALUE_CONSTRAINT'?

Or, more jargony, "PRECONDITION" and "POSTCONDITION"?

547 ↗

(On Diff #71510)

Is it ever the case that this final 'RANGE" constrains anything other than the return value? If not, can 'RET_VAL' be elided?

554 ↗

(On Diff #71510)

What is the motivation behind the use of geometric terms here (i.e., "SEG", "POINT")? Why not "INTERVAL" and "EXACT_VALUE"?

560 ↗

(On Diff #71510)

Would you be opposed to 'UNION' instead of 'U'?

[NoQ]

In D20811#544250, @dcoughlin wrote:

I think a good rule of thumb for readability is: suppose you are a maintainer and need to add a summary for a new function. Can you copy the the summary for an existing function and figure out what each component means so you can change it for the new function?

Seems i've written too many summaries to reliably use this rule :)

Could you have a look at another attempt?:

SUMMARY(isalnum, ARGUMENT_TYPES { IntTy }, RETURN_TYPE(IntTy),
        INVALIDATION_APPROACH(EvalCallAsPure))
  CASE // Boils down to isupper() or islower() or isdigit()
    PRE_CONDITION(ARG_NO(0), CONDITION_KIND(WithinRange))
      RANGE('0', '9')
      RANGE('A', 'Z')
      RANGE('a', 'z')
    END_PRE_CONDITION
    POST_CONDITION(OutOfRange)
      VALUE(0)
    END_POST_CONDITION
  END_CASE
  CASE // The locale-specific range.
    PRE_CONDITION(ARG_NO(0), CONDITION_KIND(WithinRange))
      RANGE(128, 255)
    END_PRE_CONDITION
    // No post-condition. We are completely unaware of
    // locale-specific return values.
  END_CASE
  CASE
    PRE_CONDITION(ARG_NO(0), CONDITION_KIND(OutOfRange))
      RANGE('0', '9')
      RANGE('A', 'Z')
      RANGE('a', 'z')
      RANGE(128, 255)
    END_PRE_CONDITION
    POST_CONDITION(WithinRange)
      VALUE(0)
    END_POST_CONDITION
  END_CASE
END_SUMMARY

include/clang/StaticAnalyzer/Checkers/Checkers.td

383

Hmm, i just realized what you guys were talking about :) The same checker cpp file and even the same checker object should probably produce different checker list entries here which would go into separate packages (cplusplus for C++ library functions, etc.). We could even split the specifications into different files, but the checker object would still be the same, defined in the same file. Will do.

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

537 ↗

(On Diff #71510)

Because this is where C++ initializer list syntax forces them to be. Hiding this detail is, as far as i see, only possible with the means of BEGIN_.../END_... macros (which isn't a big deal i think).

547 ↗

(On Diff #71510)

Some summaries only have pre-conditions: "for this argument constraint, any return value is possible". We should also be able to support void functions, which have no return values.

[dcoughlin]

I think this is much clearer! That said, now that I look at it with 'POSTCONDITION' alone I don't think it is clear that the provided value describes the return value. What do you think about renaming it 'RETURN_VALUE'? Or adding back the RET_VAL I asked you about removing before? :-)

Also: do you think CONDITION_KIND is needed? in PRECONDITION? Or can the bare kind be used like in POSTCONDITION?

[NoQ]

In D20811#544927, @dcoughlin wrote:

That said, now that I look at it with 'POSTCONDITION' alone I don't think it is clear that the provided value describes the return value. What do you think about renaming it 'RETURN_VALUE'? Or adding back the RET_VAL I asked you about removing before? :-)

Hmm, what about

CONSTRAIN
  ARGUMENT_VALUE(0, WithinRange)
    RANGE('0', '9')
    RANGE('A', 'Z')
    RANGE('a', 'z')
  END_ARGUMENT_VALUE
  RETURN_VALUE(OutOfRange)
    VALUE(0)
  END_RETURN_VALUE
END_CONSTRAIN

?

Something i don't like here is that the word "value" is overloaded. Maybe rename the inner VALUE back to POINT?

In D20811#544927, @dcoughlin wrote:

Also: do you think CONDITION_KIND is needed? in PRECONDITION? Or can the bare kind be used like in POSTCONDITION?

I agree that it's ok to use the bare kind, because it's quite self-explanatory.

[dcoughlin]

In D20811#544981, @NoQ wrote:

Hmm, what about

CONSTRAIN
  ARGUMENT_VALUE(0, WithinRange)
    RANGE('0', '9')
    RANGE('A', 'Z')
    RANGE('a', 'z')
  END_ARGUMENT_VALUE
  RETURN_VALUE(OutOfRange)
    VALUE(0)
  END_RETURN_VALUE
END_CONSTRAIN

?

"CONSTRAIN" is a verb. What is the direct object here? It seems to me that the thing being constrained is the return value, so it seems odd to have 'CONSTRAIN' around the conditions on the arguments.

Something i don't like here is that the word "value" is overloaded. Maybe rename the inner VALUE back to POINT?

I don't think the geometric metaphor of 'POINT' makes sense here, especially with 'RANGE' (which I think is very good). What is the analog of a range that has only a single element?

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

547 ↗

(On Diff #71510)

What does a postcondition on a void function mean in this context? Can you refer to argument values? Such as "If the the function terminates then it must have been the case that the first argument was in the rangy x..z even though we didn't know that going in? Is this useful?

[zaks.anna]

Ping? Is there something blocking progress here? This functionality is very useful and almost done.

Thanks!

[NoQ]

I thought to give it a pause to take a fresh look at how to arrange the macro-hints in the summaries.

Maybe something like that:

CASE
  ARGUMENT_CONDITION(ARG_NO(0), OutOfRange)
    RANGE('0', '9')
    RANGE('A', 'Z')
    RANGE('a', 'z')
    RANGE(128, 255)
  END_ARGUMENT_CONDITION
  RETURN_VALUE_CONDITION(WithinRange)
    SINGLE_VALUE(0)
  END_RETURN_VALUE_CONDITION
END_CASE

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

547 ↗

(On Diff #71510)

No, i don't think this is useful. There are just timeless immutable symbols about which we learn something new on every branch.

If the function doesn't terminate on certain pre-conditons, then we can model it by never mentioning these pre-conditions in any of the branches (we don't use this trick anywhere yet - all functions listed here shall terminate in all cases).

This would have been useful if we start referring to the heap shape (eg. "if the value behind the pointer passed as second argument to the call was in range [1,10] before the call, then it would be equal to 42 after the call"), but we don't do that yet.

[dcoughlin]

In D20811#575521, @NoQ wrote:

I thought to give it a pause to take a fresh look at how to arrange the macro-hints in the summaries.

Maybe something like that:

CASE
  ARGUMENT_CONDITION(ARG_NO(0), OutOfRange)
    RANGE('0', '9')
    RANGE('A', 'Z')
    RANGE('a', 'z')
    RANGE(128, 255)
  END_ARGUMENT_CONDITION
  RETURN_VALUE_CONDITION(WithinRange)
    SINGLE_VALUE(0)
  END_RETURN_VALUE_CONDITION
END_CASE

Looks great to me!

[NoQ]

Update the domain-specific language for function specs/summaries.

[dcoughlin]

This looks great!

lib/StaticAnalyzer/Checkers/StdLibraryFunctionsChecker.cpp

694 ↗

(On Diff #75446)

I think think this comment should say 'lowercase'.

702 ↗

(On Diff #75446)

Same here.

include/clang/StaticAnalyzer/Checkers/Checkers.td

 def MismatchedDeallocatorChecker : Checker<"MismatchedDeallocator">,
   HelpText<"Check for mismatched deallocators.">,
   DescFile<"MallocChecker.cpp">;
 
 def VforkChecker : Checker<"Vfork">,
   HelpText<"Check for proper usage of vfork">,
   DescFile<"VforkChecker.cpp">;
 
+def LibraryFunctionsChecker : Checker<"LibraryFunctions">,
+  HelpText<"Improve modeling of common library functions">,
+  DescFile<"LibraryFunctionsChecker.cpp">;
+
 } // end "unix"
 
 let ParentPackage = UnixAlpha in {
 
 def ChrootChecker : Checker<"Chroot">,
   HelpText<"Check improper use of chroot">,
   DescFile<"ChrootChecker.cpp">;
 

lib/StaticAnalyzer/Checkers/CMakeLists.txt

add_clang_library(clangStaticAnalyzerCheckers
   DivZeroChecker.cpp
   DynamicTypePropagation.cpp
   DynamicTypeChecker.cpp
   ExprInspectionChecker.cpp
   FixedAddressChecker.cpp
   GenericTaintChecker.cpp
   IdenticalExprChecker.cpp
   IvarInvalidationChecker.cpp
+  LibraryFunctionsChecker.cpp
   LLVMConventionsChecker.cpp
   LocalizationChecker.cpp
   MacOSKeychainAPIChecker.cpp
   MacOSXAPIChecker.cpp
   MallocChecker.cpp
   MallocOverflowSecurityChecker.cpp
   MallocSizeofChecker.cpp
   NSAutoreleasePoolChecker.cpp

lib/StaticAnalyzer/Checkers/LibraryFunctionsChecker.cpp

+//=== LibraryFunctionsChecker.cpp - integer overflows checker ---*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This checker improves modeling of a few simple library functions.
+// It does not throw warnings.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ClangSACheckers.h"
+#include "clang/StaticAnalyzer/Core/Checker.h"
+#include "clang/StaticAnalyzer/Core/CheckerManager.h"
+#include "clang/StaticAnalyzer/Core/PathSensitive/CallEvent.h"
+#include "clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h"
+
+using namespace clang;
+using namespace clang::ento;
+
+namespace {
+class LibraryFunctionsChecker : public Checker<check::PostCall, eval::Call> {
+  static const uint32_t Ret = std::numeric_limits<uint32_t>::max();
+  enum ValueRangeKindTy { Outside, Inside, ComparesToArgument };
+  enum InvalidationKindTy { Normal, Pure };
+  typedef std::vector<std::pair<int64_t, int64_t>> IntRangeVector;
+
+  class ValueRange {
+  private:
+    // ArgNo in CallExpr and CallEvent is defined is Unsigned, but
+    // obviously uint32_t should be enough for most practical purposes.
+    uint32_t ArgNo; // Argument to which we apply the range.
+    ValueRangeKindTy Kind; // Kind of range definition.
+    IntRangeVector Args; // Polymorphic arguments.
+
+  public:
+    ValueRange(uint32_t ArgNo, ValueRangeKindTy Kind,
+               const IntRangeVector &Args)
+        : ArgNo(ArgNo), Kind(Kind), Args(Args) {}
+
+    uint32_t getArgNo() const { return ArgNo; }
+    ValueRangeKindTy getKind() const { return Kind; }
+
+    BinaryOperator::Opcode getOpcode() const {
+      assert(Kind == ComparesToArgument);
+      assert(Args.size() == 1);
+      BinaryOperator::Opcode Op =
+          static_cast<BinaryOperator::Opcode>(Args[0].first);
+      assert(BinaryOperator::isComparisonOp(Op) &&
+             "Only comparison ops are supported for ComparesToArgument");
+      return Op;
+    }
+
+    uint32_t getOtherArgNo() const {
+      assert(Kind == ComparesToArgument);
+      assert(Args.size() == 1);
+      return static_cast<uint32_t>(Args[0].second);
+    }
+
+    const IntRangeVector &getRanges() const {
+      assert(Kind != ComparesToArgument);
+      return Args;
+    }
+  };
+
+  typedef std::vector<ValueRange> ValueRangeSet;
+
+  // This spec is a bit more specific than CallDescription.
+  struct FunctionSpec {
+    const std::vector<QualType> ArgTypes;
+    const QualType RetType;
+    const InvalidationKindTy InvalidationKind;
+    const std::vector<ValueRangeSet> Ranges;
+  };
+
+  typedef std::map<std::string, FunctionSpec> FunctionSpecMapTy;
+  mutable FunctionSpecMapTy FunctionSpecMap;
+
+  static QualType getArgType(const FunctionSpec &Spec, uint32_t ArgNo) {
+    return ArgNo == Ret ? Spec.RetType : Spec.ArgTypes[ArgNo];
+  }
+  static QualType getArgType(const CallEvent &Call, uint32_t ArgNo) {
+    return ArgNo == Ret ? Call.getResultType().getCanonicalType()
+                        : Call.getArgExpr(ArgNo)->getType().getCanonicalType();
+  }
+  static QualType getArgType(const CallExpr *CE, uint32_t ArgNo) {
+    return ArgNo == Ret ? CE->getType().getCanonicalType()
+                        : CE->getArg(ArgNo)->getType().getCanonicalType();
+  }
+  static SVal getArgSVal(const CallEvent &Call, uint32_t ArgNo) {
+    return ArgNo == Ret ? Call.getReturnValue() : Call.getArgSVal(ArgNo);
+  }
+
+public:
+  void checkPostCall(const CallEvent &Call, CheckerContext &C) const;
+  bool evalCall(const CallExpr *CE, CheckerContext &C) const;
+
+private:
+  Optional<FunctionSpec> findFunctionSpec(const FunctionDecl *FD,
+                                          const CallExpr *CE,
+                                          CheckerContext &C) const;
+
+  void initFunctionSpecs(BasicValueFactory &BVF) const;
+};
+} // end of anonymous namespace
+
+void LibraryFunctionsChecker::checkPostCall(const CallEvent &Call,
+                                            CheckerContext &C) const {
+  const FunctionDecl *FD = dyn_cast_or_null<FunctionDecl>(Call.getDecl());
+  if (!FD)
+    return;
+
+  const CallExpr *CE = dyn_cast_or_null<CallExpr>(Call.getOriginExpr());
+  if (!CE)
+    return;
+
+  Optional<FunctionSpec> FoundSpec = findFunctionSpec(FD, CE, C);
+  if (!FoundSpec)
+    return;
+
+  const FunctionSpec &Spec = *FoundSpec;
+
+  // Now apply ranges.
+
+  SValBuilder &SVB = C.getSValBuilder();
+  BasicValueFactory &BVF = SVB.getBasicValueFactory();
+  ConstraintManager &CM = C.getConstraintManager();
+  ProgramStateRef State = C.getState();
+  QualType CondT = SVB.getConditionType();
+
+  for (const auto &VRS: Spec.Ranges) {
+
+    ProgramStateRef NewState = State;
+
+    for (const auto &VR: VRS) {
+      QualType T = getArgType(Spec, VR.getArgNo());
+      SVal V = getArgSVal(Call, VR.getArgNo());
+
+      // In case we encounter unknown or undefined values,
+      // we would not be able to add ranges on them anyway,
+      // but other ranges are still worth adding.
+      switch (VR.getKind()) {
+      case ComparesToArgument: {
+        BinaryOperator::Opcode Op = VR.getOpcode();
+        uint32_t OtherArg = VR.getOtherArgNo();
+        SVal OtherV = getArgSVal(Call, OtherArg);
+        QualType OtherT = getArgType(Call, OtherArg);
+        // Note: we avoid integral promotion for comparison.
+        OtherV = SVB.evalCast(OtherV, T, OtherT);
+        if (auto CompV = SVB.evalBinOp(NewState, Op, V, OtherV, CondT)
+                             .getAs<DefinedOrUnknownSVal>())
+          NewState = NewState->assume(*CompV, true);
+        break;
+      }
+      case Outside:
+        if (auto N = V.getAs<NonLoc>()) {
+          const IntRangeVector &R = VR.getRanges();
+          size_t E = R.size();
+          for (size_t I = 0; I != E; ++I) {
+            const llvm::APSInt &Min = BVF.getValue(R[I].first, T);
+            const llvm::APSInt &Max = BVF.getValue(R[I].second, T);
+            assert(Min <= Max);
+            NewState =
+                CM.assumeWithinInclusiveRange(NewState, *N, Min, Max, false);
+            if (!NewState)
+              break;
+          }
+        }
+        break;
+      case Inside:
+        // "Inside R" is treated as "outside [T_MIN, T_MAX] \ R".
+        // We cut off [T_MIN, min(R) - 1] and [max(R) + 1, T_MAX] if necessary,
+        // and then cut away all holes in R one by one.
+        if (auto N = V.getAs<NonLoc>()) {
+          const IntRangeVector &R = VR.getRanges();
+          size_t E = R.size();
+
+          const llvm::APSInt &MinusInf = BVF.getMinValue(T);
+          const llvm::APSInt &PlusInf = BVF.getMaxValue(T);
+
+          const llvm::APSInt &Left = BVF.getValue(R[0].first - 1, T);
+          if (Left != PlusInf) {
+            assert(MinusInf <= Left);
+            NewState = CM.assumeWithinInclusiveRange(NewState, *N,
+                                                     MinusInf, Left, false);
+            if (!NewState)
+              break;
+          }
+
+          const llvm::APSInt &Right = BVF.getValue(R[E - 1].second + 1, T);
+          if (Right != MinusInf) {
+            assert(Right <= PlusInf);
+            NewState = CM.assumeWithinInclusiveRange(NewState, *N,
+                                                     Right, PlusInf, false);
+            if (!NewState)
+              break;
+          }
+
+          for (size_t I = 1; I != E; ++I) {
+            const llvm::APSInt &Min = BVF.getValue(R[I - 1].second + 1, T);
+            const llvm::APSInt &Max = BVF.getValue(R[I].first - 1, T);
+            assert(Min <= Max);
+            NewState = CM.assumeWithinInclusiveRange(NewState, *N, Min, Max,
+                                                     false);
+            if (!NewState)
+              break;
+          }
+        }
+        break;
+      }
+
+      if (!NewState)
+        break;
+    }
+
+    if (NewState)
+      C.addTransition(NewState);
+  }
+}
+
+bool LibraryFunctionsChecker::evalCall(const CallExpr *CE,
+                                       CheckerContext &C) const {
+  const FunctionDecl *FD = dyn_cast_or_null<FunctionDecl>(CE->getCalleeDecl());
+  if (!FD)
+    return false;
+
+  Optional<FunctionSpec> FoundSpec = findFunctionSpec(FD, CE, C);
+  if (!FoundSpec)
+    return false;
+
+  const FunctionSpec &Spec = *FoundSpec;
+  switch (Spec.InvalidationKind) {
+  case Pure: {
+    ProgramStateRef State = C.getState();
+    const LocationContext *LC = C.getLocationContext();
+    SVal V = C.getSValBuilder().conjureSymbolVal(
+        CE, LC, CE->getType().getCanonicalType(), C.blockCount());
+    State = State->BindExpr(CE, LC, V);
+    C.addTransition(State);
+    return true;
+  }
+  case Normal:
+    return false;
+  }
+  llvm_unreachable("Unknown invalidation kind!");
+}
+
+Optional<LibraryFunctionsChecker::FunctionSpec>
+LibraryFunctionsChecker::findFunctionSpec(const FunctionDecl *FD,
+                                          const CallExpr *CE,
+                                          CheckerContext &C) const {
+  // Note: we cannot always obtain FD from CE (eg. virtual call).
+  assert(CE);
+
+  if (!FD)
+    return None;
+
+  SValBuilder &SVB = C.getSValBuilder();
+  BasicValueFactory &BVF = SVB.getBasicValueFactory();
+  initFunctionSpecs(BVF);
+
+  std::string Name = FD->getQualifiedNameAsString();
+  if (Name.empty() || !C.isCLibraryFunction(FD, Name))
+    return None;
+
+  auto FSMI = FunctionSpecMap.find(Name);
+  if (FSMI == FunctionSpecMap.end())
+    return None;
+
+  // Verify that function signature matches the spec in advance,
+  // so that we didn't have to roll back if anything goes wrong.
+
+  // Check number of arguments:
+  const FunctionSpec &Spec = FSMI->second;
+  if (CE->getNumArgs() != Spec.ArgTypes.size())
+    return None;
+
+  // Check return type if relevant:
+  if (!Spec.RetType.isNull() &&
+      Spec.RetType != CE->getType().getCanonicalType())
+    return None;
+
+  // Check argument types when relevant:
+  for (size_t I = 0, E = Spec.ArgTypes.size(); I != E; ++I) {
+    QualType FormalT = Spec.ArgTypes[I];
+    // Null type marks irrelevant arguments.
+    if (FormalT.isNull())
+      continue;
+    assert(!FormalT->isVoidType() &&
+           "We should have had no significant void types in the spec");
+    assert(FormalT.isCanonical() &&
+           "We should only have canonical types in the spec");
+    // FIXME: lift this assert (but not the ones above!)
+    assert(FormalT->isIntegralOrEnumerationType() &&
+           "We only support integral ranges in the spec");
+    QualType ActualT = getArgType(CE, I);
+    assert(ActualT.isCanonical());
+    if (ActualT != FormalT)
+      return None;
+  }
+  return FSMI->second;
+}
+
+void LibraryFunctionsChecker::initFunctionSpecs(BasicValueFactory &BVF) const {
+  if (!FunctionSpecMap.empty())
+    return;
+
+  ASTContext &ACtx = BVF.getContext();
+
+  QualType Irrelevant;
+  QualType IntTy = ACtx.IntTy;
+  QualType SizeTy = ACtx.getSizeType();
+  QualType SSizeTy = ACtx.getIntTypeForBitwidth(ACtx.getTypeSize(SizeTy), true);
+
+  // Don't worry about truncation here, it'd be cast back to SIZE_MAX when used.
+  int64_t SizeMax = BVF.getMaxValue(SizeTy).getLimitedValue();
+  int64_t SSizeMax = BVF.getMaxValue(SSizeTy).getLimitedValue();
+
+  // NOTE: The signature needs to have the correct number of arguments.
+  // NOTE: However, insert Irrelevant when the type is insignificant.
+  // NOTE: Argument ranges should always cover all variants. If return value
+  //       is completely unknown, omit it from the respective range set.
+  // NOTE: Upon comparing to another argument, the other argument is casted
+  //       to the current argument's type. This avoids proper promotion but
+  //       seems useful.
+  // NOTE: All types in the spec need to be canonical.
+  //
+  // The format is as follows:
+  //
+  //{ "function name",
+  //  { spec:
+  //    { argument types list, ... }, return type, purity, { range sets list:
+  //      { ranges list:
+  //        { argument index, inside or outside, {{from, to}, ...} },
+  //        { argument index, compares to argument, {{how, which}} },
+  //        ...
+  //      }
+  //    }
+  //  }
+  //}
+  FunctionSpecMap = {
+    // The isascii() family of functions.
+    { "isalnum",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // isupper() or islower() or isdigit()
+            { 0U, Inside, {{'0', '9'}, {'A', 'Z'}, {'a', 'z'}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // The locale-specific range.
+            { 0U, Inside, {{128, 255}} }
+          },
+          { // Other.
+            { 0U, Outside, {{'0', '9'}, {'A', 'Z'}, {'a', 'z'}, {128, 255}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isalpha",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // isupper() or islower(). Note that 'Z' is less than 'a'.
+            { 0U, Inside, {{'A', 'Z'}, {'a', 'z'}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // The locale-specific range.
+            { 0U, Inside, {{128, 255}} },
+          },
+          { // Other.
+            { 0U, Outside, {{'A', 'Z'}, {'a', 'z'}, {128, 255}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isascii",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // Is ASCII.
+            { 0U, Inside, {{0, 127}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // Is not ASCII.
+            { 0U, Outside, {{0, 127}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isblank",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // Is tab or space.
+            { 0U, Inside, {{'\t', '\t'}, {' ', ' '}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // Other.
+            { 0U, Outside, {{'\t', '\t'}, {' ', ' '}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "iscntrl",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // 0..31 or 127
+            { 0U, Inside, {{0, 32}, {127, 127}} },
+            { Ret, Outside, {{0, 0}} },
+          },
+          {
+            { 0U, Outside, {{0, 32}, {127, 127}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isdigit",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // Is a digit.
+            { 0U, Inside, {{'0', '9'}} },
+            { Ret, Outside, {{0, 0}} },
+          },
+          {
+            { 0U, Outside, {{'0', '9'}} },
+            { Ret, Inside, {{0, 0}} },
+          }
+        }
+      }
+    },
+    { "isgraph",
+      {
+        { IntTy }, IntTy, Pure, {
+          {
+            { 0U, Inside, {{33, 126}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          {
+            { 0U, Outside, {{33, 126}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "islower",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // Is certainly uppercase.
+            { 0U, Inside, {{'a', 'z'}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // Is ascii but not uppercase.
+            { 0U, Inside, {{0, 127}} },
+            { 0U, Outside, {{'a', 'z'}} },
+            { Ret, Inside, {{0, 0}} }
+          },
+          { // The locale-specific range.
+            { 0U, Inside, {{128, 255}} }
+          },
+          { // Is not an unsigned char.
+            { 0U, Outside, {{0, 255}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isprint",
+      {
+        { IntTy }, IntTy, Pure, {
+          {
+            { 0U, Inside, {{32, 126}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          {
+            { 0U, Outside, {{32, 126}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "ispunct",
+      {
+        { IntTy }, IntTy, Pure, {
+          {
+            { 0U, Inside, {{'!', '/'}, {':', '@'}, {'[', '`'}, {'{', '~'}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          {
+            { 0U, Outside, {{'!', '/'}, {':', '@'}, {'[', '`'}, {'{', '~'}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isspace",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // Space, '\f', '\n', '\r', '\t', '\v'.
+            { 0U, Inside, {{9, 13}, {' ', ' '}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // The locale-specific range.
+            { 0U, Inside, {{128, 255}} }
+          },
+          {
+            { 0U, Outside, {{9, 13}, {' ', ' '}, {128, 255}} },
+            { Ret, Inside, {{0, 0}} }
+          },
+        }
+      }
+    },
+    { "isupper",
+      {
+        { IntTy }, IntTy, Pure, {
+          { // Is certainly uppercase.
+            { 0U, Inside, {{'A', 'Z'}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          { // The locale-specific range.
+            { 0U, Inside, {{128, 255}} }
+          },
+          { // Other.
+            { 0U, Outside, {{'A', 'Z'}, {128, 255}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+    { "isxdigit",
+      {
+        { IntTy }, IntTy, Pure, {
+          {
+            { 0U, Inside, {{'0', '9'}, {'A', 'F'}, {'a', 'f'}} },
+            { Ret, Outside, {{0, 0}} }
+          },
+          {
+            { 0U, Outside, {{'0', '9'}, {'A', 'F'}, {'a', 'f'}} },
+            { Ret, Inside, {{0, 0}} }
+          }
+        }
+      }
+    },
+
+    // The getc() family of functions that returns either a char or an EOF.
+    { "getc",
+      {
+        { Irrelevant }, IntTy, Normal, {
+          { // FIXME: EOF is assumed to be defined as -1.
+            { Ret, Inside, {{-1, 255}} }
+          }
+        }
+      }
+    },
+    { "fgetc",
+      {
+        { Irrelevant }, IntTy, Normal, {
+          { // FIXME: EOF is assumed to be defined as -1.
+            { Ret, Inside, {{-1, 255}} }
+          }
+        }
+      }
+    },
+    { "getchar",
+      {
+        { }, IntTy, Normal, {
+          { // FIXME: EOF is assumed to be defined as -1.
+            { Ret, Inside, {{-1, 255}} }
+          }
+        }
+      }
+    },
+
+    // read()-like functions that never return more than buffer size.
+    { "read",
+      {
+        { Irrelevant, Irrelevant, SizeTy }, SSizeTy, Normal, {
+          {
+            { Ret, ComparesToArgument, {{BO_LE, 2U}} },
+            { Ret, Inside, {{-1, SSizeMax}} }
+          },
+        }
+      }
+    },
+    { "write",
+      {
+        { Irrelevant, Irrelevant, SizeTy }, SSizeTy, Normal, {
+          {
+            { Ret, ComparesToArgument, {{BO_LE, 2U}} },
+            { Ret, Inside, {{-1, SSizeMax}} }
+          },
+        }
+      }
+    },
+    { "fread",
+      {
+        { Irrelevant, Irrelevant, SizeTy, Irrelevant }, SizeTy, Normal, {
+          {
+            { Ret, Inside, {{0, SizeMax}} },
+            { Ret, ComparesToArgument, {{BO_LE, 2U}} }
+          }
+        }
+      }
+    },
+    { "fwrite",
+      {
+        { Irrelevant, Irrelevant, SizeTy, Irrelevant }, SizeTy, Normal, {
+          {
+            { Ret, Inside, {{0, SizeMax}} },
+            { Ret, ComparesToArgument, {{BO_LE, 2U}} }
+          }
+        }
+      }
+    },
+
+    // getline()-like functions either fail or read at least the delimiter.
+    { "getline",
+      {
+        { Irrelevant, Irrelevant, Irrelevant }, SSizeTy, Normal, {
+          {
+            { Ret, Inside, {{-1, -1}, {1, SSizeMax}} }
+          },
+        }
+      }
+    },
+    { "getdelim",
+      {
+        { Irrelevant, Irrelevant, Irrelevant, Irrelevant }, SSizeTy, Normal, {
+          {
+            { Ret, Inside, {{-1, -1}, {1, SSizeMax}} }
+          },
+        }
+      }
+    }
+  };
+}
+
+void ento::registerLibraryFunctionsChecker(CheckerManager &mgr) {
+  mgr.registerChecker<LibraryFunctionsChecker>();
+}

test/Analysis/library-functions.c

+// RUN: %clang_cc1 -analyze -analyzer-checker=unix.LibraryFunctions,debug.ExprInspection -verify %s
+
+void clang_analyzer_eval(int);
+int glob;
+
+typedef struct FILE FILE;
+int getc(FILE *);
+#define EOF -1
+void test_getc(FILE *fp) {
+  int x;
+  while ((x = getc(fp)) != EOF) {
+    clang_analyzer_eval(x > 255); // expected-warning{{FALSE}}
+    clang_analyzer_eval(x >= 0); // expected-warning{{TRUE}}
+  }
+}
+
+typedef unsigned long size_t;
+typedef signed long ssize_t;
+ssize_t write(int, const void *, size_t);
+void test_write(int fd, char *buf) {
+  glob = 1;
+  ssize_t x = write(fd, buf, 10);
+  clang_analyzer_eval(glob); // expected-warning{{UNKNOWN}}
+  if (x >= 0)
+    clang_analyzer_eval(x <= 10); // expected-warning{{TRUE}}
+  else
+    clang_analyzer_eval(x == -1); // expected-warning{{TRUE}}
+}
+
+size_t fread(void *, size_t, size_t, FILE *);
+void test_fread(FILE *fp, int *buf) {
+  size_t x = fread(buf, sizeof(int), 10, fp);
+  clang_analyzer_eval(x <= 10); // expected-warning{{TRUE}}
+}
+
+ssize_t getline(char **, size_t *, FILE *);
+void test_getline(FILE *fp) {
+  char *line = 0;
+  size_t n = 0;
+  ssize_t len;
+  while ((len = getline(&line, &n, fp)) != -1) {
+    clang_analyzer_eval(len == 0); // expected-warning{{FALSE}}
+  }
+}
+
+int isascii(int);
+void test_isascii(int x) {
+  clang_analyzer_eval(isascii(123)); // expected-warning{{TRUE}}
+  clang_analyzer_eval(isascii(-1)); // expected-warning{{FALSE}}
+  if (isascii(x)) {
+    clang_analyzer_eval(x < 128); // expected-warning{{TRUE}}
+    clang_analyzer_eval(x >= 0); // expected-warning{{TRUE}}
+  } else {
+    if (x > 42)
+      clang_analyzer_eval(x >= 128); // expected-warning{{TRUE}}
+    else
+      clang_analyzer_eval(x < 0); // expected-warning{{TRUE}}
+  }
+  glob = 1;
+  isascii('a');
+  clang_analyzer_eval(glob); // expected-warning{{TRUE}}
+}
+
+int islower(int);
+void test_islower(int x) {
+  clang_analyzer_eval(islower('x')); // expected-warning{{TRUE}}
+  clang_analyzer_eval(islower('X')); // expected-warning{{FALSE}}
+  if (islower(x))
+    clang_analyzer_eval(x < 'a'); // expected-warning{{FALSE}}
+}
+
+int getchar(void);
+void test_getchar() {
+  int x = getchar();
+  if (x == EOF)
+    return;
+  clang_analyzer_eval(x < 0); // expected-warning{{FALSE}}
+  clang_analyzer_eval(x < 256); // expected-warning{{TRUE}}
+}
+
+int isalpha(int);
+void test_isalpha() {
+  clang_analyzer_eval(isalpha(']')); // expected-warning{{FALSE}}
+  clang_analyzer_eval(isalpha('Q')); // expected-warning{{TRUE}}
+  clang_analyzer_eval(isalpha(128)); // expected-warning{{UNKNOWN}}
+}
+
+int isalnum(int);
+void test_alnum() {
+  clang_analyzer_eval(isalnum('1')); // expected-warning{{TRUE}}
+  clang_analyzer_eval(isalnum(')')); // expected-warning{{FALSE}}
+}
+
+int isblank(int);
+void test_isblank() {
+  clang_analyzer_eval(isblank('\t')); // expected-warning{{TRUE}}
+  clang_analyzer_eval(isblank(' ')); // expected-warning{{TRUE}}
+  clang_analyzer_eval(isblank('\n')); // expected-warning{{FALSE}}
+}
+
+int ispunct(int);
+void test_ispunct(int x) {
+  clang_analyzer_eval(ispunct(' ')); // expected-warning{{FALSE}}
+  clang_analyzer_eval(ispunct(-1)); // expected-warning{{FALSE}}
+  clang_analyzer_eval(ispunct('#')); // expected-warning{{TRUE}}
+  clang_analyzer_eval(ispunct('_')); // expected-warning{{TRUE}}
+  if (ispunct(x))
+    clang_analyzer_eval(x < 127); // expected-warning{{TRUE}}
+}