This is an archive of the discontinued LLVM Phabricator instance.

[analyzer][solver] Iterate to a fixpoint during symbol simplification with constants
ClosedPublic

Authored by martong on Jul 26 2021, 1:58 PM.

Details

Summary

D103314 introduced symbol simplification when a new constant constraint is
added. Currently, we simplify existing equivalence classes by iterating over
all existing members of them and trying to simplify each member symbol with
simplifySVal.

At the end of such a simplification round we may end up introducing a
new constant constraint. Example:

if (a + b + c != d)
  return;
if (c + b != 0)
  return;
// Simplification starts here.
if (b != 0)
  return;

The c == 0 constraint is the result of the first simplification iteration.
However, we could do another round of simplification to reach the conclusion
that a == d. Generally, we could do as many new iterations until we reach a
fixpoint.

We can reach to a fixpoint by recursively calling State->assume on the
newly simplified symbol. By calling State->assume we re-ignite the
whole assume machinery (along e.g with adjustment handling).

Why should we do this? By reaching a fixpoint in simplification we are capable
of discovering infeasible states at the moment of the introduction of the
first constant constraint.
Let's modify the previous example just a bit, and consider what happens without
the fixpoint iteration.

if (a + b + c != d)
  return;
if (c + b != 0)
  return;
// Adding a new constraint.
if (a == d)
  return;
// This brings in a contradiction.
if (b != 0)
  return;
clang_analyzer_warnIfReached(); // This produces a warning.
            // The path is already infeasible...
if (c == 0) // ...but we realize that only when we evaluate `c == 0`.
  return;

What happens currently, without the fixpoint iteration? As the inline comments
suggest, without the fixpoint iteration we are doomed to realize that we are on
an infeasible path only after we are already walking on that. With fixpoint
iteration we can detect that before stepping on that. With fixpoint iteration,
the clang_analyzer_warnIfReached does not warn in the above example b/c
during the evaluation of b == 0 we realize the contradiction. The engine and
the checkers do rely on that either assume(Cond) or assume(!Cond) should be
feasible. This is in fact assured by the so called expensive checks
(LLVM_ENABLE_EXPENSIVE_CHECKS). The StdLibraryFuncionsChecker is notably one of
the checkers that has a very similar assertion.
(Actually, recognizing an infeasible parent state could have happened even
before simplification have been introduced. Because of the imperfection of the
solver, adding a new constraint can highlight that the parent state had been
infeasible already.)

Before this patch, we simply added the simplified symbol to the equivalence
class. In this patch, after we have added the simplified symbol, we remove the
old (more complex) symbol from the members of the equivalence class
(ClassMembers). Removing the old symbol is beneficial because during the next
iteration of the simplification we don't have to consider again the old symbol.

Contrary to how we handle ClassMembers, we don't remove the old Sym->Class
relation from the ClassMap. This is important for two reasons: The
constraints of the old symbol can still be found via it's equivalence class
that it used to be the member of (1). We can spare one removal and thus one
additional tree in the forest of ClassMap (2).

Performance and complexity: Let us assume that in a State we have N non-trivial
equivalence classes and that all constraints and disequality info is related to
non-trivial classes. In the worst case, we can simplify only one symbol of one
class in each iteration. The number of symbols in one class cannot grow b/c we
replace the old symbol with the simplified one. Also, the number of the
equivalence classes can decrease only, b/c the algorithm does a merge operation
optionally. We need N iterations in this case to reach the fixpoint. Thus, the
steps needed to be done in the worst case is proportional to N*N. Empirical
results (attached) show that there is some hardly noticeable run-time and peak
memory discrepancy compared to the baseline. In my opinion, these differences
could be the result of measurement error.
This worst case scenario can be extended to that cases when we have trivial
classes in the constraints and in the disequality map are transforming to such
a State where there are only non-trivial classes, b/c the algorithm does merge
operations. A merge operation on two trivial classes results in one non-trivial
class.

Diff Detail

Event Timeline

martong created this revision.Jul 26 2021, 1:58 PM
martong requested review of this revision.Jul 26 2021, 1:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptJul 26 2021, 1:58 PM
Herald added a subscriber: cfe-commits. · View Herald Transcript

For more details please check out this html file:

martong updated this revision to Diff 361793.Jul 26 2021, 2:05 PM
  • Rename test functions
martong edited the summary of this revision. (Show Details)Jul 27 2021, 8:41 AM

Ping. @vsavchenko Could you please take a look?

Looking great!
I have a couple of nit picks and I kind of want to check that it doesn't affect the performance on a different set of projects as well.

clang/lib/StaticAnalyzer/Core/RangeConstraintManager.cpp
2148–2149

Maybe after removing you can check that ClsMembers is not empty?
I just don't like relying on getHeight because it looks like an implementation detail and shouldn't be used. We use it only in one place in the whole codebase (in equivalence classes :) ), but since we can re-write this assertion to have a simpler condition, I think that it should be preferred.

clang/test/Analysis/symbol-simplification-fixpoint-iteration-unreachable-code.cpp
21

Do we need to print states in this test?

clang/test/Analysis/symbol-simplification-fixpoint-one-iteration.cpp
33

Same question here

clang/test/Analysis/symbol-simplification-fixpoint-two-iterations.cpp
37

OK, I understand the next two classes.
But how did we produce this one?

martong marked 4 inline comments as done.Aug 5 2021, 1:05 AM

Thanks for the review!

clang/lib/StaticAnalyzer/Core/RangeConstraintManager.cpp
2148–2149

Okay, I've also found it inconvenient to use getHeight but the API of immutable maps is quite sparse, I mean there is no way to query the size. The solution you suggest is better in the sense we don't have to use the internal API, so I've changed it that way, though, it has the disadvantage that we must check the precondition of the function in the middle of it which is weird to read.
What about having a free function that takes a tree as a param and returns whether it has two members? Or we could even extend the API of the immutable AVL tree with a new member function.

clang/test/Analysis/symbol-simplification-fixpoint-iteration-unreachable-code.cpp
21

Good point, I've accidentally left it here.

clang/test/Analysis/symbol-simplification-fixpoint-one-iteration.cpp
33

OK, I understand the next two classes. But how did we produce this one?

Simplification is done on each equivalence class we can find in the state, no matter if they are non-trivial or trivial classes.

Here is what happens in this case: We skim through the constraints and try to simplify all equivalence classes there. During this we start simplifying the trivial equivalence class ((reg_$0<int a>) + (reg_$1<int b>)) != (reg_$2<int c>). The first and only member of this class can be simplified with (reg_$0<int a>) != (reg_$2<int c>) because b==0. Now, we merge the two trivial classes of the original non-simplified and the new simplified symbols. At this point we receive a non-trivial class with two members: ((reg_$0<int a>) + (reg_$1<int b>)) != (reg_$2<int c>) and (reg_$0<int a>) != (reg_$2<int c>). And then we remove the old symbol from the class. That results in a non-trivial class with one member: (reg_$0<int a>) != (reg_$2<int c>).

clang/test/Analysis/symbol-simplification-fixpoint-two-iterations.cpp
37

I am giving an answer to this in the previous test file, because that case is shorter and easier to explain.

martong updated this revision to Diff 364373.Aug 5 2021, 1:05 AM
martong marked 4 inline comments as done.
martong edited the summary of this revision. (Show Details)
  • Remove superfluous clang_analyzer_printState
  • Assert on isEmpty instead of using getHeight
martong added inline comments.Oct 1 2021, 7:01 AM
clang/lib/StaticAnalyzer/Core/RangeConstraintManager.cpp
2144

Remove const

2159

Comment that this is a precondition.

2234

TODO add Performance and complexity essay here.

I'm gonna get back to this on Monday.

clang/lib/StaticAnalyzer/Core/RangeConstraintManager.cpp
1741–1742

IMO we should have a llvm::Statistic here, tracking the maximum iteration count to reach the fixed point and an average iteration count.

1775

I'd love to see a coverage report of the tests you add with this patch.

clang/test/Analysis/expr-inspection-printState-eq-classes.c
11

Why do you need to change this?

clang/test/Analysis/symbol-simplification-fixpoint-iteration-unreachable-code.cpp
17

Is it important to have this instead of b + c?

martong updated this revision to Diff 384117.Nov 2 2021, 8:25 AM
martong marked 3 inline comments as done.
  • Reach the fixpoint by recursively calling State->assume on the simplified symbol.
  • Address review nits.
clang/lib/StaticAnalyzer/Core/RangeConstraintManager.cpp
1741–1742

We can't do this once we reach the fixpoint with recursive assume calls.

1775

Ok, I am going to check the coverage and add the missing cases.

clang/test/Analysis/expr-inspection-printState-eq-classes.c
11

We don't need it, I removed.

clang/test/Analysis/symbol-simplification-fixpoint-iteration-unreachable-code.cpp
17

No, I changed it to b+c as you suggested.

martong updated this revision to Diff 384122.Nov 2 2021, 8:34 AM
  • Add essay about complexity.
steakhal added inline comments.Nov 3 2021, 2:55 AM
clang/lib/StaticAnalyzer/Core/RangeConstraintManager.cpp
1775

Ok, I am going to check the coverage and add the missing cases.

As of diff 5, line 1767 and all the code in the block at line 2184 are uncovered by the tests you provided.

martong updated this revision to Diff 384802.Nov 4 2021, 10:36 AM
  • Add new tests to cover missing line coverage.

As of diff 5, line 1767 and all the code in the block at line 2184 are uncovered by the tests you provided.

Thanks, I've added new tests that cover the re-assume logic (the block of line 2184).

However, I was unable to add a test that covers the case when the simplification of a trivial symbol in the disequality info results an infeasible state (line 1767). Here is how I tried: I had changed the line to an assertion and then initiated the static analysis of the following opensource projects: memcached,tmux,curl,twin,redis,vim,openssl,sqlite,ffmpeg,postgresql,tinyxml2,libwebm,xerces,bitcoin,protobuf. My idea had been, if the assertion would fired then I would use creduce to the create the test case (btw, this is how I added the other infeasible state test case). However, it did not fired.

IMHO, having a defensive check at L1767 is correct b/c there is a slight chance of reaching an infeasible state there. Although the chance is minimal, I cannot prove that is 0.

martong edited the summary of this revision. (Show Details)Nov 5 2021, 3:00 AM
martong edited the summary of this revision. (Show Details)Nov 5 2021, 3:18 AM

There are no runtime or peak memory usage growth with this patch (actually, the runtime decreases with a few %).
I am attaching the measurements results of the latest Diff.

steakhal accepted this revision.Nov 5 2021, 9:35 AM

As of diff 5, line 1767 and all the code in the block at line 2184 are uncovered by the tests you provided.

Thanks, I've added new tests that cover the re-assume logic (the block of line 2184).

However, I was unable to add a test that covers the case when the simplification of a trivial symbol in the disequality info results an infeasible state (line 1767). Here is how I tried: I had changed the line to an assertion and then initiated the static analysis of the following opensource projects: memcached,tmux,curl,twin,redis,vim,openssl,sqlite,ffmpeg,postgresql,tinyxml2,libwebm,xerces,bitcoin,protobuf. My idea had been, if the assertion would fired then I would use creduce to the create the test case (btw, this is how I added the other infeasible state test case). However, it did not fired.

IMHO, having a defensive check at L1767 is correct b/c there is a slight chance of reaching an infeasible state there. Although the chance is minimal, I cannot prove that is 0.

It's fine by me. Thanks for the investigation.

There are no runtime or peak memory usage growth with this patch (actually, the runtime decreases with a few %).
I am attaching the measurements results of the latest Diff.

Great!

The tests refer to reg_$0 by spelling the id number, which is unfortunate, but I suspect these tests will break for the slightest changes in the solver anyway so I'm not too bothered with this.

Please wait a week for the rest of the members of the community to have a look before committing.
@NoQ @Szelethus @ASDenysPetrov

This revision is now accepted and ready to land.Nov 5 2021, 9:35 AM