[Review request][analyzer] Duplicate '0 size allocation' check from unix.API in unix.Malloc.
Needs ReviewPublic

Authored by ayartsev on Nov 7 2014, 4:11 PM.

Details

Reviewers
zaks.anna
Summary

This duplication was motivated by the following:

  • FIXME from unix.API stating "Eventually these should be rolled into the MallocChecker, but right now they're more basic and valuable for widespread use."
  • an idea to extend the check to handle zero size allocation by 'new' and 'new[]' in the sequel.
  • create the base for implementation of undefbehavior.ZeroAllocDereference checker.

The patch is a scratch, may it make sense to make '0 size allocation' the separate checker to at least preserve handling of zero-size realloc unchanged.

I suggest the following logic for handling zero-size realloc:

  1. if the returned value of realloc(p, 0) is not assigned then silently treat it as simple 'free' (the old behavior of unix.Malloc)
  2. if the returned value is assigned then fire '0 size allocation' warning as the usage of the returned pointer is not safe.

Specification C11 n1570, 7.22.3.1 says: "If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object."
Usage of the returned pointer is planned to be a matter of other checkers (undefbehavior.ZeroAllocDereference + maybe another checker that checks for usage of returned pointer in comparisons)

Please review!

Diff Detail

ayartsev retitled this revision from to [Review request][analyzer] Duplicate '0 size allocation' check from unix.API in unix.Malloc..Nov 7 2014, 4:11 PM
ayartsev updated this object.
ayartsev edited the test plan for this revision. (Show Details)
ayartsev added a reviewer: jordan_rose.
ayartsev added subscribers: Unknown Object (MLST), zaks.anna, krememek.

Hi Anton,

Looks like the right direction. See a few comments throughout the code.

How do you plan on staging this? The current patch cannot be committed as is for the following 2 reasons:

  • Some warnings will be duplicated since we warn here and in the unix.api check.
  • We should not warn on realloc(p,0).

The proposed treatment of realloc seems fine to me. Maybe you could just start with skipping the warning on realloc to stage the commits. What do you think?

lib/StaticAnalyzer/Checkers/MallocChecker.cpp
222–226

Ho about BT_MallocZero -> BT_ZerroAllocation ?

259

Please, add oxygen comment for this and "CheckCallocZero".

261

I'd rename "SizeArg" to make it more clear it's the allocation size; for example, something like "AllocationSizeArg".

819

This comment is probably copied from the unix.api checker, where it tells which APIs are being processed below. It feels out of place here.

825

TalseState -> FalseState

Are the True and False states backwards? I would expect TrueState to mean zero allocation; but by looking at the return it seems that it's the opposite..

836

Does -> Performs

"most of the bellow"? where below?

Missing "."

847

Does it make sense to pass an extra argument just for sanity check?

868

Why is CheckCallocZero different from BasicAllocationCheck? (I feel like I am missing something..)

1502

Please, use more descriptive state name.

test/Analysis/malloc.c
953

Could you add a couple of more interesting zero allocation tests (where we are not dealing with zero constant).

Theoretically, the inlined defensive checks could be a problem here, right?
(This is where the value is assumed to be zero by an inlined function, but known not be be zero in a caller.)

ayartsev updated this revision to Diff 17135.Dec 10 2014, 6:11 PM
ayartsev edited reviewers, added: zaks.anna; removed: jordan_rose.
ayartsev edited subscribers, added: jordan_rose; removed: zaks.anna.

Addressed the comments, updated the patch, please review!

Updated the patch.

lib/StaticAnalyzer/Checkers/MallocChecker.cpp
825

Inverted.

847

Removed.

868

Hm, really. Removed CheckCallocZero, added two calls to BasicAllocationCheck instead.

test/Analysis/malloc.c
953

Added a path-sensitive test (testMallocPathZero) and an additional tests for 'calloc' (testCalloc1Unknown2Zero - covers the case when the first parameter to 'calloc' is an UnknownVal).

Yes, but only with 'suppress-inlined-defensive-checks=false' given to analyzer.

Hi Anton,

Thanks for the updated patch!

"Yes, but only with 'suppress-inlined-defensive-checks=false' given to analyzer."
Have you added a test to make sure that is the case? (If yes, what's the function name. I could not easily find it.) It would be similar to the path-sensitive check you've added but the check against '0' would be done in an inlined function.

Looks like you've removed the checks from the unix checker along with the tests. Can you add the tests back? I suggest adding your checker to the test file that contains the existing tests and check that it produces the same exact output. (If there are differences, I'd like to see what they are.) We can later remove the redundant tests as a separate commit.

/*
TODO: Will be uncommented when 'realloc' is handled properly (see D6178 for
details).
void testReallocZero(char *ptr) {

char *foo = realloc(ptr, 0); // TODO:expected-warning{{Call to 'realloc()' has an allocation size of 0 bytes}}
free(foo);

}
*/

  1. We should not use "/*" for comments.
  2. Why should we warn in this case? It is possible that someone is trying to free the memory by calling "realloc(ptr, 0)".. Or are you saying that if literal '0' is passed we should warn? This could potentially introduce false positives..
ayartsev updated this revision to Diff 17662.Dec 28 2014, 6:28 PM

"Have you added a test to make sure that is the case? (If yes, what's the function name. I could not easily find it.) It would be similar to the path-sensitive check you've added but the check against '0' would be done in an inlined function."
Done, 'void testMallocIdc(int i)' added.

"Looks like you've removed the checks from the unix checker along with the tests. Can you add the tests back? I suggest adding your checker to the test file that contains the existing tests and check that it produces the same exact output."
Done.

"char *foo = realloc(ptr, 0); // TODO:expected-warning{{Call to 'realloc()' has an allocation size of 0 bytes}}
Why should we warn in this case? It is possible that someone is trying to free the memory by calling "realloc(ptr, 0)"
It is unsafe to use the value returned from an allocation function that was asked for a zero-allocation ( C11 n1570, 7.22.3.1 ).
I changed the checkers behavior to those we've discussed above:

  1. if the returned value of realloc(p, 0) is not assigned then silently treat it as simple 'free' (the old behavior of unix.Malloc)
  2. if the returned value is assigned then fire '0 size allocation' warning as the usage of the returned pointer is not safe.

Additional changes in the new patch:

  • renamed 'BasicAllocationCheck' to 'ZeroAllocationCheck' (MallocChecker.cpp)
  • cut old zero-allocation tests that depended on the memory returned from zero-realloc:
  • reallocSizeZero1(), reallocSizeZero2() - tested specific case when we try to free memory that was already freed by zero-realloc, in the case when reallocation failed.
  • reallocSizeZero3, reallocSizeZero4, reallocSizeZero5 - tested that we don't track memory returned from zero-allocated realloc().

If the patch gets in as is all this tests will produce 'zero-size allocation warning' at the point 'char *r = realloc(p, 0);'
There is no need to modify an old code that handles 'realloc', all the code is still in use. Nothing special is done to stop tracking memory returned from zero-realloc, in case of zero-realloc we just return from the function that handles 'realloc' (MallocChecker::ReallocMem()) before the code that allocates memory and processes the return value.

Quuxplusone added inline comments.
lib/StaticAnalyzer/Checkers/MallocChecker.cpp
222–226

This should presumably be "Zero", not "Zerro".

test/Analysis/malloc-annotations.c
51 ↗(On Diff #17662)

FWIW, this maybe should give the same warning as realloc(p,1); on a line by itself: namely, because the returned pointer is never freed, this code has a memory leak on implementations where realloc(p,0) returns a non-null pointer.

If the returned pointer is captured (q = realloc(p,0);) and later freed (free(q);), there is no bug. It's not unsafe to use the return value of realloc; it's perfectly safe and well-defined. The only implementation-defined aspect of realloc is whether the returned pointer is null or not. Either way, *using* the returned pointer is fine — it's not like using a freed pointer, which is indeed undefined behavior.

ayartsev added inline comments.Dec 28 2014, 8:36 PM
lib/StaticAnalyzer/Checkers/MallocChecker.cpp
222–226

Thanks!

test/Analysis/malloc-annotations.c
51 ↗(On Diff #17662)

FWIW, this maybe should give the same warning as realloc(p,1); on a line by itself: namely, because the returned pointer is never freed, this code has a memory leak on implementations where realloc(p,0) returns a non-null pointer.
Agree.

It's not unsafe to use the return value of realloc; it's perfectly safe and well-defined. *using* the returned pointer is fine — it's not like using a freed pointer, which is indeed undefined behavior.
It's not a safety check, just a warning that there may be an error in the code. The usage of a pointer returned from realloc(p,0) is suspicious, any bytes in the new object have indeterminate values also modifying the contents the returned memory is an error.
Zero size may be requested by reason of an error when the second parameter to realloc() is a variable.

Anton, Can you add the quote from the standard somewhere in the comments explaining the zero allocation warning?

test/Analysis/malloc-annotations.c
51 ↗(On Diff #17662)

I don't think we should be warning when free is called on the returned value from realloc(p,0). We should try and stay away from reporting issues that would lead to known false positives.

We could whitelist various ways the pointer could be dereferenced and warn on those or just not warn on realloc(p,0). (I think passing '0' to other allocation functions would not likely trigger false positives.)

Anton, what do you think?

Added the comment (inlined comment at MallocChecker.cpp, Line 259). OK to commit?

lib/StaticAnalyzer/Checkers/MallocChecker.cpp
259

Additional comment:

The request for zero bytes is suspicious.
C11 n1570, 7.22.3.1 "If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object."

test/Analysis/malloc-annotations.c
51 ↗(On Diff #17662)

Currently we do not warn whether free() is called on the value returned from realloc(p, 0) or not. This is an old unmodified behavior.
The proposition is to warn if free() is *not* called on the returned value from realloc(p, 0) (other allocation functions as well) if the value is non-zero. May it is not a memory leak but I guess it may be some kind of a resource leak in case of a nonzero return because an information about a zero-allocated memory must be stored somewhere. Perhaps it is a good idea for a further enhancement.

We could whitelist various ways the pointer could be dereferenced and warn on those or just not warn on realloc(p,0). (I think passing '0' to other allocation functions would not likely trigger false positives.)
The usage of a pointer returned from a zero-allocation request is a matter of another potentional checker - ZeroAllocDereference which I plan to develop in the future. The goal of the current check is just to detect a zero-allocation. The current patch just moves '0 size allocation' check from unix.API to unix.Malloc with a small improvement - do not warn in the case when a pointer is not consumed. Perhaps as a next step it make sense to separate the check from the global unix.Malloc to a separate check - ZeroAlloc in order to make it switchable.