[analyzer] Fix extent modeling for casted operator new values.

[NoQ]

Summary

This continues the series of fine-tuning of how everything behaves in -analyzer-config c++-allocator-inlining=true mode with respects to casts (ElementRegions with index 0) around the return value of operator new(). Old operator new() was always returning &HeapSymRegion{conj_$N{C *}}, where C is the type of the allocated object. In the new mode, if the operator is evaluated conservatively, this value changes to &element{T, 0 S64b, HeapSymRegion{conj_$N{C *}}}, which, as pointed out in D41250#959755, is not quite intended (but not addressed yet). However, if the operator is inlined, it is likely to become &element{T, 0 S64b, HeapSymRegion{conj_$N{void *}}} (note the void), where the cast is definitely intended. It means that regardless of how do we want to treat the no-op cast in the non-inlined case, the checker does not have a right to rely on the cast being absent.

This patch fixes the region extent modeling under -analyzer-config c++-allocator-inlining=true, which is performed by MallocChecker and consumed by other checkers such as ArrayBoundChecker. It is pointless to model the extent of the element region, when in fact we're trying to model the extent of the whole array. Note that behavior of the array operator new[] changes slightly, even if we're not trying to model it.

[NoQ]

is not quite intended (but not addressed yet)

D41250#971888 addresses the issue.

However, if the operator is inlined

It isn't. Because if it is inlined, MallocChecker bails out immediately and doesn't try to model the extent - believing that we'd do a better job at that while modeling the actual allocator in the inlined function.

lib/StaticAnalyzer/Checkers/MallocChecker.cpp

if (NE->isArray()) {
     // Store the extent size for the (symbolic)region
     // containing the elements.
     Region = Target.getAsRegion()
                  ->getAs<SubRegion>()
                  ->getSuperRegion()
                  ->getAs<SubRegion>();
   } else {
     ElementCount = svalBuilder.makeIntVal(1, true);
-    Region = Target.getAsRegion()->getAs<SubRegion>();
+    Region = cast<SubRegion>(Target.getAsRegion()->StripCasts());
   }
   assert(Region);
 
   // Set the region's extent equal to the Size in Bytes.
   QualType ElementType = NE->getAllocatedType();
   ASTContext &AstContext = C.getASTContext();
   CharUnits TypeSize = AstContext.getTypeSizeInChars(ElementType);
 

test/Analysis/out-of-bounds-new.cpp

 // RUN: %clang_analyze_cc1 -std=c++11 -Wno-array-bounds -analyzer-checker=unix,core,alpha.security.ArrayBoundV2 -verify %s
+// RUN: %clang_analyze_cc1 -std=c++11 -Wno-array-bounds -analyzer-checker=unix,core,alpha.security.ArrayBoundV2 -analyzer-config c++-allocator-inlining=true -verify %s
 
 // Tests doing an out-of-bounds access after the end of an array using:
 // - constant integer index
 // - constant integer size for buffer
 void test1(int x) {
   int *buf = new int[100];
   buf[100] = 1; // expected-warning{{Out of bound memory access}}
 }