Page MenuHomePhabricator

Even more warnings utilizing gsl::Owner/gsl::Pointer annotations
ClosedPublic

Authored by xazax.hun on Jul 22 2019, 5:37 PM.

Details

Summary

This patch introduces even more warnings some of them are very compelling. Look at the loop example, it supposed to catch a lot of errors early.

These warnings has been run on more than 300 open source projects. They found a few true positives and no false positives so far.

Diff Detail

Repository
rL LLVM

Event Timeline

xazax.hun created this revision.Jul 22 2019, 5:37 PM

I have run this successfully on ~192 open source projects.

The results so far:

1.
https://github.com/ANTsX/ANTs/blob/master/Examples/sccan.cxx#L2899
/home/tgahor/data/projects/ANTs/Examples/sccan.cxx:2899:34: warning: object backing the pointer will be destroyed at the end of the full-expression [-Wdangling]
        const char *longName = ( ( *it )->GetLongName() ).c_str();
                                 ^~~~~~~~~~~~~~~~~~~~~~
This seems to be a true positive given how LongName is defined with this macro: https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Core/Common/include/itkMacro.h#L944
Here: https://github.com/ANTsX/ANTs/blob/15f4e4013ed33b9d226c8d1b7d9509b2a8b19ba2/Utilities/antsCommandLineOption.h#L179
This problem has also other instances in this project. True positives, yay!

2.
https://github.com/mgbellemare/Arcade-Learning-Environment/blob/master/ale_python_interface/ale_c_wrapper.h#L10
In file included from /home/tgahor/data/projects/Arcade-Learning-Environment/ale_python_interface/ale_c_wrapper.cpp:1:
/home/tgahor/data/projects/Arcade-Learning-Environment/ale_python_interface/ale_c_wrapper.h:10:68: warning: returning address of local temporary object [-Wreturn-stack-address]
  const char *getString(ALEInterface *ale, const char *key){return ale->getString(key).c_str();}
                                                                   ^~~~~~~~~~~~~~~~~~
This also is a true positive!

3.
https://github.com/assimp/assimp/blob/master/code/Step/STEPFile.h#L910
/home/tgahor/data/projects/assimp/./code/Step/STEPFile.h:910:26: warning: returning address of local temporary object [-Wreturn-stack-address]
                return (*it).second;
                         ^~
/home/tgahor/data/projects/assimp/./code/Step/STEPFile.h:908:45: note: via initialization of variable 'it' here
            const ObjectMap::const_iterator it = objects.find(id);

This one seems to be a false positive, will look into what is the root cause. Very similar false positives appear on jsoncpp, bamtools, gtest, cppcheck, Urho3D projects.

4.
https://github.com/openscenegraph/OpenSceneGraph/blob/master/include/osg/io_utils#L69
/home/tgahor/data/projects/OpenSceneGraph/include/osg/io_utils:69:51: warning: returning address of local temporary object [-Wreturn-stack-address]
        inline const char* c_str() const { return sstream.str().c_str(); }
                                                  ^~~~~~~~~~~~~
This also is a true positive!

Conclusion: it does find true positives! All of the false positives should have the same root cause and I will look into fixing that soon. After the fix I will rerun this on even more projects, but I think this already starts to show the value of these warnings. I believe we will be able to find all the errors I reported and have 0 false positives.

xazax.hun edited the summary of this revision. (Show Details)
  • Rebase, add the results of testing on real world projects to the description.
xazax.hun marked an inline comment as done.Jul 29 2019, 11:35 AM
xazax.hun added inline comments.
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

If we want to relax the warnings to give more results we could extend the checking of these overloaded operators for annotated types. But this would imply that the user need to have the expected semantics for those types and can only suppress false positives by removing some gsl:::owner/poinnter annotations.

mgehre added inline comments.Jul 30 2019, 10:59 AM
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

I see those options:

  • Either gsl::Owner implies some specific form of those operators (and if that does not hold for a class, then one should not annotate it with gsl::Owner)
  • or gsl::Owner only implies some specific behavior for the "gsl::Pointer constructed from gsl::Owner" case and everything else requires additional annotation

I expect that we will experiment a bit in the future to see what works well for real-world code.

gribozavr added inline comments.Aug 6 2019, 4:02 AM
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

I understand the difficulty, but I don't think it is appropriate to experiment by ourselves -- these attributes are defined in a spec, and if something is not clear, the spec should be clarified.

xazax.hun marked an inline comment as done.Aug 6 2019, 6:56 AM
xazax.hun added inline comments.
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

This is exactly what is going to happen but I think it would be unfortunate to stall the progress until the new version of the spec materializes. The idea is to keep the implementations and the specs in sync, but as Herb has other projects too, it takes some time to channel the experience back into the spec. As the current version of the warnings found true positives in real world projects and we have yet to see any false positives I would prefer to move forward to maximize utility.

gribozavr added inline comments.Aug 6 2019, 7:43 AM
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

I don't understand how different implementations can ever converge in that case.

If this language extension is not sufficiently designed yet, maybe it is not ready for inclusion in Clang?

xazax.hun marked an inline comment as done.Aug 6 2019, 8:32 AM
xazax.hun added inline comments.
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

The MSVC implementation does not support user defined annotations yet, so we are the first one to ask such questions like is it valid for an user to annotate a type as gsl::Pointer and have an overloaded deref operator with functionality other than accessing the pointee. We already forwarded these concerns to Herb, and he promised to clarify these things in the paper. Once it is clarified, MSVC will also follow it. Since this code will not reach the wider audience until Clang 10 is released and it is pretty easy to change this detail I do not see the justification to postpone the inclusion.

If we postpone the inclusion over and over we will never get enough experience from real world users to ever have enough confidence.

xazax.hun marked an inline comment as done.Aug 8 2019, 12:46 PM
xazax.hun added inline comments.
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

Ok, after discussing this with Herb the conclusion is the following. The paper does not have any requirements for any of the methods for Pointers or Owners. If a method has a semantics that is not a good match with the default rules the user can annotate the methods. As method annotations are coming later, the right approach is to only rely on the semantics of these operators for STL types for now, exactly what is implemented in this patch.

And again, just to make it clear, the reason why we want to add this first rather than the annotations because the latter is a rather large patch and we want to gain more real world experience first before committing to a specific approach. Yet, all of the true positives we found so far needs these assumptions about STL types, so it really is useful to have this until we add the support for annotations.

gribozavr accepted this revision.Aug 9 2019, 6:12 AM
gribozavr added inline comments.
clang/lib/Sema/SemaInit.cpp
6581 ↗(On Diff #212207)

Okay, since this code is introducing new behavior only for std and builtin types, I think we can do it even though it is not in the spec.

This revision is now accepted and ready to land.Aug 9 2019, 6:12 AM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptAug 9 2019, 10:11 AM