This is an archive of the discontinued LLVM Phabricator instance.

[clang][analyzer] Fix empty enum handling in EnumCastOutOfRange checker
ClosedPublic

Authored by gamesh411 on Jun 28 2023, 1:38 AM.

Details

Summary

The alpha.cplusplus.EnumCastOutOfRange checker previously gave many
false positives because a warning was given if the initializer value
did not appear in the enumerator list.
The strict handling caused std::byte to always give a warning, as it
is implemented as an enum class without any declarators.

Diff Detail

Event Timeline

gamesh411 created this revision.Jun 28 2023, 1:38 AM
Herald added a reviewer: NoQ. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
gamesh411 requested review of this revision.Jun 28 2023, 1:38 AM
Herald added a project: Restricted Project. · View Herald TranscriptJun 28 2023, 1:38 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript
gamesh411 retitled this revision from Relax alpha.cplusplusEnumCastOutOfRange This checker previously gave many false positives, because only the enum to Relax alpha.cplusplus.EnumCastOutOfRange checker.Jun 28 2023, 1:40 AM
gamesh411 edited the summary of this revision. (Show Details)
gamesh411 retitled this revision from Relax alpha.cplusplus.EnumCastOutOfRange checker to [WIP][clang][analyzer] Relax alpha.cplusplus.EnumCastOutOfRange checker.Jun 28 2023, 2:13 AM
gamesh411 edited the summary of this revision. (Show Details)
shafik added a subscriber: shafik.Jun 28 2023, 10:00 AM

I did not look at this in detail but I don't think this approach is correct. I fixed this for constant evaluation the other day and you can find the details here: D130058

The test suite I wrote should be sufficient for you to validate your approach against.

I think the old "report when the value stored in an enum type is not equal to one of the enumerators" behavior would be useful as an opt-in checker that could be adopted by some projects; while the new "relaxed" version is too bland to be really useful. I'd suggest keeping the old behavior in the general case, eliminating the "obvious" false positives like std::byte (don't report types that have no enumerators) and moving this checker towards the optin group.

I think the old "report when the value stored in an enum type is not equal to one of the enumerators" behavior would be useful as an opt-in checker that could be adopted by some projects; while the new "relaxed" version is too bland to be really useful. I'd suggest keeping the old behavior in the general case, eliminating the "obvious" false positives like std::byte (don't report types that have no enumerators) and moving this checker towards the optin group.

I would agree that after ignoring the obvious cases (like no enumerators), a checker like this could be useful as an "optin" checker, however, I'd not recommend moving the actual implementation anywhere.
You can create "virtual" checkers like it's done in many cases, eg. MismatchedDeallocatorChecker.

FYI I haven't checked the actual implementation thoroughly, because I had higher-level remarks inline.

clang/lib/StaticAnalyzer/Checkers/EnumCastOutOfRangeChecker.cpp
42

I can see llvm::APSInt used a few places. Consider using namespace llvm;

65–69

Why do you fold the "Pred" ExplodedNode?
I'd say, you probably didn't want to use addTransition here.
Why don't you assume the subsequent assumptions and transition only once?

Anyway, I think it's better to have the addTransition closer to the outer scope (where the error reporting is done), so that we can easily see how many ways we branch off, etc.

clang/test/Analysis/enum-cast-out-of-range.cpp
44

void sink(...); would have achieved the same and I'd say it's less complex.
sink(unscoped, scoped);

[...] I'd not recommend moving the actual implementation anywhere.

I agree, I mostly spoke figuratively, suggesting that this checker could end up in the optin group when it's eventually brought out of alpha.

clang/test/Analysis/enum-cast-out-of-range.cpp
44

Or just (void)unscoped; (void)scoped; if we're bikeshedding this test...

gamesh411 updated this revision to Diff 543413.Jul 24 2023, 1:39 AM
gamesh411 edited the summary of this revision. (Show Details)

The checker now retains the original detection logic, but only whitelists empty
enums.

As a future step the checker is moved into the optin package.

gamesh411 marked 4 inline comments as done.Jul 24 2023, 1:59 AM

I did not look at this in detail but I don't think this approach is correct. I fixed this for constant evaluation the other day and you can find the details here: D130058

The test suite I wrote should be sufficient for you to validate your approach against.

@shafik, @donat.nagy Thanks for looking at this patch.
I have checked the linked improvements in Sema, and this initial modification would only lead to a ClangSA implementation of the same error-detection logic.
Due to symbolic execution, the set of potentially detectable errors is bigger, but this is maybe not the right place for this checker.

My goal is to improve the checker and eventually bring it out of alpha.

I see one main problem with this checker:

It does not support enums with fixed underlying type, namely std::byte, which is implemented like this:

enum class byte: unsigned char {};

As std::byte has no enumerators, the checker would say any otherwise allowed non-default initialization of a std::byte instance is bad, i.e.:

std::byte b {42}; // the checker gives a warning for this

Aside from this, in the optin package, there is a place for the current state of the checker (namely that only the value mentioned in the enumerator list are OK).
I will create a separate patch to move it out of alpha and into optin package.

clang/lib/StaticAnalyzer/Checkers/EnumCastOutOfRangeChecker.cpp
42

Again, good point, I just abandoned the direction where many APSInt mentions are introduced, and the original only has 2 mentions of APSInt.

65–69

Thanks! This is a very good point; I just changed the general direction of the checker, and this implementation is no longer touched by the change.

gamesh411 retitled this revision from [WIP][clang][analyzer] Relax alpha.cplusplus.EnumCastOutOfRange checker to [clang][analyzer] Fix empty enum handling in EnumCastOutOfRange checker.Jul 24 2023, 1:59 AM
gamesh411 edited the summary of this revision. (Show Details)
gamesh411 marked 2 inline comments as done.Jul 31 2023, 9:23 AM
donat.nagy accepted this revision.Aug 3 2023, 5:13 AM

LGTM, with a little bikeshedding ;-)

clang/test/Analysis/enum-cast-out-of-range.cpp
49

Consider using "specified" and "unspecified" instead of "fixed" and "unfixed", because the rest of the test file uses them and in my opinion "unfixed" looks strange in this context. (I know that e.g. https://en.cppreference.com/w/cpp/language/enum speaks about "fixed" underlying context, but it doesn't use "unfixed".)

This revision is now accepted and ready to land.Aug 3 2023, 5:13 AM
steakhal accepted this revision.Aug 3 2023, 5:47 AM

LGTM

clang/test/Analysis/enum-cast-out-of-range.cpp
49

How about calling it "plain" or "common"?

58–60
donat.nagy added inline comments.Aug 3 2023, 5:54 AM
clang/test/Analysis/enum-cast-out-of-range.cpp
49

"plain" is also a good alternative for "unfixed" (although it would still be inconsistent with the earlier cases); but "common" is confusing, because in addition to the "plain, regular, normal" meaning it can also mean "mutual; shared by more than one <something>".

gamesh411 updated this revision to Diff 548593.Aug 9 2023, 6:10 AM

minor review fixups

This revision was landed with ongoing or failed builds.Aug 9 2023, 6:12 AM
This revision was automatically updated to reflect the committed changes.