Page MenuHomePhabricator

[clang-tidy] Extend 'bugprone-easily-swappable-parameters' with mixability because of implicit conversions
Needs ReviewPublic

Authored by whisperity on Feb 24 2020, 4:55 AM.

Details

Summary

Adds a relaxation option ModelImplicitConversions which will make the check report for cases where parameters refer to types that are implicitly convertible to one another.

Example:

struct IntBox { IntBox(int); operator int(); };
void foo(int i, double d, IntBox ib) {}

Implicit conversions are the last to model in the set of things that are reasons for the possibility of a function being called the wrong way which is not always immediately apparent when looking at the function (signature or call).
Implicit conversions are approximated in both directions between the parameter types, and if there is a bidirectional swap possibility, the report is made. (I.e. void bar(Base*, Derived*) {} is not reported, because a bar(DP, BP) would be a compile error.)

When a "weak" interface is diagnosed and implicit conversions are involved, the user is given a detailed breakdown (TX -> T1 -> T2 -> TB, TB -> TA, etc.) on the types the value goes through during the conversion. User-defined conversion methods are highlighted at the location with a note tag.

Unfortunately, there isn't a sensible way (at least I found none) to ask Sema at "frontend time" whether or not the two parameters could be implicitly converted between one another, as we do not have the Sema instance anymore. Hence, most of this modelling we have to do ourselves, especially for implicit conversions. Luckily, just like the original patch, it's a few lines of juggling the AST per case.

Diagnosing implicit conversions necessarily increases the number of results (both the reported length and the number of functions matched). We found in our evaluation that this increase is roughly 10%. There were a few oddballs like Bitcoin or Protobuf that produced a 20% climb. Naturally, C projects are a lot worse in this regard due to the much more proliferate use of builtin types, making a 20% increase the norm, rather than the exception.

When it comes to showing developers where their type safety is lacking, the possibility of implicit conversions is the single violator. One can argue about the impossibility of making a better design for functions that take an int, int and can't do better (this is what the Filtering patches try to capture and discard from the results). but there is hardly an excuse for an int, bool if someone wants a stronger safety.

For this very reason, the suggested default value for the check option is true, i.e. implicit conversions are considered, checked, and diagnosed.

Diff Detail

Event Timeline

whisperity created this revision.Feb 24 2020, 4:55 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 24 2020, 4:55 AM
whisperity planned changes to this revision.Feb 24 2020, 4:55 AM
whisperity added subscribers: vingeldal, zporky.

WIP because there's a lot of debug stuff that should be cleared out from here. This is a crude patch. It works fancy, but the code is crude.

whisperity retitled this revision from [WIP][clang-tidy] Approximate implicit conversion issues for the 'cppcoreguidelines-avoid-adjacent-arguments-of-same-type' check to [clang-tidy] Approximate implicit conversion issues for the 'experimental-cppcoreguidelines-avoid-adjacent-arguments-of-the-same-type' check.
whisperity edited the summary of this revision. (Show Details)
whisperity retitled this revision from [clang-tidy] Approximate implicit conversion issues for the 'experimental-cppcoreguidelines-avoid-adjacent-arguments-of-the-same-type' check to [clang-tidy] Approximate implicit conversion issues in 'experimental-cppcoreguidelines-avoid-adjacent-arguments-of-the-same-type'.
  • Re-organised code, removed debug prints, rebased, the usual tidy-up.
  • Bug fixes on certain crashes like incomplete types, conversion operator templates, etc.
  • Even more bug fixes against crashes, hopefully... sorry I lost the individual commits in a squash I think
whisperity marked 2 inline comments as done.Apr 22 2020, 9:57 AM
whisperity added inline comments.
clang-tools-extra/clang-tidy/experimental/CppcoreguidelinesAvoidAdjacentParametersOfTheSameTypeCheck.cpp
747 ↗(On Diff #259324)

'chute, I hate merge conflicts...

clang-tools-extra/docs/clang-tidy/checks/experimental-cppcoreguidelines-avoid-adjacent-parameters-of-the-same-type.rst
113–120 ↗(On Diff #259324)

This change to check the "left half" of a full graph moved to the main checker patch D69560 and there is no significant (few seconds, on large projects like LLVM) time difference between the modes at all even on large projects... so this fearmongering text should be removed.

whisperity marked 2 inline comments as not done.Apr 22 2020, 9:59 AM
whisperity retitled this revision from [clang-tidy] Approximate implicit conversion issues in 'experimental-cppcoreguidelines-avoid-adjacent-arguments-of-the-same-type' to [clang-tidy] Approximate implicit conversion issues in 'experimental-cppcoreguidelines-avoid-adjacent-parameters-of-the-same-type'.Sep 24 2020, 6:23 AM
whisperity edited the summary of this revision. (Show Details)
whisperity updated this revision to Diff 315384.Jan 8 2021, 6:48 AM

Ignore one-way implicit conversions

One-way implicit conversions introduce too much noise, and except in very odd circumstances, do not provide a meaningful result.

The stalwart example

void f(const Base& bp, const Derived& dp)

contains just one conversion, dp -> bp, which, if called in a swapped way, will 99.9...% of the cases will result in a compile error (*)

Base b;
Derived d;
f(d, b);

Thus, there is no reason to model this, at all.

(*): Unless an operator Derived() exists in Base.

whisperity planned changes to this revision.Jan 21 2021, 8:45 AM

This patch will be replaced with the refactoring of the base check.

whisperity updated this revision to Diff 327743.Mar 3 2021, 5:31 AM
whisperity retitled this revision from [clang-tidy] Approximate implicit conversion issues in 'experimental-cppcoreguidelines-avoid-adjacent-parameters-of-the-same-type' to [clang-tidy] Extend 'bugprone-easily-swappable-parameters' with mixability because of implicit conversions.
whisperity edited the summary of this revision. (Show Details)
whisperity added reviewers: hokein, alexfh, njames93.
whisperity set the repository for this revision to rG LLVM Github Monorepo.
  • Rebased over updated, rewritten, newer base patch version, vastly increased the quality of the code.
  • Tidied the tests and made them more self-explanatory.
  • Implemented handling function pointer conversion (implicit losing of noexcept). This was missing from the previous uploaded patch.
  • Ensured that implicit conversion sequences are diagnosed properly, even when typedefs are involved. This means that if the conversion takes 4-5-6 logical steps (e.g. instead of SomeFancyIntTypedef -> DoubleBoxingType, emit SomeFancyIntTypedef -> int -> double -> const double -> DoubleBoxingType (user defined ctor)) it's all explained properly.
  • Respect the user's decision about the mixability of T and const T (the previous patch D96355) even in the context of implicit conversions (i.e. if the user did not allow that, do not consider a user-defined type which constructor takes a const mixable with a non-const other parameter).
  • Put a diagnostic note to the location of the FuntionDecl of the conversion method involved in the mixability.
whisperity updated this revision to Diff 327761.Mar 3 2021, 5:53 AM

After reading through the diff as rendered by Phabricateur, I realised I left in a few crappy debug prints and commented-out values that are not needed in the end. This is fixed now.

whisperity added inline comments.Mar 16 2021, 12:17 PM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
216

Actually this is one of those debug prints that should be removed and remained in here by accident.

686–696

@aaron.ballman Getting ahead of the curve here. I understand this is ugly. Unfortunately, no matter how much I read the standard, I don't get anything of "canonical type" mentioned, it seems to me this concept is something inherent to the model of Clang.

Basically why this is here: imagine a void (*)() noexcept and a void (*)(). One's noexcept, the other isn't. Inside the AST, this is a ParenType of a PointerType to a FunctionProtoType. There exists a one-way implicit conversion from the noexcept to the non-noexcept ("noexceptness can be discarded", similarly to how "constness can be gained")
Unfortunately, because this is a one-way implicit conversion, it won't return on the code path earlier for implicit conversions.

Now because of this, the recursive descent in our code will reach the point when the two innermost FunctionProtoTypes are in our hands. As a fallback case, we simply ask Clang "Hey, do you think these two are the same?". And for some weird reason, Clang will say "Yes."... While one of them is a noexcept function and the other one isn't.

I do not know the innards of the AST well enough to even have a glimpse of whether or not this is a bug. So that's the reason why I went ahead and implemented this side-stepping logic. Basically, as the comment says, it'lll force the information of "No matter what you do, do NOT consider these mixable!" back up the recursion chain, and handle it appropriately later.

clang-tools-extra/test/clang-tidy/checkers/bugprone-easily-swappable-parameters-implicits.cpp
251–261

Now here the warning is justified, however. All these disinfectant-stinking examples are to ensure that. In this case, in our hands, on the first level, we got the function pointer, and the class. And it is the implicit conversion modeller that will eventually ask "Can I give that noexcept function to that constructor taking a non-noexcept?" and in that case, on the level of function pointers, the answer is yes, so there will be a warning made.

clang-tools-extra/test/clang-tidy/checkers/bugprone-easily-swappable-parameters-len2.cpp
341–342

Without this side-stepping logic mentioned above, even without this patch (so back to the "Ye Olde Strict Type Equality" mode), this particular function would be warned about, which is a definite false positive.

aaron.ballman added inline comments.Mar 18 2021, 8:54 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
401

Should this be MIX_WorkaroundDisableCanonicalEquivalence instead of MIX_None?

511–512
529–530
549–551

What about a DecayedType?

686–696

Unfortunately, no matter how much I read the standard, I don't get anything of "canonical type" mentioned, it seems to me this concept is something inherent to the model of Clang.

It is more of a Clang-centric concept. Basically, a canonical type is one that's had all of the typedefs stripped off it.

Now because of this, the recursive descent in our code will reach the point when the two innermost FunctionProtoTypes are in our hands. As a fallback case, we simply ask Clang "Hey, do you think these two are the same?". And for some weird reason, Clang will say "Yes."... While one of them is a noexcept function and the other one isn't.

I think a confounding factor in this area is that noexcept did not used to be part of the function type until one day it started being a part of the function type. So my guess is that this is fallout from this sort of thing: https://godbolt.org/z/EYfj8z (which may be worth keeping in mind when working on the check).

787
whisperity marked 5 inline comments as done.Mar 18 2021, 9:17 AM
whisperity added inline comments.
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
401

In this patch I changed the value of None to be 2, and Workaround... became 1. So everything above None means it's a valid mixability (somehow), of course only after sanitize() so Workaround... | CanonicalType becomes None.

aaron.ballman added inline comments.Mar 31 2021, 11:12 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
212
216

Agreed, this one should be removed.

whisperity set the repository for this revision to rG LLVM Github Monorepo.
  • Fixed a few nits.
  • Rebased, and added highlighting (under-squiggly) the parameter/return type of conversion operators for extra emphasis.
whisperity marked 3 inline comments as done.Apr 10 2021, 11:30 AM

Bump. 🙂 This is the last part that's still unreviewed and also the most complex.

martong added inline comments.Wed, Jun 2, 9:34 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
686–696

About noexcept: we've faced a similar problem in the ASTImporter library. We could not import a noexcept function's definition if we already had one without the noexcept specifier.

Thus, in ASTStructuralEquivalence.cpp we do differentiate the function types based on their noexcept specifier (and we even check the noexcept expression).:

TEST_F(StructuralEquivalenceFunctionTest, Noexcept) {
  auto t = makeNamedDecls("void foo();",
                          "void foo() noexcept;", Lang_CXX11);
  EXPECT_FALSE(testStructuralMatch(t));
}
TEST_F(StructuralEquivalenceFunctionTest, NoexceptNonMatch) {
  auto t = makeNamedDecls("void foo() noexcept(false);",
                          "void foo() noexcept(true);", Lang_CXX11);
  EXPECT_FALSE(testStructuralMatch(t));
}

May be in these special cases it would worth to reuse the ASTStructuralEquivalenceContext class?

whisperity added inline comments.Thu, Jun 3, 12:17 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
686–696

Definitely, good catch, @martong, thank you very much! @aaron.ballman, what do you think? If I see this right, StructuralEquivalenceContext is part of libClangAST so should be readily available.

The only issue I'm seeing is that this class takes non-const ASTContext and Decl nodes...

martong added inline comments.Thu, Jun 3, 6:29 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
122

FB stands for FunnyBitmask? Could you please either describe that in a comment or just spell it out?

182

Maybe it's just me, but AfterPre sounds ambiguous and AfterPost seems redundant.

210

So, we can never return with a vector with size > 4?

clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.h
42

"qualified"
Do you consider volatile here as well, or just const?

whisperity added inline comments.Thu, Jun 3, 6:33 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
686–696

Well, I started looking into putting up const whereever possible into the aforementioned type, but I hit a hurdle. When checking equivalence of records, the algorithm tries to ASTImport-complete the definition of a record if it's not fully defined yet... which is not a const method... I'll look into it some more, maybe we can do an overload on whether you're Structual equivalence checking in a const context or a non-const context.

whisperity added inline comments.Thu, Jun 3, 6:40 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
122

FlagBit. @alexfh suggested in the base check to use hexa literals. I'm not too sold about that, but maybe we can cut the macro out and keep the bit shift instructions in. Given that the check has more or less earned its final form (for now), we know how many bits we need!

182

Should I use "AfterFirstStandard" or "AfterFirstStd"?

AfterPost isn't redundant. An implicit conversion sequence might be of 3 steps. Consider in a hypothetical void f(Complex x, const double d).

Complex ---(PRE: std qual adjustment)--> const Complex ---(UD: user defined conv)--> double ---(POST: std qual adjustment)--> const double

And because in case of many conversions a lot of combinations need to be checked, we can't have AfterPost be the same as End. First, because of the combinations, second, because of the other things the check is doing.

void g(ComplexTypedef CT, ConstDoubleTypedef CDT);

In this case, Start and End are the typedefs, and the inner sequence is the same as before. And in order to generate the diagnostic, we also need to have both pieces of information.

210

N in SmallVector<T, N> only specifies the pre-allocated smallness size. AFAIK, if you have >N elements, it will just put the vector out onto the heap.

clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.h
42

Const, volatile, and in C mode, restrict, too.

martong added inline comments.Fri, Jun 4, 6:48 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
122

FlagBit, I like it, perhaps we should rename FB to FlagBit?

182

Thanks for this explanation, it makes clearer! However, this highlights that a way better description (in the form of comments) is needed for these QualType members. Especially this line could really increase the readability.

Complex ---(PRE: std qual adjustment)--> const Complex ---(UD: user defined conv)--> double ---(POST: std qual adjustment)--> const double

In this case, Start and End are the typedefs, and the inner sequence is the same as before. And in order to generate the diagnostic, we also need to have both pieces of information.

I think it would be useful to document the cases with examples when End is not equal to AfterPost and such.

210

I meant the number 4, not SmallVector. So, is there a case when the conversion sequence is longer than 4? If it can be longer, then where do you store the types, you have 4 QualType members if I am not wrong. (Also, I am not an expert of conversions.)

To be honest, it is so terrible that we cannot reuse clang::StandardConversionSequence

clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.h
42

Great!

Perhaps all conversion related logic should go into their own implementation file? Seems like it adds up to roughly 1000 lines.

clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
1171

I think that these overloaded operators are not elevating the readability, since we are not dealing with mathematical classes.
IMHO, we could simply use update or something that is less crypto.

martong added inline comments.Fri, Jun 4, 7:02 AM
clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
550–551

Is this still WIP or do you use the DEBUG printouts in the tests? If not then could you please create a Diff without the DEBUG printouts, that could increase readability and ease the review.

whisperity marked 5 inline comments as done.Fri, Jun 4, 7:08 AM

Perhaps all conversion related logic should go into their own implementation file? Seems like it adds up to roughly 1000 lines.

That's against the usual conventions of the project (1 TU for 1 check implementation). The methods are grouped by namespace, you can fold it up in your editor.

clang-tools-extra/clang-tidy/bugprone/EasilySwappableParametersCheck.cpp
550–551

As discussed earlier (and perhaps in the previous patches), these debug printouts are needed and shall stay here. The output is not used in automation, but it's intended for people who might pick up on further developing the check later. The recursive descent is too complex to be handleable in your mind without the debug information.