This is an archive of the discontinued LLVM Phabricator instance.

[clang-tidy] Add the abseil-duration-subtraction check
ClosedPublic

Authored by hwright on Dec 3 2018, 7:30 PM.

Details

Summary
foreach ($list as $item) {
  work_miracles($item);
}

This check uses the context of a subtraction expression as well as knowledge about the Abseil Time types, to infer the type of the second operand of some subtraction expressions in Duration conversions. For example:

absl::ToDoubleSeconds(duration) - foo

can become

absl::ToDoubleSeconds(duration - absl::Seconds(foo))

This ensures that time calculations are done in the proper domain, and also makes it easier to further deduce the types of the second operands to these expressions.

Diff Detail

Repository
rL LLVM

Event Timeline

hwright created this revision.Dec 3 2018, 7:30 PM
hwright updated this revision to Diff 176536.Dec 3 2018, 7:33 PM

Fix docs.

JonasToth retitled this revision from Add the abseil-duration-subtraction check to [clang-tidy] Add the abseil-duration-subtraction check.Dec 4 2018, 4:51 AM
JonasToth edited the summary of this revision. (Show Details)
JonasToth added reviewers: alexfh, hokein.
JonasToth set the repository for this revision to rCTE Clang Tools Extra.
JonasToth added a project: Restricted Project.
hwright edited the summary of this revision. (Show Details)Dec 4 2018, 6:35 AM
Eugene.Zelenko added inline comments.
docs/clang-tidy/checks/abseil-duration-subtraction.rst
7 ↗(On Diff #176536)

Please fix double space.

hwright updated this revision to Diff 176757.Dec 4 2018, 7:42 PM
hwright marked an inline comment as done.

Fix double space.

Ping.

I assume I've got the right reviewers here, but I've also been sending a bunch of stuff your way lately, so if I'm overwhelming review capacity, please let me know.

Ping.

I assume I've got the right reviewers here, but I've also been sending a bunch of stuff your way lately, so if I'm overwhelming review capacity, please let me know.

Hi hyrum, you have the right reviews. I do not have review capacity today, but at the end of the week. We usually ping after a few days (~1 week), e.g. for me its a hobby and I need to work :)
If you have more patches in the pipeline, you can already work on them/push them to review. It is easier to review a bit more in one piece and generates higher throughput too.

If other reviewers have time, go for it :)

hokein accepted this revision.Dec 7 2018, 2:34 AM
hokein added subscribers: astrelni, ahedberg.

The check looks good from my side, except one nit.

I assume I've got the right reviewers here, but I've also been sending a bunch of stuff your way lately, so if I'm overwhelming review capacity, please let me know.

I think in long run, it would be desired to have Abseil experts participate in reviews of abseil-related checks (upstream clang-tidy reviewers don't have much knowledge about Abseil). Abseil team members (@astrelni, @ahedberg) might be more interested in theses checks which could accelerate the review process.

test/clang-tidy/abseil-duration-subtraction.cpp
4 ↗(On Diff #176757)

We have multiple implementations in other abseil-duration-* tests, I think we could pull them out (in test/clang-tidy/Inputs/absl/time/time.h).

This revision is now accepted and ready to land.Dec 7 2018, 2:34 AM

Please always upload all patches with the full context (-U99999)

clang-tidy/abseil/DurationRewriter.cpp
35 ↗(On Diff #176757)

https://llvm.org/docs/ProgrammersManual.html#other-map-like-container-options

We never use hash_set and unordered_set because they are generally very expensive (each insertion requires a malloc) and very non-portable.

Since the key is an enum, [[ https://llvm.org/docs/ProgrammersManual.html#llvm-adt-indexedmap-h | llvm/ADT/IndexedMap.h ]] should be a much better fit.

68 ↗(On Diff #176757)

if (const auto *MaybeCallArg

68 ↗(On Diff #176757)

Early return?

74 ↗(On Diff #176757)

So you generate fix-it, and then immediately degrade it into a string. Weird.

hwright updated this revision to Diff 177266.Dec 7 2018, 11:41 AM
hwright marked 8 inline comments as done.
hwright added inline comments.
clang-tidy/abseil/DurationRewriter.cpp
35 ↗(On Diff #176757)

It doesn't look like IndexedMap has a constructor which takes an initializer list. Without it, this code get a bit more difficult to read, and I'd prefer to optimize for readability here.

68 ↗(On Diff #176757)

I'm not quite sure what you mean by Early return? Are you suggesting that the call to selectFirst should be pulled out of the if conditional, and then the inverse checked to return llvm::None first?

74 ↗(On Diff #176757)

This doesn't generate a fix-it, it just fetches the text of the given node as a StringRef but we're returning a string, so we need to convert.

Is there a more canonical method I should use to fetch a node's text?

lebedev.ri added inline comments.Dec 7 2018, 12:06 PM
clang-tidy/abseil/DurationRewriter.cpp
156 ↗(On Diff #177266)

Are you very sure this shouldn't be [[ https://llvm.org/docs/ProgrammersManual.html#llvm-adt-stringmap-h | StringMap ]]?

35 ↗(On Diff #176757)

The manual still 'recommends' not to use them.
Simple solution: immediately invoked lambda
Better solution: try to add such constructor to IndexedMap.

68 ↗(On Diff #176757)

Ah, nevermind then.

74 ↗(On Diff #176757)

I don't know the answer, but have you tried looking what tooling::fixit::getText() does internally?

hwright updated this revision to Diff 177325.Dec 7 2018, 2:39 PM
hwright marked 7 inline comments as done.

Use an IndexedMap instead of an std::unordered_map

clang-tidy/abseil/DurationRewriter.cpp
156 ↗(On Diff #177266)

Nope. Thanks for the catch!

35 ↗(On Diff #176757)

In hopes of not making this too much of a yak shave, I've gone with the immediately invoked lambda.

74 ↗(On Diff #176757)

I have. It calls internal::getText, which, from the namespace, I'm hesitant to call in this context.

Reminder: I'll need somebody to submit this for me, since I don't have subversion access.

lebedev.ri added inline comments.Dec 10 2018, 4:26 AM
clang-tidy/abseil/DurationRewriter.cpp
20–39 ↗(On Diff #177325)

You should not need this if you change the enum instead.

66–77 ↗(On Diff #177325)

I was thinking of more like

for(std::pair<??, ??> e : std::initializer_list({
                           {DurationScale::Hours, std::make_pair("::absl::ToDoubleHours", "::absl::ToInt64Hours")}.
                           }))
  InverseMap[e.first] = e.second;
clang-tidy/abseil/DurationRewriter.h
22–23 ↗(On Diff #177325)
enum class DurationScale : std::int8_t {
  Hours = 0,
  Minutes,
...
hwright updated this revision to Diff 177509.Dec 10 2018, 7:35 AM
hwright marked 5 inline comments as done.

Use static_cast instead of a switch for IndexedMap lookup.

clang-tidy/abseil/DurationRewriter.cpp
20–39 ↗(On Diff #177325)

The function is still required; the switch can be removed with a static_cast.

66–77 ↗(On Diff #177325)

The compilable construction looks something like:

for (const auto &e : {std::make_pair(DurationScale::Hours,
                                    std::make_pair("::absl::ToDoubleHours",
                                                   "::absl::ToInt64Hours"))})

Which is a bit more verbose than just assigning values to the map (and not any more efficient), so I've just left this bit as-is.

JonasToth added inline comments.Dec 10 2018, 8:34 AM
clang-tidy/abseil/DurationRewriter.cpp
21 ↗(On Diff #177509)

Are you using argument_type? Browser searching did only show one result.

23 ↗(On Diff #177509)

Why not std::uint8_t as its the underlying type for the enum?

73 ↗(On Diff #177509)

maybe in the name is redundant, as its return type is Optional

77 ↗(On Diff #177509)

In Principle the Node could have multiple expressions that are a call if there is nesting.
The transformation is correct from what I see right now, but might result in the necessity of multiple passes for the check. (Is the addition version affected from that too?)

Given the recursive nature of the matcher you could construct a nesting with the inner part being a subtraction, too. The generated fixes would conflict and none of them would be applied. At least thats what I would expect right now. Please take a look at this issue.

test/clang-tidy/Inputs/absl/time/time.h
1 ↗(On Diff #177509)

I think having the extraction of the common test-stuff into this header as one commit would be better. Would you prepare such a patch? I can commit for you. It probably makes sense if you ask for commit access (https://llvm.org/docs/DeveloperPolicy.html#obtaining-commit-access). Do as you wish.

test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

From this example starting:

  • The RHS should be a nested expression with function calls, as the RHS is transformed to create the adversary example i mean in the transformation function above.
absl::ToDoubleSeconds(d) - absl::ToDoubleSeconds(absl::ToDoubleSeconds(d) - absl::ToDoubleSeconds(d1));

I think you need the proper conversion function, as the result of the expression is double and you need a Duration, right?

But in principle starting from this idea the transformation might break.

hwright updated this revision to Diff 177590.Dec 10 2018, 1:03 PM
hwright marked 9 inline comments as done.

Add tests

clang-tidy/abseil/DurationRewriter.cpp
21 ↗(On Diff #177509)

This is required by IndexedMap, if I understand correctly.

23 ↗(On Diff #177509)

This is required by IndexedMap, if I understand correctly.

77 ↗(On Diff #177509)

There isn't an imminent addition version at this point.

This matcher isn't recursive: it's just looking at the entire node to see if it is a call to the inverse function. If an inverse is embedded as part of a deeper expression, it won't see it (e.g., there no hasDescendant in this matcher).

test/clang-tidy/Inputs/absl/time/time.h
1 ↗(On Diff #177509)

I can do this, but it might take a bit to get the commit bit turned on.

test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

I think there may be some confusion here (and that's entirely my fault. :) )

We should never get this expression as input to the check, since it doesn't compile (as you point out):

absl::ToDoubleSeconds(d) - absl::ToDoubleSeconds(absl::ToDoubleSeconds(d) - absl::ToDoubleSeconds(d1));

Since absl::ToDoubleSeconds requires that its argument is an absl::Duration, but the expression absl::ToDoubleSeconds(d) - absl::ToDoubleSeconds(d1) results in a double, we can't get this as input.

There may be other expressions which could be input, but in practice they don't really happen. I've added a contrived example to the tests, but at some point the tests get too complex and confuse the fix matching infrastructure.

hwright marked an inline comment as done.Dec 10 2018, 1:03 PM
JonasToth added inline comments.Dec 10 2018, 1:16 PM
clang-tidy/abseil/DurationRewriter.cpp
77 ↗(On Diff #177509)

Matchers are recursive. There will be a next match of the inner nodes below this node, just by AST traversal.

clang-tidy/abseil/DurationSubtractionCheck.cpp
38 ↗(On Diff #177590)

Loc.isInvalid() too. You can pass in macros from the command line that result in invalid sourcelocations.

test/clang-tidy/Inputs/absl/time/time.h
1 ↗(On Diff #177509)

Maybe, I (or someone else) have no problem commiting for you.

test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

Your last sentence is the thing ;) Murphies Law will hit this check, too. In my opinion wrong transformations are very unfortunate and should be avoided if possible (in this case possible).
You can simply require that the expression of type double does not contain any duration subtraction calls.

This is even possible in the matcher-part of the check.

hwright updated this revision to Diff 177749.Dec 11 2018, 11:28 AM
hwright marked 6 inline comments as done.

Rebase

test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

I've written a test (which the testing infrastructure fails to handle well, so I haven't included it in the diff), and it produces these results:

   //
   //
-  x = absl::ToDoubleSeconds(d) - (absl::ToDoubleSeconds(d1) - 5);
+  x = absl::ToDoubleSeconds(d - absl::Seconds(absl::ToDoubleSeconds(d1) - 5));
   //
   //
-  x = absl::ToDoubleSeconds(d - absl::Seconds(absl::ToDoubleSeconds(d1) - 5));
+  x = absl::ToDoubleSeconds(d - absl::Seconds(absl::ToDoubleSeconds(d1 - absl::Seconds(5))));

Those results are correct. There is a cosmetic issue of round tripping through the double conversion in the absl::Seconds(absl::ToDoubleSeconds(...)) phrase, but untangling that is 1) difficult (because of order of operations issues) and thus; 2) probably the subject of a separate check.

This is still such a rare case (as in, we've not encountered it in Google's codebase), that I'm not really concerned. But if it's worth it to explicitly exclude it from the traversal matcher, I can do that.

JonasToth added inline comments.Dec 12 2018, 9:05 AM
test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

Can you say what the direct issue is? I would bet its the overlapping?
A note in the documentation would be ok from my side. When the conflicting transformations are tried to be applied clang-tidy does not crash but print a nice diagnostic and continue its life?

hwright marked 2 inline comments as done.Dec 12 2018, 6:36 PM
hwright added inline comments.
test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

Another example I've verified:

-  x = absl::ToDoubleSeconds(d) - absl::ToDoubleSeconds(absl::Seconds(absl::ToDoubleSeconds(d1) - 5));
+  x = absl::ToDoubleSeconds(d - absl::Seconds(absl::ToDoubleSeconds(d1) - 5));

This a nested case, and while clang-tidy finds both of them, it only applies the outer most one (presumably the one it finds first in its AST traversal):

note: this fix will not be applied because it overlaps with another fix

The new code can then be checked again to fix the internal instance.

It's not possible to express this case in a test because testing infrastructure uses regular expressions, and the repeated strings in the test expectation cause it to get a bit confused.

Given all the of the above, I'm unsure what content would go in the documentation which is specific to this check.

Did you rebase the check onto the new master with your refactorings in?

test/clang-tidy/abseil-duration-subtraction.cpp
12 ↗(On Diff #177509)

Yes, that should be the outermost and the first instance it finds.
IMHO the documentation should say something along the lines of
Because it is possible the timing functions can be nested (as in one of your example) not all occurences in this single expression can be transformed in one run. Running clang-tidy multiple times will fix the nested instances, too.

The issue does not come from your check but the user will notice it through your check. And a short note for that won't hurt as it is not unreasonable someone might want to nest such expressions, even if uncommon.

hwright updated this revision to Diff 178039.Dec 13 2018, 5:29 AM
hwright marked an inline comment as done.

Rebase and update documentation

hwright marked an inline comment as done.Dec 13 2018, 5:30 AM

I've updated the documentation, and the rebased to master.

JonasToth accepted this revision.Dec 13 2018, 5:58 AM

LGTM with the doc-nit.

docs/clang-tidy/checks/abseil-duration-subtraction.rst
33 ↗(On Diff #178039)

I think it the doc should be explicit that this is the case for nesting the subtraction expression the issue can occur. Otherwise it might be a bit confusing.

Thanks for reviewing, I'll go ahead and commit.

I've also removed the hash specialization, since we moved to llvm::IndexedMap instead of std::unordered_map inside of getInverseForScale.

This revision was automatically updated to reflect the committed changes.