This is an archive of the discontinued LLVM Phabricator instance.

[clang-tidy] modernize: Find usage of random_shuffle and replace it with shuffle.
ClosedPublic

Authored by madsravn on Feb 19 2017, 11:11 AM.

Details

Summary

random_shuffle was deprecated by C++14 and will be removed by C++17. This check will find and replace usage of random_shuffle with its modern counterpart (shuffle).

Diff Detail

Repository
rL LLVM

Event Timeline

madsravn created this revision.Feb 19 2017, 11:11 AM
Eugene.Zelenko added inline comments.
docs/ReleaseNotes.rst
78 ↗(On Diff #89071)

Please add std:: and enclose random_shuffle in ``.

docs/clang-tidy/checks/modernize-replace-random-shuffle.rst
6 ↗(On Diff #89071)

I think std:: should be added.

Nice check! Thank you for working on this!

clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
30 ↗(On Diff #89071)

Maybe it would be worth to reflow this literal.

70 ↗(On Diff #89071)

What about accessing the buffer of the source file instead of pretty printing?

81 ↗(On Diff #89071)

Wouldn't just using the original source range of the argument work?
What about preserving the comments next to the arguments?

101 ↗(On Diff #89071)

You do not want to do fixits for code that is the result of macro expansions.

docs/clang-tidy/checks/modernize-replace-random-shuffle.rst
24 ↗(On Diff #89071)

Isn't it a performance problem in general to always reinitialize a random engine? Maybe the documentation should contain a notice that in case of performance critical code the user might want to factor the last parameter out somewhere.

test/clang-tidy/modernize-replace-random-shuffle.cpp
50 ↗(On Diff #89071)

This check-fix might match the previous fix instead of this one. You might want to make the fixes unique, e.g.: with a comment after a line. Note that it is also worth to test that line ending comments are preserved.

Also, are you sure you want to do the fixit when a custom random function is passed to random_shuffle?

madsravn added inline comments.Feb 20 2017, 1:49 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
30 ↗(On Diff #89071)

It seems a little weird, but this is the result of clang-format.

70 ↗(On Diff #89071)

How would you do this? printPretty was the best that I could find fitting my needs. If you have something better fitting, please share :)

81 ↗(On Diff #89071)

I am not sure what you mean about the original source range. Can I just put that onto the Stream?

Do you have an idea for the comments? Do you mean something like

std::random_shuffle(
   vec.begin(), // Comment
   vec.end() // Comment
);
101 ↗(On Diff #89071)

Why not? And how would I fix that?

xazax.hun added inline comments.Feb 20 2017, 1:58 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
30 ↗(On Diff #89071)

I usually just make it one big line and than clang format will do a better reflow. By default it do not remove linebreaks, even if it could improve the layout.

70 ↗(On Diff #89071)

I was thinking about something like getSourceText of Lexer.

81 ↗(On Diff #89071)

Or even:

std::random_shuffle(vec.begin(), /*inlinecomment*/ vec.end());
101 ↗(On Diff #89071)

Because that might cause the code not to compile when the macro is expanded in a different context. It is a conservative approach but other tidy checkers usually do this as well. You can use the isMacroID() method of the source locations.

madsravn added inline comments.Feb 20 2017, 2:31 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
81 ↗(On Diff #89071)

Thanks for you other comments. Can you elaborate on how I might fix this?

xazax.hun added inline comments.Feb 20 2017, 2:35 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
81 ↗(On Diff #89071)

You might do a different strategy, like not touching the arguments at all, but only inserting a new argument before the closing paren of the function call. This way all the comments should be preserved.

madsravn added inline comments.Feb 20 2017, 2:42 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
81 ↗(On Diff #89071)

I could try that. So just change the name of the function, random_shuffle to shuffle, and then remove the third argument if present and insert third argument. I guess it will work, but it will make the code less elegant.

madsravn updated this revision to Diff 89543.Feb 23 2017, 12:00 PM

Updated the code based on comments received.

madsravn marked 18 inline comments as done.Feb 23 2017, 12:03 PM
madsravn added inline comments.
test/clang-tidy/modernize-replace-random-shuffle.cpp
50 ↗(On Diff #89071)

I re-arranged them. This way the check-fixes does not say the same twice in a row. I am not sure what you mean by 'line ending comments are preserved'. Why shouldn't they be?

The fixit should also be done when a custom random function is passed. random_shuffle will be removed and the signature of the custom random function will be changed. It's hard to do much differently than just issuing a warning.

clang-tidy/modernize/ReplaceRandomShuffleCheck.h
21 ↗(On Diff #89543)

Typo - should be occurrences.
Same in 2 other places.

madsravn updated this revision to Diff 89817.Feb 26 2017, 2:02 PM
madsravn marked an inline comment as done.

Made small changes based on comments.

madsravn marked an inline comment as done.Feb 26 2017, 2:03 PM
madsravn updated this revision to Diff 89919.Feb 27 2017, 12:40 PM

Minor update to fix spelling mistake.

madsravn marked 2 inline comments as done.Feb 27 2017, 12:41 PM
aaron.ballman added inline comments.Feb 27 2017, 3:13 PM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
25 ↗(On Diff #89919)

Diagnostics are not full sentences. This should probably be: 'std::random_shuffle' has been removed in C++17; use 'std::shuffle' instead.

27 ↗(On Diff #89919)

I'd feel more comfortable if both of these were const char[] rather than StringRef, because StringRef doesn't typically own the underlying memory and should not be long-lived.

28 ↗(On Diff #89919)

Instead of adding additional text in this case, would it make sense to use a different diagnostic? e.g., 'std::random_shuffle' has been removed in C++17; use an alternative mechanism instead (or something along those lines). Regardless, it shouldn't be using full sentences.

71 ↗(On Diff #89919)

No need for this to be a separate variable.

72 ↗(On Diff #89919)

No need for this to be a separate variable.

77 ↗(On Diff #89919)

Is there a reason this needs to capture everything by copy? Also, no need for the empty parens. Actually, is the lambda even necessary at all?

105 ↗(On Diff #89919)

Please don't use auto as the type is not specified in the initialization.

docs/clang-tidy/checks/modernize-replace-random-shuffle.rst
8 ↗(On Diff #89919)

is -> are

20 ↗(On Diff #89919)

Both these -> Both of these
with -> with:

madsravn updated this revision to Diff 90148.Mar 1 2017, 3:21 AM

Updated patch according to comments by Ballman.

madsravn marked 9 inline comments as done.Mar 1 2017, 3:26 AM
madsravn added inline comments.
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

Is it OK to capture by reference then? Or how do we want it in llvm?

We need the lambda, because first I need to create the diag with a message based on the count of arguments and then I need to find fixits based on the same count. Example:

string message = "Message for 2 arguments";
if(argumentCount == 3) {
  message = "Message for 3 arguments";
}
auto Diag = diag(startLoc(), message);
if(argumentCount == 3) {
  Diag << FixitHint::FixForThreeArguments();
} else {
  Diag << FixitHint::FixForTwoArguments();
}

So the idea with the lambda is to avoid doing the same if-statement twice.

aaron.ballman added inline comments.Mar 1 2017, 10:45 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
89 ↗(On Diff #90148)

Should be NewName instead.

77 ↗(On Diff #89919)

But you call the lambda immediately rather than store it and reuse it? It seems like you should be able to hoist a DiagnosticBuilder variable outside of the if statement and skip the lambda entirely.

madsravn marked an inline comment as done.Mar 1 2017, 11:16 AM
madsravn added inline comments.
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

I am not sure what you mean by this. Can you elaborate? Can you give a short example how I would hoist a DiagnosticBuilder out?

I think I tried something like that, but it was not an option.

aaron.ballman added inline comments.Mar 1 2017, 11:21 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

It's entirely possible I'm missing something (I'm distracted with meetings this week), but I was envisioning:

DiagnosticBuilder Diag;
if (MatchedCallExpr->getNumArgs() == 3) {
  Diag =
      diag(MatchedCallExpr->getLocStart(),
           "'std::random_shuffle' has been removed in C++17; use "
           "'std::shuffle' and an alternative random mechanism instead");
  Diag << FixItHint::CreateReplacement(
      MatchedArgumentThree->getSourceRange(),
      "std::mt19937(std::random_device()())");
} else {
  Diag = diag(MatchedCallExpr->getLocStart(),
                    "'std::random_shuffle' has been removed in C++17; use "
                    "'std::shuffle' instead");
  Diag << FixItHint::CreateInsertion(
      MatchedCallExpr->getRParenLoc(),
      ", std::mt19937(std::random_device()())");
}
madsravn added inline comments.Mar 1 2017, 11:49 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

The constructor for DiagnosticBuilder is private. So I cannot do that. The idea had crossed my mind, but I think the lambda expression is nicer to look at.

Should I investigate if there is another way to hoist the DiagnosticBuilder out, like using diag() to make a dummy DiagnosticBuilder outside and then use the copy constructor to assign inside the if-statement? Or can we live with the lambda expression?

aaron.ballman added inline comments.Mar 1 2017, 11:51 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

Ah, okay, that was the bit I was missing. Thank you for being patient. I think the lambda (with the reference capture) is fine as-is.

madsravn added inline comments.Mar 1 2017, 11:57 AM
clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

Thank you for being patient.

Right back at you. We are working towards the same goal after all :)

For future reference: Should I try to avoid lambda expressions like this?

aaron.ballman edited edge metadata.Mar 1 2017, 12:11 PM

Out of curiosity, have you run this over libc++ or libstdc++ test suites involving std::random_shuffle? If so, were the results acceptable?

I think this generally LGTM (aside from the minor nit about naming), but I'd like to hear from @mclow.lists where this meets his needs.

clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
77 ↗(On Diff #89919)

No, this sort of expression is fine when it makes the code more simple.

madsravn updated this revision to Diff 90223.Mar 1 2017, 1:40 PM

Last small changes based on comments.

madsravn marked 8 inline comments as done.Mar 1 2017, 1:41 PM

Out of curiosity, have you run this over libc++ or libstdc++ test suites involving std::random_shuffle? If so, were the results acceptable?

I haven't. Good idea. I will get onto that. I don't have either, so I will just fetch them and set them up.

Looks good for the two tests the are for random_shuffle in llvm libc++.

madsravn marked an inline comment as done.Mar 1 2017, 2:22 PM

Looks good for the two tests the are for random_shuffle in llvm libc++.

There were a lot more some time ago, before @mclow.lists performed this transformation on libc++'s testsuite. You might want to try this on the revision before that happened.

Any updates on this?

Any updates on this?

Have you run it over the test suite on the revision before random_shuffle was removed from libc++?

Any updates on this?

Have you run it over the test suite on the revision before random_shuffle was removed from libc++?

I can't find any more tests for random_shuffle. I have looked in the SVN log for from december 2014 until now. It works on the three tests currently in trunk.

Any updates on this?

Have you run it over the test suite on the revision before random_shuffle was removed from libc++?

I can't find any more tests for random_shuffle. I have looked in the SVN log for from december 2014 until now. It works on the three tests currently in trunk.

Try just before r294328.

Any updates on this?

Have you run it over the test suite on the revision before random_shuffle was removed from libc++?

I can't find any more tests for random_shuffle. I have looked in the SVN log for from december 2014 until now. It works on the three tests currently in trunk.

Try just before r294328.

I can't see any random_shuffle tests in the libc++ repo before r294328 either. I don't know if they aren't there or if I just can't find them. Would you be able to point them out for me with revision and path?

Any updates on this?

Have you run it over the test suite on the revision before random_shuffle was removed from libc++?

I can't find any more tests for random_shuffle. I have looked in the SVN log for from december 2014 until now. It works on the three tests currently in trunk.

Try just before r294328.

I can't see any random_shuffle tests in the libc++ repo before r294328 either. I don't know if they aren't there or if I just can't find them. Would you be able to point them out for me with revision and path?

Did you look at the diff from the commit I mentioned? That's the one that removed them.

Any updates on this?

Have you run it over the test suite on the revision before random_shuffle was removed from libc++?

I can't find any more tests for random_shuffle. I have looked in the SVN log for from december 2014 until now. It works on the three tests currently in trunk.

Try just before r294328.

I can't see any random_shuffle tests in the libc++ repo before r294328 either. I don't know if they aren't there or if I just can't find them. Would you be able to point them out for me with revision and path?

Did you look at the diff from the commit I mentioned? That's the one that removed them.

I found them and I have tested them all. This check works fine on them.

aaron.ballman accepted this revision.Apr 10 2017, 3:30 PM

There are a few small nits I've mentioned, but otherwise LGTM.

clang-tidy/modernize/ReplaceRandomShuffleCheck.cpp
34–35 ↗(On Diff #90223)

No need to bind either of these -- the bound name never gets used.

36 ↗(On Diff #90223)

Can likely pick a better name to bind than "three". ;-) I would recommend "RandomFunc" since that's what the parameter represents.

70 ↗(On Diff #90223)

Wrap for 80-column (here and elsewhere).

This revision is now accepted and ready to land.Apr 10 2017, 3:30 PM
madsravn updated this revision to Diff 96303.Apr 23 2017, 4:37 AM

Small changes to code due to comments.

This continues to LGTM; do you need someone to land the patch for you?

test/clang-tidy/modernize-replace-random-shuffle.cpp
40 ↗(On Diff #96303)

Can remove the extra newline.

This continues to LGTM; do you need someone to land the patch for you?

I will remove the extra newline and land the patch tomorrow. Thanks! :)

This revision was automatically updated to reflect the committed changes.