Page MenuHomePhabricator

[ADT][Expensive checks] Create a /dev/random seed only once when shuffling the range before sorting it to avoid bot timeouts
ClosedPublic

Authored by arphaman on Nov 14 2019, 7:19 PM.

Details

Summary

The expensive checks enabled bot on green dragon (http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/) has been failing to pass the libFuzzer :: large.test ever since the test was enabled on Darwin (i.e. REQUIRES: linux was removed). It's failing because it's timing out, and it's timing out because the compiler is too slow. I sampled the clang invocations and was shocked to see that the majority of the time was spent in calls to std::random_device inside of llvm::sort.

This patch changes the expensive checking behavior to compute a single std::random_device shuffle seed, per an invocation of clang, rather than computing it for every sort. This fixes the test failure on the bot. On my machine, this improved the performance of building compiler-rt with clang that has expensive checks enabled by 50-100x!

Diff Detail

Event Timeline

arphaman created this revision.Nov 14 2019, 7:19 PM
Herald added a project: Restricted Project. · View Herald TranscriptNov 14 2019, 7:19 PM

haha, nice catch.

Random idea: Maybe the clang build-bots needs to run some background stack-sampling by default, and generate a flame-graph for people to check?

mgrang accepted this revision.Nov 15 2019, 10:51 AM

Thanks for fixing this. LGTM.

This revision is now accepted and ready to land.Nov 15 2019, 10:51 AM

Unfortunately my jubilation was premature. While this speed up the build of compiler-rt and the test file as well dramatically, the test file still takes way too long to compile and still times out. Most of the time now is spend in IR/MIR verifiers when building the test file. I'll commit this and keep looking into why they're so slow for that file.

This revision was automatically updated to reflect the committed changes.