This is an archive of the discontinued LLVM Phabricator instance.

[OpenMP] Split OpenMP/target_map_codegen test [NFC]
ClosedPublic

Authored by tra on Aug 7 2020, 1:14 PM.

Details

Summary

The test file is the single longest test among clang's tests and ends up about
doubling the wall time of clang tests on machines with high number of cores.

The test appears to consist of multiple independent subtests and does not have
to be in one file. Splitting it into smaller parts reduces test time on my
machine from ~80s down to ~45.

See before/after comparison here:
https://gist.github.com/Artem-B/d0b05c2e98a49158c02de23f7f4f0279

Diff Detail

Event Timeline

tra created this revision.Aug 7 2020, 1:14 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 7 2020, 1:14 PM
tra requested review of this revision.Aug 7 2020, 1:14 PM

Wow, cool. I imagine it was hard to split this given the manual check lines. We really need to start using the upgrade scripts here.

I'm fine with this, @ABataev WDYT?

tra added a comment.Aug 7 2020, 1:29 PM

Wow, cool. I imagine it was hard to split this given the manual check lines. We really need to start using the upgrade scripts here.

I'm fine with this, @ABataev WDYT?

It was not that hard. The file had pretty uniform structure, so all it took is trivial edit to add // CK to #endif lines of individual sections and the following two commands did the rest:

csplit -f './test/OpenMP/target_map_codegen_' -b '%02d.cp'  ./test/OpenMP/target_map_codegen.cpp '/#endif \/\/\ CK/1' '{*}' 
for f in ./test/OpenMP/target_map_codegen_*.cp; do (head -4 ./test/OpenMP/target_map_codegen.cpp; cat $f; echo '#endif') > ${f}p; done
In D85551#2203758, @tra wrote:

Wow, cool. I imagine it was hard to split this given the manual check lines. We really need to start using the upgrade scripts here.

I'm fine with this, @ABataev WDYT?

It was not that hard. The file had pretty uniform structure, so all it took is trivial edit to add // CK to #endif lines of individual sections and the following two commands did the rest:

csplit -f './test/OpenMP/target_map_codegen_' -b '%02d.cp'  ./test/OpenMP/target_map_codegen.cpp '/#endif \/\/\ CK/1' '{*}' 
for f in ./test/OpenMP/target_map_codegen_*.cp; do (head -4 ./test/OpenMP/target_map_codegen.cpp; cat $f; echo '#endif') > ${f}p; done

Very cool. Thanks for sharing this!

tra added inline comments.Aug 7 2020, 1:34 PM
clang/test/OpenMP/target_map_codegen_18.cpp
6

This file is still among the longest tests in clang and could use some further splitting, if possible.
Between this and arm-cortex-cpus.c tests there are another 10 seconds we could shave off the test time.

This revision is now accepted and ready to land.Aug 7 2020, 1:36 PM
jdoerfert added inline comments.Aug 7 2020, 1:45 PM
clang/test/OpenMP/target_map_codegen_18.cpp
6

Our target codegen is sometimes incredibly slow. I think there is an underlying problem that needs to be addressed anyway.

This revision was landed with ongoing or failed builds.Aug 7 2020, 1:48 PM
This revision was automatically updated to reflect the committed changes.
tra added inline comments.Aug 10 2020, 4:00 PM
clang/test/OpenMP/target_map_codegen_18.cpp
6

Turns out it's actually FileCheck & regex that's the bottleneck. https://reviews.llvm.org/D85695