This is an archive of the discontinued LLVM Phabricator instance.

[test-suite] Adding HPCCG benchmark
ClosedPublic

Authored by homerdin on Aug 10 2017, 8:38 AM.

Details

Summary

Description:

HPCCG: A simple conjugate gradient benchmark code for a 3D chimney
domain on an arbitrary number of processors.

This simple benchmark code is a self-contained piece of C++ software
that generates a 27-point finite difference matrix with a user-prescribed
sub-block size on each processor.

Links

Web: https://mantevo.org/packages/
github: https://github.com/Mantevo/HPCCG

On my machine:
compile_time: 56.9292 
exec_time: 0.7010 
Maximum resident set size (kbytes): 193952

Diff Detail

Event Timeline

homerdin created this revision.Aug 10 2017, 8:38 AM

Hi @homerdin,

Great to see you taking the effort to make the test-suite more relevant!
I think it'd be helpful if you could also add a little explanation of how adding this code makes the test-suite more relevant.
I'm assuming that this kind of code is common in some domain (HPC?) and that there is no other benchmark already in the test-suite with similar enough properties?

On my machine:
compile_time: 56.9292
exec_time: 0.7010
Maximum resident set size (kbytes): 193952

With my experience so far on using the test-suite to track the performance evolution of LLVM, an execution time of about 1 second should be roughly what we aim for to have a good balance between not having too much noise in the execution time vs the benchmark producing results quickly. I'm just wondering roughly what the characteristics of your machine is?

Other than that, I just think that you'll also need to add this new directory as an exception near the end of top-level LICENSE.TXT file.

Thanks!

Kristof

hfinkel edited edge metadata.Aug 11 2017, 10:00 AM

Hi @homerdin,

Great to see you taking the effort to make the test-suite more relevant!
I think it'd be helpful if you could also add a little explanation of how adding this code makes the test-suite more relevant.
I'm assuming that this kind of code is common in some domain (HPC?) and that there is no other benchmark already in the test-suite with similar enough properties?

Just to provide some context: In D27311, I added one of our HPC benchmarks (XSBench), and this is a second one. Brian and several other students working for me this summer have gone through all 40 of the possible additions on our current list (https://gitlab.com/llvm-doe/public/wikis/DOEProxyApps) and narrowed the list down to approximately 15 that are appropriate for inclusion into the test suite. They're all now working on preparing patches for those. This will increase our coverage of relevant HPC algorithms and code patterns.

HPCCG represents a conjugate gradients calculation with a 27-point stencil. It's probably most similar to Benchmarks/ASCI_Purple/SMG2000, but HPCCG's implementation looks different (and is in C++, not C, and is a decade+ newer). FWIW, given the relatively-small size of our test suite, I'm not too concerned about similarity (so long as the codes actually are different).

Thanks!

...

homerdin updated this revision to Diff 110775.Aug 11 2017, 11:14 AM

Thanks for the feedback. I have updated the LICENSE.TXT file to include this application. The machine I used was an IBM x3550 M4.

Let me know if there anything else you would like to know,
Brian

Thanks for the feedback. I have updated the LICENSE.TXT file to include this application. The machine I used was an IBM x3550 M4.

To elaborate, that machine has 2.2 GHz Intel Xeon cores (E5-2660).

Let me know if there anything else you would like to know,
Brian

Hi Brian, Hal,

Hi @homerdin,

Great to see you taking the effort to make the test-suite more relevant!
I think it'd be helpful if you could also add a little explanation of how adding this code makes the test-suite more relevant.
I'm assuming that this kind of code is common in some domain (HPC?) and that there is no other benchmark already in the test-suite with similar enough properties?

Just to provide some context: In D27311, I added one of our HPC benchmarks (XSBench), and this is a second one. Brian and several other students working for me this summer have gone through all 40 of the possible additions on our current list (https://gitlab.com/llvm-doe/public/wikis/DOEProxyApps) and narrowed the list down to approximately 15 that are appropriate for inclusion into the test suite. They're all now working on preparing patches for those. This will increase our coverage of relevant HPC algorithms and code patterns.

HPCCG represents a conjugate gradients calculation with a 27-point stencil. It's probably most similar to Benchmarks/ASCI_Purple/SMG2000, but HPCCG's implementation looks different (and is in C++, not C, and is a decade+ newer). FWIW, given the relatively-small size of our test suite, I'm not too concerned about similarity (so long as the codes actually are different).

Thanks!

Great, thanks for the context, that makes a lot of sense!

Thanks for the feedback. I have updated the LICENSE.TXT file to include this application. The machine I used was an IBM x3550 M4.

Let me know if there anything else you would like to know,
Brian

No thanks, that's enough. I was just curious about roughly what kind of machine you did your timing measurements on.

Overall, this looks good to me, I've just got 2 more minor remarks after looking in more detail at the code.

I'm also assuming that you've attempted to keep the code close to the upstream version at https://gitlab.com/llvm-doe/public/wikis/DOEProxyApps, as it seems there's quite a bit of dead code in here for how this code will be used in the test-suite (YAML, mytimer, dump_matlab_matrix, ...). If my assumption about staying close to the upstream version: that makes sense to me.
Please note that I'm not qualified to judge the actual kernel implemented here, but I'm assuming that has been reviewed before enough by qualified people.

Thanks!

Kristof

MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/Makefile
5

I can't seem to find -DVERIFICATION as a CPPFLAGS in the CMakelists.txt, which seems inconsistent.
Should -DVERIFICATION be added to both Makefile and CMakelists.txt or neither?

MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/README
58–63

I would hope the default will be to run without MPI support enabled?
Is this part of the README wrong, or does USE_MPI= have to be enabled somewhere?
Or I guess that the makefile referred to is the Makefile at https://gitlab.com/llvm-doe/public/wikis/DOEProxyApps, instead of the custom ones here?
Maybe it'd be best overall to start this README by explaining that the original source is at https://gitlab.com/llvm-doe/public/wikis/DOEProxyApps; and that the rest of the README describes that original source? And maybe with the note that the Makefile/CMakeLists.txt is the main difference from the originals (if that really is the case)?
I think leaving the README here is very valuable, there some great explanations in here in case someone needs to explore this program further.

homerdin updated this revision to Diff 110820.Aug 11 2017, 3:00 PM

Thanks again, The -DVERIFICATION should not have been in the Makefile (not used in this app). I added a short note at the top of the README to make things clearer.

Brian

kristof.beyls accepted this revision.Aug 11 2017, 10:52 PM

Thanks again, The -DVERIFICATION should not have been in the Makefile (not used in this app). I added a short note at the top of the README to make things clearer.

Brian

Thanks Brian, LGTM now.

This revision is now accepted and ready to land.Aug 11 2017, 10:52 PM
This revision was automatically updated to reflect the committed changes.