This is an archive of the discontinued LLVM Phabricator instance.

[test-suite] Add XSBench
ClosedPublic

Authored by hfinkel on Dec 1 2016, 10:29 AM.

Details

Summary

The US Department of Energy (DOE) has open-sourced a significant number of so-called proxy applications (i.e. small applications, representative of our larger workloads in some respects, intended for use as benchmarks and test beds for various kind of porting exercises). Many of these are useful as compiler tests, and I'd like to add these to our test suite. Doing so will increase the representation within our test suite of HPC/scientific applications and allow us to better track how Clang/LLVM is doing in this area. Scientific applications are certainly a diverse group in themselves, and so my long term goal is to add a significant number of these proxies to ensure good coverage.

Some months ago, I compiled a list of our open-source proxy applications released by the various DOE laboratories (https://gitlab.com/llvm-doe/public/wikis/DOEProxyApps). By my estimate, of the approximately 40 proxy applications on that list, around half could be reasonably added to our test suite. We're continuing to develop new proxies, and so the number of suitable applications should grow somewhat in the future.

In any case, here's a patch to add the first one: XSBench. This is a C application with a straightforward reference output. It is designed to represent a key computational kernel of the Monte-Carlo neutronics application OpenMC. The builtin "small" test configuration is suitable for our test suite (it runs in about a minute on my build system). The only complication in adding this application is its dependence on rand(). The developer's provided known-good output will work only for glibc's implementation of rand (other rand implementation obviously may produce a different series of random numbers given the same initial seed). Luckily, the algorithm that glibc uses for rand() is reasonably well described by various web pages, and so I constructed a small independent implementation. This is in the glibc_compat_rand.{c,h} files.

As a reflection of my desire to add more of these applications, I've place this in a subdirectory named DOE-ProxyApps-C (the idea being that the C++ proxy applications will get a DOE-ProxyApps-C++ directory).

Thanks again!

Diff Detail

Event Timeline

hfinkel updated this revision to Diff 79937.Dec 1 2016, 10:29 AM
hfinkel retitled this revision from to [test-suite] Add XSBench.
hfinkel updated this object.
hfinkel added reviewers: MatzeB, jmolloy.
hfinkel added a subscriber: llvm-commits.
jmolloy accepted this revision.Dec 1 2016, 10:39 AM
jmolloy edited edge metadata.

Hi Hal,

This LGTM, thanks!

This revision is now accepted and ready to land.Dec 1 2016, 10:39 AM
MatzeB accepted this revision.Dec 1 2016, 10:58 AM
MatzeB edited edge metadata.

LGTM, if there is a way to further reduce the benchmark runtime that would be appreciated. I think Kristof Beyls measured that at around half a second measurement noise is good enough.

MultiSource/Benchmarks/DOE-ProxyApps-C/XSBench/Makefile
6

Should this be -s small like the CMakeLists.txt?

This revision was automatically updated to reflect the committed changes.