Index: docs/Proposals/TestSuite.rst =================================================================== --- /dev/null +++ docs/Proposals/TestSuite.rst @@ -0,0 +1,310 @@ +===================== +Test-Suite Extentions +===================== + +.. contents:: + :depth: 1 + :local: + +Abstract +======== + +These are ideas for additional programs, benchmarks, applications and +algorithms that could be added to the LLVM Test-Suite. +The test-suite could be much larger than it is now, which would help us +detecting compiler errors (crashes, miscompiles) during development. + +Most probably, the reason why the programs below have not been added to +the test-suite yet is that nobody has found time to do it. But there +might be other issues as well, such as + + * Licensing (Support can still be added as external module, + like for the SPEC benchmarks) + + * Language (in particular, there is no official LLVM frontend + for FORTRAN yet) + + * Parallelism (currently, all programs in test-suite use + one thread only) + +Benchmarks +========== + +SPEC CPU 2017 +------------- +https://www.spec.org/cpu2017/ + +The following have not been included yet because they contain Fortran +code. + +In case of cactuBSSN only a small portion is Fortran. The hosts's +Fortran compiler could be used for these parts. + +Note that CMake's Ninja generator has difficulties with Fortran. See the +`CMake documentation `_ +for details. + + * 503.bwaves_r/603.bwaves_s + * 507.cactuBSSN_r + * 521.wrf_r/621.wrf_s + * 527.cam4_r/627.cam4_s + * 628.pop2_s + * 548.exchange2_r/648.exchange2_s + * 549.fotonik3d_r/649.fotonik3d_s + * 554.roms_r/654.roms_s + +SPEC OMP2012 +------------ +https://www.spec.org/omp2012/ + + * 350.md + * 351.bwaves + * 352.nab + * 357.bt331 + * 358.botsalgn + * 359.botsspar + * 360.ilbdc + * 362.fma3d + * 363.swim + * 367.imagick + * 370.mgrid331 + * 371.applu331 + * 372.smithwa + * 376.kdtree + +OpenCV +------ +https://opencv.org/ + +OpenMP 4.x SIMD Benchmarks +-------------------------- +https://github.com/flwende/simd_benchmarks + +PWM-benchmarking +---------------- +https://github.com/tbepler/PWM-benchmarking + +SLAMBench +--------- +https://github.com/pamela-project/slambench + +FireHose +-------- +http://firehose.sandia.gov/ + +A Benchmark for the C/C++ Standard Library +------------------------------------------ +https://github.com/hiraditya/std-benchmark + +OpenBenchmarking.org CPU / Processor Suite +------------------------------------------ +https://openbenchmarking.org/suite/pts/cpu + +This is a subset of the +`Phoronix Test Suite `_ +and is itself a collection of benchmark suites + +Parboil Benchmarks +------------------ +http://impact.crhc.illinois.edu/parboil/parboil.aspx + +MachSuite +--------- +https://breagen.github.io/MachSuite/ + +Rodinia +------- +http://lava.cs.virginia.edu/Rodinia/download_links.htm + +Rodinia has already been partially included in +MultiSource/Benchmarks/Rodinia. Benchmarks still missing are: + + * streamcluster + * particlefilter + * nw + * nn + * myocyte + * mummergpu + * lud + * leukocyte + * lavaMD + * kmeans + * hotspot3D + * heartwall + * cfd + * bfs + * b+tree + +vecmathlib tests harness +------------------------ +https://bitbucket.org/eschnett/vecmathlib/wiki/Home + +PARSEC +------ +http://parsec.cs.princeton.edu/ + +Graph500 reference implementations +---------------------------------- +https://github.com/graph500/graph500/tree/v2-spec + +NAS Parallel Benchmarks +----------------------- +https://www.nas.nasa.gov/publications/npb.html + +The official benchmark is written in Fortran, but an unofficial +C-translation is available as well: +https://github.com/benchmark-subsetting/NPB3.0-omp-C + +DARPA HPCS SSCA#2 C/OpenMP reference implementation +--------------------------------------------------- +http://www.highproductivity.org/SSCABmks.htm + +This web site does not exist any more, but there seems to be a copy of +some of the benchmarks +https://github.com/gtcasl/hpc-benchmarks/tree/master/SSCA2v2.2 + +Kokkos +------ +https://github.com/kokkos/kokkos-kernels/tree/master/perf_test +https://github.com/kokkos/kokkos/tree/master/benchmarks + +PolyMage +-------- +https://github.com/bondhugula/polymage-benchmarks + +PolyBench +--------- +https://sourceforge.net/projects/polybench/ + +A modified version of Polybench 3.2 is already presented in +SingleSource/Benchmarks/Polybench. A newer version 4.2.1 is available. + +High Performance Geometric Multigrid +------------------------------------ +https://crd.lbl.gov/departments/computer-science/PAR/research/hpgmg/ + +RAJA Performance Suite +---------------------- +https://github.com/LLNL/RAJAPerf + +CORAL-2 Benchmarks +------------------ +https://asc.llnl.gov/coral-2-benchmarks/ + +Many of its programs have already been integreated in +MultiSource/Benchmarks/DOE-ProxyApps-C and +MultiSource/Benchmarks/DOE-ProxyApps-C++. + + * Nekbone + * QMCPack + * LAMMPS + * Kripke + * Quicksilver + * PENNANT + * Big Data Analytic Suite + * Deep Learning Suite + * Stream + * Stride + * ML/DL micro-benchmark + * Pynamic + * ACME + * VPIC + * Laghos + * Parallel Integer Sort + * Havoq + +NWChem +------ +http://www.nwchem-sw.org/index.php/Benchmarks + +TVM +---- +https://github.com/dmlc/tvm/tree/master/apps/benchmark + +HydroBench +---------- +https://github.com/HydroBench/Hydro + +ParRes +------ +https://github.com/ParRes/Kernels/tree/master/Cxx11 + +Applications/Libraries +====================== + +GnuPG +----- +https://gnupg.org/ + +Blitz++ +------- +https://sourceforge.net/projects/blitz/ + +FFmpeg +------ +https://ffmpeg.org/ + +FreePOOMA +--------- +http://www.nongnu.org/freepooma/ + +FTensors +-------- +http://www.wlandry.net/Projects/FTensor + +Generic Algorithms +================== + +Image processing +---------------- + +Resampling +`````````` + + * Bilinear + * Bicubic + * Lanczos + +Dither +`````` + + * Threshold + * Random + * Halftone + * Bayer + * Floyd-Steinberg + * Jarvis + * Stucki + * Burkes + * Sierra + * Atkinson + * Gradient-based + +Feature detection +````````````````` + + * Harris + * Histogram of Oriented Gradients + +Color conversion +```````````````` + + * RGB to grayscale + * HSL to RGB + +Graph +----- + +Search Algorithms +````````````````` + + * Breadth-First-Search + * Depth-First-Search + * Dijkstra's algorithm + * A-Star + +Spanning Tree +````````````` + + * Kruskal's algorithm + * Prim's algorithm Index: docs/index.rst =================================================================== --- docs/index.rst +++ docs/index.rst @@ -559,6 +559,7 @@ CodeOfConduct Proposals/GitHubMove + Proposals/TestSuite Proposals/VectorizationPlan :doc:`CodeOfConduct` @@ -568,6 +569,9 @@ :doc:`Proposals/GitHubMove` Proposal to move from SVN/Git to GitHub. +:doc:`Proposals/TestSuite` + Proposals for additional benchmarks/programs for llvm's test-suite. + :doc:`Proposals/VectorizationPlan` Proposal to model the process and upgrade the infrastructure of LLVM's Loop Vectorizer.