[test-suite] SPEC2017 CPU Fotonik3d floating point tests.

[naromero77]

Summary

Add SPEC2017 CPU Fotonik3d floating point rate and speed tests.

[Meinersbur]

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

12–14

What does this do without a command to run?

31–32

Does fpcmd work with both tolerances in one invocation?

64

Files in this dir requires to run the install script, where I up to now just extracted the archive. (specxz is in \tools\bin\$platform). In addition, the platform on which cmake runs, llvm-lit is invoked, and the tests executables are running (TEST_SUITE_REMOTE_HOST) might not be the same. For which of these architecture is specxz supposed to be?

Is there any possibility to not require installation?

External/SPEC/CFP2017rate/CMakeLists.txt

13–14

External/SPEC/CFP2017speed/CMakeLists.txt

10–11

[naromero77]

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

12–14

The test is running:

/home/naromero/testsuite-backup/build-gcc/tools/timeit-target --limit-core 0 --limit-cpu 7200 --timeout 7200 --limit-file-size 104857600 --limit-rss-size 838860800 --redirect-input /dev/null --chdir /home/naromero/testsuite-backup/build-g\
cc/External/SPEC/CFP2017rate/549.fotonik3d_r/run_train --summary /home/naromero/testsuite-backup/build-gcc/External/SPEC/CFP2017rate/549.fotonik3d_r/Output/549.fotonik3d_r.test.time ../549.fotonik3d_r

The name of the inputs files and output files are hard-coded into this test. Does not need anything to read anything from STDIN.

31–32

Good question, I will check.

64

I assume that SPEC CPU 2017 is installed by the user. The object.pm for fotonik3d includes a sub generate_inputs section which calls specxz. So, if you just run the SPEC CPU 2017 installer it will not create the needed OBJ.dat, it is created on the fly.

The specxz that is pointed to in object.pmis
'command' => ::jp($ENV{SPEC}, "bin", "specxz"),

Your point is that that I might be using the wrong specxz? This is called while the tests are being built (after cmake, during make, but before the call to llvm-lit), which is normally the host? Is ${TEST_SUITE_SPEC2017_ROOT}/bin/specxz not for the host?

[naromero77]

• Add missing TEST_SUITE_FORTRAN conditional.
• Combine into a single verify_output.

[Meinersbur]

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

64

From how I understand from install.sh, ${TEST_SUITE_SPEC2017_ROOT}/bin/specxz contains the copy of the file that corresponds to the architecture as returned by uname.

I see that this is the behavior of SPEC itself, but I liked the convenience of just dumping the spec2017 files into one directory, e.g. an NFS location to be used from multiple platforms.

Possibilities to avoid the installation:

1. Use xz as installed on the host.
2. Autodetect the architecture in the CMakeLists.txt
3. Try out all specxz executables, stop with the one that worked
4. Let cmake compile 557.xz_r (spec_xz.c) and use that executable.

I am not sure whether this would be worth the effort, but it would be nice.

[naromero77]

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

64

I think it terms of simplicity and portability my initial thought would be to go with option 4 -- I am assuming that spec_xz.c hasn't been modified in some adverse way.

I think that option 1 would be OK for Linux users, but xz doesn't seem to be part of the default Mac OS installation. It is part of homebrew which most Mac OS developers would normally have installed. This path also adds another external dependency, though a pretty easy one which is a bit of minus.

With regards to option 2, this also sounds straightforward but still requires installation of SPEC itself right which you want to avoid.

How does option 4 sound to you?

The other possibility is to wait until it becomes a serious issue for someone and then we can fix. It never occurred to me NOT to run the installation script for SPEC CPU 2017. Do you know what is the convention for the other older SPEC test suites?

I will wait to hear back from you before moving forward.

[Meinersbur]

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

64

Option 2. could be as simple as using the directory ${TEST_SUITE_SPEC2017_ROOT}/tools/bin/${CMAKE_HOST_SYSTEM_NAME}-${CMAKE_HOST_SYSTEM_PROCESSOR}/specxz. (CMAKE_HOST_SYSTEM_NAME has to be lower cased). Maybe only as a fallback if ${TEST_SUITE_SPEC2017_ROOT}/specxz is not available.

There might also be no precompiled toolset available for the target system, e.g. SPEC OpenMP 2012 has none for 64 bit ARM. How to build the tools is described here: https://www.spec.org/cpu2017/Docs/tools-build.html .

I have not required installation for the SPEC2006 or SPEC2000 Externals.

However, if it turns out too be too much work, I'd compromise to accept this patch that prints a helpful message if no installed specxz is found, and add this functionality later if needed.

[Meinersbur]

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

64

The tools-build documentation refers to the install_archives folder that I actually don't have. However, xz is open source and could be added directly: https://tukaani.org/xz/

[naromero77]

• Run specxz from tools bin.

[Meinersbur]

LGTM, thank you

External/SPEC/CFP2017rate/549.fotonik3d_r/CMakeLists.txt

+# https://www.spec.org/cpu2017/Docs/benchmarks/549.fotonik3d_r.html
+include(../../SpecCPU2017.cmake)
+if (NOT TEST_SUITE_SPEC2017_ROOT)
+  return ()
+endif ()
+
+speccpu2017_benchmark(RATE)
+
+## test ########################################################################
+
+speccpu2017_run_test(
+  RUN_TYPE test
+)
+
+## train #######################################################################
+
+speccpu2017_run_test(
+  RUN_TYPE train
+)
+
+## ref #########################################################################
+
+speccpu2017_run_test(
+  RUN_TYPE ref
+)
+
+################################################################################
+
+# fotonik3d has both relative and absolute tolerances
+speccpu2017_verify_output(RELATIVE_TOLERANCE 1.0e-10 ABSOLUTE_TOLERANCE 1.0e-27)
+
+speccpu2017_add_executable(
+  parameter.f90                                                                                                   
+  globalvar.F90                                                                                                   
+  readline.f90                                                                                                    
+  power.F90                                                                                                       
+  calcflops.F90                                                                                                   
+  communicate.F90                                                                                                 
+  huygens.F90                                                                                                     
+  update.F90                                                                                                      
+  material.F90                                                                                                    
+  mur.F90                                                                                                         
+  PlaneSource.F90                                                                                                 
+  PEC.F90                                                                                                         
+  init.F90                                                                                                        
+  leapfrog.F90                                                                                                    
+  timerRoutine.f90                                                                                                
+  UPML.F90                                                                                                        
+  yeemain.F90                                                                                                     
+  writeout.F90                                                                                                    
+  MPI_dummy.F90  
+)
+
+speccpu2017_prepare_rundir()
+
+# Generate OBJ.dat from OBJ.dat.xyz for fotonik3d as input
+
+# We do not require that the SPEC install script has been run.
+# specxz is executed during the build process on the host after
+# the build system is generated by CMake.
+string(TOLOWER ${CMAKE_HOST_SYSTEM_NAME} CMAKE_HOST_SYSTEM_NAME_LC)
+set(SPECXZ_BIN
+  "${TEST_SUITE_SPEC2017_ROOT}/tools/bin/${CMAKE_HOST_SYSTEM_NAME_LC}-${CMAKE_HOST_SYSTEM_PROCESSOR}/specxz")
+
+foreach (_run_type IN LISTS TEST_SUITE_RUN_TYPE)
+  add_custom_command(
+    OUTPUT "${RUN_${_run_type}_DIR}/OBJ.dat"
+    DEPENDS "${INPUT_${_run_type}_DIR}/OBJ.dat.xz"
+            "${SPECXZ_BIN}"
+    COMMAND "${SPECXZ_BIN}"
+            -dc "${INPUT_${_run_type}_DIR}/OBJ.dat.xz"
+            > "${RUN_${_run_type}_DIR}/OBJ.dat"
+    WORKING_DIRECTORY "${RUN_${_run_type}_DIR}"
+    COMMENT "Generating OBJ.dat from ${INPUT_${_run_type}_DIR}/OBJ.dat.xz"
+    VERBATIM
+    )
+
+  # ${PROG} must be included in custom target name in order
+  # to allow re-use of code in both RATE and SPEED benchmarks
+  add_custom_target(${PROG}_OBJ_dat_${_run_type}
+    DEPENDS "${RUN_${_run_type}_DIR}/OBJ.dat"
+  )
+  add_dependencies(${PROG} ${PROG}_OBJ_dat_${_run_type})
+endforeach ()

External/SPEC/CFP2017rate/CMakeLists.txt

 if (TEST_SUITE_FORTRAN)
   add_subdirectory(503.bwaves_r)    # Fortran
   add_subdirectory(507.cactuBSSN_r) # C++, C, Fortran
 endif ()
 add_subdirectory(508.namd_r)       # C++
 add_subdirectory(510.parest_r)     # C++
 add_subdirectory(511.povray_r)     # C++, C
 add_subdirectory(519.lbm_r)        # C
 #add_subdirectory(521.wrf_r)       # Fortran, C
 add_subdirectory(526.blender_r)    # C++, C
 #add_subdirectory(527.cam4_r)      # Fortran, C
 add_subdirectory(538.imagick_r)    # C
 add_subdirectory(544.nab_r)        # C
-#add_subdirectory(549.fotonik3d_r) # Fortran
-#add_subdirectory(554.roms_r)      # Fortran
+if (TEST_SUITE_FORTRAN)
+  add_subdirectory(549.fotonik3d_r) # Fortran
+  #add_subdirectory(554.roms_r)     # Fortran
+endif ()
 
 add_subdirectory(997.specrand_fr)  # C

External/SPEC/CFP2017speed/649.fotonik3d_s/CMakeLists.txt

+# https://www.spec.org/cpu2017/Docs/benchmarks/649.fotonik3d_s.html
+include(../../SpecCPU2017.cmake)
+if (NOT TEST_SUITE_SPEC2017_ROOT)
+  return ()
+endif ()
+
+speccpu2017_benchmark(SPEED ORIGIN 549.fotonik3d_r)
+
+speccpu2017_inherit(../../CFP2017rate/549.fotonik3d_r)

External/SPEC/CFP2017speed/CMakeLists.txt

 if (TEST_SUITE_FORTRAN)
   add_subdirectory(603.bwaves_s)    # Fortran
   add_subdirectory(607.cactuBSSN_s) # C++, C, Fortran
 endif ()
 add_subdirectory(619.lbm_s)        # C
 #add_subdirectory(621.wrf_s)       # Fortran, C
 #add_subdirectory(627.cam4_s)      # Fortran, C
 #add_subdirectory(628.pop2_s)      # Fortran, C
 add_subdirectory(638.imagick_s)    # C
 add_subdirectory(644.nab_s)        # C
-#add_subdirectory(649.fotonik3d_s) # Fortran
-#add_subdirectory(654.roms_s)      # Fortran
+if (TEST_SUITE_FORTRAN)
+  add_subdirectory(649.fotonik3d_s) # Fortran
+  #add_subdirectory(654.roms_s)     # Fortran
+endif ()
 
 add_subdirectory(996.specrand_fs)  # C