[test-suite] A number of cmake configuration fixes for External/SPEC.

[kristof.beyls]

Summary

These are the changes I had to make to make the spec 2000 and spec 2006 benchmark
compile and run correctly using the new cmake/lit-framework in the test-suite,on the following platforms, that I used for testing:

• Ubuntu 12.04LTS x86_64, gcc 4.7
• Ubuntu 12.04LTS x86_64, clang 3.5
• Debian linux AArch64 v8.0, clang 3.7
• Debian linux AArch32 v8.0, clang 3.7
• OSX Yosemite x86_64, Xcode clang 7.0.0

[MatzeB]

Thanks for working on this, this is looking good. At a first glance the changes look good (some comments below). I'll do a testrun later to see whether it still works on our platform with these changes.

External/SPEC/CINT2000/176.gcc/CMakeLists.txt

7–9

I think we can add that flag unconditionally.

External/SPEC/CINT2000/252.eon/CMakeLists.txt

8–10

Why do you need to disable this specific warning? I would assume we get many other warnings which are not disabled as well.
Also you can write it the if like this:

if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")

External/SPEC/CINT2000/253.perlbmk/CMakeLists.txt

47–49

We can add that flag unconditionally.

[MatzeB]

External/SPEC/CINT2000/252.eon/CMakeLists.txt

5–6

Without the -stdlib=libstdc++ the benchmark does not compile for me because it cannot find <iostream.h>. I'm surprised that worked with clang/OS X for you, maybe it is because you have an older version than me. I guess we need something like a global LIBSTDCXX_AVAILABLE variable that defaults to true if APPLE is set?

[kristof.beyls]

External/SPEC/CINT2000/176.gcc/CMakeLists.txt

7–9

Ah yes, indeed.
I tried to make sure that the cmake files allow to also run with gcc instead of clang, but -std=gnu89 is indeed also a gcc command line option with the same semantics.

External/SPEC/CINT2000/252.eon/CMakeLists.txt

5–6

It turns out my copy of spec2000 has been modified to work around the iostream.h issue, by adding an iostream.h in the eon source code directory, which just does:
#include <iostream>
using namespace std;

I really dislike changing the source code of the benchmark, but I also don't really like being stuck to use libstdc++ and not libc++.
I think that the ideal work-around would be to add a workaround like "#include <iostream>\nusing namespace std;", but without having to change the SPEC source code. I'm not sure what the least-convoluted way would be to achieve something like that from just adapting the 252.eon/CMakeLists.txt file. I'll think a bit more about this next week, if no one else comes up with a reasonable suggestion by then.

8–10

Strange, I thought I needed it to disable an error message, but I can't reproduce it. Let's remove this from my patch.

External/SPEC/CINT2000/253.perlbmk/CMakeLists.txt

47–49

agreed, will change that in the next version of this patch.

[MatzeB]

External/SPEC/CINT2000/252.eon/CMakeLists.txt

5–6

Well it's clearly ancient C++ code in the benchmark so working around that a bit is fair IMO. Anyway if it is indeed just iostream.h then I'd suggest we place a file like you described into the directory where the CMakeLists.txt is and add a -I flag for that. We already did similar hacks if you look at (test-suite/SPEC/CINT2006/483.xalancbmk/ppc/endian.h and test-suite/SPEC/CINT95/124.m88ksim/builtins.h)

[MatzeB]

I just pushed a r259581 which provides an iostream.h and fstream.h in the benchmark directory that just includes iostream and fstream and removes the -stdlib=libstdc++.

[kristof.beyls]

In D16688#342180, @MatzeB wrote:

I just pushed a r259581 which provides an iostream.h and fstream.h in the benchmark directory that just includes iostream and fstream and removes the -stdlib=libstdc++.

Hi Matthias,

I think I've just found out why we've been seeing differences in our runs.
It turns out that spec have made a number of dot-releases of both SPEC2000 and SPEC2006.
I've been using the latest release, i.e. SPEC2000 v1.3 (see http://spec.org/cpu2000/) and SPEC2006 v1.2 (see http://spec.org/cpu2006/).

The issue for 252.eon using iostream.h and fstream.h has been fixed/worked around in the SPEC2000 v1.3 source code (see http://spec.org/cpu2000/docs/changes_in_V1.3.html, search for 252.eon).
Next to the issue in 252.eon, I also see at least an issue with 403.gcc in SPEC2006: the input data files have been renamed, and therefore the inputs as recorded in the CMakefiles don't exist (see http://spec.org/cpu2006/Docs/changes-in-v1.2.html, search for 403.gcc for details).

So, I think before trying to fix the issues I'm seeing, we should first make a decision on which versions of spec2000 and spec2006 we'd want to support in the CMakefiles.
Would it be acceptable to only support spec2000 v1.3 and spec2006 v1.2 - i.e. the latest releases? If not, we'll probably need a few more conditionals in the CMakfiles, based on the spec release - e.g. for the changed names of the input files for 403.gcc.

What do you think?

Thanks,

Kristof

[MatzeB]

I never gave a LGTM even though there is no problem with the patch anymore, independently of the spec revision discussion.

External/SPEC/CFP2000/CMakeLists.txt

+list(APPEND LDFLAGS -lm)
 list(APPEND CPPFLAGS -DSPEC_CPU2000)
 if(CMAKE_SIZEOF_VOID_P EQUAL 8)
   list(APPEND CPPFLAGS -DSPEC_CPU2000_LP64)
 endif()
 
 macro(cfp2000_subdir)
   foreach(BENCHMARK ${ARGN})
     set(BENCHMARK_DIR ${TEST_SUITE_SPEC2000_ROOT}/benchspec/CFP2000/${BENCHMARK})

External/SPEC/CINT2000/176.gcc/CMakeLists.txt

 if(ENDIAN STREQUAL "big")
   list(APPEND CPPFLAGS -DHOST_WORDS_BIG_ENDIAN)
 endif()
 if(TARGET_OS STREQUAL "Darwin")
   list(APPEND LDFLAGS -Xlinker -stack_size -Xlinker 0x800000)
 endif()
+if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
+  list(APPEND CFLAGS -std=gnu89)
+endif()
 
 macro(test_input run_type ifile sfile)
   llvm_test_run(RUN_TYPE ${run_type}
     ${BENCHMARK_DIR}/data/${run_type}/input/${ifile}
     -o ${CMAKE_CURRENT_BINARY_DIR}/${sfile}
   )
   llvm_test_verify(RUN_TYPE ${run_type} ${FPCMP}
     ${BENCHMARK_DIR}/data/${run_type}/output/${sfile}

External/SPEC/CINT2000/252.eon/CMakeLists.txt

-list(APPEND CPPFLAGS -include errno.h -DHAS_ERRLIST -DUSE_STRERROR -DSPEC_STDCPP)
+list(APPEND CPPFLAGS -include errno.h -include string.h -DHAS_ERRLIST -DUSE_STRERROR -DSPEC_STDCPP)
 if(ARCH STREQUAL "PowerPC")
   list(APPEND CPPFLAGS -DFMAX_IS_DOUBLE)
 endif()
-list(APPEND CXXFLAGS -fno-exceptions -Wno-deprecated -fpermissive -stdlib=libstdc++)
-list(APPEND LDFLAGS -stdlib=libstdc++ -lm)
+list(APPEND CXXFLAGS -fno-exceptions -Wno-deprecated -fpermissive)
+list(APPEND LDFLAGS -lm)
 list(APPEND LIBS -lm)
+if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
+  list(APPEND CPPFLAGS -Wno-implicit-function-declaration)
+endif()
+
 include_directories(${BENCHMARK_DIR}/src)
 
 # There's more sourcefiles in the directory than necessary so enumerate the ones
 # we want.
 set(SourceNames
   ggCoverageSolidTexture.cc
   ggPathDielectricMaterial.cc
   ggBox2.cc

External/SPEC/CINT2000/253.perlbmk/CMakeLists.txt

set(SourceNames
   sv.c
   taint.c
   toke.c
   universal.c
   util.c
   MD5.c
   md5c.c
 )
+
+list(APPEND CPPFLAGS -DSPEC_CPU2000_NEED_BOOL)
+list(APPEND LDFLAGS -lm)
+
 if(TARGET_OS STREQUAL "Win32")
   list(APPEND SourceNames nt_perlmain.c win32.c win32sck.c perllib.c)
   list(APPEND CPPFLAGS -DSPEC_CPU2000_NTOS -DWIN32 -D_CONSOLE)
   include_directories(${BENCHMARK_DIR}/src)
   include_directories(${BENCHMARK_DIR}/src/win32)
 else()
   list(APPEND SourceNames unix_perlmain.c)
   list(APPEND CPPFLAGS -DSPEC_CPU2000_LINUX)
 endif()
+if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
+  list(APPEND CPPFLAGS -std=gnu89)
+endif()
 
 include(TestBigEndian)
 test_big_endian(IS_BIGENDIAN)
 if(IS_BIGENDIAN)
   if(CMAKE_SIZEOF_VOID_P EQUAL 8)
     list(APPEND CPPFLAGS -DBYTEORDER=0x87654321)
   else()
     list(APPEND CPPFLAGS -DBYTEORDER=0x4321)

External/SPEC/CINT2000/255.vortex/CMakeLists.txt

+list(APPEND LDFLAGS -lm)
+
 macro(test_input run_type input outfile)
   llvm_test_run(RUN_TYPE ${run_type}
     WORKDIR ${CMAKE_CURRENT_BINARY_DIR}/data/${run_type}/input
     ${input}
   )
   llvm_test_verify(RUN_TYPE ${run_type} ${FPCMP}
     ${BENCHMARK_DIR}/data/${run_type}/output/${outfile}
     ${CMAKE_CURRENT_BINARY_DIR}/data/${run_type}/input/${outfile}

External/SPEC/CINT2000/300.twolf/CMakeLists.txt

+list(APPEND CPPFLAGS -DHAVE_SIGNED_CHAR)
 list(APPEND LDFLAGS -lm)
 list(APPEND CFLAGS -Wno-return-type)
 
 macro(test_input run_type)
   llvm_test_run(RUN_TYPE ${run_type}
     WORKDIR ${CMAKE_CURRENT_BINARY_DIR}/data/${run_type}/input
     ${run_type}
     > ${CMAKE_CURRENT_BINARY_DIR}/data/${run_type}/input/${run_type}.stdout

External/SPEC/CINT2006/400.perlbench/CMakeLists.txt

 list(APPEND CPPFLAGS -DPERL_CORE -DI_TIME)
 list(APPEND CFLAGS -std=gnu89)
+list(APPEND LDFLAGS -lm)
 
 include_directories(${BENCHMARK_DIR}/src)
 
 set(SourceNames
   av.c
   deb.c
   doio.c
   doop.c

External/SPEC/CINT2006/445.gobmk/CMakeLists.txt

 list(APPEND CPPFLAS -DWANT_STDC_PROTO)
+list(APPEND LDFLAGS -lm)
 
 set(Source "")
 foreach(dir IN ITEMS
         src/sgf src/engine src/interface src/patterns src/utils src)
   include_directories(${BENCHMARK_DIR}/${dir})
   file(GLOB MoreSource ${BENCHMARK_DIR}/${dir}/*.c)
   list(APPEND Source ${MoreSource})
 endforeach()

External/SPEC/CINT2006/456.hmmer/CMakeLists.txt

+list(APPEND LDFLAGS -lm)
+
 macro(test_input run_type input output)
   # The benchmark modifies the data file, need to copy a fresh one
   llvm_test_run(RUN_TYPE ${run_type} EXECUTABLE cp -f
     ${BENCHMARK_DIR}/data/${run_type}/input/${input}
     ${CMAKE_CURRENT_BINARY_DIR}/${input}
   )
   llvm_test_run(RUN_TYPE ${run_type}
     ${ARGN} ${input}

External/SPEC/CINT2006/464.h264ref/CMakeLists.txt

-if(ARCH STREQUAL "PowerPC" AND TARGET_OS STREQUAL "Linux")
-  list(APPEND CFLAGS -fsigned-char)
-endif()
+list(APPEND LDFLAGS -lm)
 
 macro(test_input run_type leading level)
   llvm_test_run(RUN_TYPE ${run_type}
     -d ${leading}encoder_${level}.cfg
     > ${CMAKE_CURRENT_BINARY_DIR}/${leading}${level}_encodelog.out
     WORKDIR ${CMAKE_CURRENT_BINARY_DIR}/data/${run_type}/input
   )
   llvm_test_verify(RUN_TYPE ${run_type} ${FPCMP}