This is an archive of the discontinued LLVM Phabricator instance.

Differential D27490

Add CTMark directory to test-suite
ClosedPublic

Authored by MatzeB on Dec 6 2016, 4:14 PM.

Details

Reviewers
cmatthews
Commits
rOLDT289269: Add a CTMark directory
rL289269: Add a CTMark directory
Summary

This is a proposal to make CTMark an explicit entity in the test-suite. Some notes:

  • You can use this with cmake -DTEST_SUITE_SUBDIRS=CTMark
  • It cannot be used in combination with SingleSource or MultiSource benchmarks (duplicate targets)
  • The usage of symlinks makes the benchmarks comprising CTMark obvious while reusing the existing CMakeLists.txt of these benchmarks.
  • Symlink usage may be problematic on windows (but does the test-suite work properly on windows anyway?)
  • The way this is setup will miss intermediate CMakeLists.txt (i.e. if someone adds CPPFLAGS in say MultiSource/Benchmarks/CMakeLists.txt we will miss that because we include MultiSource/Benchmarks/tramp3d-v4/CMakeLists.txt directly)

I believe none of the problems are big enough to not do this. But would be nice to hear some opinions.

Diff Detail

Repository
rL LLVM

Event Timeline

MatzeB updated this revision to Diff 80499.Dec 6 2016, 4:14 PM
MatzeB retitled this revision from to Add CTMark directory to test-suite.
MatzeB updated this object.
MatzeB added a reviewer: cmatthews.
MatzeB set the repository for this revision to rL LLVM.
MatzeB added a subscriber: llvm-commits.
Herald added subscribers: mgorny, mcrosier. · View Herald TranscriptDec 6 2016, 4:14 PM
cmatthews accepted this revision.Dec 9 2016, 12:00 PM
cmatthews edited edge metadata.

I think we need this badly! We will be able to move towards removing all the silly conditionals towards nice well defined "suites" of tests.

This revision is now accepted and ready to land.Dec 9 2016, 12:00 PM
Closed by commit rL289269: Add a CTMark directory (authored by matze). · Explain WhyDec 9 2016, 12:58 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
test-suite/
trunk/
4 lines
CTMark/
1 line
1 line
10 lines
1 line
13 lines
1 line
1 line
1 line
1 line
1 line
1 line
1 line

Diff 80939

test-suite/trunk/CMakeLists.txt

Show First 20 Lines • Show All 172 Lines • ▼ Show 20 Lines
# directory where they will be picked up automatically. Alternatively you may # directory where they will be picked up automatically. Alternatively you may
# manually specify directories to include test-suites at external locations # manually specify directories to include test-suites at external locations
# and to leave out some of the default ones. # and to leave out some of the default ones.
if(NOT TEST_SUITE_SUBDIRS) if(NOT TEST_SUITE_SUBDIRS)
file(GLOB sub_cmakelists */CMakeLists.txt) file(GLOB sub_cmakelists */CMakeLists.txt)
set(TEST_SUITE_SUBDIRS "") set(TEST_SUITE_SUBDIRS "")
foreach(entry ${sub_cmakelists}) foreach(entry ${sub_cmakelists})
get_filename_component(subdir ${entry} DIRECTORY) get_filename_component(subdir ${entry} DIRECTORY)
if(NOT ${subdir} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/tools) # Exclude tools and CTMark from default list
if(NOT ${subdir} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/tools AND
NOT ${subdir} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/CTMark)
list(APPEND TEST_SUITE_SUBDIRS ${subdir}) list(APPEND TEST_SUITE_SUBDIRS ${subdir})
endif() endif()
endforeach() endforeach()
set(TEST_SUITE_SUBDIRS "${TEST_SUITE_SUBDIRS}" CACHE STRING set(TEST_SUITE_SUBDIRS "${TEST_SUITE_SUBDIRS}" CACHE STRING
"Semicolon separated list of directories with CMakeLists.txt to include") "Semicolon separated list of directories with CMakeLists.txt to include")
endif() endif()
mark_as_advanced(TEST_SUITE_SUBDIRS) mark_as_advanced(TEST_SUITE_SUBDIRS)
▲ Show 20 Lines • Show All 45 Lines • Show Last 20 Lines

test-suite/trunk/CTMark/7zip

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Benchmarks/7zip
No newline at end of file

test-suite/trunk/CTMark/Bullet

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Benchmarks/Bullet
No newline at end of file

test-suite/trunk/CTMark/CMakeLists.txt

add_subdirectory(7zip)
add_subdirectory(Bullet)
add_subdirectory(tramp3d-v4)
add_subdirectory(kimwitu++)
add_subdirectory(lencod)
add_subdirectory(SPASS)
add_subdirectory(ClamAV)
add_subdirectory(sqlite3)
add_subdirectory(consumer-typeset)
add_subdirectory(mafft)

test-suite/trunk/CTMark/ClamAV

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Applications/ClamAV
No newline at end of file

test-suite/trunk/CTMark/README.md

Compile Time Mark
=================
This is a collection of applications used for compiler performance testing. It
currently features a selection of larger applications from the test-suite.
You can use this with:
$ cmake -DTEST_SUITE_SUBDIRS=CTMark ...
See also:
- http://lab.llvm.org:8080/green/view/Compile%20Time/
- http://lists.llvm.org/pipermail/llvm-dev/2016-November/107087.html

test-suite/trunk/CTMark/SPASS

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Applications/SPASS
No newline at end of file

test-suite/trunk/CTMark/consumer-typeset

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Benchmarks/MiBench/consumer-typeset
No newline at end of file

test-suite/trunk/CTMark/kimwitu++

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Applications/kimwitu++
No newline at end of file

test-suite/trunk/CTMark/lencod

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Applications/JM/lencod
No newline at end of file

test-suite/trunk/CTMark/mafft

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Benchmarks/mafft
No newline at end of file

test-suite/trunk/CTMark/sqlite3

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Applications/sqlite3
No newline at end of file

test-suite/trunk/CTMark/tramp3d-v4

PropertyOld ValueNew Value
svn:specialnull*
link ../MultiSource/Benchmarks/tramp3d-v4
No newline at end of file