[compiler-rt][builtins] CTest to execute builtins testsuite

[farazs]

Summary

There doesn't seem to be a defined procedure for running the builtins testsuite. There is a 'test' shell script in test/builtins/Unit but it seems to be outdated and specific to darwin with hard-coded paths.

Comments in test/CMakeLists.txt suggest that builtins and BlockRunTime testsuites have not yet been ported to lit framework. Attached is a rudimentary CTest setup to run the unit tests in builtins, as a stop-gap until the test
sources can be modified for lit.

The new target to trigger test run is `check-builtins'. It is intended to stop after building tests for cross-builds, else build and execute for native builds. Hope it is useful!

[samsonov]

Sorry for delay, I haven't yet had a chance to review this. IIRC the rest of LLVM regression test suites doesn't really work with CTest, though. Not sure if it's a good temporary solution. Anyway, will provide more comments later.

[samsonov]

Once again, sorry for delay with reviewing this.

test/builtins/CMakeLists.txt

1

Please specify somewhere that you're actually using a host compiler to build these tests, and link them against a just-built builtins library.

184

${COMPILER_RT_SOURCE_DIR}/lib/builtins

187

Is it picked up by CMake automatically?

193

please check COMPILER_RT_CAN_EXECUTE_TESTS here instead - we don't need to add targets otherwise.

196

consider using get_filename_component instead.

209

Use ${COMPILER_RT_LIBRARY_OUTPUT_DIR} instead.

214

Why not LINK_FLAGS?

[farazs]

I don't mind the delay, please take your time. This is not exactly high priority.
Thanks for your comments & corrections, I agree with most. Few clarifications below.

test/builtins/CMakeLists.txt

187

No. Gets picked on line 199 as ${{test}_EXTRA}

193

It is checked below for add_test(line 202). My plan was to be able to cross-build the test-suite even when we can't execute it. The dev environment is usually much faster and I think it might be useful to fix all build issues before moving to the target for execution.

If this doesn't seem like a good idea, can we just remove the check from this file and modify the upper test/CMakeLists.txt instead? It already has a COMPILER_RT_CAN_EXECUTE_TESTS block. Enclosing add_subdirectory in that block should give the desired effect and be consistent with the way sanitizer tests are controlled.

214

No hard reason - just following cmake docs. Linkers are known to be finicky about the order of command-line options. My guess is, to construct a correct invocation, cmake should be able to distinguish libraries from all other linker options.

[samsonov]

FWIW, I agree that it would bring more benefit to just port these tests to lit. The RUN-line for them would be similar and trivial (smth. like RUN: %clang %s -O2 -o %t && %t), and would allow us to test just-built Clang, which seems like the right thing to do (we'll also automatically test that Clang driver appropriately links in builtins library). RUN-lines are also suited to the special cases like "compile this file with -fnested-functions").

test/builtins/CMakeLists.txt

193

Yes, the latter sounds good.

[sgundapa]

This builds the unit tests along with the library.
We want to build the unit tests when invoked with "check-builtins". Isn't it?

I am writing the lit cfg files for test/builtins and will upload it when I am done testing it.

[farazs]

In D4251#10, @sgundapa wrote:

We want to build the unit tests when invoked with "check-builtins". Isn't it?

That is exactly what this patch does.

I am writing the lit cfg files for test/builtins and will upload it when I am done testing it.

Great! In that case I won't bother doing a second revision. Should I mark this as abandoned?

[sgundapa]

You can mark this as abandoned once I push mine.
Sounds good ?

--Sumanth G

[jaydeep]

Hi Sumanth,

Any update on this?

Thanks,
Jaydeep

test/CMakeLists.txt

 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.common.configured.in
   ${CMAKE_CURRENT_BINARY_DIR}/lit.common.configured)
 
 # BlocksRuntime and builtins testsuites are not yet ported to lit.
 # add_subdirectory(BlocksRuntime)
-# add_subdirectory(builtins)
+
+# a CTest module is in place to run builtins unit testsuite
+add_subdirectory(builtins)
 
 set(SANITIZER_COMMON_LIT_TEST_DEPS)
 # When ANDROID, we build tests with the host compiler (i.e. CMAKE_C_COMPILER),
 # and run tests with tools from the host toolchain.
 if(NOT ANDROID)
   if(NOT COMPILER_RT_STANDALONE_BUILD)
     # Use LLVM utils and Clang from the same build tree.
     list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS

test/builtins/CMakeLists.txt

+# CTest for builtins Unit test-suite
+#
+# directory:Unit contains a large amount of C unit-test code for the
+# generic implementations of the core runtime library along
+# architecture-specific tests for some features.
+#
+# directory:timing contains code to measure the time take to execute
+# certain performance-critical functions. Darwin-specific, not
+# yet ported to other platforms.
+
+enable_testing()
+
+set(GENERIC_SOURCES
+  Unit/absvdi2_test.c
+  Unit/absvsi2_test.c
+  Unit/absvti2_test.c
+  Unit/adddf3vfp_test.c
+  Unit/addsf3vfp_test.c
+  Unit/addtf3_test.c
+  Unit/addvdi3_test.c
+  Unit/addvsi3_test.c
+  Unit/addvti3_test.c
+  Unit/ashldi3_test.c
+  Unit/ashlti3_test.c
+  Unit/ashrdi3_test.c
+  Unit/ashrti3_test.c
+  Unit/bswapdi2_test.c
+  Unit/bswapsi2_test.c
+  Unit/clear_cache_test.c
+  Unit/clzdi2_test.c
+  Unit/clzsi2_test.c
+  Unit/clzti2_test.c
+  Unit/cmpdi2_test.c
+  Unit/cmpti2_test.c
+  Unit/comparedf2_test.c
+  Unit/comparesf2_test.c
+  Unit/ctzdi2_test.c
+  Unit/ctzsi2_test.c
+  Unit/ctzti2_test.c
+  Unit/divdc3_test.c
+  Unit/divdf3vfp_test.c
+  Unit/divdi3_test.c
+  Unit/divmodsi4_test.c
+  Unit/divsc3_test.c
+  Unit/divsf3vfp_test.c
+  Unit/divsi3_test.c
+  Unit/divtc3_test.c
+  Unit/divtf3_test.c
+  Unit/divti3_test.c
+  Unit/divxc3_test.c
+  Unit/enable_execute_stack_test.c
+  Unit/eqdf2vfp_test.c
+  Unit/eqsf2vfp_test.c
+  Unit/eqtf2_test.c
+  Unit/extebdsfdf2vfp_test.c
+  Unit/extenddftf2_test.c
+  Unit/extendsftf2_test.c
+  Unit/ffsdi2_test.c
+  Unit/ffsti2_test.c
+  Unit/fixdfdi_test.c
+  Unit/fixdfsivfp_test.c
+  Unit/fixdfti_test.c
+  Unit/fixsfdi_test.c
+  Unit/fixsfsivfp_test.c
+  Unit/fixsfti_test.c
+  Unit/fixunsdfdi_test.c
+  Unit/fixunsdfsi_test.c
+  Unit/fixunsdfsivfp_test.c
+  Unit/fixunsdfti_test.c
+  Unit/fixunssfdi_test.c
+  Unit/fixunssfsi_test.c
+  Unit/fixunssfsivfp_test.c
+  Unit/fixunssfti_test.c
+  Unit/fixunstfdi_test.c
+  Unit/fixunsxfdi_test.c
+  Unit/fixunsxfsi_test.c
+  Unit/fixunsxfti_test.c
+  Unit/fixxfdi_test.c
+  Unit/fixxfti_test.c
+  Unit/floatdidf_test.c
+  Unit/floatdisf_test.c
+  Unit/floatdixf_test.c
+  Unit/floatsidfvfp_test.c
+  Unit/floatsisfvfp_test.c
+  Unit/floattidf_test.c
+  Unit/floattisf_test.c
+  Unit/floattixf_test.c
+  Unit/floatundidf_test.c
+  Unit/floatundisf_test.c
+  Unit/floatundixf_test.c
+  Unit/floatunssidfvfp_test.c
+  Unit/floatunssisfvfp_test.c
+  Unit/floatuntidf_test.c
+  Unit/floatuntisf_test.c
+  Unit/floatuntixf_test.c
+  Unit/gcc_personality_test.c
+  Unit/gedf2vfp_test.c
+  Unit/gesf2vfp_test.c
+  Unit/getf2_test.c
+  Unit/gtdf2vfp_test.c
+  Unit/gtsf2vfp_test.c
+  Unit/gttf2_test.c
+  Unit/ledf2vfp_test.c
+  Unit/lesf2vfp_test.c
+  Unit/letf2_test.c
+  Unit/lshrdi3_test.c
+  Unit/lshrti3_test.c
+  Unit/ltdf2vfp_test.c
+  Unit/ltsf2vfp_test.c
+  Unit/lttf2_test.c
+  Unit/moddi3_test.c
+  Unit/modsi3_test.c
+  Unit/modti3_test.c
+  Unit/muldc3_test.c
+  Unit/muldf3vfp_test.c
+  Unit/muldi3_test.c
+  Unit/mulodi4_test.c
+  Unit/mulosi4_test.c
+  Unit/muloti4_test.c
+  Unit/mulsc3_test.c
+  Unit/mulsf3vfp_test.c
+  Unit/multc3_test.c
+  Unit/multi3_test.c
+  Unit/multf3_test.c
+  Unit/mulvdi3_test.c
+  Unit/mulvsi3_test.c
+  Unit/mulvti3_test.c
+  Unit/mulxc3_test.c
+  Unit/nedf2vfp_test.c
+  Unit/negdf2vfp_test.c
+  Unit/negdi2_test.c
+  Unit/negsf2vfp_test.c
+  Unit/negti2_test.c
+  Unit/negvdi2_test.c
+  Unit/negvsi2_test.c
+  Unit/negvti2_test.c
+  Unit/nesf2vfp_test.c
+  Unit/netf2_test.c
+  Unit/paritydi2_test.c
+  Unit/paritysi2_test.c
+  Unit/parityti2_test.c
+  Unit/popcountdi2_test.c
+  Unit/popcountsi2_test.c
+  Unit/popcountti2_test.c
+  Unit/powidf2_test.c
+  Unit/powisf2_test.c
+  Unit/powitf2_test.c
+  Unit/powixf2_test.c
+  Unit/subdf3vfp_test.c
+  Unit/subsf3vfp_test.c
+  Unit/subtf3_test.c
+  Unit/subvdi3_test.c
+  Unit/subvsi3_test.c
+  Unit/subvti3_test.c
+  Unit/trampoline_setup_test.c
+  Unit/truncdfsf2vfp_test.c
+  Unit/trunctfdf2_test.c
+  Unit/trunctfsf2_test.c
+  Unit/ucmpdi2_test.c
+  Unit/ucmpti2_test.c
+  Unit/udivdi3_test.c
+  Unit/udivmoddi4_test.c
+  Unit/udivmodsi4_test.c
+  Unit/udivmodti4_test.c
+  Unit/udivsi3_test.c
+  Unit/udivti3_test.c
+  Unit/umoddi3_test.c
+  Unit/umodsi3_test.c
+  Unit/umodti3_test.c
+  Unit/unorddf2vfp_test.c
+  Unit/unordsf2vfp_test.c
+  Unit/unordtf2_test.c)
+
+set(ppc_SOURCES
+  Unit/ppc/fixtfdi_test.c
+  Unit/ppc/floatditf_test.c
+  Unit/ppc/floatunditf_test.c
+  Unit/ppc/qadd_test.c
+  Unit/ppc/qdiv_test.c
+  Unit/ppc/qmul_test.c
+  Unit/ppc/qsub_test.c)
+
+# For tests that rely on common(*_lib.h) headers
+include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../../lib/builtins)
+
+# gcc personality needs an extra source file
+set(gcc_personality_EXTRA Unit/gcc_personality_test_helper.cxx)
+
+# auto-trigger ctest
+add_custom_target(check-builtins COMMAND ${CMAKE_CTEST_COMMAND})
+
+foreach(arch x86_64 i386 arm ppc mips)
+  if(CAN_TARGET_${arch})
+    foreach (src ${GENERIC_SOURCES} ${${arch}_SOURCES})
+      # Strip leading directories and file-name suffix for executable
+      string(REGEX REPLACE "[^/]*/" "" test ${src})
+      string(REPLACE "_test.c" "" test ${test})
+
+      add_executable(${test} ${src} ${${test}_EXTRA})
+      add_dependencies(check-builtins ${test})
+      # skip execution for cross-builds
+      if(COMPILER_RT_CAN_EXECUTE_TESTS)
+        add_test(${test} ${test})
+      endif()
+
+      # Link against freshly built libclang-rt.builtins and libm
+      target_link_libraries(${test} -lm -lclang_rt.builtins-${arch})
+      set_target_properties(${test} PROPERTIES
+        LINK_FLAGS "-L${CMAKE_BINARY_DIR}/lib/${COMPILER_RT_OS_DIR}")
+    endforeach ()
+
+    set_target_properties(gcc_personality PROPERTIES
+      COMPILE_OPTIONS "-fexceptions"
+      LINK_LIBRARIES "-lstdc++")
+    set_target_properties(trampoline_setup PROPERTIES
+      COMPILE_OPTIONS "-fnested-functions")
+  endif()
+endforeach()