[test-suite] Enable MicroBenchmarks by default

[homerdin]

Summary

With the changes in https://reviews.llvm.org/rL327422 and https://reviews.llvm.org/rL327710 the MicroBenchmarks directory produces meaning data. From what I could find that was the only reason to exclude the directory by default.

[homerdin]

Ping

[MatzeB]

LGTM. Please watch the buildbots carefully (and revert in case of problems) when you push this.

[aemerson]

Looks like this change caused http://green.lab.llvm.org/green/job/test-suite-verify-machineinstrs-aarch64-O0-g/1944/ as well as others.

[homerdin]

The cmake system in the benchmark library uses cmake's try_run if CMAKE_CROSSCOMPILING is not set. It looks like this is what was causing the failures (trying to run something it cannot). The documentation for cross-compiling clang/llvm using clang/llvm lists -DCMAKE_CROSSCOMPILING=True as a needed cmake option. I'm not sure if we would want the same for the test suite. Ideally we would automatically set the variable when cross compiling. I have not been able to reproduce the issue so I'm not certain if this is the cause. Is anyone familiar with cmake and cross compilation?

CMake Error at MicroBenchmarks/libs/benchmark-1.3.0/CMakeLists.txt:181 (message):
  Failed to determine the source files for the regular expression backend

[MatzeB]

Cross compiling currently is a mess. Basically cmake only works properly when you have a complete gcc style cross toolchain, or a MacOS style toolchain on Mac. Most llvm developers currently are not in a position to set one of those two things up when they just want to test a compiler they made some changes to; to be honest when I started hacking on the llvm test-suite cmake I wasn't even aware of what assumptions are made there/what problem exist...

So today we circumvent some of the cmake conventions. That said try_run tests are just lazy and will somewhat fall short in cross compilation setting sanyway. How about forcing the googletest cmake to use C++11 and always use std::regex?

[homerdin]

I've added some definitions to the MicroBenchmarks/libs/CMakeLists.txt. Keeping these out of the benchmark library CMakeLists.txt will prevent having to make custom changes whenever the benchmark library is upgraded. I am not currently setup to test this while cross compiling. The try_run's should still compile but fail to build, but with the HAVE_STD_REGEX this will not be a fatal error.

Adding dberris and Eizan as reviewers.

[MatzeB]

LGTM

[homerdin]

Added a definition that will prevent the benchmark library from building cxx03_test which will fail when using c++11.

CMakeLists.txt

 # directory where they will be picked up automatically. Alternatively you may
 # manually specify directories to include test-suites at external locations
 # and to leave out some of the default ones.
 if(NOT TEST_SUITE_SUBDIRS)
   file(GLOB sub_cmakelists RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} */CMakeLists.txt)
   set(TEST_SUITE_SUBDIRS "")
   foreach(entry ${sub_cmakelists})
     get_filename_component(subdir ${entry} DIRECTORY)
-   # Exclude tools, CTMark, and MicroBenchmarks from default list
-    if(NOT ${subdir} STREQUAL tools AND NOT ${subdir} STREQUAL CTMark
-       AND NOT ${subdir} STREQUAL MicroBenchmarks)
+   # Exclude tools and CTMark from default list
+    if(NOT ${subdir} STREQUAL tools AND NOT ${subdir} STREQUAL CTMark)
       list(APPEND TEST_SUITE_SUBDIRS ${subdir})
     endif()
   endforeach()
   set(TEST_SUITE_SUBDIRS "${TEST_SUITE_SUBDIRS}")
 endif()
 set(TEST_SUITE_SUBDIRS "${TEST_SUITE_SUBDIRS}" CACHE STRING
     "Semicolon separated list of directories with CMakeLists.txt to include")
 

MicroBenchmarks/CMakeLists.txt

+if(NOT DEFINED DISABLE_CXX)
   file(COPY lit.local.cfg DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
 
   add_subdirectory(libs)
   add_subdirectory(XRay)
   add_subdirectory(LCALS)
+endif()

MicroBenchmarks/libs/CMakeLists.txt

+# The benchmark library build system uses try_run tests to determine regex
+# Circumvent these tests and enforce std::regex
+set(CMAKE_CXX_STANDARD 11)
+set(HAVE_STD_REGEX 1)
+# When enforcing c++11 we need to disable one of the benchmark library's tests
+set(BENCHMARK_HAS_CXX03_FLAG 0)
+
 add_subdirectory(benchmark-1.3.0)
 test_suite_add_build_dependencies(benchmark)
 test_suite_add_build_dependencies(output_test_helper)