[libcxxabi] Add lit configs for libcxxabi tests.

[danalbert]

Summary

This makes running libcxxabi tests on Linux _much_ easier.
Adds a check-libcxxabi target to cmake.

Also defaults to building a dynamic libc++abi. This is so that the
default options still test the libc++abi that is being built. There are
two problems with testing a static libc++abi. In the case of a
standalone build, the tests will link the system's libc++, which might
not have been built against our libc++abi. In the case of an in tree
build, libc++ will prefer a dynamic libc++abi from the system over a
static libc++abi from the output directory.

[danalbert]

test/CMakeLists.txt

42

The only thing this conditional is doing is adding an additional dependency on libc++ if we're building in tree... is there a way to add a dependency to a custom target after add_custom_target has been called?

[compnerd]

CMakeLists.txt

104

Did you intend to make this shared by default?

110

I think that you should use a separate variable (LIBCXX_INCLUDE_DIR perhaps?) that is set to ${CMAKE_CURRENT_BINARY_DIR}/include rather than relying on LLVM's variable.

I realize that this is a standalone build, and therefore LLVM_INCLUDE_DIR *should* point to the same location, but it makes it something that you need to reason out (at least, I had to think about it for a moment).

test/lit.cfg

2

Is this the format that the rest of the python code in LLVM uses? I don't really care at the end of the day, just curious.

[danalbert]

CMakeLists.txt

104

Yes. See the long part of the summary. I'm willing to change this back to the way it was if people think that's better.

110

I don't understand. ${CMAKE_CURRENT_BINARY_DIR}/include won't contain the libcxx headers...

test/lit.cfg

2

I assumed it was. I copied it from libcxx/test/lit.cfg. I'll look in to it.

As a somewhat related note, there's a lot of redundancy between here and the lit.cfg in libcxx. It'd be nice to factor that out at some point...

[compnerd]

CMakeLists.txt

104

I think moving the comment here would be better.

110

AIUI, thats what it will be set to. It does actually get the headers copied there (at least, it did in my local tree).

test/lit.cfg

2

WFM. As I said, it was more that I was just surprised. Sharing that would be *great*.

[danalbert]

CMakeLists.txt

110

Hmm. They aren't for mine. Not for a standalone build, anyway. I'm running exactly the command from the docs.

[danalbert]

Some cleanup: now uses add_lit_testsuiite(), use find_path() to determine libcxx include directories (which also fixes a bug caused by relative paths).
Address some review comments.

[compnerd]

Under the assumption that lit.cfg is simply copied as I didn't look at that carefully based on your previous comment, LGTM. This should definitely help with testing, so I think that getting this in earlier is better.

[danalbert]

Committed as r212672.

CMakeLists.txt

   if(EXISTS ${LLVMCONFIG_FILE})
     list(APPEND CMAKE_MODULE_PATH "${LLVM_CMAKE_PATH}")
     include(${LLVMCONFIG_FILE})
+    include("${LLVM_CMAKE_PATH}/AddLLVM.cmake")
   else()
     message(FATAL_ERROR "Not found: ${LLVMCONFIG_FILE}")
   endif()

   set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
 
   set(LIBCXXABI_BUILT_STANDALONE 1)
+else()
+  set(LLVM_LIT "${CMAKE_SOURCE_DIR}/utils/lit/lit.py")
 endif()
 
 #===============================================================================

 option(LIBCXXABI_ENABLE_ASSERTIONS "Enable assertions independent of build mode." ON)
 option(LIBCXXABI_ENABLE_PEDANTIC "Compile with pedantic enabled." ON)
 option(LIBCXXABI_ENABLE_WERROR "Fail and stop if a warning is triggered." OFF)
-option(LIBCXXABI_ENABLE_SHARED "Build libc++abi as a shared library." OFF)
+
+# Default to building a shared library so that the default options still test
+# the libc++abi that is being built. There are two problems with testing a
+# static libc++abi. In the case of a standalone build, the tests will link the
+# system's libc++, which might not have been built against our libc++abi. In the
+# case of an in tree build, libc++ will prefer a dynamic libc++abi from the
+# system over a static libc++abi from the output directory.
+option(LIBCXXABI_ENABLE_SHARED "Build libc++abi as a shared library." ON)
+
+find_path(
+  LIBCXXABI_LIBCXX_INCLUDES
+  vector
+  PATHS ${LIBCXXABI_LIBCXX_INCLUDES}
+        ${CMAKE_BINARY_DIR}/${LIBCXXABI_LIBCXX_INCLUDES}
+        ${LLVM_MAIN_SRC_DIR}/projects/libcxx/include
+        ${LLVM_INCLUDE_DIR}/c++/v1
+  )
+
+set(LIBCXXABI_LIBCXX_INCLUDES "${LIBCXXABI_LIBCXX_INCLUDES}" CACHE STRING
+    "Specify path to libc++ includes." FORCE)
 
 #===============================================================================
 # Configure System

 # Add source code. This also contains all of the logic for deciding linker flags
 # soname, etc...
 add_subdirectory(src)
+
+if(NOT LIBCXXABI_ENABLE_SHARED)
+  # TODO: Fix the libc++ cmake files so that libc++abi can be statically linked.
+  # As it is now, libc++ will prefer linking against a dynamic libc++abi in the
+  # system library paths over a static libc++abi in the out directory. This
+  # would test the system library rather than the one we just built, which isn't
+  # very helpful.
+  message(WARNING "The libc++abi tests are currently only valid when "
+                  "LIBCXXABI_ENABLE_SHARED is on, no check target will be "
+                  "available!")
+else()
+  add_subdirectory(test)
+endif()

src/CMakeLists.txt

     )
 endif()
 
-if (LIBCXXABI_LIBCXX_INCLUDES)
-  include_directories("${LIBCXXABI_LIBCXX_INCLUDES}")
-elseif (NOT LIBCXXABI_BUILT_STANDALONE)
-  include_directories("${LLVM_MAIN_SRC_DIR}/projects/libcxx/include")
-else()
-  include_directories("${LLVM_INCLUDE_DIR}/c++/v1")
-endif()
+include_directories("${LIBCXXABI_LIBCXX_INCLUDES}")
 
 # Generate library list.
 set(libraries ${LIBCXXABI_CXX_ABI_LIBRARIES})

test/CMakeLists.txt

+macro(pythonize_bool var)
+  if (${var})
+    set(${var} True)
+  else()
+    set(${var} False)
+  endif()
+endmacro()
+
+set(LIBCXXABI_COMPILER ${CMAKE_CXX_COMPILER})
+set(LIBCXXABI_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/..)
+set(LIBCXXABI_BINARY_DIR ${CMAKE_BINARY_DIR})
+pythonize_bool(LIBCXXABI_ENABLE_SHARED)
+
+set(AUTO_GEN_COMMENT "## Autogenerated by libcxxabi configuration.\n# Do not edit!")
+configure_file(
+  ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
+  ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg
+  @ONLY)
+
+set(LIBCXXABI_TEST_DEPS cxxabi)
+if (NOT LIBCXXABI_BUILT_STANDALONE)
+  list(APPEND LIBCXXABI_TEST_DEPS cxx)
+endif()
+
+add_lit_testsuite(check-libcxxabi "Running libcxxabi tests"
+  ${CMAKE_CURRENT_BINARY_DIR}
+  DEPENDS ${LIBCXXABI_TEST_DEPS}
+  )

test/lit.cfg

+# -*- Python -*- vim: set syntax=python tabstop=4 expandtab cc=80:
+
+# Configuration file for the 'lit' test runner.
+
+import errno
+import os
+import platform
+import re
+import shlex
+import signal
+import subprocess
+import sys
+import tempfile
+import time
+
+import lit.Test
+import lit.formats
+import lit.util
+
+class LibcxxabiTestFormat(lit.formats.FileBasedTest):
+    """
+    Custom test format handler for use with the test format use by libc++abi.
+    """
+
+    def __init__(self, cxx_under_test, cpp_flags, ld_flags, exec_env):
+        self.cxx_under_test = cxx_under_test
+        self.cpp_flags = list(cpp_flags)
+        self.ld_flags = list(ld_flags)
+        self.exec_env = dict(exec_env)
+
+    def execute_command(self, command, in_dir=None):
+        kwargs = {
+            'stdin' :subprocess.PIPE,
+            'stdout':subprocess.PIPE,
+            'stderr':subprocess.PIPE,
+        }
+        if in_dir:
+            kwargs['cwd'] = in_dir
+        p = subprocess.Popen(command, **kwargs)
+        out,err = p.communicate()
+        exitCode = p.wait()
+
+        # Detect Ctrl-C in subprocess.
+        if exitCode == -signal.SIGINT:
+            raise KeyboardInterrupt
+
+        return out, err, exitCode
+
+    def execute(self, test, lit_config):
+        while True:
+            try:
+                return self._execute(test, lit_config)
+            except OSError, oe:
+                if oe.errno != errno.ETXTBSY:
+                    raise
+                time.sleep(0.1)
+
+    def _execute(self, test, lit_config):
+        # Extract test metadata from the test file.
+        requires = []
+        with open(test.getSourcePath()) as f:
+            for ln in f:
+                if 'XFAIL:' in ln:
+                    items = ln[ln.index('XFAIL:') + 6:].split(',')
+                    test.xfails.extend([s.strip() for s in items])
+                elif 'REQUIRES:' in ln:
+                    items = ln[ln.index('REQUIRES:') + 9:].split(',')
+                    requires.extend([s.strip() for s in items])
+                elif not ln.startswith("//") and ln.strip():
+                    # Stop at the first non-empty line that is not a C++
+                    # comment.
+                    break
+
+        # Check that we have the required features.
+        #
+        # FIXME: For now, this is cribbed from lit.TestRunner, to avoid
+        # introducing a dependency there. What we more ideally would like to do
+        # is lift the "requires" handling to be a core lit framework feature.
+        missing_required_features = [f for f in requires
+                                     if f not in test.config.available_features]
+        if missing_required_features:
+            return (lit.Test.UNSUPPORTED,
+                    "Test requires the following features: %s" % (
+                      ', '.join(missing_required_features),))
+
+        # Evaluate the test.
+        return self._evaluate_test(test, lit_config)
+
+    def _evaluate_test(self, test, lit_config):
+        name = test.path_in_suite[-1]
+        source_path = test.getSourcePath()
+        source_dir = os.path.dirname(source_path)
+
+        # If this is a compile (failure) test, build it and check for failure.
+        exec_file = tempfile.NamedTemporaryFile(suffix="exe", delete=False)
+        exec_path = exec_file.name
+        exec_file.close()
+
+        try:
+            compile_cmd = [self.cxx_under_test, '-o', exec_path,
+                   source_path] + self.cpp_flags + self.ld_flags
+            cmd = compile_cmd
+            out, err, exitCode = self.execute_command(cmd)
+            if exitCode != 0:
+                report = """Command: %s\n""" % ' '.join(["'%s'" % a
+                                                         for a in cmd])
+                report += """Exit Code: %d\n""" % exitCode
+                if out:
+                    report += """Standard Output:\n--\n%s--""" % out
+                if err:
+                    report += """Standard Error:\n--\n%s--""" % err
+                report += "\n\nCompilation failed unexpectedly!"
+                return lit.Test.FAIL, report
+
+            cmd = []
+            if self.exec_env:
+                cmd.append('env')
+                cmd.extend('%s=%s' % (name, value)
+                           for name,value in self.exec_env.items())
+            cmd.append(exec_path)
+            if lit_config.useValgrind:
+                cmd = lit_config.valgrindArgs + cmd
+            out, err, exitCode = self.execute_command(cmd, source_dir)
+            if exitCode != 0:
+                report = """Compiled With: %s\n""" % \
+                    ' '.join(["'%s'" % a for a in compile_cmd])
+                report += """Command: %s\n""" % \
+                    ' '.join(["'%s'" % a for a in cmd])
+                report += """Exit Code: %d\n""" % exitCode
+                if out:
+                    report += """Standard Output:\n--\n%s--""" % out
+                if err:
+                    report += """Standard Error:\n--\n%s--""" % err
+                report += "\n\nCompiled test failed unexpectedly!"
+                return lit.Test.FAIL, report
+        finally:
+            try:
+                os.remove(exec_path)
+            except:
+                pass
+        return lit.Test.PASS, ""
+
+# name: The name of this test suite.
+config.name = 'libc++abi'
+
+# suffixes: A list of file extensions to treat as test files.
+config.suffixes = ['.cpp']
+
+# test_source_root: The root path where tests are located.
+config.test_source_root = os.path.dirname(__file__)
+
+# Gather various compiler parameters.
+cxx_under_test = lit_config.params.get('cxx_under_test', None)
+if cxx_under_test is None:
+    cxx_under_test = getattr(config, 'cxx_under_test', None)
+
+    # If no specific cxx_under_test was given, attempt to infer it as clang++.
+    if cxx_under_test is None:
+        clangxx = lit.util.which('clang++', config.environment['PATH'])
+        if clangxx is not None:
+            cxx_under_test = clangxx
+    lit_config.note("inferred cxx_under_test as: %r" % (cxx_under_test,))
+if cxx_under_test is None:
+    lit_config.fatal('must specify user parameter cxx_under_test '
+                     '(e.g., --param=cxx_under_test=clang++)')
+
+libcxxabi_src_root = lit_config.params.get('libcxxabi_src_root', None)
+if libcxxabi_src_root is None:
+    libcxxabi_src_root = getattr(config, 'libcxxabi_src_root', None)
+    if libcxxabi_src_root is None:
+        libcxxabi_src_root = os.path.dirname(config.test_source_root)
+
+libcxxabi_obj_root = lit_config.params.get('libcxxabi_obj_root', None)
+if libcxxabi_obj_root is None:
+    libcxxabi_obj_root = getattr(config, 'libcxxabi_obj_root', None)
+    if libcxxabi_obj_root is None:
+        libcxxabi_obj_root = libcxxabi_src_root
+
+libcxx_includes = lit_config.params.get('libcxx_includes', None)
+if libcxx_includes is None:
+    libcxx_includes = getattr(config, 'libcxx_includes', None)
+    if libcxx_includes is None:
+        lit_config.fatal("libcxx_includes must be defined")
+
+enable_shared = lit_config.params.get('enable_shared', None)
+if enable_shared is None:
+    enable_shared = getattr(config, 'enable_shared', None)
+    if enable_shared is None:
+        lit_config.fatal("enable_shared must be defined")
+
+link_flags = []
+link_flags_str = lit_config.params.get('link_flags', None)
+if link_flags_str is None:
+    link_flags_str = getattr(config, 'link_flags', None)
+    if link_flags_str is None:
+        if enable_shared:
+            link_flags += ['-lc++abi']
+        if sys.platform == 'darwin':
+            link_flags += ['-lSystem']
+        elif sys.platform == 'linux2':
+            link_flags += ['-lgcc_eh', '-lc', '-lm', '-lpthread', '-lgcc_s']
+        else:
+            lit_config.fatal("unrecognized system")
+
+        lit_config.note("inferred link_flags as: %r" % (link_flags,))
+if link_flags_str is not None:
+    link_flags += shlex.split(link_flags_str)
+
+# Configure extra compiler flags.
+include_paths = ['-I' + libcxxabi_src_root + '/include',
+                 '-I' + libcxx_includes]
+library_paths = ['-L' + libcxxabi_obj_root + '/lib']
+compile_flags = ['-std=c++11']
+
+# Configure extra linker parameters.
+exec_env = {}
+if sys.platform == 'darwin':
+    exec_env['DYLD_LIBRARY_PATH'] = os.path.join(libcxxabi_obj_root, 'lib')
+elif sys.platform == 'linux2':
+    link_flags += ['-Wl,-R', libcxxabi_obj_root + '/lib']
+else:
+    lit_config.fatal("unrecognized system")
+
+config.test_format = LibcxxabiTestFormat(
+    cxx_under_test,
+    cpp_flags = ['-nostdinc++'] + compile_flags + include_paths,
+    ld_flags = ['-nodefaultlibs'] + library_paths + ['-lc++'] + link_flags,
+    exec_env = exec_env)
+
+# Get or infer the target triple.
+config.target_triple = lit_config.params.get('target_triple', None)
+# If no target triple was given, try to infer it from the compiler under test.
+if config.target_triple is None:
+    config.target_triple = lit.util.capture(
+        [cxx_under_test, '-dumpmachine']).strip()
+    lit_config.note("inferred target_triple as: %r" % (config.target_triple,))

test/lit.site.cfg.in

+@AUTO_GEN_COMMENT@
+config.cxx_under_test        = "@LIBCXXABI_COMPILER@"
+config.libcxxabi_src_root    = "@LIBCXXABI_SOURCE_DIR@"
+config.libcxxabi_obj_root    = "@LIBCXXABI_BINARY_DIR@"
+config.python_executable     = "@PYTHON_EXECUTABLE@"
+config.enable_shared         = @LIBCXXABI_ENABLE_SHARED@
+config.libcxx_includes       = "@LIBCXXABI_LIBCXX_INCLUDES@"
+
+# Let the main config do the real work.
+lit_config.load_config(config, "@LIBCXXABI_SOURCE_DIR@/test/lit.cfg")

www/index.html

   <ul>
   <li><code>cd libcxxabi</code></li>
   <li><code>mkdir build &amp;&amp; cd build</code></li>
-  <li><code>cmake -DLIBCXXABI_LIBCXX_INCLUDES=path/to/libcxx/include .. # on linux you may need -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++</code></li>
+  <li><code>cmake -DLIBCXXABI_LIBCXX_INCLUDES=path/to/libcxx/include .. # on
+    linux you may need -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++</code></li>
   <li><code>make</code></li>
   </ul>
 
+  <p>To run the tests:</p>
+  <ul>
+  <li><code>make check-libcxxabi</code></li>
+  </ul>
+  <p>Note: in a standalone build, the system's libc++ will be used for tests. If
+  the system's libc++ was statically linked against libc++abi (or linked against
+  a different ABI library), this may interfere with test results.</p>
+
   <p>Send discussions to the
   (<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev">clang mailing list</a>).</p>