[test] Split LLDB tests into filecheck, unittest and dotest.

[JDevlieghere]

Summary

LLDB has three major testing strategies: unit tests, tests that exercise the SB API though dotest.py and what we currently call lit tests. The later is rather confusing as we're now using lit as the driver for all three types of tests. As most of this grew organically, the directory structure in the lldb repository doesn't really make this clear. The 'lit' tests are part of the root and among these tests there's a Unit and Suite folder for the unit and dotest-tests. This makes it impossible to run just the lit tests.

This patch changes the directory layout to match the 3 testing strategies, each with their own directory and their own configuration file. I've opted to name the directories after the testing tool, so dotest for the dotest.py tests and filecheck for the lit tests. This means there are now 3 directories under lit with 3 corresponding targets in addition to the usual check-lldb:

• check-lldb-unittests (replaces check-lldb-unit)
• check-lldb-dotest
• check-lldb-filecheck

Each directory defines its own lit test suite (which was already the case). The top level directory also has a lit.cfg.py which is there for check-lldb and is mostly empty.

Current layout:

lit
├── CMakeLists.txt
├── lit-lldb-init.in
├── lit.cfg.py
├── lit.site.cfg.py.in
├── Breakpoint
├── BuildScript
├── Commands
├── Driver
├── ExecControl
├── Expr
├── Heap
├── Host
├── Minidump
├── Modules
├── ObjectFile
├── Process
├── Python
├── Quit
├── Register
├── Reproducer
├── Settings
├── Suite          <- dotest
├── SymbolFile
├── Unit           <- unittests
├── Unwind
├── Watchpoint
├── dotest
├── filecheck
├── helper
└── lldb

New layout:

lit
├── CMakeLists.txt
├── lit.cfg.py
├── dotest
├── filecheck
└── unittests

My goal is to move the unit test and dotest files into their corresponding subdirectories, remove them from the root and finally rename lit to tests. This is the first step in that direction.

[jingham]

This seems much clearer to me.

[JDevlieghere]

Essentially this patch just moves all the filecheck tests in a directory called filecheck and renames the Suite and Unit directories to dotest and unittests respectively.

[thopre]

Shouldn't the CMake target remain check-lldb-unit to be consistant with check-llvm-unit/check-clang-unit etc.?

[mstorsjo]

and finally rename lit to tests

Shouldn't that be test, to be consistent with llvm/lld/clang and others?

[labath]

In D68606#1699197, @thopre wrote:

Shouldn't the CMake target remain check-lldb-unit to be consistant with check-llvm-unit/check-clang-unit etc.?

In D68606#1699219, @mstorsjo wrote:

and finally rename lit to tests

Shouldn't that be test, to be consistent with llvm/lld/clang and others?

+1 to both of these.

More generally, there are other things here which are not consistent with llvm layouts, and I am wondering how much should try to be compatible with that. On one hand, consistency is good, but on the other, llvm does not have _three_ kinds of tests to worry about. For instance, test/Unit *is* consistent with the llvm location of unit tests, and by extension test/Suite could be considered the "right place" for non-filecheck tests too (though maybe it shouldn't be called "Suite"). Speaking of that, I am not completely sure about the name "filecheck" either -- technically this should be "shtest" as that is their main distinguishing feature, though that may end up being too pedantic.

[tatyana-krasnukha]

I have longed for this change, thank you!

[JDevlieghere]

We discussed this on IRC a bit and this is my updated proposal for the directory names:

• API
• Shell
• Unit

The corresponding targets are lldb-check-unit (unchanged), lldb-check-shell (runs just the FileCheck / ShTests) and lldb-check-api (runs just the dotest.py tests). I've also added a README to both API and Unit explaining where the actual tests can be found.

[JDevlieghere]

Rename lit to test to decrease the number of renames in the git history.

[jankratochvil]

Before this patch I was getting:

#rm -rf *
cmake ~/redhat/llvm-monorepo2/llvm/ -DCMAKE_BUILD_TYPE=Release  -DLLVM_USE_LINKER=gold -DLLVM_ENABLE_PROJECTS="lldb;clang;lld"  -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_ENABLE_ASSERTIONS=ON
make check-lldb
...
llvm-lit: /home/jkratoch/redhat/llvm-monorepo2/llvm/utils/lit/lit/llvm/subst.py:134: fatal: Did not find count in /home/jkratoch/redhat/llvm-monorepo2-clangassert/./bin

After this patch I am getting:

[100%] Running lldb lit test suite
llvm-lit: /home/jkratoch/redhat/llvm-monorepo2/llvm/utils/lit/lit/discovery.py:133: warning: unable to find test suite for '/home/jkratoch/redhat/llvm-monorepo2-clangassert/tools/lldb/test'
llvm-lit: /home/jkratoch/redhat/llvm-monorepo2/llvm/utils/lit/lit/discovery.py:244: warning: input '/home/jkratoch/redhat/llvm-monorepo2-clangassert/tools/lldb/test' contained no tests
Testing Time: 0.00s
2 warning(s) in tests.
[100%] Built target check-lldb-lit
Scanning dependencies of target check-lldb
[100%] Built target check-lldb

A Fedora buildbot has not yet built this release: http://lab.llvm.org:8014/builders/lldb-x86_64-fedora?numbuilds=1000

lldb/CMakeLists.txt

if(APPLE)
         endif()
         add_lldb_test_dependency(cxx)
       endif()
     endif()
   endif()
 
   add_subdirectory(test)
   add_subdirectory(unittests)
-  add_subdirectory(lit)
   add_subdirectory(utils/lit-cpuid)
   add_subdirectory(utils/lldb-dotest)
 endif()
 
 if (NOT LLDB_DISABLE_PYTHON)
     if(NOT LLDB_BUILD_FRAMEWORK)
       set(use_python_wrapper_from_src_dir -m)
     endif()

lldb/lit/CMakeLists.txt

-# Test runner infrastructure for LLDB. This configures the LLDB test trees
-# for use by Lit, and delegates to LLVM's lit test handlers.
-
-# LLVM_BUILD_MODE is used in lit.site.cfg
-if (CMAKE_CFG_INTDIR STREQUAL ".")
-  set(LLVM_BUILD_MODE ".")
-else ()
-  set(LLVM_BUILD_MODE "%(build_mode)s")
-endif ()
-
-if (CMAKE_SIZEOF_VOID_P EQUAL 8)
-  set(LLDB_IS_64_BITS 1)
-endif()
-
-get_property(LLDB_DOTEST_ARGS GLOBAL PROPERTY LLDB_DOTEST_ARGS_PROPERTY)
-set(dotest_args_replacement ${LLVM_BUILD_MODE})
-
-set(LLDB_TEST_MODULE_CACHE_LLDB "${LLDB_TEST_BUILD_DIRECTORY}/module-cache-lldb" CACHE PATH "The Clang module cache used by the Clang embedded in LLDB while running tests.")
-set(LLDB_TEST_MODULE_CACHE_CLANG "${LLDB_TEST_BUILD_DIRECTORY}/module-cache-clang" CACHE PATH "The Clang module cache used by the Clang while building tests.")
-
-if(LLDB_BUILT_STANDALONE)
-  # In paths to our build-tree, replace CMAKE_CFG_INTDIR with our configuration name placeholder.
-  string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} config_runtime_output_dir ${LLVM_RUNTIME_OUTPUT_INTDIR})
-  string(REPLACE ${LLVM_RUNTIME_OUTPUT_INTDIR} ${config_runtime_output_dir} LLDB_DOTEST_ARGS "${LLDB_DOTEST_ARGS}")
-
-  # Remaining ones must be paths to the provided LLVM build-tree.
-  if(LLVM_CONFIGURATION_TYPES)
-    # LLDB uses single-config; LLVM multi-config; pick one and prefer Release types.
-    # Otherwise, if both use multi-config the default is fine.
-    if(NOT CMAKE_CONFIGURATION_TYPES)
-      if(RelWithDebInfo IN_LIST LLVM_CONFIGURATION_TYPES)
-        set(dotest_args_replacement RelWithDebInfo)
-      elseif(Release IN_LIST LLVM_CONFIGURATION_TYPES)
-        set(dotest_args_replacement Release)
-      else()
-        list(GET LLVM_CONFIGURATION_TYPES 0 dotest_args_replacement)
-      endif()
-    endif()
-  else()
-    # Common case: LLVM used a single-configuration generator like Ninja.
-    set(dotest_args_replacement ".")
-  endif()
-endif()
-
-string(REPLACE ${CMAKE_CFG_INTDIR} ${dotest_args_replacement} LLDB_DOTEST_ARGS "${LLDB_DOTEST_ARGS}")
-string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_LIBS_DIR ${LLVM_LIBRARY_OUTPUT_INTDIR})
-string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_TOOLS_DIR ${LLVM_RUNTIME_OUTPUT_INTDIR})
-
-add_lldb_test_dependency(
-  lit-cpuid
-  llc
-  lli
-  llvm-config
-  llvm-dwarfdump
-  llvm-nm
-  llvm-mc
-  llvm-objcopy
-  llvm-readobj
-  llvm-strip
-  )
-
-if(TARGET lld)
-  add_lldb_test_dependency(lld)
-endif()
-
-# the value is not canonicalized within LLVM
-llvm_canonicalize_cmake_booleans(
-  LLDB_DISABLE_PYTHON
-  LLVM_ENABLE_ZLIB
-  LLVM_ENABLE_SHARED_LIBS
-  LLDB_IS_64_BITS)
-
-configure_lit_site_cfg(
-  ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
-  ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
-  MAIN_CONFIG
-  ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py)
-configure_lit_site_cfg(
-  ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in
-  ${CMAKE_CURRENT_BINARY_DIR}/Unit/lit.site.cfg.py
-  MAIN_CONFIG
-  ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.cfg.py)
-configure_lit_site_cfg(
-  ${CMAKE_CURRENT_SOURCE_DIR}/Suite/lit.site.cfg.in
-  ${CMAKE_CURRENT_BINARY_DIR}/Suite/lit.site.cfg)
-
-configure_file(
-  ${CMAKE_CURRENT_SOURCE_DIR}/lit-lldb-init.in
-  ${CMAKE_CURRENT_BINARY_DIR}/lit-lldb-init)
-
-if(NOT LLDB_BUILT_STANDALONE)
-  add_lldb_test_dependency(
-    FileCheck
-    count
-    not
-  )
-endif()
-
-add_lit_testsuite(check-lldb-lit "Running lldb lit test suite"
-  ${CMAKE_CURRENT_BINARY_DIR}
-  DEPENDS lldb-test-deps
-  )
-
-set_target_properties(check-lldb-lit PROPERTIES FOLDER "lldb tests")
-
-# If we're building with an in-tree clang, then list clang as a dependency
-# to run tests.
-if (TARGET clang)
-  add_dependencies(check-lldb-lit clang)
-endif()
-
-add_lit_testsuites(LLDB
-  ${CMAKE_CURRENT_SOURCE_DIR}
-  DEPENDS lldb-test-deps
-  )

lldb/test/API/README.md

+# LLDB API Tests
+
+This directory only exists for the lit test driver. The actual tests live in
+the `tests` directory in top level LLDB directory.

lldb/test/API/lit.cfg

 # -*- Python -*-
 
 # Configuration file for the 'lit' test runner.
 
 import os
 import platform
 import shlex
 
 import lit.formats
 
 # name: The name of this test suite.
-config.name = 'lldb-Suite'
+config.name = 'lldb-api'
 
 # suffixes: A list of file extensions to treat as test files.
 config.suffixes = ['.py']
 
 # test_source_root: The root path where tests are located.
 # test_exec_root: The root path where tests should be run.
 config.test_source_root = os.path.join(config.lldb_src_root, 'packages',
                                        'Python', 'lldbsuite', 'test')

 
 if config.lldb_build_directory:
   dotest_cmd += ['--build-dir', config.lldb_build_directory]
 
 if config.lldb_module_cache:
   dotest_cmd += ['--module-cache-dir', config.lldb_module_cache]
 
 # Load LLDB test format.
-sys.path.append(os.path.join(config.lldb_src_root, "lit", "Suite"))
+sys.path.append(os.path.join(config.lldb_src_root, "test", "API"))
 import lldbtest
 
 # testFormat: The test format to use to interpret tests.
 config.test_format = lldbtest.LLDBTest(dotest_cmd)

lldb/test/API/lit.site.cfg.in

 config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.shared_libs = @LLVM_ENABLE_SHARED_LIBS@
 config.llvm_use_sanitizer = "@LLVM_USE_SANITIZER@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.lldb_build_directory = "@LLDB_TEST_BUILD_DIRECTORY@"
 config.lldb_module_cache = "@LLDB_TEST_MODULE_CACHE_LLDB@"
 config.clang_module_cache = "@LLDB_TEST_MODULE_CACHE_CLANG@"
 config.python_executable = "@PYTHON_EXECUTABLE@"
-config.dotest_path = "@LLDB_SOURCE_DIR@/test/dotest.py"
+config.dotest_path = "@LLDB_SOURCE_DIR@/test/API/dotest.py"
 config.dotest_args_str = "@LLDB_DOTEST_ARGS@"
 config.lldb_disable_python = @LLDB_DISABLE_PYTHON@
 config.dotest_lit_args_str = None
 
 # Additional dotest arguments can be passed to lit by providing a
 # semicolon-separates list: --param dotest-args="arg;arg".
 dotest_lit_args_str = lit_config.params.get('dotest-args', None)
 if dotest_lit_args_str:
     config.dotest_lit_args_str = dotest_lit_args_str
 
 # Support substitution of the tools and libs dirs with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
     config.dotest_args_str = config.dotest_args_str % lit_config.params
     config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@LLDB_SOURCE_DIR@/lit/Suite/lit.cfg")
+lit_config.load_config(config, "@LLDB_SOURCE_DIR@/test/API/lit.cfg")

lldb/test/CMakeLists.txt

-function(add_python_test_target name test_script args comment)
-  set(PYTHON_TEST_COMMAND
-    ${PYTHON_EXECUTABLE}
-    ${test_script}
-    ${args}
-    )
-
-  add_custom_target(${name}
-    COMMAND ${PYTHON_TEST_COMMAND} ${ARG_DEFAULT_ARGS}
-    COMMENT "${comment}"
-    USES_TERMINAL
-    )
-  add_dependencies(${name} lldb-test-deps)
-endfunction()
-
-# The default architecture with which to compile test executables is the default LLVM target
-# architecture, which itself defaults to the host architecture.
-string(TOLOWER "${LLVM_TARGET_ARCH}" LLDB_DEFAULT_TEST_ARCH)
-if( LLDB_DEFAULT_TEST_ARCH STREQUAL "host" )
-  string(REGEX MATCH "^[^-]*" LLDB_DEFAULT_TEST_ARCH ${LLVM_HOST_TRIPLE})
-endif ()
-
-# Allow the user to override the default by setting LLDB_TEST_ARCH
-set(LLDB_TEST_ARCH
-  ${LLDB_DEFAULT_TEST_ARCH}
-  CACHE STRING "Specify the architecture to run LLDB tests as (x86|x64).  Determines whether tests are compiled with -m32 or -m64")
-
-# Users can override LLDB_TEST_USER_ARGS to specify arbitrary arguments to pass to the script
-set(LLDB_TEST_USER_ARGS
-  ""
-  CACHE STRING "Specify additional arguments to pass to test runner. For example: '-C gcc -C clang -A i386 -A x86_64'")
-
-# The .noindex suffix is a marker for Spotlight to never index the
-# build directory.  LLDB queries Spotlight to locate .dSYM bundles
-# based on the UUID embedded in a binary, and because the UUID is a
-# hash of filename and .text section, there *will* be conflicts inside
-# the build directory.
-set(LLDB_TEST_COMMON_ARGS
-  --arch=${LLDB_TEST_ARCH}
-  -s
-  ${CMAKE_BINARY_DIR}/lldb-test-traces
-  -S nm
-  -u CXXFLAGS
-  -u CFLAGS
-  )
-
-list(APPEND LLDB_TEST_COMMON_ARGS
-  --executable ${LLDB_TEST_EXECUTABLE}
-  --dsymutil ${LLDB_TEST_DSYMUTIL}
-  --filecheck ${LLDB_TEST_FILECHECK}
-  -C ${LLDB_TEST_C_COMPILER}
-  )
+# Test runner infrastructure for LLDB. This configures the LLDB test trees
+# for use by Lit, and delegates to LLVM's lit test handlers.
 
-if ( CMAKE_SYSTEM_NAME MATCHES "Windows" )
-  # All tests are currently flaky on Windows, so rerun them all once when they fail.
-  set(LLDB_TEST_COMMON_ARGS ${LLDB_TEST_COMMON_ARGS} --rerun-all-issues)
+add_subdirectory(API)
 
-  set(LLDB_TEST_DEBUG_TEST_CRASHES
-    0
-    CACHE BOOL "(Windows only) Enables debugging of tests in the test suite by showing the crash dialog when lldb crashes")
-
-  set(LLDB_TEST_HIDE_CONSOLE_WINDOWS
-    1
-    CACHE BOOL "(Windows only) Hides the console window for an inferior when it is launched through the test suite")
-
-  if (LLDB_TEST_DEBUG_TEST_CRASHES)
-    set(LLDB_TEST_COMMON_ARGS ${LLDB_TEST_COMMON_ARGS} --enable-crash-dialog)
-  endif()
+# LLVM_BUILD_MODE is used in lit.site.cfg
+if (CMAKE_CFG_INTDIR STREQUAL ".")
+  set(LLVM_BUILD_MODE ".")
+else ()
+  set(LLVM_BUILD_MODE "%(build_mode)s")
+endif ()
 
-  if (NOT LLDB_TEST_HIDE_CONSOLE_WINDOWS)
-    set(LLDB_TEST_COMMON_ARGS ${LLDB_TEST_COMMON_ARGS} --show-inferior-console)
-  endif()
+if (CMAKE_SIZEOF_VOID_P EQUAL 8)
+  set(LLDB_IS_64_BITS 1)
 endif()
 
-if (NOT ${CMAKE_SYSTEM_NAME} MATCHES "Windows|Darwin")
-  list(APPEND LLDB_TEST_COMMON_ARGS
-    --env ARCHIVER=${CMAKE_AR} --env OBJCOPY=${CMAKE_OBJCOPY})
-endif()
+get_property(LLDB_DOTEST_ARGS GLOBAL PROPERTY LLDB_DOTEST_ARGS_PROPERTY)
+set(dotest_args_replacement ${LLVM_BUILD_MODE})
 
-if (NOT "${LLDB_LIT_TOOLS_DIR}" STREQUAL "")
-  if (NOT EXISTS "${LLDB_LIT_TOOLS_DIR}")
-    message(WARNING "LLDB_LIT_TOOLS_DIR ${LLDB_LIT_TOOLS_DIR} does not exist.")
-  endif()
-endif()
+set(LLDB_TEST_MODULE_CACHE_LLDB "${LLDB_TEST_BUILD_DIRECTORY}/module-cache-lldb" CACHE PATH "The Clang module cache used by the Clang embedded in LLDB while running tests.")
+set(LLDB_TEST_MODULE_CACHE_CLANG "${LLDB_TEST_BUILD_DIRECTORY}/module-cache-clang" CACHE PATH "The Clang module cache used by the Clang while building tests.")
 
-if(CMAKE_HOST_APPLE)
-  if(LLDB_BUILD_FRAMEWORK)
-    get_target_property(framework_build_dir liblldb LIBRARY_OUTPUT_DIRECTORY)
-    list(APPEND LLDB_TEST_COMMON_ARGS --framework ${framework_build_dir}/LLDB.framework)
-  endif()
+if(LLDB_BUILT_STANDALONE)
+  # In paths to our build-tree, replace CMAKE_CFG_INTDIR with our configuration name placeholder.
+  string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} config_runtime_output_dir ${LLVM_RUNTIME_OUTPUT_INTDIR})
+  string(REPLACE ${LLVM_RUNTIME_OUTPUT_INTDIR} ${config_runtime_output_dir} LLDB_DOTEST_ARGS "${LLDB_DOTEST_ARGS}")
 
-  # Use the same identity for testing
-  get_property(code_sign_identity_used GLOBAL PROPERTY LLDB_DEBUGSERVER_CODESIGN_IDENTITY)
-  if(code_sign_identity_used)
-    list(APPEND LLDB_TEST_COMMON_ARGS --codesign-identity "${code_sign_identity_used}")
+  # Remaining ones must be paths to the provided LLVM build-tree.
+  if(LLVM_CONFIGURATION_TYPES)
+    # LLDB uses single-config; LLVM multi-config; pick one and prefer Release types.
+    # Otherwise, if both use multi-config the default is fine.
+    if(NOT CMAKE_CONFIGURATION_TYPES)
+      if(RelWithDebInfo IN_LIST LLVM_CONFIGURATION_TYPES)
+        set(dotest_args_replacement RelWithDebInfo)
+      elseif(Release IN_LIST LLVM_CONFIGURATION_TYPES)
+        set(dotest_args_replacement Release)
+      else()
+        list(GET LLVM_CONFIGURATION_TYPES 0 dotest_args_replacement)
       endif()
-
-  if(LLDB_USE_SYSTEM_DEBUGSERVER)
-    lldb_find_system_debugserver(system_debugserver_path)
-    add_custom_target(debugserver
-      COMMAND ${CMAKE_COMMAND} -E copy_if_different
-      ${system_debugserver_path} ${LLVM_RUNTIME_OUTPUT_INTDIR}
-      COMMENT "Copying the system debugserver to LLDB's binaries directory for testing.")
-    # The custom target for the system debugserver has no install target, so we
-    # need to remove it from the LLVM_DISTRIBUTION_COMPONENTS list.
-    if (LLVM_DISTRIBUTION_COMPONENTS)
-      list(REMOVE_ITEM LLVM_DISTRIBUTION_COMPONENTS debugserver)
-      set(LLVM_DISTRIBUTION_COMPONENTS ${LLVM_DISTRIBUTION_COMPONENTS} PARENT_SCOPE)
     endif()
-    message(STATUS "LLDB tests use out-of-tree debugserver: ${system_debugserver_path}")
-    list(APPEND LLDB_TEST_COMMON_ARGS --out-of-tree-debugserver)
-    add_lldb_test_dependency(debugserver)
-  elseif(TARGET debugserver)
-    set(debugserver_path ${LLVM_RUNTIME_OUTPUT_INTDIR}/debugserver)
-    message(STATUS "LLDB Tests use just-built debugserver: ${debugserver_path}")
-    list(APPEND LLDB_TEST_COMMON_ARGS --server ${debugserver_path})
-    add_lldb_test_dependency(debugserver)
-  elseif(TARGET lldb-server)
-    set(lldb_server_path ${LLVM_RUNTIME_OUTPUT_INTDIR}/lldb-server)
-    message(STATUS "LLDB Tests use just-built lldb-server: ${lldb_server_path}")
-    list(APPEND LLDB_TEST_COMMON_ARGS --server ${lldb_server_path})
-    add_lldb_test_dependency(lldb-server)
   else()
-    message(WARNING "LLDB Tests enabled, but no server available")
+    # Common case: LLVM used a single-configuration generator like Ninja.
+    set(dotest_args_replacement ".")
   endif()
 endif()
 
-set(LLDB_DOTEST_ARGS ${LLDB_TEST_COMMON_ARGS};${LLDB_TEST_USER_ARGS})
-set_property(GLOBAL PROPERTY LLDB_DOTEST_ARGS_PROPERTY ${LLDB_DOTEST_ARGS})
-
-add_custom_target(check-lldb)
-add_dependencies(check-lldb lldb-test-deps)
-set_target_properties(check-lldb PROPERTIES FOLDER "lldb misc")
+string(REPLACE ${CMAKE_CFG_INTDIR} ${dotest_args_replacement} LLDB_DOTEST_ARGS "${LLDB_DOTEST_ARGS}")
+string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_LIBS_DIR ${LLVM_LIBRARY_OUTPUT_INTDIR})
+string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} LLDB_TOOLS_DIR ${LLVM_RUNTIME_OUTPUT_INTDIR})
+
+add_lldb_test_dependency(
+  lit-cpuid
+  llc
+  lli
+  llvm-config
+  llvm-dwarfdump
+  llvm-nm
+  llvm-mc
+  llvm-objcopy
+  llvm-readobj
+  llvm-strip
+  )
 
-# LLD is required to link test executables on Windows.
-if (CMAKE_SYSTEM_NAME MATCHES "Windows")
-  if (TARGET lld)
-    add_lldb_test_dependency(lld)
-  else ()
-    message(WARNING "lld required to test LLDB on Windows")
-  endif ()
-endif ()
+if(TARGET lld)
+endif()
+
+# the value is not canonicalized within LLVM
+llvm_canonicalize_cmake_booleans(
+  LLDB_DISABLE_PYTHON
+  LLVM_ENABLE_ZLIB
+  LLVM_ENABLE_SHARED_LIBS
+  LLDB_IS_64_BITS)
+
+configure_lit_site_cfg(
+  ${CMAKE_CURRENT_SOURCE_DIR}/Shell/lit.site.cfg.py.in
+  ${CMAKE_CURRENT_BINARY_DIR}/Shell/lit.site.cfg.py
+  MAIN_CONFIG
+  ${CMAKE_CURRENT_SOURCE_DIR}/Shell/lit.cfg.py)
+configure_lit_site_cfg(
+  ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in
+  ${CMAKE_CURRENT_BINARY_DIR}/Unit/lit.site.cfg.py
+  MAIN_CONFIG
+  ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.cfg.py)
+configure_lit_site_cfg(
+  ${CMAKE_CURRENT_SOURCE_DIR}/API/lit.site.cfg.in
+  ${CMAKE_CURRENT_BINARY_DIR}/API/lit.site.cfg)
+
+configure_file(
+  ${CMAKE_CURRENT_SOURCE_DIR}/Shell/lit-lldb-init.in
+  ${CMAKE_CURRENT_BINARY_DIR}/Shell/lit-lldb-init)
+
+if(NOT LLDB_BUILT_STANDALONE)
+  add_lldb_test_dependency(
+    FileCheck
+    count
+    not
+  )
+endif()
 
-# Run the lit-style tests and the unittests as part of the check-lldb target.
-add_dependencies(check-lldb check-lldb-lit)
+add_lit_testsuite(check-lldb-lit "Running lldb lit test suite"
+  ${CMAKE_CURRENT_BINARY_DIR}
+  DEPENDS lldb-test-deps
+  )
+
+set_target_properties(check-lldb-lit PROPERTIES FOLDER "lldb tests")
 
-# This will add LLDB's test dependencies to the dependencies for check-all and
-# include them in the test-depends target.
-set_property(GLOBAL APPEND PROPERTY LLVM_LIT_DEPENDS ${ARG_DEPENDS})
+# If we're building with an in-tree clang, then list clang as a dependency
+# to run tests.
+if (TARGET clang)
+  add_dependencies(check-lldb-lit clang)
+endif()
+
+add_lit_testsuites(LLDB
+  ${CMAKE_CURRENT_SOURCE_DIR}
+  DEPENDS lldb-test-deps
+  )

lldb/test/Shell/Python/crashlog.test

 #                                                                 -*- python -*-
 # REQUIRES: system-darwin
-# RUN: cd %S/../../examples/python && cat %s | %lldb | FileCheck %s
+# RUN: cd %S/../../../examples/python && cat %s | %lldb | FileCheck %s
 # CHECK-LABEL: {{S}}KIP BEYOND CHECKS
 script
 import crashlog
 cl = crashlog.CrashLog
 images = [
 "0x10b60b000 - 0x10f707fff com.apple.LLDB.framework (1.1000.11.38.2 - 1000.11.38.2) <96E36F5C-1A83-39A1-8713-5FDD9701C3F1> /Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/LLDB",
 # CHECK: 0x10b60b000
 # CHECK: 0x10f707fff

 
 "0x1111111 - 0x22222 +MyApp Pro (0) <01234> /tmp/MyApp Pro.app/MyApp Pro",
 # CHECK: 0x1111111
 # CHECK: 0x22222
 # CHECK: MyApp Pro
 # CHECK: (0)
 # CHECK: 01234
 # CHECK: /tmp/MyApp Pro.app/MyApp Pro
-    
 "0x7fff63f20000 - 0x7fff63f77ff7  libc++.1.dylib (400.9.4) /usr/lib/libc++.1.dylib"
 # CHECK: 0x7fff63f20000
 # CHECK: 0x7fff63f77ff7
 # CHECK: libc++.1.dylib
 # CHECK: (400.9.4)
 # CHECK: None
 # CHECK: /usr/lib/libc++.1.dylib
 ]

lldb/test/Shell/helper/toolchain.py

build_script_args = [build_script,
                         '--arch=' + str(config.lldb_bitness)]
     if config.lldb_lit_tools_dir:
         build_script_args.append('--tools-dir={0}'.format(config.lldb_lit_tools_dir))
     if config.lldb_tools_dir:
         build_script_args.append('--tools-dir={0}'.format(config.lldb_tools_dir))
     if config.llvm_libs_dir:
         build_script_args.append('--libs-dir={0}'.format(config.llvm_libs_dir))
 
-    lldb_init = os.path.join(config.test_exec_root, 'lit-lldb-init')
+    lldb_init = os.path.join(config.test_exec_root, 'Shell', 'lit-lldb-init')
 
     primary_tools = [
         ToolSubst('%lldb',
                   command=FindTool('lldb'),
                   extra_args=['--no-lldbinit', '-S', lldb_init]),
         ToolSubst('%lldb-init',
                   command=FindTool('lldb'),
                   extra_args=['-S', lldb_init]),

lldb/test/Shell/lit.cfg.py

 from lit.llvm.subst import FindTool
 from lit.llvm.subst import ToolSubst
 from distutils.spawn import find_executable
 
 site.addsitedir(os.path.dirname(__file__))
 from helper import toolchain
 
 # name: The name of this test suite.
-config.name = 'LLDB'
+config.name = 'lldb-shell'
 
 # testFormat: The test format to use to interpret tests.
 config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
 
 # suffixes: A list of file extensions to treat as test files. This is overriden
 # by individual lit.local.cfg files in the test subdirectories.
 config.suffixes = ['.test', '.cpp', '.s']
 
 # excludes: A list of directories to exclude from the testsuite. The 'Inputs'
 # subdirectories contain auxiliary inputs for various tests in their parent
 # directories.
 config.excludes = ['Inputs', 'CMakeLists.txt', 'README.txt', 'LICENSE.txt']
 
 # test_source_root: The root path where tests are located.
 config.test_source_root = os.path.dirname(__file__)
 
 # test_exec_root: The root path where tests should be run.
-config.test_exec_root = os.path.join(config.lldb_obj_root, 'lit')
+config.test_exec_root = os.path.join(config.lldb_obj_root, 'test')
 
 
 llvm_config.use_default_substitutions()
 
 toolchain.use_lldb_substitutions(config)
 
 toolchain.use_support_substitutions(config)
 

lldb/test/Shell/lit.site.cfg.py.in

 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@LLDB_SOURCE_DIR@/lit/lit.cfg.py")
+lit_config.load_config(config, "@LLDB_SOURCE_DIR@/test/Shell/lit.cfg.py")

lldb/test/Unit/README.md

+# LLDB Unit Tests
+
+This directory only exists for the lit test driver. The actual tests live in
+the `unittest` directory in top level LLDB directory.

lldb/test/Unit/lit.cfg.py

 # -*- Python -*-
 
 # Configuration file for the 'lit' test runner.
 
 import os
 import sys
 
 import lit.formats
 from lit.llvm import llvm_config
 
 # name: The name of this test suite.
-config.name = 'lldb-Unit'
+config.name = 'lldb-unit'
 
 # suffixes: A list of file extensions to treat as test files.
 config.suffixes =  []
 
 # test_source_root: The root path where unit test binaries are located.
 # test_exec_root: The root path where tests should be run.
 config.test_source_root = os.path.join(config.lldb_obj_root, 'unittests')
 config.test_exec_root = config.test_source_root

lldb/test/Unit/lit.site.cfg.py.in

 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@LLDB_SOURCE_DIR@/lit/Unit/lit.cfg.py")
+lit_config.load_config(config, "@LLDB_SOURCE_DIR@/test/Unit/lit.cfg.py")

lldb/test/lit.cfg.py

+# -*- Python -*-
+
+import os
+import platform
+import re
+import shutil
+import site
+import sys
+
+import lit.formats
+from lit.llvm import llvm_config
+from lit.llvm.subst import FindTool
+from lit.llvm.subst import ToolSubst
+from distutils.spawn import find_executable
+
+# This is the top level configuration. Most of these configuration options will
+# be overriden by individual lit configuration files in the test
+# subdirectories.
+
+# name: The name of this test suite.
+config.name = 'lldb'
+
+# testFormat: The test format to use to interpret tests.
+config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
+
+# suffixes: A list of file extensions to treat as test files. This is overriden
+# by individual lit.local.cfg files in the test subdirectories.
+config.suffixes = ['.test', '.cpp', '.s']
+
+# excludes: A list of directories to exclude from the testsuite. The 'Inputs'
+# subdirectories contain auxiliary inputs for various tests in their parent
+# directories.
+config.excludes = ['Inputs', 'CMakeLists.txt', 'README.txt', 'LICENSE.txt']
+
+# test_source_root: The root path where tests are located.
+config.test_source_root = os.path.dirname(__file__)
+
+# test_exec_root: The root path where tests should be run.
+config.test_exec_root = os.path.join(config.lldb_obj_root, 'test')
+
+llvm_config.use_default_substitutions()