Fix (benignly) incorrect GoogleTest specs in various lit configs.

[dlj]

Summary

The GoogleTest lit format accepts two parameters to its constructor: a subdirectory to find test binaries, and a required suffix for the test filenames. Typically, the config should look like this:

config.test_format = lit.formats.GoogleTest('.', 'Tests')

This will search the current directory for filenames ending with 'Tests', and run the matching binaries it finds as GoogleTests. It appears that several lit configs pass a different value for the subdirectory, however. They use "config.llvm_build_mode", which would be set by @LLVM_BUILD_MODE@, but never is. Fortunately, this means that the Python lookup passes and finds an empty string, which means that lit searches the current directory.

I can imagine cases where looking for a build-specific subdirectory might have worked in the paste, but I suspect it is no longer the right behaviour to check with cmake. So, I'm removing the (somewhat confusing) LLVM_BUILD_MODE logic altogether.

[andrewng]

Hi,

I believe that this "build mode" is intended for the Visual Studio MSVC build. This build is special in that it can produce builds for multiple configurations, e.g. Debug, Release & RelWithDebInfo, within the same top level build output directory. It is this configuration type that defines the "build mode". This means that the unit tests will only pick up the configuration that matches that of the lit that was run. Without the "build mode" I believe lit might end up running all configurations of unit tests that have been built, which is probably not the intended behaviour.

Cheers,
Andrew

[dlj]

In D34853#798699, @andrewng wrote:

Hi,

I believe that this "build mode" is intended for the Visual Studio MSVC build. This build is special in that it can produce builds for multiple configurations, e.g. Debug, Release & RelWithDebInfo, within the same top level build output directory. It is this configuration type that defines the "build mode". This means that the unit tests will only pick up the configuration that matches that of the lit that was run. Without the "build mode" I believe lit might end up running all configurations of unit tests that have been built, which is probably not the intended behaviour.

Cheers,
Andrew

Ah, excellent. I suspected something like this.

(I don't usually have access to Windows, and even then I typically use Ninja instead of VS... so cleanups like this could be hit-or-miss, I guess.)

cfe/trunk/test/Unit/lit.cfg

 # test_source_root: The root path where tests are located.
 # test_exec_root: The root path where tests should be run.
 clang_obj_root = getattr(config, 'clang_obj_root', None)
 if clang_obj_root is not None:
     config.test_exec_root = os.path.join(clang_obj_root, 'unittests')
     config.test_source_root = config.test_exec_root
 
 # testFormat: The test format to use to interpret tests.
-llvm_build_mode = getattr(config, 'llvm_build_mode', "Debug")
-config.test_format = lit.formats.GoogleTest(llvm_build_mode, 'Tests')
+config.test_format = lit.formats.GoogleTest('.', 'Tests')
 
 # Propagate the temp directory. Windows requires this because it uses \Windows\
 # if none of these are present.
 if 'TMP' in os.environ:
     config.environment['TMP'] = os.environ['TMP']
 if 'TEMP' in os.environ:
     config.environment['TEMP'] = os.environ['TEMP']
 

cfe/trunk/test/Unit/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
-config.llvm_build_mode = "@LLVM_BUILD_MODE@"
 config.clang_obj_root = "@CLANG_BINARY_DIR@"
 config.enable_shared = @ENABLE_SHARED@
 config.shlibdir = "@SHLIBDIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 
-# Support substitution of the tools_dir, libs_dirs, and build_mode with user
+# Support substitution of the tools_dir, and libs_dirs with user
 # parameters. This is used when we can't determine the tool dir at
 # configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
-    config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError:
     e = sys.exc_info()[1]
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@CLANG_SOURCE_DIR@/test/Unit/lit.cfg")

compiler-rt/trunk/test/lit.common.configured.in

 @LIT_SITE_CFG_IN_HEADER@
 
 # Set attribute value if it is unset.
 def set_default(attr, value):
   if not getattr(config, attr, None):
     setattr(config, attr, value)
 
 # Generic config options for all compiler-rt lit tests.
 set_default("target_triple", "@COMPILER_RT_DEFAULT_TARGET_TRIPLE@")
 set_default("target_cflags", "@COMPILER_RT_TEST_COMPILER_CFLAGS@")
 set_default("host_arch", "@HOST_ARCH@")
 set_default("target_arch", "@COMPILER_RT_DEFAULT_TARGET_ARCH@")
 set_default("host_os", "@HOST_OS@")
-set_default("llvm_build_mode", "@LLVM_BUILD_MODE@")
 set_default("llvm_src_root", "@LLVM_MAIN_SRC_DIR@")
 set_default("llvm_obj_root", "@LLVM_BINARY_DIR@")
 set_default("compiler_rt_src_root", "@COMPILER_RT_SOURCE_DIR@")
 set_default("compiler_rt_obj_root", "@COMPILER_RT_BINARY_DIR@")
 set_default("llvm_tools_dir", "@LLVM_TOOLS_BINARY_DIR@")
 set_default("llvm_shlib_dir", "@LLVM_LIBRARY_OUTPUT_INTDIR@")
 set_default("gold_executable", "@GOLD_EXECUTABLE@")
 set_default("clang", "@COMPILER_RT_TEST_COMPILER@")

compiler-rt/trunk/unittests/lit.common.unit.cfg

 # -*- Python -*-
 
 # Configuration file for 'lit' test runner.
 # This file contains common config setup rules for unit tests in various
 # compiler-rt testsuites.
 
 import os
 
 import lit.formats
 
 # Setup test format
-llvm_build_mode = getattr(config, "llvm_build_mode", "Debug")
-config.test_format = lit.formats.GoogleTest(llvm_build_mode, "Test")
+config.test_format = lit.formats.GoogleTest('.', "Test")
 
 # Setup test suffixes.
 config.suffixes = []
 
 # Tweak PATH to include llvm tools dir.
 llvm_tools_dir = getattr(config, 'llvm_tools_dir', None)
 if (not llvm_tools_dir) or (not os.path.exists(llvm_tools_dir)):
   lit_config.fatal("Invalid llvm_tools_dir config attribute: %r" % llvm_tools_dir)

compiler-rt/trunk/unittests/lit.common.unit.configured.in

 @LIT_SITE_CFG_IN_HEADER@
 
 # Generic config options for all compiler-rt unit tests.
 config.target_triple = "@TARGET_TRIPLE@"
 config.llvm_src_root = "@LLVM_MAIN_SRC_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_BINARY_DIR@"
 config.compiler_rt_src_root = "@COMPILER_RT_SOURCE_DIR@"
 config.compiler_rt_libdir = "@COMPILER_RT_LIBRARY_OUTPUT_DIR@"
-config.llvm_build_mode = "@LLVM_BUILD_MODE@"
 config.host_arch = "@HOST_ARCH@"
 config.host_os = "@HOST_OS@"
 
 # LLVM tools dir and build mode can be passed in lit parameters,
 # so try to apply substitution.
 try:
   config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
-  config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError as e:
   key, = e.args
   lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key, key))
 
 # Setup attributes common for all compiler-rt unit tests.
 lit_config.load_config(config, "@COMPILER_RT_SOURCE_DIR@/unittests/lit.common.unit.cfg")

lld/trunk/test/Unit/lit.cfg

 config.suffixes =  []
 
 # test_source_root: The root path where unit test binaries are located.
 # test_exec_root: The root path where tests should be run.
 config.test_source_root = os.path.join(config.lld_obj_root, 'unittests')
 config.test_exec_root = config.test_source_root
 
 # testFormat: The test format to use to interpret tests.
-if not hasattr(config, 'llvm_build_mode'):
-    lit_config.fatal("unable to find llvm_build_mode value on config")
-config.test_format = lit.formats.GoogleTest(config.llvm_build_mode, 'Tests')
+config.test_format = lit.formats.GoogleTest('.', 'Tests')

lld/trunk/test/Unit/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
-config.llvm_build_mode = "@LLVM_BUILD_MODE@"
 config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
 config.lld_obj_root = "@LLD_BINARY_DIR@"
 config.lld_src_root = "@LLD_SOURCE_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@PYTHON_EXECUTABLE@"
 
 # Support substitution of the tools and libs dirs with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
-    config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LLD_SOURCE_DIR@/test/Unit/lit.cfg")

lldb/trunk/lit/Unit/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 config.test_exec_root = "@LLVM_BINARY_DIR@" 
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
-config.llvm_build_mode = "@LLVM_BUILD_MODE@"
 config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
 config.lldb_obj_root = "@LLDB_BINARY_DIR@"
 config.lldb_src_root = "@LLDB_SOURCE_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@PYTHON_EXECUTABLE@"
 
 # Support substitution of the tools and libs dirs with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
-    config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LLDB_SOURCE_DIR@/lit/Unit/lit.cfg")

llvm/trunk/test/Unit/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
-config.llvm_build_mode = "@LLVM_BUILD_MODE@"
 config.enable_shared = @ENABLE_SHARED@
 config.shlibdir = "@SHLIBDIR@"
 
-# Support substitution of the tools_dir and build_mode with user parameters.
+# Support substitution of the tools_dir with user parameters.
 # This is used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
-    config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError:
     e = sys.exc_info()[1]
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LLVM_SOURCE_DIR@/test/Unit/lit.cfg")