[debuginfo-tests] Support moving debuginfo-tests to llvm/projects

[hintonda]

Summary

Add cmake and lit files needed to run these tests as an
external project. Also, copy test_debuginfo.pl from llvm/utils since
it's only used here. The copy in llvm/utils must be maintained as
long as bots continue to include debuginfo-tests in clang/test.

The name of lit.cfg.py was changed to lit.config.py so tests will
still run correctly clang/test -- otherwise it would be picked up by
lit automatically.

---

(see D40972)
This is one of two independent patches designed to support moving the
debuginfo-test repo from clang/test to llvm/projects. While each
patch is completely independent and can be applied seperately, both
are required to actually move the repo.

Due to the independent nature of these patches, debuginfo-test can
coexist in both clang/test and llvm/projects, so testers can add
debuginfo-test to llvm/projects without the need to immediately remove
it from clang/test..

Once both patches land and all buildbots have moved debuginfo-test to
llvm/projects, llvm/utils/test-debuginfo.pl can be removed as well the
tools reference to it in clang/test/lit.cfg.py. At which point, only
llvm/projects will be supported.

[aprantl]

I tried this locally with debuginfo-tests checked out into tools/clang/test, and as far as I can tell the debuginfo-tests are only recognized once by lit.
Tentatively LGTM, but please do watch the bots after committing.

test_debuginfo.pl

1 ↗

(On Diff #125998)

It seems weird to have this file here, but not llgdb.py. I would either copy both or none.

[hintonda]

In D40971#948999, @aprantl wrote:

I tried this locally with debuginfo-tests checked out into tools/clang/test, and as far as I can tell the debuginfo-tests are only recognized once by lit.
Tentatively LGTM, but please do watch the bots after committing.

Great, I'll commit in the morning so I can watch the bots.

test_debuginfo.pl

1 ↗

(On Diff #125998)

lldgbb.py is already in debuginfo.-tests. This just puts them both in the same place.

[zturner]

lit.config.py

1 ↗

(On Diff #125998)

I think the rename of this file is going to be a problem for the mono-repo case. In that, this will be the top-level directory tree and if there's no` lit.cfg.py`, lit won't find anything to run.

[zturner]

lit.config.py

1 ↗

(On Diff #125998)

There needs to be a way to disable lit discovery of this suite only as part of the clang suite, and not as part of anything else that might cause it to be discovered.

[hintonda]

lit.site.cfg.py.in

25 ↗

(On Diff #125998)

This is how lit find lit.config.py.

I'll test the monorepo variant and get back to you.

[zturner]

lit.site.cfg.py.in

25 ↗

(On Diff #125998)

Yes, but you need to be able to type llvm-lit.py path/to/debuginfo-tests, and if you do that, lit is going to be looking specifically for a lit.cfg.py and won't find it

[hintonda]

• Change name back to lit.cfg.py, and handle config.clang_src_dir.

[hintonda]

lit.site.cfg.py.in

25 ↗

(On Diff #125998)

Changing the name back to lit.cfg.py was required for the monorepo, but also required a change to clang, D41055, which needs to land first -- hope I set the dependency correctly.

debuginfo-tests/trunk/.arcconfig

+{
+  "conduit_uri" : "https://reviews.llvm.org/"
+}

debuginfo-tests/trunk/CMakeLists.txt

+# Debug Info tests.  These tests invoke clang to generate programs with
+# various types of debug info, and then run those programs under a debugger
+# such as GDB or LLDB to verify the results.
+
+set(DEBUGINFO_TESTS_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
+set(DEBUGINFO_TESTS_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
+
+set(DEBUGINFO_TEST_DEPS
+  clang
+  FileCheck
+  count
+  not
+  )
+
+configure_lit_site_cfg(
+  ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
+  ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
+  MAIN_CONFIG
+  ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
+  )
+
+add_lit_testsuite(check-debuginfo "Running debug info integration tests"
+  ${CMAKE_CURRENT_BINARY_DIR}
+  DEPENDS ${DEBUGINFO_TEST_DEPS}
+  )
+set_target_properties(check-debuginfo PROPERTIES FOLDER "Debug info tests")

debuginfo-tests/trunk/lit.cfg.py

+# -*- Python -*-
+
+import os
+import platform
+import re
+import subprocess
+import tempfile
+
+import lit.formats
+import lit.util
+
+from lit.llvm import llvm_config
+from lit.llvm.subst import ToolSubst
+from lit.llvm.subst import FindTool
+
+# Configuration file for the 'lit' test runner.
+
+# name: The name of this test suite.
+config.name = 'debuginfo-tests'
+
+# testFormat: The test format to use to interpret tests.
+#
+# For now we require '&&' between commands, until they get globally killed and
+# the test runner updated.
+config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
+
+# suffixes: A list of file extensions to treat as test files.
+config.suffixes = ['.c', '.cpp', '.m']
+
+# excludes: A list of directories to exclude from the testsuite. The 'Inputs'
+# subdirectories contain auxiliary inputs for various tests in their parent
+# directories.
+config.excludes = ['Inputs']
+
+# test_source_root: The root path where tests are located.
+config.test_source_root = os.path.join(config.debuginfo_tests_src_root)
+
+# test_exec_root: The root path where tests should be run.
+config.test_exec_root = config.debuginfo_tests_obj_root
+
+llvm_config.use_default_substitutions()
+
+# clang_src_dir is not used by these tests, but is required by
+# use_clang(), so set it to "".
+if not hasattr(config, 'clang_src_dir'):
+    config.clang_src_dir = ""
+llvm_config.use_clang()
+
+if config.llvm_use_sanitizer:
+    # Propagate path to symbolizer for ASan/MSan.
+    llvm_config.with_system_environment(
+        ['ASAN_SYMBOLIZER_PATH', 'MSAN_SYMBOLIZER_PATH'])
+
+tool_dirs = [config.llvm_tools_dir]
+
+tools = [
+    ToolSubst('%test_debuginfo', command=os.path.join(
+        config.debuginfo_tests_src_root, 'test_debuginfo.pl')),
+]
+
+llvm_config.add_tool_substitutions(tools, tool_dirs)
+
+lit.util.usePlatformSdkOnDarwin(config, lit_config)

debuginfo-tests/trunk/lit.site.cfg.py.in

+@LIT_SITE_CFG_IN_HEADER@
+
+import lit.util
+
+config.test_exec_root = "@CMAKE_BINARY_DIR@"
+
+config.llvm_src_root = "@LLVM_SOURCE_DIR@"
+config.llvm_obj_root = "@LLVM_BINARY_DIR@"
+config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
+config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
+config.llvm_shlib_dir = "@SHLIBDIR@"
+config.llvm_plugin_ext = "@LLVM_PLUGIN_EXT@"
+config.debuginfo_tests_obj_root = "@DEBUGINFO_TESTS_BINARY_DIR@"
+config.debuginfo_tests_src_root = "@DEBUGINFO_TESTS_SOURCE_DIR@"
+config.has_lld = lit.util.pythonize_bool("@DEBUGINFO_TESTS_HAS_LLD@")
+config.host_triple = "@LLVM_HOST_TRIPLE@"
+config.target_triple = "@TARGET_TRIPLE@"
+config.host_arch = "@HOST_ARCH@"
+
+config.llvm_use_sanitizer = "@LLVM_USE_SANITIZER@"
+
+@LIT_SITE_CFG_IN_FOOTER@
+
+# Let the main config do the real work.
+lit_config.load_config(config, "@DEBUGINFO_TESTS_SOURCE_DIR@/lit.cfg.py")

debuginfo-tests/trunk/test_debuginfo.pl

+#!/usr/bin/perl
+#
+# This script tests debugging information generated by a compiler.
+# Input arguments
+#   - Input source program. Usually this source file is decorated using
+#     special comments to communicate debugger commands.
+#   - Executable file. This file is generated by the compiler.
+#
+# This perl script extracts debugger commands from input source program
+# comments in a script. A debugger is used to load the executable file
+# and run the script generated from source program comments. Finally,
+# the debugger output is checked, using FileCheck, to validate
+# debugging information.
+#
+# On Darwin the default is to use the llgdb.py wrapper script which
+# translates gdb commands into their lldb equivalents.
+
+use File::Basename;
+use Config;
+use Cwd;
+
+my $testcase_file = $ARGV[0];
+my $executable_file = $ARGV[1];
+
+my $input_filename = basename $testcase_file;
+my $output_dir = dirname $executable_file;
+
+my $debugger_script_file = "$output_dir/$input_filename.debugger.script";
+my $output_file = "$output_dir/$input_filename.gdb.output";
+
+my %cmd_map = ();
+# Assume lldb to be the debugger on Darwin.
+my $use_lldb = 0;
+$use_lldb = 1 if ($Config{osname} eq "darwin");
+
+# Extract debugger commands from testcase. They are marked with DEBUGGER:
+# at the beginning of a comment line.
+open(INPUT, $testcase_file);
+open(OUTPUT, ">$debugger_script_file");
+while(<INPUT>) {
+    my($line) = $_;
+    $i = index($line, "DEBUGGER:");
+    if ( $i >= 0) {
+        $l = length("DEBUGGER:");
+        $s = substr($line, $i + $l);
+        print OUTPUT  "$s";
+    }
+}
+print OUTPUT "\n";
+print OUTPUT "quit\n";
+close(INPUT);
+close(OUTPUT);
+
+# setup debugger and debugger options to run a script.
+my $my_debugger = $ENV{'DEBUGGER'};
+if (!$my_debugger) {
+    if ($use_lldb) {
+        my $path = dirname(Cwd::abs_path($0));
+        $my_debugger = "/usr/bin/env python $path/llgdb.py";
+    } else {
+        $my_debugger = "gdb";
+    }
+}
+
+# quiet / exit after cmdline / no init file / execute script
+my $debugger_options = "-q -batch -n -x";
+
+# run debugger and capture output.
+system("$my_debugger $debugger_options $debugger_script_file $executable_file > $output_file 2>&1");
+
+# validate output.
+system("FileCheck", "-input-file", "$output_file", "$testcase_file");
+if ($?>>8 == 1) {
+    print "Debugger output was:\n";
+    system("cat", "$output_file");
+    exit 1;
+}
+else {
+    exit 0;
+}