[debuginfo-tests] Decouple debuginfo-tests from the clang source tree.

[zturner]

Summary

This is necessary in preparation for future work to debuginfo-tests that makes debuginfo-tests optionally able to make use of lld (if present).

The workflow now is:

1. In a mono-repo, just set -DLLVM_ENABLE_PROJECTS=debuginfo-tests on CMake and it will work. You can run ninja check-debuginfo.

2. In a multi-repo, clone debuginfo-tests into llvm/projects, and you can then run ninja check-debuginfo.

Everything else is unchanged. The previous workflow of cloning debuginfo-tests into the clang source tree and running it as part of check-clang is no longer supported. It now alawys has its own target, check-debuginfo.

[aprantl]

This change is generally fine, but since this will break the workflow that the green-dragon bots are using where the repository is checked out into llvm/tools/clang/test, we must update the bots at the same time. I think this involves both changes to zorg and the jenkins configuration. I'm marking the review as "request changes" to reflect that.

[aprantl]

I'm adding Chris Matthews and Mike Edwards who are maintaining zorg and the jenkins configuration on green dragon to the review, so we can coordinate this.

[probinson]

At Sony we build our own unified tree by pasting together several subtrees, including debuginfo-tests, and moving one will cause a bit of disruption. I remember the idea of moving debuginfo-tests has come up previously, and in principle I have no problem with it, but a more explicit announcement on llvm-dev/cfe-dev would be appropriate.

[zturner]

In D39605#915645, @probinson wrote:

At Sony we build our own unified tree by pasting together several subtrees, including debuginfo-tests, and moving one will cause a bit of disruption. I remember the idea of moving debuginfo-tests has come up previously, and in principle I have no problem with it, but a more explicit announcement on llvm-dev/cfe-dev would be appropriate.

FWIW, debuginfo-tests is not actually checked out by default. You have to explicitly clone it at a location of your choosing. Before this patch, that location has to be clang/test, and after this patch, it has to be llvm/projects. How are your trees subtrees pasted together, is it something other than a mono-repo layout?

[probinson]

In D39605#915648, @zturner wrote:

In D39605#915645, @probinson wrote:

At Sony we build our own unified tree by pasting together several subtrees, including debuginfo-tests, and moving one will cause a bit of disruption. I remember the idea of moving debuginfo-tests has come up previously, and in principle I have no problem with it, but a more explicit announcement on llvm-dev/cfe-dev would be appropriate.

FWIW, debuginfo-tests is not actually checked out by default. You have to explicitly clone it at a location of your choosing. Before this patch, that location has to be clang/test, and after this patch, it has to be llvm/projects. How are your trees subtrees pasted together, is it something other than a mono-repo layout?

Yes, it's a homegrown unified tree with maybe a half-dozen projects and other internal cruft, done with subtree merges. Our internal tree does include debuginfo-tests. The upstream mono-repo didn't exist yet when we did this.

We can cope with what you're doing, but with the cmake changes not supporting debuginfo-tests in the old location, we'll need to fiddle with some of our automation so it ends up in the correct new place. If you can give us a few days that would be great.

[zturner]

Definitely. I'm in no rush. I'll push out a message to llvm-dev before the day's over.

[sqlbyme]

I'm fine with the changes being made. We just need to coordinate a date & time for the cutover so we don't set Green Dragon ablaze.

Also, someone needs to look at the buildbot configs. It has been so long since I have looked at those configs I'm not sure which bots on that side will need to be updated (if any).

[zturner]

In D39605#915688, @sqlbyme wrote:

I'm fine with the changes being made. We just need to coordinate a date & time for the cutover so we don't set Green Dragon ablaze.

Also, someone needs to look at the buildbot configs. It has been so long since I have looked at those configs I'm not sure which bots on that side will need to be updated (if any).

After the discussion on llvm-dev about how this should continue to work in the normal configuration, are there are other blockers for this? Or are we ok to try it out? If it does break anything, I can revert.

clang/test/lit.cfg.py

 # For each occurrence of a clang tool name, replace it with the full path to
 # the build directory holding that tool.  We explicitly specify the directories
 # to search to ensure that we get the tools just built and not some random
 # tools that might happen to be in the user's PATH.
 tool_dirs = [config.clang_tools_dir, config.llvm_tools_dir]
 
 tools = [
     'c-index-test', 'clang-check', 'clang-diff', 'clang-format', 'opt',
-    ToolSubst('%test_debuginfo', command=os.path.join(
-        config.llvm_src_root, 'utils', 'test_debuginfo.pl')),
     ToolSubst('%clang_func_map', command=FindTool(
         'clang-func-mapping'), unresolved='ignore'),
 ]
 
 if config.clang_examples:
     tools.append('clang-interpreter')
 
 llvm_config.add_tool_substitutions(tools, tool_dirs)

debuginfo-tests/CMakeLists.txt

+# Debug Info tests.  These tests invoke clang to generate programs with
+# various types of debug info, and then run those programs under a debugger
+# such as GDB or LLDB to verify the results.
+
+set(DEBUGINFO_TESTS_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
+set(DEBUGINFO_TESTS_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
+
+set(DEBUGINFO_TEST_DEPS
+  clang
+  llvm-config
+  FileCheck
+  count
+  not
+  )
+
+get_target_property(CLANG_SOURCE_DIR clang SOURCE_DIR)
+
+if (TARGET lld)
+  set(DEBUGINFO_TESTS_HAS_LLD 1)
+  list(APPEND DEBUGINFO_TEST_DEPS lld)
+  get_target_property(LLD_SOURCE_DIR lld SOURCE_DIR)
+endif()
+
+configure_lit_site_cfg(
+  ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
+  ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
+  MAIN_CONFIG
+  ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
+  )
+
+add_lit_testsuite(check-debuginfo "Running debug info integration tests"
+  ${CMAKE_CURRENT_BINARY_DIR}
+  DEPENDS ${DEBUGINFO_TEST_DEPS}
+  )
+
+set_target_properties(check-debuginfo PROPERTIES FOLDER "Debug info tests")
 No newline at end of file

debuginfo-tests/lit.cfg.py

+# -*- Python -*-
+
+import os
+import platform
+import re
+import subprocess
+import tempfile
+
+import lit.formats
+import lit.util
+
+from lit.llvm import llvm_config
+from lit.llvm.subst import ToolSubst
+from lit.llvm.subst import FindTool
+
+# Configuration file for the 'lit' test runner.
+
+# name: The name of this test suite.
+config.name = 'debuginfo-tests'
+
+# testFormat: The test format to use to interpret tests.
+#
+# For now we require '&&' between commands, until they get globally killed and
+# the test runner updated.
+config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
+
+# suffixes: A list of file extensions to treat as test files.
+config.suffixes = ['.c', '.cpp', '.m']
+
+# excludes: A list of directories to exclude from the testsuite. The 'Inputs'
+# subdirectories contain auxiliary inputs for various tests in their parent
+# directories.
+config.excludes = ['Inputs']
+
+# test_source_root: The root path where tests are located.
+config.test_source_root = os.path.join(config.debuginfo_tests_src_root, 'tests')
+
+# test_exec_root: The root path where tests should be run.
+config.test_exec_root = config.debuginfo_tests_obj_root
+
+llvm_config.use_default_substitutions()
+
+llvm_config.use_clang()
+
+if config.llvm_use_sanitizer:
+    # Propagate path to symbolizer for ASan/MSan.
+    llvm_config.with_system_environment(
+        ['ASAN_SYMBOLIZER_PATH', 'MSAN_SYMBOLIZER_PATH'])
+
+tool_dirs = [config.llvm_tools_dir]
+
+tools = [
+    ToolSubst('%test_debuginfo', command=os.path.join(
+        config.llvm_src_root, 'utils', 'test_debuginfo.pl')),
+]
+
+llvm_config.add_tool_substitutions(tools, tool_dirs)

debuginfo-tests/lit.site.cfg.py.in

+@LIT_SITE_CFG_IN_HEADER@
+
+import lit.util
+
+config.test_exec_root = "@CMAKE_BINARY_DIR@"
+
+config.llvm_src_root = "@LLVM_SOURCE_DIR@"
+config.llvm_obj_root = "@LLVM_BINARY_DIR@"
+config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
+config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
+config.llvm_shlib_dir = "@SHLIBDIR@"
+config.llvm_plugin_ext = "@LLVM_PLUGIN_EXT@"
+config.debuginfo_tests_obj_root = "@DEBUGINFO_TESTS_BINARY_DIR@"
+config.debuginfo_tests_src_root = "@DEBUGINFO_TESTS_SOURCE_DIR@"
+config.has_lld = lit.util.pythonize_bool("@DEBUGINFO_TESTS_HAS_LLD@")
+config.host_triple = "@LLVM_HOST_TRIPLE@"
+config.target_triple = "@TARGET_TRIPLE@"
+config.host_arch = "@HOST_ARCH@"
+
+config.llvm_use_sanitizer = "@LLVM_USE_SANITIZER@"
+
+@LIT_SITE_CFG_IN_FOOTER@
+
+# Let the main config do the real work.
+lit_config.load_config(config, "@DEBUGINFO_TESTS_SOURCE_DIR@/lit.cfg.py")

debuginfo-tests/tests/aggregate-indirect-arg.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clangxx %target_itanium_abi_host_triple %t.o -o %t.out
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 // Radar 8945514
 // DEBUGGER: break 22
 // DEBUGGER: r
 // DEBUGGER: p v
 // CHECK: ${{[0-9]+}} =
 // CHECK:  Data ={{.*}} 0x0{{(0*)}}
 // CHECK:  Kind = 2142
 

debuginfo-tests/tests/block_var.m

 // RUN: %clang %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clang %target_itanium_abi_host_triple %t.o -o %t.out -framework Foundation
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 
 // REQUIRES: system-darwin
 
 // DEBUGGER: break 24
 // DEBUGGER: r
 // DEBUGGER: p result
 // CHECK: ${{[0-9]}} = 42
 

debuginfo-tests/tests/blocks.m

 // RUN: %clang %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clang %target_itanium_abi_host_triple %t.o -o %t.out -framework Foundation
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 
 // REQUIRES: system-darwin
 // Radar 9279956
 
 // DEBUGGER: break 31
 // DEBUGGER: r
 // DEBUGGER: p m2
 // CHECK: ${{[0-9]}} = 1
 // DEBUGGER: p dbTransaction
 // CHECK: ${{[0-9]}} = 0
 // DEBUGGER: p master
 // CHECK: ${{[0-9]}} = 0
 
 #include <Cocoa/Cocoa.h>
 
 extern void foo(void(^)(void));
 
 @interface A:NSObject @end
 @implementation A
 - (void) helper {
  int master = 0;
  __block int m2 = 0;
  __block int dbTransaction = 0;
- int (^x)(void) = ^(void) { (void) self; 
-  (void) master; 
-  (void) dbTransaction; 
+ int (^x)(void) = ^(void) { (void) self;
+  (void) master;
+  (void) dbTransaction;
   m2++;
   return m2;
   };
   master = x();
 }
 @end
 
 void foo(void(^x)(void)) {}
 
 int main() {
   A *a = [A alloc];
   [a helper];
   return 0;
 }

debuginfo-tests/tests/ctor.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clangxx %target_itanium_abi_host_triple %t.o -o %t.out
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 
 
 // DEBUGGER: break 14
 // DEBUGGER: r
 // DEBUGGER: p *this
-// CHECK-NEXT-NOT: Cannot access memory at address 
+// CHECK-NEXT-NOT: Cannot access memory at address
 
 class A {
 public:
   A() : zero(0), data(42)
   {
   }
 private:
   int zero;
   int data;
 };
 
 int main() {
   A a;
   return 0;
 }
 

debuginfo-tests/tests/foreach.m

 // RUN: %clang %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clang %target_itanium_abi_host_triple %t.o -o %t.out -framework Foundation
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 //
 // REQUIRES: system-darwin
 // Radar 8757124
 
 // DEBUGGER: break 25
 // DEBUGGER: r
 // DEBUGGER: po thing
 // CHECK: aaa
 
 #import <Foundation/Foundation.h>
 
 int main (int argc, const char * argv[]) {
 
     NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
     NSArray *things = [NSArray arrayWithObjects:@"one", @"two", @"three" , nil];
     for (NSString *thing in things) {
         NSLog (@"%@", thing);
     }
-    
     things = [NSArray arrayWithObjects:@"aaa", @"bbb", @"ccc" , nil];
     for (NSString *thing in things) {
         NSLog (@"%@", thing);
     }
     [pool release];
     return 0;
 }
 
 

debuginfo-tests/tests/sret.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clangxx %target_itanium_abi_host_triple %t.o -o %t.out
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 // Radar 8775834
 // DEBUGGER: break 62
 // DEBUGGER: r
 // DEBUGGER: p a
 // CHECK: ${{[0-9]+}} =
 // LLDB does not print artificial members.
 // CHECK:  {{(_vptr\$A =)?.*}}m_int = 12
 
 class A
 {
 public:
     A (int i=0);
     A (const A& rhs);
     const A&
     operator= (const A& rhs);
     virtual ~A() {}
 
     int get_int();
 
 protected:
     int m_int;
 };
 
-A::A (int i) : 
+A::A (int i) :
     m_int(i)
 {
 }
 
 A::A (const A& rhs) :
     m_int (rhs.m_int)
 {
 }

 {
     return m_int;
 }
 
 class B
 {
 public:
     B () {}
-    
     A AInstance();
 };
 
-A 
+A
 B::AInstance()
 {
     A a(12);
     return a;
 }
 
 int main (int argc, char const *argv[])
 {
     B b;
     int return_val = b.AInstance().get_int();
-    
     A a(b.AInstance());
     return return_val;
 }

debuginfo-tests/tests/vla.c

 // This test case verifies the debug location for variable-length arrays.
 // RUN: %clang %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clang %target_itanium_abi_host_triple %t.o -o %t.out
-// RUN: %test_debuginfo %s %t.out 
+// RUN: %test_debuginfo %s %t.out
 //
 // DEBUGGER: break 18
 // DEBUGGER: r
 // DEBUGGER: p vla[0]
 // CHECK: 23
 // DEBUGGER: p vla[1]
 // CHECK: 22
 

llvm/CMakeLists.txt

if(DYLD_LIBRARY_PATH)
     endforeach()
   endif()
 endif()
 
 # Side-by-side subprojects layout: automatically set the
 # LLVM_EXTERNAL_${project}_SOURCE_DIR using LLVM_ALL_PROJECTS
 # This allows an easy way of setting up a build directory for llvm and another
 # one for llvm+clang+... using the same sources.
-set(LLVM_ALL_PROJECTS "clang;libcxx;libcxxabi;lldb;compiler-rt;lld;polly")
+set(LLVM_ALL_PROJECTS "clang;libcxx;libcxxabi;lldb;compiler-rt;debuginfo-tests;lld;polly")
 set(LLVM_ENABLE_PROJECTS "" CACHE STRING
   "Semicolon-separated list of projects to build (${LLVM_ALL_PROJECTS}), or \"all\".")
 if( LLVM_ENABLE_PROJECTS STREQUAL "all" )
   set( LLVM_ENABLE_PROJECTS ${LLVM_ALL_PROJECTS})
 endif()
 foreach(proj ${LLVM_ENABLE_PROJECTS})
   set(PROJ_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../${proj}")
   if(NOT EXISTS "${PROJ_DIR}" OR NOT IS_DIRECTORY "${PROJ_DIR}")
if( LLVM_INCLUDE_RUNTIMES )
   add_subdirectory(runtimes)
 endif()
 
 if( LLVM_INCLUDE_EXAMPLES )
   add_subdirectory(examples)
 endif()
 
 if( LLVM_INCLUDE_TESTS )
-  if(EXISTS ${LLVM_MAIN_SRC_DIR}/projects/test-suite AND TARGET clang)
+  if(TARGET clang)
     include(LLVMExternalProjectUtils)
+    if (EXISTS ${LLVM_MAIN_SRC_DIR}/projects/test-suite)
       llvm_ExternalProject_Add(test-suite ${LLVM_MAIN_SRC_DIR}/projects/test-suite
         USE_TOOLCHAIN
+        TOOLCHAIN_TOOLS clang lld clang-cl
         EXCLUDE_FROM_ALL
         NO_INSTALL
         ALWAYS_CLEAN)
     endif()
+
+    add_llvm_external_project(debuginfo-tests projects/debuginfo-tests)
+  endif()
   add_subdirectory(utils/lit)
   add_subdirectory(test)
   add_subdirectory(unittests)
   if( LLVM_INCLUDE_UTILS )
     add_subdirectory(utils/unittest)
   endif()
 
   if (WIN32)

llvm/projects/CMakeLists.txt

 # Discover the projects that use CMake in the subdirectories.
 # Note that explicit cmake invocation is required every time a new project is
 # added or removed.
 file(GLOB entries *)
 foreach(entry ${entries})
   if(IS_DIRECTORY ${entry} AND EXISTS ${entry}/CMakeLists.txt)
     if((NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/compiler-rt) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/dragonegg) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/libcxx) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/libcxxabi) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/libunwind) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/test-suite) AND
+       (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/debuginfo-tests) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/parallel-libs) AND
        (NOT ${entry} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR}/openmp))
       add_subdirectory(${entry})
     endif()
   endif()
 endforeach(entry)
 
 # Also add in libc++ and compiler-rt trees if present (and we have

llvm/utils/lit/lit/llvm/config.py

def __init__(self, lit_config, config):
         features = config.available_features
 
         self.use_lit_shell = False
         # Tweak PATH for Win32 to decide to use bash.exe or not.
         if sys.platform == 'win32':
             # For tests that require Windows to run.
             features.add('system-windows')
 
+            lit_tools_dir = getattr(config, 'lit_tools_dir', None)
             # Seek sane tools in directories and set to $PATH.
-            path = self.lit_config.getToolsPath(config.lit_tools_dir,
+            path = self.lit_config.getToolsPath(lit_tools_dir,
                                                 config.environment['PATH'],
                                                 ['cmp.exe', 'grep.exe', 'sed.exe'])
             if path is not None:
                 self.with_environment('PATH', path, append_path=True)
             self.use_lit_shell = True
 
         # Choose between lit's internal shell pipeline runner and a real shell.  If
         # LIT_USE_INTERNAL_SHELL is in the environment, we use that as an override.
def use_clang(self, required=True):
         # an already installed llvm.
         if self.config.host_triple and self.config.host_triple != '@LLVM_HOST_TRIPLE@':
             self.config.substitutions.append(('%target_itanium_abi_host_triple',
                                               '--target=%s' % self.make_itanium_abi_triple(self.config.host_triple)))
         else:
             self.config.substitutions.append(
                 ('%target_itanium_abi_host_triple', ''))
 
+        clang_src_dir = getattr(self.config, 'clang_src_dir', None)
+        if clang_src_dir:
             self.config.substitutions.append(
-            ('%src_include_dir', self.config.clang_src_dir + '/include'))
+                ('%src_include_dir', os.path.join(clang_src_dir, 'include')))
 
         # FIXME: Find nicer way to prohibit this.
         self.config.substitutions.append(
             (' clang ', """*** Do not use 'clang' in tests, use '%clang'. ***"""))
         self.config.substitutions.append(
             (' clang\+\+ ', """*** Do not use 'clang++' in tests, use '%clangxx'. ***"""))
         self.config.substitutions.append(
             (' clang-cc ',