[cross-project-tests] XFAIL llgdb-tests when gdb can't read clang's DWARF

[Orlando]

Summary

Tests in the cross-project-tests/debuginfo-tests/llgdb-tests directory run gdb on non-darwin platforms. gdb versions less than 10.1 cannot parse the DWARF v5 emitted by clang, and DWARF v5 is now the default, so these tests fail on Linux with gdb versions less than 10.1. This patch lets us XFAIL the tests under these conditions.

Add 'gdb-clang-incompatibility' to the available_features in cross-project-tests/lit.cfg.py when clang's default DWARF version is 5 or greater and the gdb (if found) version is less than 10.1.

The patch isn't particularly pretty, but I think it might be the least-bad thing to do. We can't add XFAIL: system-linux to these tests because they will "unexpectedly pass" if a compatible gdb is installed. We could force the tests to emit v4 instead, but that undermines the point of the tests, which is to check that the platform debugger works with the just-build clang [0]. The only alternative I can think of is to add UNSUPPORTED: system-linux to these tests, but losing the coverage is undesirable.

Discourse discussion: https://llvm.discourse.group/t/gdb-10-1-cant-read-clangs-dwarf-v5/6035
[0]: https://llvm.discourse.group/t/gdb-10-1-cant-read-clangs-dwarf-v5/6035/4?u=ochyams

[probinson]

+llvm-commits

[probinson]

LGTM but I'd rather have @aprantl or @JDevlieghere review because I believe the llgdb tests are something Apple did? I'm happy to approve if I'm wrong.

[Orlando]

Thanks for taking a look @probinson. Adding Jeremy as reviewer.

[Orlando]

Ping

We've now got a native Linux builder in staging where you can see these tests failing: https://lab.llvm.org/staging/#/builders/205/builds/91

[jmorse]

SGTM with a couple of nits. From the discussion on discourse, I think it's a deliberate choice that there's going to be incompatible configurations out there, in which case as you say the least-worst-option is xfailing these tests.

In addition, I think it'll be worth printing a warning to the developer when the tests start running that because their gdb version is out of date, they won't get full test coverage. That gives developers a sporting chance to spot the problem, otherwise we're silently suppressing information.

cross-project-tests/lit.cfg.py

212–215

Would this benefit from a pokemon exception handler? Given that we're calling another process, there's a nonzero possibility we'll get a CalledProcessError or some other unexpected behaviour. I'd suggest that if gdb itself is broken, we should catch the resulting exceptions here, return None, and let the gdb tests run and possibly fail. IMO, that's better than presenting an exception in llvm-lit to the developer.

220

Having the same length-check for [2] as on the earlier few lines would future proof this code.

[Orlando]

Thanks @jmorse

+ Catch-em-all when trying to run gdb
+ Print a message when the XFAIL condition is detected.
+ Fix stderr -> sys.stderr in a print call.

cross-project-tests/lit.cfg.py

212–215

SGTM, done.

220

partition returns a 3-tuple of strings (docs).

[Orlando]

+ Make gdb-version-check more robust by checking version string length

[Orlando]

cross-project-tests/lit.cfg.py

220

I have now updated this to check that the partition isn't zero-length though.

[jmorse]

LGTM

cross-project-tests/debuginfo-tests/llgdb-tests/asan-blocks.c

 // RUN: %clang -fblocks %target_itanium_abi_host_triple -arch x86_64 %s -o %t.out -g -fsanitize=address
 // RUN: %test_debuginfo %s %t.out
 // FIXME: Remove system-darwin when we build BlocksRuntime everywhere.
 // REQUIRES: !asan, system-darwin
 //           Zorg configures the ASAN stage2 bots to not build the asan
 //           compiler-rt. Only run this test on non-asanified configurations.
+// XFAIL: !system-darwin && gdb-clang-incompatibility
+
 void b();
 struct S {
   int a[8];
 };
 
 int f(struct S s, unsigned i) {
-  // DEBUGGER: break 17
+  // DEBUGGER: break 19
   // DEBUGGER: r
   // DEBUGGER: p s
   // CHECK: a = ([0] = 0, [1] = 1, [2] = 2, [3] = 3, [4] = 4, [5] = 5, [6] = 6, [7] = 7)
   return s.a[i];
 }
 
 int main(int argc, const char **argv) {
   struct S s = {{0, 1, 2, 3, 4, 5, 6, 7}};
   if (f(s, 4) == 4) {
-    // DEBUGGER: break 27
+    // DEBUGGER: break 29
     // DEBUGGER: c
     // DEBUGGER: p s
     // CHECK: a = ([0] = 0, [1] = 1, [2] = 2, [3] = 3, [4] = 4, [5] = 5, [6] = 6, [7] = 7)
     b();
   }
   return 0;
 }
 
 void c() {}
 
 void b() {
-  // DEBUGGER: break 40
+  // DEBUGGER: break 42
   // DEBUGGER: c
   // DEBUGGER: p x
   // CHECK: 42
   __block int x = 42;
   c();
 }

cross-project-tests/debuginfo-tests/llgdb-tests/asan-deque.cpp

 // RUN: %clangxx -arch x86_64 %target_itanium_abi_host_triple -O1 -g %s -o %t.out -fsanitize=address
 // RUN: %test_debuginfo %s %t.out
 // REQUIRES: !asan
 //           Zorg configures the ASAN stage2 bots to not build the asan
 //           compiler-rt. Only run this test on non-asanified configurations.
 // UNSUPPORTED: apple-lldb-pre-1000
+// XFAIL: !system-darwin && gdb-clang-incompatibility
+
 #include <deque>
 
 struct A {
   int a;
   A(int a) : a(a) {}
 };
 
 using log_t = std::deque<A>;
 
 static void __attribute__((noinline, optnone)) escape(log_t &log) {
   static volatile log_t *sink;
   sink = &log;
 }
 
 int main() {
   log_t log;
   log.push_back(1234);
   log.push_back(56789);
   escape(log);
-  // DEBUGGER: break 25
+  // DEBUGGER: break 27
   while (!log.empty()) {
     auto record = log.front();
     log.pop_front();
     escape(log);
-    // DEBUGGER: break 30
+    // DEBUGGER: break 32
   }
 }
 
 // DEBUGGER: r
 
-// (at line 25)
+// (at line 27)
 // DEBUGGER: p log
 // CHECK: 1234
 // CHECK: 56789
 
 // DEBUGGER: c
 
-// (at line 30)
+// (at line 32)
 // DEBUGGER: p log
 // CHECK: 56789

cross-project-tests/debuginfo-tests/llgdb-tests/asan.c

 // RUN: %clang -fblocks %target_itanium_abi_host_triple -arch x86_64 %s -o %t.out -g -fsanitize=address
 // RUN: %test_debuginfo %s %t.out
 // REQUIRES: !asan
 //           Zorg configures the ASAN stage2 bots to not build the asan
 //           compiler-rt. Only run this test on non-asanified configurations.
-//
+// XFAIL: !system-darwin && gdb-clang-incompatibility
 
 struct S {
   int a[8];
 };
 
 int f(struct S s, unsigned i) {
   // DEBUGGER: break 14
   return s.a[i];

cross-project-tests/debuginfo-tests/llgdb-tests/nested-struct.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %test_debuginfo %s %t.o
+// XFAIL: !system-darwin && gdb-clang-incompatibility
 // Radar 9440721
 // If debug info for my_number() is emitted outside function foo's scope
 // then a debugger may not be able to handle it. At least one version of
 // gdb crashes in such cases.
 
 // DEBUGGER: ptype foo
 // CHECK: int (void)
 

cross-project-tests/debuginfo-tests/llgdb-tests/nrvo-string.cpp

 // This ensures that DW_OP_deref is inserted when necessary, such as when NRVO
 // of a string object occurs in C++.
 //
 // RUN: %clangxx -O0 -fno-exceptions %target_itanium_abi_host_triple %s -o %t.out -g
 // RUN: %test_debuginfo %s %t.out
 // RUN: %clangxx -O1 -fno-exceptions %target_itanium_abi_host_triple %s -o %t.out -g
 // RUN: %test_debuginfo %s %t.out
-//
+// XFAIL: !system-darwin && gdb-clang-incompatibility
 // PR34513
 volatile int sideeffect = 0;
 void __attribute__((noinline)) stop() { sideeffect++; }
 
 struct string {
   string() {}
   string(int i) : i(i) {}
   ~string() {}

cross-project-tests/debuginfo-tests/llgdb-tests/safestack.c

 // RUN: %clang %target_itanium_abi_host_triple -arch x86_64 %s -o %t.out -g -fsanitize=safe-stack
 // RUN: %test_debuginfo %s %t.out
 // UNSUPPORTED: system-darwin
 // REQUIRES: !asan
 //           Zorg configures the ASAN stage2 bots to not build the
 //           safestack compiler-rt.  Only run this test on
 //           non-asanified configurations.
-
+// XFAIL: !system-darwin && gdb-clang-incompatibility
 struct S {
   int a[8];
 };
 
 int f(struct S s, unsigned i);
 
 int main(int argc, const char **argv) {
   struct S s = {{0, 1, 2, 3, 4, 5, 6, 7}};

cross-project-tests/debuginfo-tests/llgdb-tests/sret.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -c -o %t.o
 // RUN: %clangxx %target_itanium_abi_host_triple %t.o -o %t.out
 // RUN: %test_debuginfo %s %t.out
+// XFAIL: !system-darwin && gdb-clang-incompatibility
 // Radar 8775834
-// DEBUGGER: break 62
+// DEBUGGER: break 63
 // DEBUGGER: r
 // DEBUGGER: p a
 // CHECK: ${{[0-9]+}} =
 // LLDB does not print artificial members.
 // CHECK:  {{(_vptr\$A =)?.*}}m_int = 12
 
 class A
 {

cross-project-tests/debuginfo-tests/llgdb-tests/static-member-2.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -o %t -c
 // RUN: %clangxx %target_itanium_abi_host_triple %t -o %t.out
 // RUN: %test_debuginfo %s %t.out
 
 // FIXME: LLDB finds the wrong symbol for "C". rdar://problem/14933867
-// XFAIL: darwin
+// XFAIL: darwin, gdb-clang-incompatibility
 
 // DEBUGGER: delete breakpoints
 // DEBUGGER: break static-member.cpp:33
 // DEBUGGER: r
 // DEBUGGER: ptype C
 // CHECK:      {{struct|class}} C {
 // CHECK:      static const int a;
 // CHECK-NEXT: static int b;

cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp

 // RUN: %clangxx %target_itanium_abi_host_triple -O0 -g %s -o %t -c
 // RUN: %clangxx %target_itanium_abi_host_triple %t -o %t.out
 // RUN: %test_debuginfo %s %t.out
-
+// XFAIL: !system-darwin && gdb-clang-incompatibility
 // DEBUGGER: delete breakpoints
 // DEBUGGER: break static-member.cpp:33
 // DEBUGGER: r
 // DEBUGGER: ptype MyClass
 // CHECK:      {{struct|class}} MyClass {
 // CHECK:      static const int a;
 // CHECK-NEXT: static int b;
 // CHECK-NEXT: static int c;

cross-project-tests/lit.cfg.py

 import os
 import platform
 import re
 import subprocess
 import sys
+from distutils.version import StrictVersion
 
 import lit.formats
 import lit.util
 
 from lit.llvm import llvm_config
 from lit.llvm.subst import ToolSubst
 
 # Configuration file for the 'lit' test runner.

 if platform.system() == 'Darwin':
     xcode_lldb_vers = subprocess.check_output(['xcrun', 'lldb', '--version']).decode("utf-8")
     match = re.search('lldb-(\d+)', xcode_lldb_vers)
     if match:
         apple_lldb_vers = int(match.group(1))
         if apple_lldb_vers < 1000:
             config.available_features.add('apple-lldb-pre-1000')
 
+def get_gdb_version_string():
+  """Return gdb's version string, or None if gdb cannot be found or the
+  --version output is formatted unexpectedly.
+  """
+  # See if we can get a gdb version, e.g.
+  #   $ gdb --version
+  #   GNU gdb (GDB) 10.2
+  #   ...More stuff...
+  try:
+    gdb_vers_lines = subprocess.check_output(['gdb', '--version']).decode().splitlines()
+  except:
+    return None # We coudln't find gdb or something went wrong running it.
+  if len(gdb_vers_lines) < 1:
+    print("Unkown GDB version format (too few lines)", file=sys.stderr)
+    return None
+  string = gdb_vers_lines[0].strip().partition('GNU gdb (GDB) ')[2]
+  if len(string) == 0:
+    print("Unkown GDB version format", file=sys.stderr)
+    return None
+  return string
+
+def get_clang_default_dwarf_version_string(triple):
+  """Return the default dwarf version string for clang on this (host) platform
+  or None if we can't work it out.
+  """
+  # Get the flags passed by the driver and look for -dwarf-version.
+  cmd = f'{llvm_config.use_llvm_tool("clang")} -g -xc  -c - -v -### --target={triple}'
+  stderr = subprocess.run(cmd.split(), stderr=subprocess.PIPE).stderr.decode()
+  match = re.search('-dwarf-version=(\d+)', stderr)
+  if match is None:
+    print("Cannot determine default dwarf version", file=sys.stderr)
+    return None
+  return match.group(1)
+
+# Some cross-project-tests use gdb, but not all versions of gdb are compatible
+# with clang's dwarf. Add feature `gdb-clang-incompatibility` to signal that
+# there's an incompatibility between clang's default dwarf version for this
+# platform and the installed gdb version.
+dwarf_version_string = get_clang_default_dwarf_version_string(config.host_triple)
+gdb_version_string = get_gdb_version_string()
+if dwarf_version_string and gdb_version_string:
+  if int(dwarf_version_string) >= 5:
+    if StrictVersion(gdb_version_string) < StrictVersion('10.1'):
+      # Example for llgdb-tests, which use lldb on darwin but gdb elsewhere:
+      # XFAIL: !system-darwin && gdb-clang-incompatibility
+      config.available_features.add('gdb-clang-incompatibility')
+      print("XFAIL some tests: use gdb version >= 10.1 to restore test coverage", file=sys.stderr)
+
 llvm_config.feature_config(
     [('--build-mode', {'Debug|RelWithDebInfo': 'debug-info'})]
 )