[dotest] Add a dotest flag `--force-dwarf` to override the tested DWARF version.

[JDevlieghere]

Summary

On the matrix bot on GreenDragon [1] we want to run the test suite against different DWARF versions. The idea here is not to replace targeted tests for certain DWARF features, but rather to provide an easy way to support this configuration.

[1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/

[aprantl]

This is definitely more elegant than the current solution!

lldb/packages/Python/lldbsuite/test/dotest.py

353 ↗

(On Diff #215702)

Does this actually work with tests whose Makefiles manually set CFLAGS_EXTRAS? I.e., do we need to distinguish between extra CFLAGS that come in from the layer above and ones that are meant to be customized in the individual test Makefiles?

[jankratochvil]

For DWARF >= 4 it also matters for testing whether one is using -fdebug-types-section or not.

lldb/packages/Python/lldbsuite/test/dotest.py

373 ↗

(On Diff #215702)

intentional?

[labath]

This looks fine to me. We spoke about this on irc, but I'll just repeat what I said here for the record. Running the dotest tests with different compilers/architectures/dwarf versions/etc. is very useful for increasing coverage, finding new bugs, etc. However, I don't think we should be using this facility as a first or only line of defence against regressions. Specifically, I'd like to avoid arguments like "there's no need to add tests for this because this code is exercised when running the test suite with gcc-47 and dwarf-13 on a Tuesday wearing shorts". New features/bugfixes should be testable in the default configuration, and this facility should be the "extra mile".

lldb/packages/Python/lldbsuite/test/dotest.py

353 ↗

(On Diff #215702)

If makefiles do CFLAGS_EXTRAS += ... instead of CFLAGS_EXTRAS = foo (which I think most do), then everything should be fine.

lldb/packages/Python/lldbsuite/test/dotest_args.py

129 ↗

(On Diff #215702)

Maybe just call the argument --dwarf-version, just like the source variable?

lldb/trunk/packages/Python/lldbsuite/test/configuration.py

 
 # Test suite repeat count.  Can be overwritten with '-# count'.
 count = 1
 
 # The 'arch' and 'compiler' can be specified via command line.
 arch = None        # Must be initialized after option parsing
 compiler = None    # Must be initialized after option parsing
 
+# The overriden dwarf verison.
+dwarf_version = 0
+
 # Path to the FileCheck testing tool. Not optional.
 filecheck = None
 
 # The arch might dictate some specific CFLAGS to be passed to the toolchain to build
 # the inferior programs.  The global variable cflags_extras provides a hook to do
 # just that.
 cflags_extras = ''
 

lldb/trunk/packages/Python/lldbsuite/test/decorators.py

def fn(self):
         skip_for_py_version = (
             py_version is None) or _check_expected_version(
             py_version[0], py_version[1], sys.version_info)
         skip_for_macos_version = (macos_version is None) or (
             (platform.mac_ver()[0] != "") and (_check_expected_version(
                 macos_version[0],
                 macos_version[1],
                 platform.mac_ver()[0])))
-        skip_for_dwarf_version = (
-             dwarf_version is None) or (
-                 (self.getDebugInfo() is 'dwarf') and
-                 _check_expected_version(
-                     dwarf_version[0], dwarf_version[1], self.getDwarfVersion()))
+        skip_for_dwarf_version = (dwarf_version is None) or (
+            _check_expected_version(dwarf_version[0], dwarf_version[1],
+                                    self.getDwarfVersion()))
 
         # For the test to be skipped, all specified (e.g. not None) parameters must be True.
         # An unspecified parameter means "any", so those are marked skip by default.  And we skip
         # the final test if all conditions are True.
         conditions = [(oslist, skip_for_os, "target o/s"),
                       (hostoslist, skip_for_hostos, "host o/s"),
                       (compiler, skip_for_compiler, "compiler or version"),
                       (archs, skip_for_arch, "architecture"),

lldb/trunk/packages/Python/lldbsuite/test/dotest.py

if args.categoriesList:
         configuration.useCategories = True
     else:
         configuration.categoriesList = []
 
     if args.skipCategories:
         configuration.skipCategories += test_categories.validate(
             args.skipCategories, False)
 
+    cflags_extras = ""
     if args.E:
-        cflags_extras = args.E
-        os.environ['CFLAGS_EXTRAS'] = cflags_extras
+        cflags_extras += args.E
+
+    if args.dwarf_version:
+        configuration.dwarf_version = args.dwarf_version
+        cflags_extras += '-gdwarf-{}'.format(args.dwarf_version)
+
 
     if args.d:
         sys.stdout.write(
             "Suspending the process %d to wait for debugger to attach...\n" %
             os.getpid())
         sys.stdout.flush()
         os.kill(os.getpid(), signal.SIGSTOP)
 

lldb/trunk/packages/Python/lldbsuite/test/dotest_args.py

group.add_argument(
         metavar='server-path',
         help='The path to the debug server executable to use')
     group.add_argument(
         '--out-of-tree-debugserver',
         dest='out_of_tree_debugserver',
         action='store_true',
         help='A flag to indicate an out-of-tree debug server is being used')
     group.add_argument(
+        '--dwarf-version',
+        metavar='dwarf_version',
+        dest='dwarf_version',
+        type=int,
+        help='Override the DWARF version.')
+    group.add_argument(
         '-s',
         metavar='name',
         help='Specify the name of the dir created to store the session files of tests with errored or failed status. If not specified, the test driver uses the timestamp as the session dir name')
     group.add_argument(
         '-S',
         '--session-file-format',
         default=configuration.session_file_format,
         metavar='format',

lldb/trunk/packages/Python/lldbsuite/test/lldbtest.py

def getCompilerVersion(self):
         for line in version_output.split(os.linesep):
             m = re.search('version ([0-9\.]+)', line)
             if m:
                 version = m.group(1)
         return version
 
     def getDwarfVersion(self):
         """ Returns the dwarf version generated by clang or '0'. """
+        if configuration.dwarf_version:
+            return str(configuration.dwarf_version)
         if 'clang' in self.getCompiler():
             try:
                 driver_output = check_output(
                     [self.getCompiler()] + '-g -c -x c - -o - -###'.split(),
                     stderr=STDOUT)
                 for line in driver_output.split(os.linesep):
                     m = re.search('dwarf-version=([0-9])', line)
                     if m: