This is an archive of the discontinued LLVM Phabricator instance.

Differential D49110

[testsuite] Implement a category to skip libstdcxx tests
ClosedPublic

Authored by davide on Jul 9 2018, 5:10 PM.

Details

Reviewers
labath
zturner
JDevlieghere
Commits
rG87951b6693bf: [testsuite] Implement a category to skip libstdcxx tests
rLLDB336724: [testsuite] Implement a category to skip libstdcxx tests
rL336724: [testsuite] Implement a category to skip libstdcxx tests
Summary

On systems where it's not supported.
As far as I understand Linux is the only systems which now ships with libstdcxx (maybe NetBSD?, but I'm not entirely sure of the state of lldb on the platform).
We could make this more fine grained looking for the header as we do for libcxx. This is a little tricky as there's no such thing as /usr/include/c++/v1, but libstdcxx encodes the version number in the path (i.e. /usr/include/c++/5.4). I guess we might match a regex, but it seems fragile to me.

@labath I hope this is more in line with what you had in mind, thanks for your feedback!

Diff Detail

Event Timeline

davide created this revision.Jul 9 2018, 5:10 PM
Herald added a subscriber: krytarowski. · View Herald TranscriptJul 9 2018, 5:10 PM
davide added a reviewer: zturner.Jul 10 2018, 9:36 AM
davide added a comment.Jul 10 2018, 9:39 AM

@labath this is breaking several bots for us internally so I would really love to get this in.
As this seems to be reasonably close to what you ask, I might consider checking this in before end of day (PST) today and we can iterate from there.

JDevlieghere accepted this revision.Jul 10 2018, 10:00 AM
JDevlieghere added a subscriber: JDevlieghere.

LGTM

This revision is now accepted and ready to land.Jul 10 2018, 10:00 AM
labath accepted this revision.Jul 10 2018, 11:35 AM

Sorry for the slow response, I am OOO officially :P.

I think this is fine in general. As for the list of platforms, android should be on that list but not for long (which is why one of the things I was planning to do next week was to implement something like this). I am not sure about NetBSD, but @krytarowski should be able to add that easily if that is the case.

For the category markup, I was hoping you would just place a .categories file in the appropriate subfolder, but after checking I see that this is actually consistent with how libc++ tests are marked up, so that is not a big deal either. I'll probably convert both of them to the filesystem-based markup next week.

davide added a comment.Jul 10 2018, 11:37 AM

Thank you Pavel!

Closed by commit rL336724: [testsuite] Implement a category to skip libstdcxx tests (authored by davide). · Explain WhyJul 10 2018, 1:42 PM
This revision was automatically updated to reflect the committed changes.
Herald added a subscriber: llvm-commits. · View Herald TranscriptJul 10 2018, 1:42 PM
krytarowski added a comment.Jul 10 2018, 2:59 PM

NetBSD ships with GCC style and Clang style userland. The former one ships with GNU libstdc++, the latter with libc++.

krytarowski added a comment.Jul 10 2018, 3:00 PM

Probably the former with ship GDB at least for long time, the latter will switch to LLDB.

krytarowski added a comment.Jul 10 2018, 3:03 PM

Through pkgsrc a user can install LLDB with either style of distribution, including GCC with libstdc++ one (I'm using this myself also for the development of LLVM related code).

BTW. NetBSD uses GPLv3 GNU toolchain.

Revision Contents

PathSize
lldb/
packages/
Python/
lldbsuite/
test/
18 lines
functionalities/
data-formatter/
data-formatter-stl/
libstdcpp/
iterator/
3 lines
list/
3 lines
map/
4 lines
smart_ptr/
5 lines
string/
3 lines
tuple/
5 lines
unique_ptr/
5 lines
vbool/
7 lines
vector/
4 lines
1 line

Diff 154737

lldb/packages/Python/lldbsuite/test/dotest.py

Show First 20 Lines • Show All 492 Lines • ▼ Show 20 Lines
result, reason = canRunLibcxxTests() result, reason = canRunLibcxxTests()
if result: if result:
return # libc++ supported return # libc++ supported
if "libc++" in configuration.categoriesList: if "libc++" in configuration.categoriesList:
return # libc++ category explicitly requested, let it run. return # libc++ category explicitly requested, let it run.
print("Libc++ tests will not be run because: " + reason) print("Libc++ tests will not be run because: " + reason)
configuration.skipCategories.append("libc++") configuration.skipCategories.append("libc++")
def canRunLibstdcxxTests():
from lldbsuite.test import lldbplatformutil
platform = lldbplatformutil.getPlatform()
if platform == "linux":
return True, "libstdcxx always present"
return False, "Don't know how to build with libstdcxx on %s" % platform
def checkLibstdcxxSupport():
result, reason = canRunLibstdcxxTests()
if result:
return # libstdcxx supported
if "libstdcxx" in configuration.categoriesList:
return # libstdcxx category explicitly requested, let it run.
print("libstdcxx tests will not be run because: " + reason)
configuration.skipCategories.append("libstdcxx")
def checkDebugInfoSupport(): def checkDebugInfoSupport():
import lldb import lldb
platform = lldb.DBG.GetSelectedPlatform().GetTriple().split('-')[2] platform = lldb.DBG.GetSelectedPlatform().GetTriple().split('-')[2]
compiler = configuration.compiler compiler = configuration.compiler
skipped = [] skipped = []
for cat in test_categories.debug_info_categories: for cat in test_categories.debug_info_categories:
if cat in configuration.categoriesList: if cat in configuration.categoriesList:
▲ Show 20 Lines • Show All 108 Lines • ▼ Show 20 Lines
# Note that it's not dotest's job to clean this directory. # Note that it's not dotest's job to clean this directory.
import lldbsuite.test.lldbutil as lldbutil import lldbsuite.test.lldbutil as lldbutil
build_dir = configuration.test_build_dir build_dir = configuration.test_build_dir
lldbutil.mkdir_p(build_dir) lldbutil.mkdir_p(build_dir)
target_platform = lldb.DBG.GetSelectedPlatform().GetTriple().split('-')[2] target_platform = lldb.DBG.GetSelectedPlatform().GetTriple().split('-')[2]
checkLibcxxSupport() checkLibcxxSupport()
checkLibstdcxxSupport()
checkDebugInfoSupport() checkDebugInfoSupport()
# Don't do debugserver tests on anything except OS X. # Don't do debugserver tests on anything except OS X.
configuration.dont_do_debugserver_test = "linux" in target_platform or "freebsd" in target_platform or "windows" in target_platform configuration.dont_do_debugserver_test = "linux" in target_platform or "freebsd" in target_platform or "windows" in target_platform
# Don't do lldb-server (llgs) tests on anything except Linux. # Don't do lldb-server (llgs) tests on anything except Linux.
configuration.dont_do_llgs_test = not ("linux" in target_platform) configuration.dont_do_llgs_test = not ("linux" in target_platform)
▲ Show 20 Lines • Show All 136 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/iterator/TestDataFormatterStdIterator.py

Show All 17 Linesclass StdIteratorDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
def setUp(self): def setUp(self):
# Call super's setUp(). # Call super's setUp().
TestBase.setUp(self) TestBase.setUp(self)
# Find the line number to break at. # Find the line number to break at.
self.line = line_number('main.cpp', '// Set break point at this line.') self.line = line_number('main.cpp', '// Set break point at this line.')
@skipIfWindows # libstdcpp not ported to Windows @add_test_categories(["libstdcxx"])
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
"""Test that libstdcpp iterators format properly.""" """Test that libstdcpp iterators format properly."""
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_file_and_line( lldbutil.run_break_set_by_file_and_line(
self, "main.cpp", self.line, num_expected_locations=-1) self, "main.cpp", self.line, num_expected_locations=-1)
Show All 40 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/list/TestDataFormatterStdList.py

Show All 21 Lines def setUp(self):
TestBase.setUp(self) TestBase.setUp(self)
# Find the line numbers to break at for the different tests. # Find the line numbers to break at for the different tests.
self.line = line_number('main.cpp', '// Set break point at this line.') self.line = line_number('main.cpp', '// Set break point at this line.')
self.optional_line = line_number( self.optional_line = line_number(
'main.cpp', '// Optional break point at this line.') 'main.cpp', '// Optional break point at this line.')
self.final_line = line_number( self.final_line = line_number(
'main.cpp', '// Set final break point at this line.') 'main.cpp', '// Set final break point at this line.')
@skipIfWindows # libstdcpp not ported to Windows @add_test_categories(["libstdcxx"])
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
"""Test that that file and class static variables display correctly.""" """Test that that file and class static variables display correctly."""
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_file_and_line( lldbutil.run_break_set_by_file_and_line(
self, "main.cpp", self.line, num_expected_locations=-1) self, "main.cpp", self.line, num_expected_locations=-1)
▲ Show 20 Lines • Show All 171 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/map/TestDataFormatterStdMap.py

Show All 17 Linesclass StdMapDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
def setUp(self): def setUp(self):
# Call super's setUp(). # Call super's setUp().
TestBase.setUp(self) TestBase.setUp(self)
# Find the line number to break at. # Find the line number to break at.
self.line = line_number('main.cpp', '// Set break point at this line.') self.line = line_number('main.cpp', '// Set break point at this line.')
@skipIfWindows # libstdcpp not ported to Windows @add_test_categories(["libstdcxx"])
@skipIfFreeBSD
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
"""Test that that file and class static variables display correctly.""" """Test that that file and class static variables display correctly."""
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_source_regexp( lldbutil.run_break_set_by_source_regexp(
self, "Set break point at this line.") self, "Set break point at this line.")
▲ Show 20 Lines • Show All 300 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/smart_ptr/TestDataFormatterStdSmartPtr.py

Show All 9 Lines
from lldbsuite.test.decorators import * from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import * from lldbsuite.test.lldbtest import *
from lldbsuite.test import lldbutil from lldbsuite.test import lldbutil
class StdSmartPtrDataFormatterTestCase(TestBase): class StdSmartPtrDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
@skipIfFreeBSD @add_test_categories(["libstdcxx"])
@skipIfWindows # libstdcpp not ported to Windows
@skipIfDarwin # doesn't compile on Darwin
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_source_regexp( lldbutil.run_break_set_by_source_regexp(
self, "Set break point at this line.") self, "Set break point at this line.")
self.runCmd("run", RUN_SUCCEEDED) self.runCmd("run", RUN_SUCCEEDED)
Show All 21 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/string/TestDataFormatterStdString.py

Show All 18 Linesclass StdStringDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
def setUp(self): def setUp(self):
# Call super's setUp(). # Call super's setUp().
TestBase.setUp(self) TestBase.setUp(self)
# Find the line number to break at. # Find the line number to break at.
self.line = line_number('main.cpp', '// Set break point at this line.') self.line = line_number('main.cpp', '// Set break point at this line.')
@skipIfWindows # libstdcpp not ported to Windows @add_test_categories(["libstdcxx"])
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
"""Test that that file and class static variables display correctly.""" """Test that that file and class static variables display correctly."""
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_file_and_line( lldbutil.run_break_set_by_file_and_line(
self, "main.cpp", self.line, num_expected_locations=-1) self, "main.cpp", self.line, num_expected_locations=-1)
▲ Show 20 Lines • Show All 46 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py

Show All 9 Lines
from lldbsuite.test.decorators import * from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import * from lldbsuite.test.lldbtest import *
from lldbsuite.test import lldbutil from lldbsuite.test import lldbutil
class StdTupleDataFormatterTestCase(TestBase): class StdTupleDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
@skipIfFreeBSD @add_test_categories(["libstdcxx"])
@skipIfWindows # libstdcpp not ported to Windows
@skipIfDarwin # doesn't compile on Darwin
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_source_regexp( lldbutil.run_break_set_by_source_regexp(
self, "Set break point at this line.") self, "Set break point at this line.")
self.runCmd("run", RUN_SUCCEEDED) self.runCmd("run", RUN_SUCCEEDED)
Show All 21 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/unique_ptr/TestDataFormatterStdUniquePtr.py

Show All 9 Lines
from lldbsuite.test.decorators import * from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import * from lldbsuite.test.lldbtest import *
from lldbsuite.test import lldbutil from lldbsuite.test import lldbutil
class StdUniquePtrDataFormatterTestCase(TestBase): class StdUniquePtrDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
@skipIfFreeBSD @add_test_categories(["libstdcxx"])
@skipIfWindows # libstdcpp not ported to Windows
@skipIfDarwin # doesn't compile on Darwin
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_source_regexp( lldbutil.run_break_set_by_source_regexp(
self, "Set break point at this line.") self, "Set break point at this line.")
self.runCmd("run", RUN_SUCCEEDED) self.runCmd("run", RUN_SUCCEEDED)
▲ Show 20 Lines • Show All 69 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/vbool/TestDataFormatterStdVBool.py

Show All 17 Linesclass StdVBoolDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
def setUp(self): def setUp(self):
# Call super's setUp(). # Call super's setUp().
TestBase.setUp(self) TestBase.setUp(self)
# Find the line number to break at. # Find the line number to break at.
self.line = line_number('main.cpp', '// Set break point at this line.') self.line = line_number('main.cpp', '// Set break point at this line.')
@expectedFailureAll( @add_test_categories(["libstdcxx"])
oslist=['freebsd'],
bugnumber='llvm.org/pr20548 fails to build on lab.llvm.org buildbot')
@skipIfWindows # libstdcpp not ported to Windows.
@skipIfDarwin
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
"""Test that that file and class static variables display correctly.""" """Test that that file and class static variables display correctly."""
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_file_and_line( lldbutil.run_break_set_by_file_and_line(
self, "main.cpp", self.line, num_expected_locations=-1) self, "main.cpp", self.line, num_expected_locations=-1)
▲ Show 20 Lines • Show All 44 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/vector/TestDataFormatterStdVector.py

Show All 17 Linesclass StdVectorDataFormatterTestCase(TestBase):
mydir = TestBase.compute_mydir(__file__) mydir = TestBase.compute_mydir(__file__)
def setUp(self): def setUp(self):
# Call super's setUp(). # Call super's setUp().
TestBase.setUp(self) TestBase.setUp(self)
# Find the line number to break at. # Find the line number to break at.
self.line = line_number('main.cpp', '// Set break point at this line.') self.line = line_number('main.cpp', '// Set break point at this line.')
@skipIfFreeBSD @add_test_categories(["libstdcxx"])
@skipIfWindows # libstdcpp not ported to Windows
@skipIfwatchOS # libstdcpp not ported to watchos
def test_with_run_command(self): def test_with_run_command(self):
"""Test that that file and class static variables display correctly.""" """Test that that file and class static variables display correctly."""
self.build() self.build()
self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET) self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
lldbutil.run_break_set_by_source_regexp( lldbutil.run_break_set_by_source_regexp(
self, "Set break point at this line.") self, "Set break point at this line.")
▲ Show 20 Lines • Show All 182 Lines • Show Last 20 Lines

lldb/packages/Python/lldbsuite/test/test_categories.py

Show All 20 Lines
all_categories = { all_categories = {
'dataformatters': 'Tests related to the type command and the data formatters subsystem', 'dataformatters': 'Tests related to the type command and the data formatters subsystem',
'dwarf': 'Tests that can be run with DWARF debug information', 'dwarf': 'Tests that can be run with DWARF debug information',
'dwo': 'Tests that can be run with DWO debug information', 'dwo': 'Tests that can be run with DWO debug information',
'dsym': 'Tests that can be run with DSYM debug information', 'dsym': 'Tests that can be run with DSYM debug information',
'gmodules': 'Tests that can be run with -gmodules debug information', 'gmodules': 'Tests that can be run with -gmodules debug information',
'expression': 'Tests related to the expression parser', 'expression': 'Tests related to the expression parser',
'libc++': 'Test for libc++ data formatters', 'libc++': 'Test for libc++ data formatters',
'libstdcxx': 'Test for libstdcxx data formatters',
'objc': 'Tests related to the Objective-C programming language support', 'objc': 'Tests related to the Objective-C programming language support',
'pyapi': 'Tests related to the Python API', 'pyapi': 'Tests related to the Python API',
'basic_process': 'Basic process execution sniff tests.', 'basic_process': 'Basic process execution sniff tests.',
'cmdline': 'Tests related to the LLDB command-line interface', 'cmdline': 'Tests related to the LLDB command-line interface',
'dyntype': 'Tests related to dynamic type support', 'dyntype': 'Tests related to dynamic type support',
'stresstest': 'Tests related to stressing lldb limits', 'stresstest': 'Tests related to stressing lldb limits',
'flakey': 'Flakey test cases, i.e. tests that do not reliably pass at each execution', 'flakey': 'Flakey test cases, i.e. tests that do not reliably pass at each execution',
'lldb-mi': 'lldb-mi tests', 'lldb-mi': 'lldb-mi tests',
▲ Show 20 Lines • Show All 53 Lines • Show Last 20 Lines