[test-suite] Add a decorator for the lack of libstdcxx on the system.

[davide]

Summary

commit 9fed17ef5633194ba7d444451cce70fe5711c84e (HEAD -> master)
Author: Davide Italiano <[e-mail]>
Date: Fri Jul 6 09:41:29 2018 -0700

This generalizes a bunch of target-specific tests. MacOS has no
libstdcxx anymore, and neither does FreeBSD (or Windows).

<rdar://problem/41896105>

[davide]

I went ahead and committed this because it was breaking a downstream bot:

rL336463: [test-suite] Add a decorator for the lack of libstdcxx on the system..

Post-review commit appreciated of course.

[jasonmolenda]

Is this going to work with non-clang compilers? ('$CXX -x c++ -stdlib=libstdc++ ') I can imagine gcc supporting these same flags, but will this work on windows?

[davide]

In D49033#1154767, @jasonmolenda wrote:

Is this going to work with non-clang compilers? ('$CXX -x c++ -stdlib=libstdc++ ') I can imagine gcc supporting these same flags, but will this work on windows?

I checked and gcc supports the flags.
I'm not sure about Windows (cc: @stella.stamenova) and I don't have a machine to check I'm afraid.
FWIW, there are a bunch of other decorators with do this (pass a random flag) so we might consider fixing them all at once (if there's any need to).

Thanks!

[labath]

Besides the problems I mention in inline comments, the other issue I have with this patch is that it does things completely different than how we implement the check for libc++, which is an identical problem. (libc++ detection is done by marking all libc++ tests as with a special category, and then deciding in dotest.py (checkLibcxxSupport) whether to run the category as a whole).

So, I think this should be reverted and replaced by a solution mirroring the libc++ approach. I'm not saying that one is perfect, but it is in many ways better than this one (e.g., if the default skipping logic gets things wrong, you can override it by manually skipping the corresponding category), and most importantly, it makes things consistent.

lldb/packages/Python/lldbsuite/test/decorators.py

696

This won't work in several scenarios:

• gcc: I'm not sure how you tested on gcc, but my gcc(-7.3.0) gives me an "unrecognised command line option" on the -stdlib argument.
• cross-compilation: just because the default target has/hasn't libstdc++ headers+libs available, it does not mean the target we selected for cross compilation will be the same.

On top of all of that you have a mismatched ' inside the executed command...

[stella.stamenova]

cl (the visual studio compiler) does not support all of the flags:

cl : Command line warning D9002 : ignoring unknown option '-x'
cl : Command line warning D9002 : ignoring unknown option '-stdlib=libstdc++'
cl : Command line warning D9002 : ignoring unknown option '-'

[davide]

In D49033#1155743, @labath wrote:

Besides the problems I mention in inline comments, the other issue I have with this patch is that it does things completely different than how we implement the check for libc++, which is an identical problem. (libc++ detection is done by marking all libc++ tests as with a special category, and then deciding in dotest.py (checkLibcxxSupport) whether to run the category as a whole).

So, I think this should be reverted and replaced by a solution mirroring the libc++ approach. I'm not saying that one is perfect, but it is in many ways better than this one (e.g., if the default skipping logic gets things wrong, you can override it by manually skipping the corresponding category), and most importantly, it makes things consistent.

OK, I'm going to revert this and work on a better solution today.

[davide]

This was reverted at:

Davides-Mac-Pro:lldb davide$ git llvm push
Pushing 1 commit:
  116b1ec8e32 Rollback [test-suite] Add a decorator for the lack of libstdcxx on the system.
Sending        lldb/trunk/packages/Python/lldbsuite/test/decorators.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/iterator/TestDataFormatterStdIterator.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/list/TestDataFormatterStdList.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/map/TestDataFormatterStdMap.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/smart_ptr/TestDataFormatterStdSmartPtr.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/string/TestDataFormatterStdString.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/unique_ptr/TestDataFormatterStdUniquePtr.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/vbool/TestDataFormatterStdVBool.py
Sending        lldb/trunk/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/vector/TestDataFormatterStdVector.py
Transmitting file data ..........done
Committing transaction...
Committed revision 336608.
Committed 116b1ec8e32 to svn.

@labath I'll send another review with the approach you proposed soon.

[davide]

Was implemented differently. Closing.

lldb/packages/Python/lldbsuite/test/decorators.py

def compiler_doesnt_support_struct_attribute(self):
         cmd = [self.getCompiler(), "-x", "c++", "-c", "-o", f.name, "-"]
         p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
         stdout, stderr = p.communicate('struct __attribute__((%s)) Test {};'%attr)
         if attr in stderr:
             return "Compiler does not support attribute %s"%(attr)
         return None
     return skipTestIfFn(compiler_doesnt_support_struct_attribute)
 
+def skipUnlessLibstdcxxAvailable(func):
+    """Decorate the item to skip test unless libstdc++ is available on the system."""
+    def compiler_doesnt_support_libstdcxx(self):
+        compiler_path = self.getCompiler()
+        f = tempfile.NamedTemporaryFile()
+        f = tempfile.NamedTemporaryFile()
+        cmd = "echo '#include <string> | %s -x c++ -stdlib=libstdc++ -o %s -" % (compiler_path, f.name)
+        if os.popen(cmd).close() is not None:
+            return "libstdcxx not available on the sytem"
+        return None
+    return skipTestIfFn(compiler_doesnt_support_libstdcxx)(func)
+
 def skipUnlessThreadSanitizer(func):
     """Decorate the item to skip test unless Clang -fsanitize=thread is supported."""
 
     def is_compiler_clang_with_thread_sanitizer(self):
         compiler_path = self.getCompiler()
         compiler = os.path.basename(compiler_path)
         if not compiler.startswith("clang"):
             return "Test requires clang as compiler"

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/iterator/TestDataFormatterStdIterator.py

class StdIteratorDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', '// Set break point at this line.')
 
-    @skipIfWindows  # libstdcpp not ported to Windows
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         """Test that libstdcpp iterators format properly."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=-1)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/list/TestDataFormatterStdList.py

def setUp(self):
         TestBase.setUp(self)
         # Find the line numbers to break at for the different tests.
         self.line = line_number('main.cpp', '// Set break point at this line.')
         self.optional_line = line_number(
             'main.cpp', '// Optional break point at this line.')
         self.final_line = line_number(
             'main.cpp', '// Set final break point at this line.')
 
-    @skipIfWindows  # libstdcpp not ported to Windows
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         """Test that that file and class static variables display correctly."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=-1)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/map/TestDataFormatterStdMap.py

class StdMapDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', '// Set break point at this line.')
 
-    @skipIfWindows  # libstdcpp not ported to Windows
-    @skipIfFreeBSD
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         """Test that that file and class static variables display correctly."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_source_regexp(
             self, "Set break point at this line.")
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/smart_ptr/TestDataFormatterStdSmartPtr.py

 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class StdSmartPtrDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
-    @skipIfFreeBSD
-    @skipIfWindows  # libstdcpp not ported to Windows
     @skipIfDarwin  # doesn't compile on Darwin
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_source_regexp(
             self, "Set break point at this line.")
         self.runCmd("run", RUN_SUCCEEDED)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/string/TestDataFormatterStdString.py

class StdStringDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', '// Set break point at this line.')
 
-    @skipIfWindows  # libstdcpp not ported to Windows
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         """Test that that file and class static variables display correctly."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=-1)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py

 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class StdTupleDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
-    @skipIfFreeBSD
-    @skipIfWindows  # libstdcpp not ported to Windows
     @skipIfDarwin  # doesn't compile on Darwin
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_source_regexp(
             self, "Set break point at this line.")
         self.runCmd("run", RUN_SUCCEEDED)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/unique_ptr/TestDataFormatterStdUniquePtr.py

 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class StdUniquePtrDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
-    @skipIfFreeBSD
-    @skipIfWindows  # libstdcpp not ported to Windows
     @skipIfDarwin  # doesn't compile on Darwin
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_source_regexp(
             self, "Set break point at this line.")
         self.runCmd("run", RUN_SUCCEEDED)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/vbool/TestDataFormatterStdVBool.py

class StdVBoolDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', '// Set break point at this line.')
 
-    @expectedFailureAll(
-        oslist=['freebsd'],
-        bugnumber='llvm.org/pr20548 fails to build on lab.llvm.org buildbot')
-    @skipIfWindows  # libstdcpp not ported to Windows.
     @skipIfDarwin
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         """Test that that file and class static variables display correctly."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=-1)
 

lldb/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/vector/TestDataFormatterStdVector.py

class StdVectorDataFormatterTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', '// Set break point at this line.')
 
-    @skipIfFreeBSD
-    @skipIfWindows  # libstdcpp not ported to Windows
-    @skipIfwatchOS  # libstdcpp not ported to watchos
+    @skipUnlessLibstdcxxAvailable
     def test_with_run_command(self):
         """Test that that file and class static variables display correctly."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_source_regexp(
             self, "Set break point at this line.")