[vscode] Skip some of the vscode tests on Linux and fix one

[stella.stamenova]

Summary

These are already skipped on Darwin because they cause build bot failures. Both on the build bots as well as in our testing we have seen a number of these tests fail and hang. This change skips the failing/hanging tests on Linux and also fixes one of the test - the test needs the thread library to build.

[teemperor]

packages/Python/lldbsuite/test/tools/lldb-vscode/attach/TestVSCode_attach.py

50 ↗

(On Diff #162430)

"is well as" -> "as well as" (same for the other comments).

[stella.stamenova]

packages/Python/lldbsuite/test/tools/lldb-vscode/attach/TestVSCode_attach.py

50 ↗

(On Diff #162430)

Apparently I can't spell today. Thanks for catching this!

[aprantl]

Looks like we *really* need to figure out what the underlying bug is here, because there's not many platforms left after this patch.
This patch is fine as a stopgap, but we can't have a feature like this be in the source code while being virtually untested, that's just asking for trouble.

Greg, were you able to reproduce this?
Stella, do you have any logging available that could help Greg in tracking down the root cause?

[stella.stamenova]

In D51227#1212838, @aprantl wrote:

Looks like we *really* need to figure out what the underlying bug is here, because there's not many platforms left after this patch.
This patch is fine as a stopgap, but we can't have a feature like this be in the source code while being virtually untested, that's just asking for trouble.

Greg, were you able to reproduce this?
Stella, do you have any logging available that could help Greg in tracking down the root cause?

It's fairly easy for me to reproduce, so if Greg has time to investigate, I can collect the logs he needs.

[stella.stamenova]

I spent some more time on this to isolate the failing tests (and reduce the number of disabled tests). I was also able to figure out what one of the failures is - the test needs the thread library to build. The update change reflects that.

lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-vscode/attach/TestVSCode_attach.py

def set_and_hit_breakpoint(self, continueToExit=True):
                          "expect correct number of breakpoints")
         self.continue_to_breakpoints(breakpoint_ids)
         if continueToExit:
             self.continue_to_exit()
 
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_by_pid(self):
         '''
             Tests attaching to a process by process ID.
         '''
         self.build_and_create_debug_adaptor()
         program = self.getBuildArtifact("a.out")
         self.process = subprocess.Popen([program],
                                         stdin=subprocess.PIPE,
                                         stdout=subprocess.PIPE,
                                         stderr=subprocess.PIPE)
         self.attach(pid=self.process.pid)
         self.set_and_hit_breakpoint(continueToExit=True)
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_by_name(self):
         '''
             Tests attaching to a process by process name.
         '''
         self.build_and_create_debug_adaptor()
         orig_program = self.getBuildArtifact("a.out")
         # Since we are going to attach by process name, we need a unique
def test_by_name(self):
                                         stderr=subprocess.PIPE)
         # Wait for a bit to ensure the process is launched
         time.sleep(5)
         self.attach(program=program)
         self.set_and_hit_breakpoint(continueToExit=True)
 
     @skipUnlessDarwin
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_by_name_waitFor(self):
         '''
             Tests attaching to a process by process name and waiting for the
             next instance of a process to be launched, ingoring all current
             ones.
         '''
         self.build_and_create_debug_adaptor()
         program = self.getBuildArtifact("a.out")
         self.spawn_thread = threading.Thread(target=spawn_and_wait,
                                              args=(program, 1.0,))
         self.spawn_thread.start()
         self.attach(program=program, waitFor=True)
         self.set_and_hit_breakpoint(continueToExit=True)
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_commands(self):
         '''
             Tests the "initCommands", "preRunCommands", "stopCommands",
             "exitCommands", and "attachCommands" that can be passed during
             attach.
 
             "initCommands" are a list of LLDB commands that get executed

lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-vscode/launch/TestVSCode_launch.py

 
 
 class TestVSCode_launch(lldbvscode_testcase.VSCodeTestCaseBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_default(self):
         '''
             Tests the default launch of a simple program. No arguments,
             environment, or anything else is specified.
         '''
         program = self.getBuildArtifact("a.out")
         self.build_and_launch(program)
         self.continue_to_exit()
         # Now get the STDOUT and verify our program argument is correct
         output = self.get_stdout()
         self.assertTrue(output and len(output) > 0,
                         "expect program output")
         lines = output.splitlines()
         self.assertTrue(program in lines[0],
                         "make sure program path is in first argument")
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_stopOnEntry(self):
         '''
             Tests the default launch of a simple program that stops at the
             entry point instead of continuing.
         '''
         program = self.getBuildArtifact("a.out")
         self.build_and_launch(program, stopOnEntry=True)
         self.set_function_breakpoints(['main'])
         stopped_events = self.continue_to_next_stop()
         for stopped_event in stopped_events:
             if 'body' in stopped_event:
                 body = stopped_event['body']
                 if 'reason' in body:
                     reason = body['reason']
                     self.assertTrue(
                         reason != 'breakpoint',
                         'verify stop isn\'t "main" breakpoint')
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_cwd(self):
         '''
             Tests the default launch of a simple program with a current working
             directory.
         '''
         program = self.getBuildArtifact("a.out")
         program_parent_dir = os.path.split(os.path.split(program)[0])[0]
def test_cwd(self):
                 found = True
                 self.assertTrue(quote_path in line,
                                 "working directory '%s' not in '%s'" % (
                                     program_parent_dir, line))
         self.assertTrue(found, "verified program working directory")
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_debuggerRoot(self):
         '''
             Tests the "debuggerRoot" will change the working directory of
             the lldb-vscode debug adaptor.
         '''
         program = self.getBuildArtifact("a.out")
         program_parent_dir = os.path.split(os.path.split(program)[0])[0]
def test_debuggerRoot(self):
                 self.assertTrue(program_parent_dir == line[len(prefix):],
                                 "lldb-vscode working dir '%s' == '%s'" % (
                                     program_parent_dir, line[6:]))
         self.assertTrue(found, "verified lldb-vscode working directory")
         self.continue_to_exit()
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_sourcePath(self):
         '''
             Tests the "sourcePath" will set the target.source-map.
         '''
         program = self.getBuildArtifact("a.out")
         program_dir = os.path.split(program)[0]
         self.build_and_launch(program,
def test_sourcePath(self):
                 self.assertTrue(quoted_path == line[len(prefix):],
                                 "lldb-vscode working dir %s == %s" % (
                                     quoted_path, line[6:]))
         self.assertTrue(found, 'found "sourcePath" in console output')
         self.continue_to_exit()
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_disableSTDIO(self):
         '''
             Tests the default launch of a simple program with STDIO disabled.
         '''
         program = self.getBuildArtifact("a.out")
         self.build_and_launch(program,
                               disableSTDIO=True)
         self.continue_to_exit()
         # Now get the STDOUT and verify our program argument is correct
         output = self.get_stdout()
         self.assertTrue(output is None or len(output) == 0,
                         "expect no program output")
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_shellExpandArguments_enabled(self):
         '''
             Tests the default launch of a simple program with shell expansion
             enabled.
         '''
         program = self.getBuildArtifact("a.out")
         program_dir = os.path.split(program)[0]
def test_shellExpandArguments_enabled(self):
             quote_path = '"%s"' % (program)
             if line.startswith("arg[1] ="):
                 self.assertTrue(quote_path in line,
                                 'verify "%s" expanded to "%s"' % (
                                     glob, program))
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_shellExpandArguments_disabled(self):
         '''
             Tests the default launch of a simple program with shell expansion
             disabled.
         '''
         program = self.getBuildArtifact("a.out")
         program_dir = os.path.split(program)[0]
def test_shellExpandArguments_disabled(self):
             quote_path = '"%s"' % (glob)
             if line.startswith("arg[1] ="):
                 self.assertTrue(quote_path in line,
                                 'verify "%s" stayed to "%s"' % (
                                     glob, glob))
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_args(self):
         '''
             Tests launch of a simple program with arguments
         '''
         program = self.getBuildArtifact("a.out")
         args = ["one", "with space", "'with single quotes'",
                 '"with double quotes"']
def test_args(self):
         # Make sure arguments we specified are correct
         for (i, arg) in enumerate(args):
             quoted_arg = '"%s"' % (arg)
             self.assertTrue(quoted_arg in lines[i],
                             'arg[%i] "%s" not in "%s"' % (i+1, quoted_arg, lines[i]))
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_environment(self):
         '''
             Tests launch of a simple program with environment variables
         '''
         program = self.getBuildArtifact("a.out")
         env = ["NO_VALUE", "WITH_VALUE=BAR", "EMPTY_VALUE=",
                "SPACE=Hello World"]
def test_environment(self):
                     found = True
                     break
             self.assertTrue(found,
                             '"%s" must exist in program environment (%s)' % (
                                 var, lines))
 
     @skipIfWindows
     @skipIfDarwin # Skip this test for now until we can figure out why tings aren't working on build bots
+    @skipIfLinux # This test is timing out and/or failing on Linux as well as Darwin
     @no_debug_info_test
     def test_commands(self):
         '''
             Tests the "initCommands", "preRunCommands", "stopCommands" and
             "exitCommands" that can be passed during launch.
 
             "initCommands" are a list of LLDB commands that get executed
             before the targt is created.

lldb/trunk/packages/Python/lldbsuite/test/tools/lldb-vscode/step/Makefile

 LEVEL = ../../../make
 
+ENABLE_THREADS := YES
+
 CXX_SOURCES := main.cpp
 
 include $(LEVEL)/Makefile.rules