[lldb] [test] Workaround symlink-related test failures

[mgorny]

Summary

Use realpath() when spawning the executable create_after_attach
to workaround a FreeBSD plugin (and possibly others) problem.
If the executable is started via a path containing a symlink, it is
added to the module list twice -- via the real and apparent path.
This in turn cases the requested breakpoint to resolve twice.

Use realpath() for main program path in lldb-vscode breakpoint tests
to workaround a similar problem. If the passed path does not match
the realpath, lldb-vscode does not report the breakpoints as verified
and causes tests to fail.

Since the underlying problems are non-trivial to fix and the purpose
of these tests is not to reproduce symlink problems, let's apply
trivial workarounds to make them pass.

[mgorny]

Added a similar hack for tools/lldb-vscode/breakpoint/TestVSCode_setBreakpoints.py test.

[emaste]

I think it's valuable to avoid testing this (symlink issue) as a side effect of these test cases, but we should have an explicit test for this; it is an issue users could hit.

Do you know if this is a bug specific to lldb's FreeBSD code (or do we just have a default symlink in the paths used here where other OS's don/t)?

[mgorny]

In D97230#2585139, @emaste wrote:

I think it's valuable to avoid testing this (symlink issue) as a side effect of these test cases, but we should have an explicit test for this; it is an issue users could hit.

Do you know if this is a bug specific to lldb's FreeBSD code (or do we just have a default symlink in the paths used here where other OS's don/t)?

I think the issue may be more widespread but I need to test. The module list problem probably depends on dyld behavior but I suppose lldb-vscode may be generic.

[emaste]

OK; I think this change is reasonable, I just want to make sure we don't completely lose the underlying issue.

Let's make sure that we at least have a bug report to track it (i.e., let's not close llvm.org/pr48376 and llvm.org/pr48421 with "no longer fails" without having a new bug for this problem)

[labath]

Sounds reasonable. Having a canonical bug for the symlink issue would be reasonable too...

[mgorny]

I've edited both bugs (https://bugs.llvm.org/show_bug.cgi?id=48421 and https://bugs.llvm.org/show_bug.cgi?id=48376) to cover the generic problem. I can't promise I'll have time to work on them in the next ~2 months but if you have any suggestion how to solve them, I can try looking at them as time permits.

lldb/test/API/functionalities/thread/create_after_attach/TestCreateAfterAttach.py

def setUp(self):
         # Find the line numbers for our breakpoints.
         self.break_1 = line_number('main.cpp', '// Set first breakpoint here')
         self.break_2 = line_number('main.cpp', '// Set second breakpoint here')
         self.break_3 = line_number('main.cpp', '// Set third breakpoint here')
 
     # Occasionally hangs on Windows, may be same as other issues.
     @skipIfWindows
     @skipIfiOSSimulator
-    @expectedFailureAll(oslist=["freebsd"], bugnumber="llvm.org/pr48376")
     @expectedFailureNetBSD
     def test_create_after_attach(self):
         """Test thread creation after process attach."""
         self.build(dictionary=self.getBuildFlags(use_cpp11=False))
         exe = self.getBuildArtifact("a.out")
 
         # Spawn a new process
-        popen = self.spawnSubprocess(exe)
+        # use realpath to workaround llvm.org/pr48376
+        popen = self.spawnSubprocess(os.path.realpath(exe))
         pid = popen.pid
 
         # Attach to the spawned process
         self.runCmd("process attach -p " + str(pid))
 
         target = self.dbg.GetSelectedTarget()
 
         process = target.GetProcess()

lldb/test/API/tools/lldb-vscode/breakpoint/TestVSCode_setBreakpoints.py

 class TestVSCode_setBreakpoints(lldbvscode_testcase.VSCodeTestCaseBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         lldbvscode_testcase.VSCodeTestCaseBase.setUp(self)
 
         self.main_basename = 'main-copy.cpp'
-        self.main_path = self.getBuildArtifact(self.main_basename)
+        self.main_path = os.path.realpath(self.getBuildArtifact(self.main_basename))
 
     @skipIfWindows
     @skipIfRemote
-    @expectedFailureAll(oslist=["freebsd"], bugnumber="llvm.org/pr48421")
     def test_source_map(self):
         self.build_and_create_debug_adaptor()
 
         other_basename = 'other-copy.c'
         other_path = self.getBuildArtifact(other_basename)
 
         source_folder = os.path.dirname(self.main_path)
 
def test_source_map(self):
         breakpoint = breakpoints[0]
         self.assertEqual(breakpoint['line'], other_line)
         self.assertTrue(breakpoint['verified'])
         self.assertEqual(other_basename, breakpoint['source']['name'])
         self.assertEqual(new_other_path, breakpoint['source']['path'])
 
     @skipIfWindows
     @skipIfRemote
-    @expectedFailureAll(oslist=["freebsd"], bugnumber="llvm.org/pr48421")
     def test_set_and_clear(self):
         '''Tests setting and clearing source file and line breakpoints.
            This packet is a bit tricky on the debug adaptor side since there
            is no "clearBreakpoints" packet. Source file and line breakpoints
            are set by sending a "setBreakpoints" packet with a source file
            specified and zero or more source lines. If breakpoints have been
            set in the source file before, any existing breakpoints must remain
            set, and any new breakpoints must be created, and any breakpoints
def test_set_and_clear(self):
             for breakpoint in breakpoints:
                 line = breakpoint['line']
                 self.assertIn(line, lines, "line expected in lines array")
                 self.assertTrue(breakpoint['verified'],
                                 "expect breakpoint still verified")
 
     @skipIfWindows
     @skipIfRemote
-    @expectedFailureAll(oslist=["freebsd"], bugnumber="llvm.org/pr48421")
     def test_clear_breakpoints_unset_breakpoints(self):
         '''Test clearing breakpoints like test_set_and_clear, but clear
            breakpoints by omitting the breakpoints array instead of sending an
            empty one.'''
         lines = [line_number('main.cpp', 'break 12'),
                  line_number('main.cpp', 'break 13')]
 
         # Visual Studio Code Debug Adaptors have no way to specify the file
def test_clear_breakpoints_unset_breakpoints(self):
 
         # Verify with the target that all breakpoints have been cleared.
         response = self.vscode.request_testGetTargetBreakpoints()
         breakpoints = response['body']['breakpoints']
         self.assertEquals(len(breakpoints), 0, "expect no source breakpoints")
 
     @skipIfWindows
     @skipIfRemote
-    @expectedFailureAll(oslist=["freebsd"], bugnumber="llvm.org/pr48421")
     def test_functionality(self):
         '''Tests hitting breakpoints and the functionality of a single
            breakpoint, like 'conditions' and 'hitCondition' settings.'''
         loop_line = line_number('main.cpp', '// break loop')
 
         program = self.getBuildArtifact("a.out")
         self.build_and_launch(program)
         # Set a breakpoint at the loop line with no condition and no