[lldb] Enable FreeBSDRemote plugin by default and update test status

[mgorny]

Summary

The new FreeBSDRemote plugin has reached feature parity on i386
and amd64 targets. Use it by default on these architectures, while
allowing the use of the legacy plugin via FREEBSD_LEGACY_PLUGIN envvar.

Revisit the method of switching plugins. Apparently, the return value
of PlatformFreeBSD::CanDebugProcess() is what really decides whether
the legacy or the new plugin is used.

Update the test status. Reenable the tests that were previously
disabled on FreeBSD and do not cause hangs or are irrelevant to FreeBSD.
Mark all tests that fail reliably as expectedFailure. For now, tests
that are flaky (i.e. produce unstable results) are left enabled
and cause unpredictable test failures.

[mgorny]

lldb/source/Plugins/Platform/FreeBSD/PlatformFreeBSD.cpp

250–262

@labath, any clue if I am missing something or is this really sufficient to switch between plugins?

lldb/test/API/functionalities/avoids-fd-leak/TestFdLeak.py

13

I've removed this entirely since LLDB does not support Python 2 anymore.

[mgorny]

Missed one test.

[labath]

lldb/source/Plugins/Platform/FreeBSD/PlatformFreeBSD.cpp

250–262

Yeah, this should be fine. I was thinking it should be possible to switch this on from a single place all along....

263–269

I think this ought to go into the if(host) block, as it's really not applicable to remote debugging.

lldb/test/API/functionalities/gdb_remote_client/TestProcessConnect.py

39

These may be related to the if(host) thingy.

lldb/test/API/functionalities/inferior-crashing/TestInferiorCrashingStep.py

53–54

I'm pretty sure the root cause here is the same for net/free bsd as it is for linux (it comes down to macos catching the "crashes" specially, before they even get turned to a SEGV -- something that's not possible elsewhere). I marked it skip because that's not something we should support, ever. I don't care that much which decorator (skip vs. xfail) is used here, but I think they should be consistent.

lldb/test/API/functionalities/inferior-crashing/recursive-inferior/TestRecursiveInferiorStep.py

30

ditto

lldb/test/API/tools/lldb-server/thread-name/TestGdbRemoteThreadName.py

32

What's up with this? I though you fixed thread names already..

[mgorny]

lldb/test/API/functionalities/gdb_remote_client/TestProcessConnect.py

39

Doesn't seem to have changed after changing it. I will look into it while reiterating the whole test suite.

lldb/test/API/tools/lldb-server/thread-name/TestGdbRemoteThreadName.py

32

The test code is buggy. I want to fix it separately not to mix things up too much.

[mgorny]

Applied changes as requested.

[emaste]

I added comments in the now-dereferenced bugs linking back to this review - most of them were submitted by me, and I'll double check and close them once this lands.

One minor update needed (restoring a comment that was partially deleted)

lldb/packages/Python/lldbsuite/test/dotest.py

955

It's a shame that these variables and the comments are inverted sense / double negatives, but not an issue in this patch.

lldb/test/API/commands/register/register/register_command/TestRegisters.py

31

there's no (existing) PR for the failure?

lldb/test/API/commands/watchpoints/multiple_threads/TestWatchpointMultipleThreads.py

25

was this one already failing, or is this new?

lldb/test/API/functionalities/exec/TestExec.py

23

So hangs on FreeBSD but reliably fails on NetBSD?

lldb/test/API/functionalities/thread/create_after_attach/TestCreateAfterAttach.py

24–25

part of comment here disappeared

[mgorny]

In D90757#2374087, @emaste wrote:

I added comments in the now-dereferenced bugs linking back to this review - most of them were submitted by me, and I'll double check and close them once this lands.

I was planning to go through them once this is accepted. But if you could double check my results, that would be great.

[emaste]

When running locally with this change applied I get:

********************
Failed Tests (8):
  lldb-api :: commands/expression/multiline-completion/TestMultilineCompletion.py
  lldb-api :: functionalities/breakpoint/breakpoint_set_restart/TestBreakpointSetRestart.py
  lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointThreads.py
  lldb-api :: iohandler/completion/TestIOHandlerCompletion.py
  lldb-api :: python_api/watchpoint/watchlocation/TestSetWatchlocation.py
  lldb-api :: python_api/watchpoint/watchlocation/TestTargetWatchAddress.py
  lldb-api :: tools/lldb-server/thread-name/TestGdbRemoteThreadName.py
  lldb-unit :: tools/lldb-server/tests/./LLDBServerTests/StandardStartupTest.TestStopReplyContainsThreadPcs

********************
Unexpectedly Passed Tests (2):
  lldb-api :: api/multiple-debuggers/TestMultipleDebuggers.py
  lldb-api :: lang/c/conflicting-symbol/TestConflictingSymbol.py


Testing Time: 367.34s
  Unsupported        :  422
  Passed             : 1741
  Expectedly Failed  :    5
  Failed             :    8
  Unexpectedly Passed:    2

[mgorny]

lldb/packages/Python/lldbsuite/test/dotest.py

955

Yes, I had to look at it for a while to make sure I'm doing it right. I'll change them in a followup commit.

lldb/test/API/commands/register/register/register_command/TestRegisters.py

31

No. I'm planning to file bugs during the next stage, after trying to fix the trivial ones, if that's ok with you.

lldb/test/API/commands/watchpoints/multiple_threads/TestWatchpointMultipleThreads.py

25

Yes, it is new. I must've attributed it to unstable tests when I reported no regressions yesterday. However, today I've confirmed it to fail reliably. Hopefully, it won't be hard to fix once I get to it.

lldb/test/API/functionalities/exec/TestExec.py

23

Yes, or at least it reliably failed on NetBSD the last time I checked.

lldb/test/API/functionalities/thread/create_after_attach/TestCreateAfterAttach.py

24–25

Thanks for noticing. Restored it and move nearer to the new skip.

[mgorny]

Fixed test comment.

[mgorny]

I had accidentally included a commented out xfail for testing.

[mgorny]

In D90757#2374533, @emaste wrote:

When running locally with this change applied I get:

********************
Failed Tests (8):
  lldb-api :: commands/expression/multiline-completion/TestMultilineCompletion.py
  lldb-api :: functionalities/breakpoint/breakpoint_set_restart/TestBreakpointSetRestart.py

These two pass reliably for me. You can check if it could be related to parallel testing. To start them individually:

ninja check-lldb-api-commands-expression-multiline-completion
ninja check-lldb-api-functionalities-breakpoint-breakpoint_set_restart

lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointThreads.py

This one's unstable.

lldb-api :: iohandler/completion/TestIOHandlerCompletion.py
lldb-api :: python_api/watchpoint/watchlocation/TestSetWatchlocation.py
lldb-api :: python_api/watchpoint/watchlocation/TestTargetWatchAddress.py

All three pass for me reliably right now but I recall that watchlocation test could be fragile to parallel runs.

lldb-api :: tools/lldb-server/thread-name/TestGdbRemoteThreadName.py

This one I've commented out for testing per earlier comments here and forget to revert.

lldb-unit :: tools/lldb-server/tests/./LLDBServerTests/StandardStartupTest.TestStopReplyContainsThreadPcs

This one I know about. There's no easy way to xfail an unittest, and the hack to skip it is kinda ugly, so I kept it for now not to forget about it.

********************
Unexpectedly Passed Tests (2):
  lldb-api :: api/multiple-debuggers/TestMultipleDebuggers.py

I think this one's unstable in long runs. I recall I marked it xfail first but then removed the xfail because it succeeded. But then, it failed in a few consecutive, so I've xfailed it ;-).

lldb-api :: lang/c/conflicting-symbol/TestConflictingSymbol.py

This one also reliably XFAILs for me.

Testing Time: 367.34s
  Unsupported        :  422
  Passed             : 1741
  Expectedly Failed  :    5
  Failed             :    8
  Unexpectedly Passed:    2

What do you suggest? Is it fine to go with my results for as long as I work on it? I can update it to match CI/buildbot results later.

[labath]

Looks ok to me. I'll leave it to you to sort out the specific annotations on individual tests.

lldb/test/API/tools/lldb-server/thread-name/TestGdbRemoteThreadName.py

32

Fixing it here is not a good idea, yes. But I think it would have been good to fix in the patch that was implementing thread name support.

[mgorny]

lldb/test/API/tools/lldb-server/thread-name/TestGdbRemoteThreadName.py

32

I'm planning to run the test suite a lot more once this lands, and we have more predictable results.

[emaste]

What do you suggest? Is it fine to go with my results for as long as I work on it? I can update it to match CI/buildbot results later.

Yes absolutely. I pasted my results mainly as an FYI/for comparison, since I recall there were many intermittent or flaky tests. I definitely want to see this go in and we can iterate on the individual annotations.

[emaste]

lldb/packages/Python/lldbsuite/test/dotest.py

955

Thanks.

lldb/test/API/commands/register/register/register_command/TestRegisters.py

31

Yes absolutely. I was just curious because this was already failing for NetBSD

lldb/test/API/functionalities/inferior-crashing/TestInferiorCrashingStep.py

53–54

Sounds reasonable, we may want to change the comment above to make it clear this is an explicit decision.

[theraven]

Does the new plugin work with processes that are created with pdfork? The last time I tried this, it caused the old plugin to lock up the debugger entirely. Please can you ensure that there are tests that cover pdfork and cap_enter in the child? These are currently quite badly broken.

[krytarowski]

In D90757#2378404, @theraven wrote:

Does the new plugin work with processes that are created with pdfork? The last time I tried this, it caused the old plugin to lock up the debugger entirely. Please can you ensure that there are tests that cover pdfork and cap_enter in the child? These are currently quite badly broken.

Can you show an example of a program (ideally minimal) that is supposed to work but hangs? Does GDB support pdfork-ed children?

There is also TRAP_CAP and I'm not sure what should we do upon it.

lldb/packages/Python/lldbsuite/test/dotest.py

def run_suite():
 
     # Don't do debugserver tests on anything except OS X.
     configuration.dont_do_debugserver_test = (
             "linux" in target_platform or
             "freebsd" in target_platform or
             "netbsd" in target_platform or
             "windows" in target_platform)
 
-    # Don't do lldb-server (llgs) tests on anything except Linux and Windows.
+    # Don't do lldb-server (llgs) tests on platforms not supporting it.
     configuration.dont_do_llgs_test = not (
+            "freebsd" in target_platform or
             "linux" in target_platform or
             "netbsd" in target_platform or
             "windows" in target_platform)
 
     for testdir in configuration.testdirs:
         for (dirpath, dirnames, filenames) in os.walk(testdir):
             visit('Test', dirpath, filenames)
 

lldb/source/Plugins/Platform/FreeBSD/PlatformFreeBSD.cpp

 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/FileSpec.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/State.h"
 #include "lldb/Utility/Status.h"
 #include "lldb/Utility/StreamString.h"
 
+#include "llvm/ADT/Triple.h"
+#include "llvm/Support/Host.h"
+
 // Define these constants from FreeBSD mman.h for use when targeting remote
 // FreeBSD systems even when host has different values.
 #define MAP_PRIVATE 0x0002
 #define MAP_ANON 0x1000
 
 using namespace lldb;
 using namespace lldb_private;
 using namespace lldb_private::platform_freebsd;
if (bp_site->SetTrapOpcode(g_arm_breakpoint_opcode, trap_opcode_size))
       return trap_opcode_size;
   }
     LLVM_FALLTHROUGH;
   default:
     return Platform::GetSoftwareBreakpointTrapOpcode(target, bp_site);
   }
 }
 
 bool PlatformFreeBSD::CanDebugProcess() {
-  if (getenv("FREEBSD_REMOTE_PLUGIN")) {
-    if (IsHost()) {
-      return true;
-    } else {
-      // If we're connected, we can debug.
-      return IsConnected();
-    }
-  }
-  return false;
-}
+    llvm::Triple host_triple{llvm::sys::getProcessTriple()};
+    bool use_legacy_plugin;
+
+    switch (host_triple.getArch()) {
+      case llvm::Triple::x86:
+      case llvm::Triple::x86_64:
+        // FreeBSDRemote plugin supports x86 only at the moment
+        use_legacy_plugin = !!getenv("FREEBSD_LEGACY_PLUGIN");
+        break;
+      default:
+        use_legacy_plugin = true;
+    }
+
+    return !use_legacy_plugin;
 
 void PlatformFreeBSD::CalculateTrapHandlerSymbolNames() {
   m_trap_handlers.push_back(ConstString("_sigtramp"));
 }
 
 MmapArgList PlatformFreeBSD::GetMmapArgumentList(const ArchSpec &arch,
                                                  addr_t addr, addr_t length,
                                                  unsigned prot, unsigned flags,

lldb/source/Plugins/Process/FreeBSD/ProcessFreeBSD.cpp

ProcessFreeBSD::CreateInstance(lldb::TargetSP target_sp,
   lldb::ProcessSP process_sp;
   if (crash_file_path == NULL)
     process_sp.reset(
         new ProcessFreeBSD(target_sp, listener_sp, GetFreeBSDSignals()));
   return process_sp;
 }
 
 void ProcessFreeBSD::Initialize() {
-  if (!getenv("FREEBSD_REMOTE_PLUGIN")) {
-    static llvm::once_flag g_once_flag;
 
-    llvm::call_once(g_once_flag, []() {
-      PluginManager::RegisterPlugin(GetPluginNameStatic(),
-                                    GetPluginDescriptionStatic(), CreateInstance);
-    });
-  }
-}
 
 lldb_private::ConstString ProcessFreeBSD::GetPluginNameStatic() {
   static ConstString g_name("freebsd");
   return g_name;
 }
 
 const char *ProcessFreeBSD::GetPluginDescriptionStatic() {
   return "Process plugin for FreeBSD";

lldb/test/API/api/multiple-debuggers/TestMultipleDebuggers.py

 class TestMultipleSimultaneousDebuggers(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     NO_DEBUG_INFO_TESTCASE = True
 
     @skipIfNoSBHeaders
     @skipIfWindows
-    @expectedFailureAll(oslist=['freebsd'])
+    @expectedFailureAll(oslist=["freebsd"])
     def test_multiple_debuggers(self):
         env = {self.dylibPath: self.getLLDBLibraryEnvVal()}
 
         self.driver_exe = self.getBuildArtifact("multi-process-driver")
         self.buildDriver('multi-process-driver.cpp', self.driver_exe)
         self.addTearDownHook(lambda: os.remove(self.driver_exe))
         self.signBinary(self.driver_exe)
 

lldb/test/API/commands/expression/call-restarts/TestCallThatRestarts.py

class ExprCommandThatRestartsTestCase(TestBase):
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
 
         self.main_source = "lotta-signals.c"
         self.main_source_spec = lldb.SBFileSpec(self.main_source)
 
-    @skipIfFreeBSD  # llvm.org/pr19246: intermittent failure
     @skipIfDarwin  # llvm.org/pr19246: intermittent failure
     @skipIfWindows  # Test relies on signals, unsupported on Windows
     @expectedFlakeyAndroid(bugnumber="llvm.org/pr19246")
     @expectedFailureNetBSD
     def test(self):
         """Test calling function that hits a signal and restarts."""
         self.build()
         self.call_function()

lldb/test/API/commands/expression/formatters/TestFormatters.py

class ExprFormattersTestCase(TestBase):
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break for main.cpp.
         self.line = line_number('main.cpp',
                                 '// Stop here')
 
-    @skipIfFreeBSD  # llvm.org/pr24691 skipping to avoid crashing the test runner
-    @expectedFailureAll(
-        oslist=['freebsd'],
-        bugnumber='llvm.org/pr19011 Newer Clang omits C1 complete object constructor')
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr21765")
     @skipIfTargetAndroid()  # skipping to avoid crashing the test runner
     @expectedFailureAndroid('llvm.org/pr24691')  # we hit an assertion in clang
     def test(self):
         """Test expr + formatters for good interoperability."""
         self.build()
 
         # This is the function to remove the custom formats in order to have a

lldb/test/API/commands/expression/no-deadlock/TestExprDoesntBlock.py

 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class ExprDoesntDeadlockTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
-    @expectedFailureAll(oslist=['freebsd'], bugnumber='llvm.org/pr17946')
     @add_test_categories(["basic_process"])
     @skipIfReproducer # Timeouts are not currently modeled.
     def test_with_run_command(self):
         """Test that expr will time out and allow other threads to run if it blocks."""
         self.build()
         exe = self.getBuildArtifact("a.out")
 
         # Create a target by the debugger.

lldb/test/API/commands/register/register/register_command/TestRegisters.py

def setUp(self):
         self.has_teardown = False
 
     def tearDown(self):
         self.dbg.GetSelectedTarget().GetProcess().Destroy()
         TestBase.tearDown(self)
 
     @skipIfiOSSimulator
     @skipIf(archs=no_match(['amd64', 'arm', 'i386', 'x86_64']))
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     def test_register_commands(self):
         """Test commands related to registers, in particular vector registers."""
         self.build()
         self.common_setup()
 
         # verify that logging does not assert
         self.log_enable("registers")
 
class RegisterCommandsTestCase(TestBase):
     def test_fp_register_write(self):
         """Test commands that write to registers, in particular floating-point registers."""
         self.build()
         self.fp_register_write()
 
     @skipIfiOSSimulator
     # "register read fstat" always return 0xffff
     @expectedFailureAndroid(archs=["i386"])
-    @skipIfFreeBSD  # llvm.org/pr25057
     @skipIf(archs=no_match(['amd64', 'i386', 'x86_64']))
     @skipIfOutOfTreeDebugserver
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr37995")
     @expectedFailureNetBSD
     def test_fp_special_purpose_register_read(self):
         """Test commands that read fpu special purpose registers."""
         self.build()
         self.fp_special_purpose_register_read()

lldb/test/API/commands/watchpoints/multiple_threads/TestWatchpointMultipleThreads.py

class WatchpointForMultipleThreadsTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
     NO_DEBUG_INFO_TESTCASE = True
     main_spec = lldb.SBFileSpec("main.cpp", False)
 
     def test_watchpoint_before_thread_start(self):
         """Test that we can hit a watchpoint we set before starting another thread"""
         self.do_watchpoint_test("Before running the thread")
 
+    @expectedFailureAll(oslist=["freebsd"])
     def test_watchpoint_after_thread_start(self):
         """Test that we can hit a watchpoint we set after starting another thread"""
         self.do_watchpoint_test("After running the thread")
 
     def do_watchpoint_test(self, line):
         self.build()
         lldbutil.run_to_source_breakpoint(self, line, self.main_spec)
 

lldb/test/API/functionalities/avoids-fd-leak/TestFdLeak.py

 """
 Test whether a process started by lldb has no extra file descriptors open.
 """
 
 
 
 import lldb
 from lldbsuite.test import lldbutil
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test.decorators import *
 
 
-def python_leaky_fd_version(test):
-    import sys
-    # Python random module leaks file descriptors on some versions.
-    if sys.version_info >= (2, 7, 8) and sys.version_info < (2, 7, 10):
-        return "Python random module leaks file descriptors in this python version"
-    return None
-
-
 class AvoidsFdLeakTestCase(TestBase):
 
     NO_DEBUG_INFO_TESTCASE = True
 
     mydir = TestBase.compute_mydir(__file__)
 
-    @expectedFailureIfFn(python_leaky_fd_version, "bugs.freebsd.org/197376")
     # The check for descriptor leakage needs to be implemented differently
     # here.
     @skipIfWindows
     @skipIfTargetAndroid()  # Android have some other file descriptors open by the shell
     @skipIfDarwinEmbedded # <rdar://problem/33888742>  # debugserver on ios has an extra fd open on launch
     def test_fd_leak_basic(self):
         self.do_test([])
 
-    @expectedFailureIfFn(python_leaky_fd_version, "bugs.freebsd.org/197376")
-    @expectedFailureAll(
-        oslist=['freebsd'],
-        bugnumber="llvm.org/pr25624 still failing with Python 2.7.10")
     # The check for descriptor leakage needs to be implemented differently
     # here.
     @skipIfWindows
     @skipIfTargetAndroid()  # Android have some other file descriptors open by the shell
     @skipIfDarwinEmbedded # <rdar://problem/33888742>  # debugserver on ios has an extra fd open on launch
     def test_fd_leak_log(self):
         self.do_test(["log enable -f '/dev/null' lldb commands"])
 
def do_test(self, commands):
 
         self.assertTrue(
             process.GetState() == lldb.eStateExited,
             "Process should have exited.")
         self.assertTrue(
             process.GetExitStatus() == 0,
             "Process returned non-zero status. Were incorrect file descriptors passed?")
 
-    @expectedFailureIfFn(python_leaky_fd_version, "bugs.freebsd.org/197376")
-    @expectedFailureAll(
-        oslist=['freebsd'],
-        bugnumber="llvm.org/pr25624 still failing with Python 2.7.10")
     # The check for descriptor leakage needs to be implemented differently
     # here.
     @skipIfWindows
     @skipIfTargetAndroid()  # Android have some other file descriptors open by the shell
     @skipIfDarwinEmbedded # <rdar://problem/33888742>  # debugserver on ios has an extra fd open on launch
     def test_fd_leak_multitarget(self):
         self.build()
         exe = self.getBuildArtifact("a.out")

lldb/test/API/functionalities/data-formatter/data-formatter-python-synth/TestDataFormatterPythonSynth.py

 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class PythonSynthDataFormatterTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
-    @skipIfFreeBSD  # llvm.org/pr20545 bogus output confuses buildbot parser
     def test_with_run_command(self):
         """Test data formatter commands."""
         self.build()
         self.data_formatter_commands()
 
     def test_rdar10960550_with_run_command(self):
         """Test data formatter commands."""
         self.build()

lldb/test/API/functionalities/data-formatter/data-formatter-synthtype/TestDataFormatterSynthType.py

class DataFormatterSynthTypeTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', 'break here')
 
-    @skipIfFreeBSD  # llvm.org/pr20545 bogus output confuses buildbot parser
     def test_with_run_command(self):
         """Test using Python synthetic children provider to provide a typename."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=1, loc_exact=True)
 

lldb/test/API/functionalities/data-formatter/data-formatter-synthval/TestDataFormatterSynthVal.py

class DataFormatterSynthValueTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', 'break here')
 
-    @skipIfFreeBSD  # llvm.org/pr20545 bogus output confuses buildbot parser
     def test_with_run_command(self):
         """Test using Python synthetic children provider to provide a value."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=1, loc_exact=True)
 

lldb/test/API/functionalities/data-formatter/varscript_formatting/TestDataFormatterVarScriptFormatting.py

class DataFormatterVarScriptFormatting(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break at.
         self.line = line_number('main.cpp', ' // Set breakpoint here.')
 
-    @skipIfFreeBSD  # llvm.org/pr20545 bogus output confuses buildbot parser
     def test_with_run_command(self):
         """Test using Python synthetic children provider."""
         self.build()
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
 
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=1, loc_exact=True)
 

lldb/test/API/functionalities/deleted-executable/TestDeletedExecutable.py

 from lldbsuite.test import lldbutil
 
 class TestDeletedExecutable(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
     NO_DEBUG_INFO_TESTCASE = True
 
     @skipIfWindows # cannot delete a running executable
-    @expectedFailureAll(oslist=["linux"],
+    @expectedFailureAll(oslist=["freebsd", "linux", "netbsd"],
         triple=no_match('aarch64-.*-android'))
         # determining the architecture of the process fails
-    @expectedFailureNetBSD
     @skipIfReproducer # File synchronization is not supported during replay.
     def test(self):
         self.build()
         exe = self.getBuildArtifact("a.out")
 
         # Use a file as a synchronization point between test and inferior.
         pid_file_path = lldbutil.append_to_process_working_directory(self,
             "token_pid_%d" % (int(os.getpid())))

lldb/test/API/functionalities/exec/TestExec.py

class ExecTestCase(TestBase):
     NO_DEBUG_INFO_TESTCASE = True
 
     mydir = TestBase.compute_mydir(__file__)
 
     @expectedFailureAll(archs=['i386'], bugnumber="rdar://28656532")
     @expectedFailureAll(oslist=["ios", "tvos", "watchos", "bridgeos"], bugnumber="rdar://problem/34559552") # this exec test has problems on ios systems
     @expectedFailureNetBSD
     @skipIfAsan # rdar://problem/43756823
+    @skipIfFreeBSD  # hangs
     @skipIfWindows
     def test_hitting_exec (self):
         self.do_test(False)
 
     @expectedFailureAll(archs=['i386'], bugnumber="rdar://28656532")
     @expectedFailureAll(oslist=["ios", "tvos", "watchos", "bridgeos"], bugnumber="rdar://problem/34559552") # this exec test has problems on ios systems
     @expectedFailureNetBSD
     @skipIfAsan # rdar://problem/43756823
+    @skipIfFreeBSD  # hangs
     @skipIfWindows
     def test_skipping_exec (self):
         self.do_test(True)
 
     def do_test(self, skip_exec):
         self.build()
         exe = self.getBuildArtifact("a.out")
         secondprog = self.getBuildArtifact("secondprog")

lldb/test/API/functionalities/gdb_remote_client/TestProcessConnect.py

def test_gdb_remote_async(self):
                         matching=False,
                         substrs=['Process', 'stopped'])
             lldbutil.expect_state_changes(self, self.dbg.GetListener(),
                                           self.process(), [lldb.eStateStopped])
         finally:
             self.dbg.GetSelectedPlatform().DisconnectRemote()
 
     @skipIfWindows
+    @expectedFailureAll(oslist=["freebsd"])
     def test_process_connect_sync(self):
         """Test the gdb-remote command in synchronous mode"""
         try:
             self.dbg.SetAsync(False)
             self.expect("process connect connect://" +
                         self.server.get_connect_address(),
                         substrs=['Process', 'stopped'])
         finally:
             self.dbg.GetSelectedPlatform().DisconnectRemote()
 
     @skipIfWindows
+    @expectedFailureAll(oslist=["freebsd"])
     @skipIfReproducer # Reproducer don't support async.
     def test_process_connect_async(self):
         """Test the gdb-remote command in asynchronous mode"""
         try:
             self.dbg.SetAsync(True)
             self.expect("process connect connect://" +
                         self.server.get_connect_address(),
                         matching=False,
                         substrs=['Process', 'stopped'])
             lldbutil.expect_state_changes(self, self.dbg.GetListener(),
                                           self.process(), [lldb.eStateStopped])
         finally:
             self.dbg.GetSelectedPlatform().DisconnectRemote()

lldb/test/API/functionalities/inferior-crashing/TestInferiorCrashingStep.py

class CrashingInferiorStepTestCase(TestBase):
     @skipIfTargetAndroid()  # debuggerd interferes with this test on Android
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24778")
     def test_inferior_crashing_step_after_break(self):
         """Test that lldb functions correctly after stepping through a crash."""
         self.build()
         self.inferior_crashing_step_after_break()
 
     # Inferior exits after stepping after a segfault. This is working as
     # intended IMHO.
-    @skipIfLinux
-    @skipIfFreeBSD
-    @expectedFailureNetBSD
+    @skipIf(oslist=["freebsd", "linux", "netbsd"])
     def test_inferior_crashing_expr_step_and_expr(self):
         """Test that lldb expressions work before and after stepping after a crash."""
         self.build()
         self.inferior_crashing_expr_step_expr()
 
     def set_breakpoint(self, line):
         lldbutil.run_break_set_by_file_and_line(
             self, "main.c", line, num_expected_locations=1, loc_exact=True)

lldb/test/API/functionalities/inferior-crashing/recursive-inferior/TestRecursiveInferiorStep.py

class CrashingRecursiveInferiorStepTestCase(TestBase):
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24778")
     def test_recursive_inferior_crashing_step_after_break(self):
         """Test that lldb functions correctly after stepping through a crash."""
         self.build()
         self.recursive_inferior_crashing_step_after_break()
 
     # Inferior exits after stepping after a segfault. This is working as
     # intended IMHO.
-    @skipIfLinux
-    @skipIfFreeBSD
-    @expectedFailureNetBSD
+    @skipIf(oslist=["freebsd", "linux", "netbsd"])
     def test_recursive_inferior_crashing_expr_step_and_expr(self):
         """Test that lldb expressions work before and after stepping after a crash."""
         self.build()
         self.recursive_inferior_crashing_expr_step_expr()
 
     def set_breakpoint(self, line):
         lldbutil.run_break_set_by_file_and_line(
             self, "main.c", line, num_expected_locations=1, loc_exact=True)

lldb/test/API/functionalities/load_unload/TestLoadUnload.py

def setSvr4Support(self, enabled):
         self.runCmd(
             "settings set plugin.process.gdb-remote.use-libraries-svr4 {enabled}".format(
                 enabled="true" if enabled else "false"
             )
         )
 
     # libloadunload_d.so does not appear in the image list because executable
     # dependencies are resolved relative to the debuggers PWD. Bug?
-    @expectedFailureAll(oslist=["linux"])
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
+    @expectedFailureAll(oslist=["freebsd", "linux", "netbsd"])
     @not_remote_testsuite_ready
     @skipIfWindows  # Windows doesn't have dlopen and friends, dynamic libraries work differently
-    @expectedFailureNetBSD
     @skipIfReproducer # VFS is a snapshot.
     def test_modules_search_paths(self):
         """Test target modules list after loading a different copy of the library libd.dylib, and verifies that it works with 'target modules search-paths add'."""
         if self.platformIsDarwin():
             dylibName = 'libloadunload_d.dylib'
         else:
             dylibName = 'libloadunload_d.so'
 
def test_modules_search_paths(self):
 
         self.expect(
             "target modules list",
             "LLDB successfully locates the relocated dynamic library",
             substrs=[new_dylib])
 
     # libloadunload_d.so does not appear in the image list because executable
     # dependencies are resolved relative to the debuggers PWD. Bug?
-    @expectedFailureAll(oslist=["linux"])
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
+    @expectedFailureAll(oslist=["freebsd", "linux", "netbsd"])
     @expectedFailureAndroid  # wrong source file shows up for hidden library
     @skipIfWindows  # Windows doesn't have dlopen and friends, dynamic libraries work differently
     @skipIfDarwinEmbedded
-    @expectedFailureNetBSD
     def test_dyld_library_path(self):
         """Test (DY)LD_LIBRARY_PATH after moving libd.dylib, which defines d_function, somewhere else."""
         self.copy_shlibs_to_remote(hidden_dir=True)
 
         exe = self.getBuildArtifact("a.out")
         self.runCmd("file " + exe, CURRENT_EXECUTABLE_SET)
 
         # Shut off ANSI color usage so we don't get ANSI escape sequences
def test_dyld_library_path(self):
 
         # This time, the hidden library should be picked up.
         self.expect("run", substrs=["return", "12345"])
 
     @expectedFailureAll(
         bugnumber="llvm.org/pr25805",
         hostoslist=["windows"],
         triple='.*-android')
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     @expectedFailureAll(oslist=["windows"]) # process load not implemented
     def test_lldb_process_load_and_unload_commands(self):
         self.setSvr4Support(False)
         self.run_lldb_process_load_and_unload_commands()
 
     @expectedFailureAll(
         bugnumber="llvm.org/pr25805",
         hostoslist=["windows"],
         triple='.*-android')
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     @expectedFailureAll(oslist=["windows"]) # process load not implemented
     def test_lldb_process_load_and_unload_commands_with_svr4(self):
         self.setSvr4Support(True)
         self.run_lldb_process_load_and_unload_commands()
 
     @skipIfReproducer # FIXME: Unexpected packet during (passive) replay
     def run_lldb_process_load_and_unload_commands(self):
         """Test that lldb process load/unload command work correctly."""
def run_lldb_process_load_and_unload_commands(self):
             "%s unloaded correctly" %
             dylibName,
             patterns=[
                 "Unloading .* with index %s.*ok" %
                 index])
 
         self.runCmd("process continue")
 
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     @expectedFailureAll(oslist=["windows"]) # breakpoint not hit
     def test_load_unload(self):
         self.setSvr4Support(False)
         self.run_load_unload()
 
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     @expectedFailureAll(oslist=["windows"]) # breakpoint not hit
     def test_load_unload_with_svr4(self):
         self.setSvr4Support(True)
         self.run_load_unload()
 
     def run_load_unload(self):
         """Test breakpoint by name works correctly with dlopen'ing."""
         self.copy_shlibs_to_remote()
def run_load_unload(self):
                     substrs=['stopped',
                              'a_function',
                              'stop reason = breakpoint'])
 
         # The breakpoint should have a hit count of 2.
         self.expect("breakpoint list -f", BREAKPOINT_HIT_ONCE,
                     substrs=[' resolved, hit count = 2'])
 
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     def test_step_over_load(self):
         self.setSvr4Support(False)
         self.run_step_over_load()
 
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     def test_step_over_load_with_svr4(self):
         self.setSvr4Support(True)
         self.run_step_over_load()
 
     def run_step_over_load(self):
         """Test stepping over code that loads a shared library works correctly."""
         self.copy_shlibs_to_remote()
 
def run_step_over_load(self):
 
         # The stop reason should be step end.
         self.expect("thread list", "step over succeeded.",
                     substrs=['stopped',
                              'stop reason = step over'])
 
     # We can't find a breakpoint location for d_init before launching because
     # executable dependencies are resolved relative to the debuggers PWD. Bug?
-    @expectedFailureAll(oslist=["linux"], triple=no_match('aarch64-.*-android'))
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "linux", "netbsd"], triple=no_match('aarch64-.*-android'))
     def test_static_init_during_load(self):
         """Test that we can set breakpoints correctly in static initializers"""
         self.copy_shlibs_to_remote()
 
         exe = self.getBuildArtifact("a.out")
         self.runCmd("file " + exe, CURRENT_EXECUTABLE_SET)
 
         a_init_bp_num = lldbutil.run_break_set_by_symbol(

lldb/test/API/functionalities/load_using_paths/TestLoadUsingPaths.py

def setUp(self):
         if self.platformIsDarwin():
             ext = 'dylib'
         self.lib_name = 'libloadunload.' + ext
 
         self.wd = os.path.realpath(self.getBuildDir())
         self.hidden_dir = os.path.join(self.wd, 'hidden')
         self.hidden_lib = os.path.join(self.hidden_dir, self.lib_name)
 
-    @skipIfFreeBSD  # llvm.org/pr14424 - missing FreeBSD Makefiles/testcase support
     @not_remote_testsuite_ready
     @skipIfWindows  # Windows doesn't have dlopen and friends, dynamic libraries work differently
     @expectedFlakeyNetBSD
     @expectedFailureAll(oslist=["linux"], archs=['arm'], bugnumber="llvm.org/pr45894")
     @skipIfReproducer # FIXME: Unexpected packet during (passive) replay
     def test_load_using_paths(self):
         """Test that we can load a module by providing a set of search paths."""
         if self.platformIsDarwin():

lldb/test/API/functionalities/longjmp/TestLongjmp.py

 from lldbsuite.test import lldbutil
 
 
 class LongjmpTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     @skipIfDarwin  # llvm.org/pr16769: LLDB on Mac OS X dies in function ReadRegisterBytes in GDBRemoteRegisterContext.cpp
-    @skipIfFreeBSD  # llvm.org/pr17214
-    @expectedFailureAll(oslist=["linux"], bugnumber="llvm.org/pr20231")
+    @expectedFailureAll(oslist=["freebsd", "linux"], bugnumber="llvm.org/pr20231")
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24778")
     @expectedFlakeyNetBSD
     def test_step_out(self):
         """Test stepping when the inferior calls setjmp/longjmp, in particular, thread step-out."""
         self.build()
         self.step_out()
 
     @skipIfDarwin  # llvm.org/pr16769: LLDB on Mac OS X dies in function ReadRegisterBytes in GDBRemoteRegisterContext.cpp
-    @skipIfFreeBSD  # llvm.org/pr17214
-    @expectedFailureAll(oslist=["linux"], bugnumber="llvm.org/pr20231")
+    @expectedFailureAll(oslist=["freebsd", "linux"], bugnumber="llvm.org/pr20231")
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24778")
     @skipIfNetBSD
     def test_step_over(self):
         """Test stepping when the inferior calls setjmp/longjmp, in particular, thread step-over a longjmp."""
         self.build()
         self.step_over()
 
     @skipIfDarwin  # llvm.org/pr16769: LLDB on Mac OS X dies in function ReadRegisterBytes in GDBRemoteRegisterContext.cpp
-    @skipIfFreeBSD  # llvm.org/pr17214
-    @expectedFailureAll(oslist=["linux"], bugnumber="llvm.org/pr20231")
+    @expectedFailureAll(oslist=["freebsd", "linux"], bugnumber="llvm.org/pr20231")
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24778")
     @expectedFlakeyNetBSD
     def test_step_back_out(self):
         """Test stepping when the inferior calls setjmp/longjmp, in particular, thread step-out after thread step-in."""
         self.build()
         self.step_back_out()
 
     def start_test(self, symbol):

lldb/test/API/functionalities/plugins/python_os_plugin/stepping_plugin_threads/TestOSPluginStepping.py

 
 
 class TestOSPluginStepping(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
     NO_DEBUG_INFO_TESTCASE = True
 
     @skipIfWindows
+    @skipIfFreeBSD  # hangs
     def test_python_os_plugin(self):
         """Test that stepping works when the OS Plugin doesn't report all
            threads at every stop"""
         self.build()
         self.main_file = lldb.SBFileSpec('main.cpp')
         self.run_python_os_step_missing_thread(False)
 
     @skipIfWindows
+    @skipIfFreeBSD  # hangs
     def test_python_os_plugin_prune(self):
         """Test that pruning the unreported PlanStacks works"""
         self.build()
         self.main_file = lldb.SBFileSpec('main.cpp')
         self.run_python_os_step_missing_thread(True)
 
     def get_os_thread(self):
         return self.process.GetThreadByID(0x111111111)

lldb/test/API/functionalities/signal/TestSendSignal.py

class SendSignalTestCase(TestBase):
     mydir = TestBase.compute_mydir(__file__)
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break inside main().
         self.line = line_number('main.c', 'Put breakpoint here')
 
-    @expectedFailureAll(
-        oslist=['freebsd'],
-        bugnumber="llvm.org/pr23318: does not report running state")
     @expectedFailureNetBSD(bugnumber='llvm.org/pr43959')
     @skipIfWindows  # Windows does not support signals
     @skipIfReproducer # FIXME: Unexpected packet during (active) replay
     def test_with_run_command(self):
         """Test that lldb command 'process signal SIGUSR1' sends a signal to the inferior process."""
         self.build()
         exe = self.getBuildArtifact("a.out")
 

lldb/test/API/functionalities/signal/raise/TestRaise.py

class RaiseTestCase(TestBase):
     @skipIfNetBSD  # Hangs on NetBSD
     def test_sigstop(self):
         self.build()
         self.signal_test('SIGSTOP', False)
         # passing of SIGSTOP is not correctly handled, so not testing that
         # scenario: https://llvm.org/bugs/show_bug.cgi?id=23574
 
     @skipIfDarwin  # darwin does not support real time signals
+    @skipIfFreeBSD  # hangs
     @skipIfTargetAndroid()
     def test_sigsigrtmin(self):
         self.build()
         self.signal_test('SIGRTMIN', True)
 
     @skipIfNetBSD  # Hangs on NetBSD
+    @skipIfFreeBSD  # hangs
     def test_sigtrap(self):
         self.build()
         self.signal_test('SIGTRAP', True)
 
     def launch(self, target, signal):
         # launch the process, do not stop at entry point.
         process = target.LaunchSimple(
             [signal], None, self.get_process_working_directory())

lldb/test/API/functionalities/thread/create_after_attach/TestCreateAfterAttach.py

class CreateAfterAttachTestCase(TestBase):
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line numbers for our breakpoints.
         self.break_1 = line_number('main.cpp', '// Set first breakpoint here')
         self.break_2 = line_number('main.cpp', '// Set second breakpoint here')
         self.break_3 = line_number('main.cpp', '// Set third breakpoint here')
 
-    @skipIfFreeBSD  # Hangs.  May be the same as Linux issue llvm.org/pr16229 but
-    # not yet investigated.  Revisit once required functionality
-    # is implemented for FreeBSD.
     # Occasionally hangs on Windows, may be same as other issues.
     @skipIfWindows
     @skipIfiOSSimulator
-    @expectedFailureNetBSD
+    # FreeBSD: Hangs.  May be the same as Linux issue llvm.org/pr16229
+    # but not yet investigated.  Revisit once required functionality is
+    # implemented for FreeBSD.
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     def test_create_after_attach(self):
         """Test thread creation after process attach."""
         self.build(dictionary=self.getBuildFlags(use_cpp11=False))
         exe = self.getBuildArtifact("a.out")
 
         # Spawn a new process
         popen = self.spawnSubprocess(exe)
         pid = popen.pid

lldb/test/API/functionalities/thread/exit_during_step/TestExitDuringStep.py

 """
 Test number of threads.
 """
 
 
 
 import lldb
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class ExitDuringStepTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
-    @skipIfFreeBSD  # llvm.org/pr21411: test is hanging
     @skipIfWindows # This is flakey on Windows: llvm.org/pr38373
     def test(self):
         """Test thread exit during step handling."""
         self.build(dictionary=self.getBuildFlags())
         self.exit_during_step_base(
             "thread step-inst -m all-threads",
             'stop reason = instruction step',
             True)
 
-    @skipIfFreeBSD  # llvm.org/pr21411: test is hanging
     @skipIfWindows # This is flakey on Windows: llvm.org/pr38373
     def test_step_over(self):
         """Test thread exit during step-over handling."""
         self.build(dictionary=self.getBuildFlags())
         self.exit_during_step_base(
             "thread step-over -m all-threads",
             'stop reason = step over',
             False)
 
-    @skipIfFreeBSD  # llvm.org/pr21411: test is hanging
     @skipIfWindows # This is flakey on Windows: llvm.org/pr38373
     def test_step_in(self):
         """Test thread exit during step-in handling."""
         self.build(dictionary=self.getBuildFlags())
         self.exit_during_step_base(
             "thread step-in -m all-threads",
             'stop reason = step in',
             False)

lldb/test/API/functionalities/thread/state/TestThreadStates.py

def test_state_after_breakpoint(self):
         """Test thread state after breakpoint."""
         self.build(dictionary=self.getBuildFlags(use_cpp11=False))
         self.thread_state_after_breakpoint_test()
 
     @skipIfDarwin  # 'llvm.org/pr23669', cause Python crash randomly
     @expectedFailureAll(
         oslist=lldbplatformutil.getDarwinOSTriples(),
         bugnumber="llvm.org/pr23669")
-    @expectedFailureAll(oslist=["freebsd"], bugnumber="llvm.org/pr15824")
     @expectedFailureAll(oslist=["windows"], bugnumber="llvm.org/pr24660")
     def test_state_after_continue(self):
         """Test thread state after continue."""
         self.build(dictionary=self.getBuildFlags(use_cpp11=False))
         self.thread_state_after_continue_test()
 
     @skipIfDarwin  # 'llvm.org/pr23669', cause Python crash randomly
     @expectedFailureDarwin('llvm.org/pr23669')

lldb/test/API/lang/c/modules/TestCModules.py

 """Test that importing modules in C works as expected."""
 
 
 
 import os
 
 import lldb
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class CModulesTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
-    @skipIfFreeBSD
     @expectedFailureAll(
-        oslist=["linux"],
+        oslist=["freebsd", "linux"],
         bugnumber="http://llvm.org/pr23456 'fopen' has unknown return type")
     @expectedFailureAll(
         oslist=["windows"],
         bugnumber="llvm.org/pr24489: Name lookup not working correctly on Windows")
     @skipIf(macos_version=["<", "10.12"])
     @expectedFailureNetBSD
     def test_expr(self):
         self.build()

lldb/test/API/lit.cfg.py

 
 # Load LLDB test format.
 sys.path.append(os.path.join(config.lldb_src_root, "test", "API"))
 import lldbtest
 
 # testFormat: The test format to use to interpret tests.
 config.test_format = lldbtest.LLDBTest(dotest_cmd)
 
-# Propagate FREEBSD_REMOTE_PLUGIN
-if 'FREEBSD_REMOTE_PLUGIN' in os.environ:
-  config.environment['FREEBSD_REMOTE_PLUGIN'] = os.environ[
-      'FREEBSD_REMOTE_PLUGIN']
+# Propagate FREEBSD_LEGACY_PLUGIN
+if 'FREEBSD_LEGACY_PLUGIN' in os.environ:
+  config.environment['FREEBSD_LEGACY_PLUGIN'] = os.environ[
+      'FREEBSD_LEGACY_PLUGIN']

lldb/test/API/python_api/event/TestEvents.py

def test_wait_for_event(self):
         my_thread.start()
 
         # Wait until the 'MyListeningThread' terminates.
         my_thread.join()
 
         self.assertTrue(event,
                         "My listening thread successfully received an event")
 
-    @skipIfFreeBSD  # llvm.org/pr21325
     @add_test_categories(['pyapi'])
     @expectedFailureAll(
         oslist=["linux"],
         bugnumber="llvm.org/pr23617 Flaky, fails ~1/10 cases")
     @skipIfWindows # This is flakey on Windows AND when it fails, it hangs: llvm.org/pr38373
+    @expectedFailureAll(oslist=["freebsd"])
     @expectedFlakeyNetBSD
     def test_add_listener_to_broadcaster(self):
         """Exercise some SBBroadcaster APIs."""
         self.build()
         exe = self.getBuildArtifact("a.out")
 
         self.dbg.SetAsync(True)
 

lldb/test/API/tools/lldb-server/TestGdbRemote_qThreadStopInfo.py

class TestGdbRemote_qThreadStopInfo(gdbremote_testcase.GdbRemoteTestCaseBase):
     def test_qThreadStopInfo_only_reports_one_thread_stop_reason_during_interrupt_debugserver(
             self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qThreadStopInfo_only_reports_one_thread_stop_reason_during_interrupt(
             self.THREAD_COUNT)
 
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_qThreadStopInfo_only_reports_one_thread_stop_reason_during_interrupt_llgs(
             self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qThreadStopInfo_only_reports_one_thread_stop_reason_during_interrupt(
             self.THREAD_COUNT)

lldb/test/API/tools/lldb-server/TestLldbGdbServer.py

def qRegisterInfo_contains_avx_registers(self):
         # Collect all generics found.
         register_sets = {
             reg_info['set']: 1 for reg_info in reg_infos if 'set' in reg_info}
         self.assertEqual(
             self.targetHasAVX(),
             "Advanced Vector Extensions" in register_sets)
 
     @expectedFailureAll(oslist=["windows"]) # no avx for now.
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_qRegisterInfo_contains_avx_registers_llgs(self):
         self.init_llgs_test()
         self.build()
         self.qRegisterInfo_contains_avx_registers()
 
     def qThreadInfo_contains_thread(self):
         procs = self.prep_debug_monitor_and_inferior()
class LldbGdbServerTestCase(gdbremote_testcase.GdbRemoteTestCaseBase, DwarfOpcodeParser):
     @skipIfDarwinEmbedded # <rdar://problem/34539270> lldb-server tests not updated to work on ios etc yet
     def test_p_returns_correct_data_size_for_each_qRegisterInfo_launch_debugserver(
             self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         self.p_returns_correct_data_size_for_each_qRegisterInfo()
 
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_p_returns_correct_data_size_for_each_qRegisterInfo_launch_llgs(
             self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.p_returns_correct_data_size_for_each_qRegisterInfo()
 
     @debugserver_test
     @skipIfDarwinEmbedded # <rdar://problem/34539270> lldb-server tests not updated to work on ios etc yet
     def test_p_returns_correct_data_size_for_each_qRegisterInfo_attach_debugserver(
             self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_attach()
         self.p_returns_correct_data_size_for_each_qRegisterInfo()
 
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_p_returns_correct_data_size_for_each_qRegisterInfo_attach_llgs(
             self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_attach()
         self.p_returns_correct_data_size_for_each_qRegisterInfo()
 
def test_Hc_then_Csignal_signals_correct_thread_launch_debugserver(self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         # Darwin debugserver translates some signals like SIGSEGV into some gdb
         # expectations about fixed signal numbers.
         self.Hc_then_Csignal_signals_correct_thread(self.TARGET_EXC_BAD_ACCESS)
 
     @skipIfWindows # no SIGSEGV support
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_Hc_then_Csignal_signals_correct_thread_launch_llgs(self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.Hc_then_Csignal_signals_correct_thread(
             lldbutil.get_signal_number('SIGSEGV'))
 
class LldbGdbServerTestCase(gdbremote_testcase.GdbRemoteTestCaseBase, DwarfOpcodeParser):
     @skipIfDarwinEmbedded # <rdar://problem/34539270> lldb-server tests not updated to work on ios etc yet
     def test_qMemoryRegionInfo_is_supported_debugserver(self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_is_supported()
 
     @llgs_test
+    @expectedFailureAll(oslist=["freebsd"])
     def test_qMemoryRegionInfo_is_supported_llgs(self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_is_supported()
 
     def qMemoryRegionInfo_reports_code_address_as_executable(self):
         # Start up the inferior.
class LldbGdbServerTestCase(gdbremote_testcase.GdbRemoteTestCaseBase, DwarfOpcodeParser):
     def test_qMemoryRegionInfo_reports_code_address_as_executable_debugserver(
             self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_reports_code_address_as_executable()
 
     @skipIfWindows # No pty support to test any inferior output
+    @expectedFailureAll(oslist=["freebsd"])
     @llgs_test
     def test_qMemoryRegionInfo_reports_code_address_as_executable_llgs(self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_reports_code_address_as_executable()
 
     def qMemoryRegionInfo_reports_stack_address_as_readable_writeable(self):

     def test_qMemoryRegionInfo_reports_stack_address_as_readable_writeable_debugserver(
             self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_reports_stack_address_as_readable_writeable()
 
     @skipIfWindows # No pty support to test any inferior output
+    @expectedFailureAll(oslist=["freebsd"])
     @llgs_test
     def test_qMemoryRegionInfo_reports_stack_address_as_readable_writeable_llgs(
             self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_reports_stack_address_as_readable_writeable()
 

     def test_qMemoryRegionInfo_reports_heap_address_as_readable_writeable_debugserver(
             self):
         self.init_debugserver_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_reports_heap_address_as_readable_writeable()
 
     @skipIfWindows # No pty support to test any inferior output
+    @expectedFailureAll(oslist=["freebsd"])
     @llgs_test
     def test_qMemoryRegionInfo_reports_heap_address_as_readable_writeable_llgs(
             self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.qMemoryRegionInfo_reports_heap_address_as_readable_writeable()
 

lldb/test/API/tools/lldb-server/commandline/TestStubSetSID.py

class TestStubSetSIDTestCase(gdbremote_testcase.GdbRemoteTestCaseBase):
     def test_sid_is_same_without_setsid_debugserver(self):
         self.init_debugserver_test()
         self.set_inferior_startup_launch()
         self.sid_is_same_without_setsid()
 
     @skipIfWindows
     @llgs_test
     @skipIfRemote  # --setsid not used on remote platform and currently it is also impossible to get the sid of lldb-platform running on a remote target
-    @expectedFailureAll(oslist=['freebsd'])
     def test_sid_is_same_without_setsid_llgs(self):
         self.init_llgs_test()
         self.set_inferior_startup_launch()
         self.sid_is_same_without_setsid()
 
     @debugserver_test
     @skipIfRemote  # --setsid not used on remote platform and currently it is also impossible to get the sid of lldb-platform running on a remote target
     def test_sid_is_different_with_setsid_debugserver(self):

lldb/test/API/tools/lldb-server/libraries-svr4/TestGdbRemoteLibrariesSvr4Support.py

def libraries_svr4_libs_present(self):
         libraries_svr4_names = []
         for child in xml_root:
             name = child.attrib.get("name")
             libraries_svr4_names.append(os.path.realpath(name))
         for lib in self.get_expected_libs():
             self.assertIn(os.path.realpath(self.getBuildDir() + "/" + lib), libraries_svr4_names)
 
     @llgs_test
-    @skipUnlessPlatform(["linux", "android", "netbsd"])
+    @skipUnlessPlatform(["linux", "android", "freebsd", "netbsd"])
     def test_supports_libraries_svr4(self):
         self.setup_test()
         self.assertTrue(self.has_libraries_svr4_support())
 
     @llgs_test
-    @skipUnlessPlatform(["linux", "android", "netbsd"])
+    @skipUnlessPlatform(["linux", "android", "freebsd", "netbsd"])
     @expectedFailureNetBSD
     def test_libraries_svr4_well_formed(self):
         self.setup_test()
         self.libraries_svr4_well_formed()
 
     @llgs_test
-    @skipUnlessPlatform(["linux", "android", "netbsd"])
-    @expectedFailureNetBSD
+    @skipUnlessPlatform(["linux", "android", "freebsd", "netbsd"])
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     def test_libraries_svr4_load_addr(self):
         self.setup_test()
         self.libraries_svr4_has_correct_load_addr()
 
     @llgs_test
-    @skipUnlessPlatform(["linux", "android", "netbsd"])
+    @skipUnlessPlatform(["linux", "android", "freebsd", "netbsd"])
     @expectedFailureNetBSD
     def test_libraries_svr4_libs_present(self):
         self.setup_test()
         self.libraries_svr4_libs_present()

lldb/test/API/tools/lldb-server/register-reading/TestGdbRemoteGPacket.py

def g_returns_correct_data(self, with_suffix):
             ['0x424446484a4c4e41', '0x434547494b4d4f40'], get_reg_value('xmm12'))
         self.assertEqual(
             ['0x525456585a5c5e51', '0x535557595b5d5f50'], get_reg_value('xmm13'))
         self.assertEqual(
             ['0x626466686a6c6e61', '0x636567696b6d6f60'], get_reg_value('xmm14'))
         self.assertEqual(
             ['0x727476787a7c7e71', '0x737577797b7d7f70'], get_reg_value('xmm15'))
 
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_g_returns_correct_data_with_suffix_llgs(self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.g_returns_correct_data(True)
 
-    @expectedFailureNetBSD
+    @expectedFailureAll(oslist=["freebsd", "netbsd"])
     @llgs_test
     def test_g_returns_correct_data_no_suffix_llgs(self):
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         self.g_returns_correct_data(False)

lldb/test/API/tools/lldb-server/thread-name/TestGdbRemoteThreadName.py

def run_and_check_name(self, expected_name):
         self.assertIsNotNone(context)
 
         sigint = lldbutil.get_signal_number("SIGINT")
         self.assertEqual(sigint, int(context.get("signal"), 16))
         kv_dict = self.parse_key_val_dict(context.get("key_vals_text"))
         self.assertEqual(expected_name, kv_dict.get("name"))
 
     @skipIfWindows # the test is not updated for Windows.
+    @expectedFailureAll(oslist=["freebsd"])
     @llgs_test
     def test(self):
         """ Make sure lldb-server can retrieve inferior thread name"""
         self.init_llgs_test()
         self.build()
         self.set_inferior_startup_launch()
         procs = self.prep_debug_monitor_and_inferior()
 
         self.run_and_check_name("hello world")
         self.run_and_check_name("goodbye world")

lldb/test/Shell/ExecControl/StopHook/stop-hook-threads.test

 # RUN: %clangxx_host -std=c++11 %p/Inputs/stop-hook-threads.cpp -g -o %t
 # RUN: %lldb -b -s %p/Inputs/stop-hook-threads-1.lldbinit -s %s -f %t \
 # RUN:     | FileCheck --check-prefix=CHECK --check-prefix=CHECK-NO-FILTER %s
 # RUN: %lldb -b -s %p/Inputs/stop-hook-threads-2.lldbinit -s %s -f %t \
 # RUN:     | FileCheck --check-prefix=CHECK --check-prefix=CHECK-FILTER %s
+# XFAIL: system-freebsd
 # XFAIL: system-netbsd
 # UNSUPPORTED: system-windows
 # This test is flakey and hangs on windows periodically: llvm.org/pr38373
 # UNSUPPORTED: system-linux, system-darwin
 
 thread list
 break set -f stop-hook-threads.cpp -p "Set break point at this line"
 target stop-hook list

lldb/test/Shell/Recognizer/assert.test

 # XFAIL: target-arm && linux-gnu
+# XFAIL: system-freebsd
 # UNSUPPORTED: system-windows
 # RUN: %clang_host -g -O0 %S/Inputs/assert.c -o %t.out
 # RUN: %lldb -b -s %s %t.out | FileCheck %s
 run
 # CHECK: thread #{{.*}}stop reason = hit program assert
 frame info
 # CHECK: frame #{{.*}}`main at assert.c
 frame recognizer info 0
 # CHECK: frame 0 is recognized by Assert StackFrame Recognizer
 set set thread-format "{${thread.stop-reason-raw}}\n"
 thread info
 # CHECK: signal SIGABRT
 q

lldb/test/Shell/lit.cfg.py

try:
         if output != "1":
             can_set_dbregs = False
     except subprocess.CalledProcessError:
         can_set_dbregs = False
 if can_set_dbregs:
     config.available_features.add('dbregs-set')
 
 # pass control variable through
-llvm_config.with_system_environment('FREEBSD_REMOTE_PLUGIN')
+llvm_config.with_system_environment('FREEBSD_LEGACY_PLUGIN')