@infrandomness Nicely suggested using one of their machines running FreeBSD. We've set up a BuildKite agent and this patch should allow targeting the agent in our CI.
We are aware that some tests are failing on FreeBSD, and it would be nice if the FreeBSD maintainers could take a look. Also, it would be nice to add at least one other FreeBSD runner that is not backed by an individual to ensure that our CI setup is robust. If @infrandomness can't provide the machine anymore in the future, or even if they simply have a power outage, it would be nice for our CI setup to still work. But this is an excellent start!
@infrandomness thanks a lot for providing this CI runner!
I had a look at the failed tests and it seems the number isn't too large and most of them seem to be filesystem and locales.
Two areas where I'm not surprised that there are issues.
@ldionne Do we want to add XFAIL to the failing tests or add a soft_fail entry?
It would be great when you can fix these tests! We can indeed make them XFAIL now and remove that in patches that fixes the tests.
We do the same for AIX, they have some people submitting fixes for their failing locale tests.
This situation is exactly why we have the support policy that we have: we are interested in supporting platforms that want to be supported, i.e. that want to invest the time required to make things work. People have volunteered machines to test on FreeBSD, and have volunteered time to get this patch up. All it needs is someone (probably someone who cares first hand about FreeBSD support) to finish it up and start fixing the LIBCXX-FREEBSD-FIXMEs added here.
I would be happy to address these FIXMEs, but I would have to do so outside working hours so my time is somewhat limited. I recently ran the testsuite on my Arm Morello system running CheriBSD and it seems like a lot of the filesystem ones are fixed on this FreeBSD 14 snapshot.
Which version of FreeBSD is the builder running? This would allow me to reproduce and attach a debugger on a similar system.