lib Fuzzer FreeBSD support

[devnexen]

Summary

Patch by David CARLIER

[devnexen]

From llvm as if I m correct lib buzzer does not belong to compiler rt anymore.

[krytarowski]

What are the results for check-fuzzer?

[krytarowski]

lib/fuzzer/FuzzerUtilPosix.cpp

121

Please add || LIBFUZZER_NETBSD too.

[devnexen]

Updated accordingly + one forgotten change to be able to build the target.

[devnexen]

In D41642#966655, @krytarowski wrote:

What are the results for check-fuzzer?

Sorry for late reply.
In my FreeBSD machines I m unable to build the clang target (linkage ... machine hanging) but all the rest does (e.g. I m able on Linux with similar hw). I tried the patch on libfuzzer alone (with h2o) and seemed working ok.

[krytarowski]

lib/fuzzer/tests/CMakeLists.txt

18 ↗

(On Diff #128875)

Why -lpthread here? Is it needed for a particular test only? Or all of them?

Maybe this should be moved to the clang frontend.

[morehouse]

lib/fuzzer/build.sh

1

Is this change necessary? All our shell scripts use #!/bin/sh or #!/bin/bash.

lib/fuzzer/tests/CMakeLists.txt

18 ↗

(On Diff #129328)

Can you combine this into the original if expression with OR?

[krytarowski]

lib/fuzzer/build.sh

1

This is necessary, better to rewrite it to /bin/sh (POSIX shell) if possible. The env fallback is usually good enough for scripts with bash-specific features.

There is no bash in the basesystem of BSD. It can be installed optionally and if so, it's located in a different location dedicated for external software - in the NetBSD case, it's /usr/pkg/bin/bash.

[devnexen]

With this change (alongside with this one https://reviews.llvm.org/D41809) I m finally able to build fuzz binaries.

[krytarowski]

Can you run 'check-fuzzer'?

[devnexen]

The tests pass with those changes.

[devnexen]

In D41642#973343, @krytarowski wrote:

Can you run 'check-fuzzer'?

I got 147 tests but had to disable some.

lib/fuzzer/tests/CMakeLists.txt

18 ↗

(On Diff #129328)

Yes sure I just feared either it would make the expression too long or ... in case we might need to add later on some FreeBSD specificities.

18 ↗

(On Diff #128875)

When I did the change I think one test required it indeed but might not be necessary now I ve update the tree locally but still struggle to compile the whole :-) we ll see

[krytarowski]

test/fuzzer/strcmp.test

1 ↗

(On Diff #129455)

I think that the style is to put UNSUPPORTED after RUN:

[devnexen]

test/fuzzer/strcmp.test

1 ↗

(On Diff #129455)

Oh ok I looked at others and were put first.

[krytarowski]

I can land this for you (and for the benefit of NetBSD), assuming that it will be accepted by other developer. Maybe @george.karpenkov can have a look?

[george.karpenkov]

@krytarowski CMake changes look good to me, but I'm not the code owner, so probably @kcc (or @morehouse) would have to approve it first.
It's interesting that tests requiring intrinsics instrumentation are failing, is it hard to port those as well? (though that could be done in a separate patch)

[morehouse]

LGTM

[dim]

lib/fuzzer/build.sh

1

I see nothing in this script that requires bash. Maybe just use /bin/sh here?

But indeed, /bin/bash is normally only available on Linux. BSDs put it in a ports or pkgsrc specific installation directory, and that directory will be in the user's PATH.

Something similar goes for e.g. perl and python, one should *never* use /usr/bin/perl or /usr/bin/python, always /usr/bin/env perl and /usr/bin/env python.

[krytarowski]

lib/fuzzer/build.sh

1

Can you prepare a patch please?

[devnexen]

lib/fuzzer/build.sh

1

Hope it s ok https://reviews.llvm.org/D42008

lib/fuzzer/FuzzerDefs.h

 #define LIBFUZZER_FUCHSIA 0
 #define LIBFUZZER_LINUX 1
 #define LIBFUZZER_NETBSD 0
+#define LIBFUZZER_FREEBSD 0
 #define LIBFUZZER_WINDOWS 0
 #elif __APPLE__
 #define LIBFUZZER_APPLE 1

 #define LIBFUZZER_FUCHSIA 0
 #define LIBFUZZER_LINUX 0
 #define LIBFUZZER_NETBSD 0
+#define LIBFUZZER_FREEBSD 0
 #define LIBFUZZER_WINDOWS 0
 #elif __NetBSD__
 #define LIBFUZZER_APPLE 0

 #define LIBFUZZER_FUCHSIA 0
 #define LIBFUZZER_LINUX 0
 #define LIBFUZZER_NETBSD 1
+#define LIBFUZZER_FREEBSD 0
 #define LIBFUZZER_WINDOWS 0
+#elif __FreeBSD__
+#define LIBFUZZER_APPLE 0
+#define LIBFUZZER_FUCHSIA 0
+#define LIBFUZZER_LINUX 0
+#define LIBFUZZER_NETBSD 0
+#define LIBFUZZER_FREEBSD 1
+#define LIBFUZZER_WINDOWS 0
 #elif _WIN32
 #define LIBFUZZER_APPLE 0
 #define LIBFUZZER_FUCHSIA 0
 #define LIBFUZZER_LINUX 0
 #define LIBFUZZER_NETBSD 0
+#define LIBFUZZER_FREEBSD 0
 #define LIBFUZZER_WINDOWS 1
 #elif __Fuchsia__
 #define LIBFUZZER_APPLE 0

lib/fuzzer/FuzzerExtFunctionsWeak.cpp

 // to clients right now.
 //===----------------------------------------------------------------------===//
 #include "FuzzerDefs.h"
-#if LIBFUZZER_LINUX || LIBFUZZER_NETBSD || LIBFUZZER_FUCHSIA
+#if LIBFUZZER_LINUX || LIBFUZZER_NETBSD || LIBFUZZER_FUCHSIA || LIBFUZZER_FREEBSD
 
 #include "FuzzerExtFunctions.h"
 #include "FuzzerIO.h"

 
 } // namespace fuzzer
 
-#endif // LIBFUZZER_LINUX || LIBFUZZER_NETBSD
+#endif // LIBFUZZER_LINUX || LIBFUZZER_NETBSD || LIBFUZZER_FUSCHIA || LIBFUZZER_FREEBSD

lib/fuzzer/FuzzerUtilLinux.cpp

 // Misc utils for Linux.
 //===----------------------------------------------------------------------===//
 #include "FuzzerDefs.h"
-#if LIBFUZZER_LINUX || LIBFUZZER_NETBSD
+#if LIBFUZZER_LINUX || LIBFUZZER_NETBSD || LIBFUZZER_FREEBSD
 #include "FuzzerCommand.h"
 
 #include <stdlib.h>

 
 } // namespace fuzzer
 
-#endif // LIBFUZZER_LINUX || LIBFUZZER_NETBSD
+#endif // LIBFUZZER_LINUX || LIBFUZZER_NETBSD || LIBFUZZER_FREEBSD

lib/fuzzer/FuzzerUtilPosix.cpp

   struct rusage usage;
   if (getrusage(RUSAGE_SELF, &usage))
     return 0;
-  if (LIBFUZZER_LINUX) {
+  if (LIBFUZZER_LINUX || LIBFUZZER_FREEBSD) {
     // ru_maxrss is in KiB
     return usage.ru_maxrss >> 10;
   } else if (LIBFUZZER_APPLE) {

lib/fuzzer/build.sh

-#!/bin/bash
+#!/usr/bin/env bash
 LIBFUZZER_SRC_DIR=$(dirname $0)
 CXX="${CXX:-clang}"
 for f in $LIBFUZZER_SRC_DIR/*.cpp; do