[libc] add exception to atof differential fuzz

[michaelrj]

Summary

The differential fuzzer for atof found a bug in glibc's handling of
hexadecimal rounding. Since we can't easily update glibc and we want to
avoid false positives when running the fuzzer, I've added an exception
to skip all hexadecimal subnormal cases.

[sivachandra]

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

41

Can there be a way to do this without using __llvm_libc::atof?

[michaelrj]

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

41

Not really. We need to use some version of atof, either the system's or ours. The thing we're trying to detect here is hexadecimal subnormals, and the process for checking that is exactly the process for converting a string to a float.

[sivachandra]

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

41

Let me rephrase my question: can you detect problem inputs using the system libc atof?

[michaelrj]

switch to system atof for exception detection.

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

41

yes, either atof will work.

[sivachandra]

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

22

A normal way to do this would be to:

#ifdef LLVM_LIBC_...
...
#else
...
#endif

And, let users define that macro in the build system. Also, the name of the macro can be more suggestive I think: LLVM_LIBC_ATOF_DIF_FUZZ_SKIP_GLIBC_HEX_SUBNORMAL_ERR.

54

If the skip check function can take the size argument, can we skip before making the copy.

[michaelrj]

address comments

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

54

not really, we need to add the null terminator to the string and to do that we need to copy it into something we can modify.

libc/fuzzing/stdlib/CMakeLists.txt

 add_libc_fuzzer(
   qsort_fuzz
   SRCS
     qsort_fuzz.cpp
   DEPENDS
     libc.src.stdlib.qsort
 )
 
 add_libc_fuzzer(
   atof_differential_fuzz
   SRCS
     atof_differential_fuzz.cpp
   HDRS
     StringParserOutputDiff.h
   DEPENDS
     libc.src.stdlib.atof
+  COMPILE_OPTIONS
+    -DLLVM_LIBC_ATOF_DIF_FUZZ_SKIP_GLIBC_HEX_SUBNORMAL_ERR
 )
 
 add_libc_fuzzer(
   strtofloat_fuzz
   SRCS
     strtofloat_fuzz.cpp
   DEPENDS
     libc.src.stdlib.atof

libc/fuzzing/stdlib/atof_differential_fuzz.cpp

 //===----------------------------------------------------------------------===//
 #include "src/stdlib/atof.h"
 #include <stddef.h>
 #include <stdint.h>
 #include <stdlib.h>
 
 #include "fuzzing/stdlib/StringParserOutputDiff.h"
 
+// TODO: Remove this once glibc fixes hex subnormal rounding. See
+// https://sourceware.org/bugzilla/show_bug.cgi?id=30220
+#ifdef LLVM_LIBC_ATOF_DIF_FUZZ_SKIP_GLIBC_HEX_SUBNORMAL_ERR
+#include <ctype.h>
+constexpr double MIN_NORMAL = 0x1p-1022;
+
+bool has_hex_prefix(const uint8_t *str) {
+  size_t index = 0;
+
+  // Skip over leading whitespace
+  while (isspace(str[index])) {
+    ++index;
+  }
+  // Skip over sign
+  if (str[index] == '-' || str[index] == '+') {
+    ++index;
+  }
+  return str[index] == '0' && (tolower(str[index + 1])) == 'x';
+}
+
+bool should_be_skipped(const uint8_t *str) {
+  double init_result = ::atof(reinterpret_cast<const char *>(str));
+  if (init_result < 0) {
+    init_result = -init_result;
+  }
+  if (init_result < MIN_NORMAL && init_result != 0) {
+    return has_hex_prefix(str);
+  }
+  return false;
+}
+#else
+bool should_be_skipped(const uint8_t *) { return false; }
+#endif // LLVM_LIBC_ATOF_DIF_FUZZ_SKIP_GLIBC_HEX_SUBNORMAL_ERR
+
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
   uint8_t *container = new uint8_t[size + 1];
   if (!container)
     __builtin_trap();
   size_t i;
 
   for (i = 0; i < size; ++i)
     container[i] = data[i];
   container[size] = '\0'; // Add null terminator to container.
 
-  StringParserOutputDiff<double>(&__llvm_libc::atof, &::atof, container, size);
+  if (!should_be_skipped(container)) {
+    StringParserOutputDiff<double>(&__llvm_libc::atof, &::atof, container,
+                                   size);
+  }
   delete[] container;
   return 0;
 }