[libfuzzer][MSVC] Make calls to builtin functions work with MSVC

[metzman]

Summary

Replace calls to builtin functions with macros or functions that call the
Windows-equivalents when targeting windows and call the original
builtin functions everywhere else.
This change makes more parts of libFuzzer buildable with MSVC.

[metzman]

• less casting and use intrisics.

[metzman]

• fmt

[metzman]

• include <stdlib.>
• reword

[metzman]

• add nonwin implementations

[metzman]

• fix spacing
• remove newline

[metzman]

Vitaly, could you please take a look?

I think Kostya wants to try making LF compilable with MSVC so he asked me to split up https://reviews.llvm.org/D56363

This change specifically handles replacing calls to builtin functions with things that can be compiled with MSVC

Thank you

compiler-rt/lib/fuzzer/FuzzerDefs.h

143 ↗

(On Diff #180676)

I added reinterpret_cast<uintptr_t>() to GET_CALLER_PC since every caller casts the result, was this the right call?

206 ↗

(On Diff #180676)

In my current approach I defined wrapper functions for Windows and non-Windows (which call the builtins).
An alternative is to continue calling __builtin_* everywhere and simply define these when compiling with MSVC. I thought that approach was too hacky and that this is better WDYT?
Another approach is using macros, but I think the inline functions are preferable (though I'll do what ever you think is best).

[metzman]

CC @rnk and @thakis (since rnk is OOO)

[vitalybuka]

compiler-rt/lib/fuzzer/FuzzerDefs.h

197 ↗

(On Diff #180681)

Could you please move them into:
fuzzer/FuzzerBuiltins.h
fuzzer/FuzzerBuiltinsMsvc.h

include thsese headers unconditionally
but inside of the header put

#if defined(_MSC_VER)
...
and
#if !defined(_MSC_VER)

143 ↗

(On Diff #180676)

GET_CALLER_PC is defined in sanitizer_common, so it should probably defined there
also if we probably want to keep __builtin_return_address if we compile with clang,

[metzman]

compiler-rt/lib/fuzzer/FuzzerDefs.h

143 ↗

(On Diff #180676)

GET_CALLER_PC is defined in sanitizer_common, so it should probably defined there

Can libFuzzer depend on sanitizer_common?

also if we probably want to keep __builtin_return_address if we compile with clang,

I'll do this, but I figured it would be simpler to do things one way on Windows and another way everywhere else instead of having two ways to do things on Windows.

[metzman]

• Move builtin wrappers to their own files.

[metzman]

• format

[metzman]

• move Bswap to FuzzerBuiltin headers for consistency.

[metzman]

compiler-rt/lib/fuzzer/FuzzerDefs.h

197 ↗

(On Diff #180681)

Done.

143 ↗

(On Diff #180676)

Made both of these changes.

[metzman]

• define LIBFUZZER_MSVC

[vitalybuka]

LGTM

compiler-rt/lib/fuzzer/FuzzerBuiltins.h

22 ↗

(On Diff #180880)

could you please separate namespace from the other code with emty line
same for closing } and other files
just for consistency

compiler-rt/lib/fuzzer/FuzzerBuiltinsMsvc.h

40 ↗

(On Diff #180880)

#error ?

[metzman]

• less casting and use intrisics.
• fmt
• include properly
• reword
• add nonwin implementations
• fix spacing
• remove newline
• Move builtin wrappers to their own files.
• fix headers
• fix
• fix
• fmt
• fmt
• move Bswap to FuzzerBuiltin headers for consistency.
• define LIBFUZZER_MSVC
• add spacing and ensure not on x86

[metzman]

• Add space before end of namespace

[metzman]

compiler-rt/lib/fuzzer/FuzzerBuiltins.h

22 ↗

(On Diff #180880)

Done in both builtins headers.

compiler-rt/lib/fuzzer/FuzzerBuiltinsMsvc.h

40 ↗

(On Diff #180880)

Done.

[• ham99922]

This comment was removed by asl.

lib/fuzzer/CMakeLists.txt

set(LIBFUZZER_SOURCES
   FuzzerUtil.cpp
   FuzzerUtilDarwin.cpp
   FuzzerUtilFuchsia.cpp
   FuzzerUtilLinux.cpp
   FuzzerUtilPosix.cpp
   FuzzerUtilWindows.cpp)
 
 set(LIBFUZZER_HEADERS
+  FuzzerBuiltins.h
+  FuzzerBuiltinsMsvc.h
   FuzzerCommand.h
   FuzzerCorpus.h
   FuzzerDataFlowTrace.h
   FuzzerDefs.h
   FuzzerDictionary.h
   FuzzerExtFunctions.def
   FuzzerExtFunctions.h
   FuzzerFlags.def

lib/fuzzer/FuzzerBuiltins.h

+//===- FuzzerBuiltins.h - Internal header for builtins ----------*- C++ -* ===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+// Wrapper functions and marcos around builtin functions.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_FUZZER_BUILTINS_H
+#define LLVM_FUZZER_BUILTINS_H
+
+#include "FuzzerDefs.h"
+
+#if !LIBFUZZER_MSVC
+#include <cstdint>
+
+#define GET_CALLER_PC() __builtin_return_address(0)
+
+namespace fuzzer {
+
+inline uint8_t  Bswap(uint8_t x)  { return x; }
+inline uint16_t Bswap(uint16_t x) { return __builtin_bswap16(x); }
+inline uint32_t Bswap(uint32_t x) { return __builtin_bswap32(x); }
+inline uint64_t Bswap(uint64_t x) { return __builtin_bswap64(x); }
+
+inline uint32_t Clzll(unsigned long long X) { return __builtin_clzll(X); }
+inline uint32_t Clz(unsigned long long X) { return __builtin_clz(X); }
+inline int Popcountll(unsigned long long X) { return __builtin_popcountll(X); }
+
+}  // namespace fuzzer
+
+#endif  // !LIBFUZZER_MSVC
+#endif  // LLVM_FUZZER_BUILTINS_H

lib/fuzzer/FuzzerBuiltinsMsvc.h

+//===- FuzzerBuiltinsMSVC.h - Internal header for builtins ------*- C++ -* ===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+// Wrapper functions and marcos that use intrinsics instead of builtin functions
+// which cannot be compiled by MSVC.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_FUZZER_BUILTINS_MSVC_H
+#define LLVM_FUZZER_BUILTINS_MSVC_H
+
+#include "FuzzerDefs.h"
+
+#if LIBFUZZER_MSVC
+#if !defined(_M_ARM) && !defined(_M_X64)
+#error "_BitScanReverse64 unavailable on this platform so MSVC is unsupported."
+#endif
+#include <intrin.h>
+#include <cstdint>
+#include <cstdlib>
+
+// __builtin_return_address() cannot be compiled with MSVC. Use the equivalent
+// from <intrin.h>
+#define GET_CALLER_PC() reinterpret_cast<uintptr_t>(_ReturnAddress())
+
+namespace fuzzer {
+
+inline uint8_t  Bswap(uint8_t x)  { return x; }
+// Use alternatives to __builtin functions from <stdlib.h> and <intrin.h> on
+// Windows since the builtins are not supported by MSVC.
+inline uint16_t Bswap(uint16_t x) { return _byteswap_ushort(x); }
+inline uint32_t Bswap(uint32_t x) { return _byteswap_ulong(x); }
+inline uint64_t Bswap(uint64_t x) { return _byteswap_uint64(x); }
+
+// The functions below were mostly copied from
+// compiler-rt/lib/builtins/int_lib.h which defines the __builtin functions used
+// outside of Windows.
+inline uint32_t Clzll(uint64_t X) {
+  unsigned long LeadZeroIdx = 0;
+  if (_BitScanReverse64(&LeadZeroIdx, X)) return 63 - LeadZeroIdx;
+  return 64;
+}
+
+inline uint32_t Clz(uint32_t X) {
+  unsigned long LeadZeroIdx = 0;
+  if (_BitScanReverse(&LeadZeroIdx, X)) return 31 - LeadZeroIdx;
+  return 32;
+}
+
+inline int Popcountll(unsigned long long X) { return __popcnt64(X); }
+
+}  // namespace fuzzer
+
+#endif  // LIBFUZER_MSVC
+#endif  // LLVM_FUZZER_BUILTINS_MSVC_H

lib/fuzzer/FuzzerDefs.h

 #define LIBFUZZER_NETBSD 0
 #define LIBFUZZER_FREEBSD 0
 #define LIBFUZZER_OPENBSD 0
 #define LIBFUZZER_WINDOWS 0
 #else
 #error "Support for your platform has not been implemented"
 #endif
 
+#if defined(_MSC_VER) && !defined(__clang__)
+// MSVC compiler is being used.
+#define LIBFUZZER_MSVC 1
+#else
+#define LIBFUZZER_MSVC 0
+#endif
+
 #ifndef __has_attribute
 #  define __has_attribute(x) 0
 #endif
 
 #define LIBFUZZER_POSIX                                                        \
   (LIBFUZZER_APPLE || LIBFUZZER_LINUX || LIBFUZZER_NETBSD ||                   \
    LIBFUZZER_FREEBSD || LIBFUZZER_OPENBSD)
 

 using Set = std::set<T, std::less<T>, fuzzer_allocator<T>>;
 
 typedef Vector<uint8_t> Unit;
 typedef Vector<Unit> UnitVector;
 typedef int (*UserCallback)(const uint8_t *Data, size_t Size);
 
 int FuzzerDriver(int *argc, char ***argv, UserCallback Callback);
 
-inline uint8_t  Bswap(uint8_t x)  { return x; }
-inline uint16_t Bswap(uint16_t x) { return __builtin_bswap16(x); }
-inline uint32_t Bswap(uint32_t x) { return __builtin_bswap32(x); }
-inline uint64_t Bswap(uint64_t x) { return __builtin_bswap64(x); }
-
 uint8_t *ExtraCountersBegin();
 uint8_t *ExtraCountersEnd();
 void ClearExtraCounters();
 
 extern bool RunningUserCallback;
 
 }  // namespace fuzzer
 
 #endif  // LLVM_FUZZER_DEFS_H

lib/fuzzer/FuzzerTracePC.cpp

 //===- FuzzerTracePC.cpp - PC tracing--------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 // Trace PCs.
 // This module implements __sanitizer_cov_trace_pc_guard[_init],
 // the callback required for -fsanitize-coverage=trace-pc-guard instrumentation.
 //
 //===----------------------------------------------------------------------===//
 
 #include "FuzzerTracePC.h"
+#include "FuzzerBuiltins.h"
+#include "FuzzerBuiltinsMsvc.h"
 #include "FuzzerCorpus.h"
 #include "FuzzerDefs.h"
 #include "FuzzerDictionary.h"
 #include "FuzzerExtFunctions.h"
 #include "FuzzerIO.h"
 #include "FuzzerUtil.h"
 #include "FuzzerValueBitMap.h"
 #include <set>

 ATTRIBUTE_TARGET_POPCNT ALWAYS_INLINE
 ATTRIBUTE_NO_SANITIZE_ALL
 void TracePC::HandleCmp(uintptr_t PC, T Arg1, T Arg2) {
   uint64_t ArgXor = Arg1 ^ Arg2;
   if (sizeof(T) == 4)
       TORC4.Insert(ArgXor, Arg1, Arg2);
   else if (sizeof(T) == 8)
       TORC8.Insert(ArgXor, Arg1, Arg2);
-  uint64_t HammingDistance = __builtin_popcountll(ArgXor); // [0,64]
-  uint64_t AbsoluteDistance =
-      (Arg1 == Arg2 ? 0 : __builtin_clzll(Arg1 - Arg2) + 1);
+  uint64_t HammingDistance = Popcountll(ArgXor);  // [0,64]
+  uint64_t AbsoluteDistance = (Arg1 == Arg2 ? 0 : Clzll(Arg1 - Arg2) + 1);
   ValueProfileMap.AddValue(PC * 128 + HammingDistance);
   ValueProfileMap.AddValue(PC * 128 + 64 + AbsoluteDistance);
 }
 
 static size_t InternalStrnlen(const char *S, size_t MaxLen) {
   size_t Len = 0;
   for (; Len < MaxLen && S[Len]; Len++) {}
   return Len;

 }
 
 } // namespace fuzzer
 
 extern "C" {
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 void __sanitizer_cov_trace_pc_guard(uint32_t *Guard) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   uint32_t Idx = *Guard;
   __sancov_trace_pc_pcs[Idx] = PC;
   __sancov_trace_pc_guard_8bit_counters[Idx]++;
 }
 
 // Best-effort support for -fsanitize-coverage=trace-pc, which is available
 // in both Clang and GCC.
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 void __sanitizer_cov_trace_pc() {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   uintptr_t Idx = PC & (((uintptr_t)1 << fuzzer::TracePC::kTracePcBits) - 1);
   __sancov_trace_pc_pcs[Idx] = PC;
   __sancov_trace_pc_guard_8bit_counters[Idx]++;
 }
 
 ATTRIBUTE_INTERFACE
 void __sanitizer_cov_trace_pc_guard_init(uint32_t *Start, uint32_t *Stop) {
   fuzzer::TPC.HandleInit(Start, Stop);
 }
 
 ATTRIBUTE_INTERFACE
 void __sanitizer_cov_8bit_counters_init(uint8_t *Start, uint8_t *Stop) {
   fuzzer::TPC.HandleInline8bitCountersInit(Start, Stop);
 }
 
 ATTRIBUTE_INTERFACE
 void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
                               const uintptr_t *pcs_end) {
   fuzzer::TPC.HandlePCsInit(pcs_beg, pcs_end);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 void __sanitizer_cov_trace_pc_indir(uintptr_t Callee) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCallerCallee(PC, Callee);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_cmp8(uint64_t Arg1, uint64_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 // Now the __sanitizer_cov_trace_const_cmp[1248] callbacks just mimic
 // the behaviour of __sanitizer_cov_trace_cmp[1248] ones. This, however,
 // should be changed later to make full use of instrumentation.
 void __sanitizer_cov_trace_const_cmp8(uint64_t Arg1, uint64_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_cmp4(uint32_t Arg1, uint32_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_const_cmp4(uint32_t Arg1, uint32_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_cmp2(uint16_t Arg1, uint16_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_const_cmp2(uint16_t Arg1, uint16_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_cmp1(uint8_t Arg1, uint8_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_const_cmp1(uint8_t Arg1, uint8_t Arg2) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Arg1, Arg2);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_switch(uint64_t Val, uint64_t *Cases) {
   uint64_t N = Cases[0];
   uint64_t ValSizeInBits = Cases[1];
   uint64_t *Vals = Cases + 2;
   // Skip the most common and the most boring case.
   if (Vals[N - 1]  < 256 && Val < 256)
     return;
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   size_t i;
   uint64_t Token = 0;
   for (i = 0; i < N; i++) {
     Token = Val ^ Vals[i];
     if (Val < Vals[i])
       break;
   }
 
   if (ValSizeInBits == 16)
     fuzzer::TPC.HandleCmp(PC + i, static_cast<uint16_t>(Token), (uint16_t)(0));
   else if (ValSizeInBits == 32)
     fuzzer::TPC.HandleCmp(PC + i, static_cast<uint32_t>(Token), (uint32_t)(0));
   else
     fuzzer::TPC.HandleCmp(PC + i, Token, (uint64_t)(0));
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_div4(uint32_t Val) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Val, (uint32_t)0);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_div8(uint64_t Val) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Val, (uint64_t)0);
 }
 
 ATTRIBUTE_INTERFACE
 ATTRIBUTE_NO_SANITIZE_ALL
 ATTRIBUTE_TARGET_POPCNT
 void __sanitizer_cov_trace_gep(uintptr_t Idx) {
-  uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
+  uintptr_t PC = reinterpret_cast<uintptr_t>(GET_CALLER_PC());
   fuzzer::TPC.HandleCmp(PC, Idx, (uintptr_t)0);
 }
 
 ATTRIBUTE_INTERFACE ATTRIBUTE_NO_SANITIZE_MEMORY
 void __sanitizer_weak_hook_memcmp(void *caller_pc, const void *s1,
                                   const void *s2, size_t n, int result) {
   if (!fuzzer::RunningUserCallback) return;
   if (result == 0) return;  // No reason to mutate.

lib/fuzzer/FuzzerUtil.h

 //===- FuzzerUtil.h - Internal header for the Fuzzer Utils ------*- C++ -* ===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 // Util functions.
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_FUZZER_UTIL_H
 #define LLVM_FUZZER_UTIL_H
 
-#include "FuzzerDefs.h"
+#include "FuzzerBuiltins.h"
+#include "FuzzerBuiltinsMsvc.h"
 #include "FuzzerCommand.h"
+#include "FuzzerDefs.h"
 
 namespace fuzzer {
 
 void PrintHexArray(const Unit &U, const char *PrintAfter = "");
 
 void PrintHexArray(const uint8_t *Data, size_t Size,
                    const char *PrintAfter = "");
 

 }
 
 std::string DisassembleCmd(const std::string &FileName);
 
 std::string SearchRegexCmd(const std::string &Regex);
 
 size_t SimpleFastHash(const uint8_t *Data, size_t Size);
 
-inline uint32_t Log(uint32_t X) { return 32 - __builtin_clz(X) - 1; }
+inline uint32_t Log(uint32_t X) { return 32 - Clz(X) - 1; }
 
 }  // namespace fuzzer
 
 #endif  // LLVM_FUZZER_UTIL_H