Integrate Kostya's clang-proto-fuzzer with LLVM.

[morehouse]

Summary

The clang-proto-fuzzer models a subset of C++ as a protobuf and
uses libprotobuf-mutator to generate interesting mutations of C++
programs. Clang-proto-fuzzer has already found several bugs in
Clang (e.g., https://bugs.llvm.org/show_bug.cgi?id=33747,
https://bugs.llvm.org/show_bug.cgi?id=33749).

As with clang-fuzzer, clang-proto-fuzzer requires the following
cmake flags:

• CMAKE_C_COMPILER=clang
• CMAKE_CXX_COMPILER=clang++
• LLVM_USE_SANITIZE_COVERAGE=YES // needed for libFuzzer
• LLVM_USE_SANITIZER=Address // needed for libFuzzer

In addition, clang-proto-fuzzer requires:

• CLANG_ENABLE_PROTO_FUZZER=ON

clang-proto-fuzzer also requires the following dependencies:

• binutils // needed for libprotobuf-mutator
• liblzma-dev // needed for libprotobuf-mutator
• libz-dev // needed for libprotobuf-mutator
• docbook2x // needed for libprotobuf-mutator
• Recent version of protobuf [3.3.0 is known to work]

A working version of libprotobuf-mutator will automatically be
downloaded and built as an external project.

Implementation of clang-proto-fuzzer provided by Kostya
Serebryany.

https://bugs.llvm.org/show_bug.cgi?id=33829

[thakis]

Why should this be part of llvm? This seems to come with very heavy dependencies (protobuf), and LLVM has historically tried to minimize the number of things it depends on.

[kcc]

In D36324#832271, @thakis wrote:

Why should this be part of llvm? This seems to come with very heavy dependencies (protobuf), and LLVM has historically tried to minimize the number of things it depends on.

This fuzzer has already uncovered a few llvm bugs, so I hope it can be useful directly.
But more than that, I hope that a ready-to-use integration with structure-aware fuzzing will allow other researchers to experiment.
Having it as a side patch (bit-rotten in a few weeks after creation) will discourage most of the potential researchers from experiments.

I agree we don't want to bring heavy deps to LLVM, but this patch (AFAICT) doesn't bring any new deps to the default build. (at least this is the intent)

[vitalybuka]

clang/cmake/modules/ProtobufMutator.cmake

13 ↗

(On Diff #109761)

Why this is debug?

clang/tools/clang-fuzzer/CMakeLists.txt

12 ↗

(On Diff #109761)

You already download mutator, so maybe just DOWNLOAD_PROTOBUF and simplify this piece?

clang/tools/clang-fuzzer/ClangFuzzer.cpp

20 ↗

(On Diff #109761)

Do we want replace this fuzzer? Why not just add another one?

clang/tools/clang-fuzzer/cxx_proto.proto

93 ↗

(On Diff #109761)

message CxxInput {

required Function f = 1;
required int/enum opt_level = 2;
required enum tripple = 3;
required scalar-evolution-max-arith-depth ...

}

clang/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp

22 ↗

(On Diff #109761)

Not sure that macro here is justified
could you please replace with
std::ostream &operator<<(std::ostream &os, const BinaryOp& x) {

...

[morehouse]

clang/cmake/modules/ProtobufMutator.cmake

13 ↗

(On Diff #109761)

I was just using what the libprotobuf-mutator readme suggested. But I can change it to use CMAKE_BUILD_TYPE instead.

clang/tools/clang-fuzzer/CMakeLists.txt

12 ↗

(On Diff #109761)

That would be simpler if only protobuf-mutator needed protobuf. But since we need protobuf for some of the source files here, it would actually make this CMakeLists.txt more complicated since it would have to fish for the paths where protobuf mutator builds protobuf and then redefine variables.

clang/tools/clang-fuzzer/ClangFuzzer.cpp

20 ↗

(On Diff #109761)

The idea was to keep ClangFuzzer and add ClangProtoFuzzer alongside it. However, the two share a fair amount of code, which was factored out into HandleCXX.

clang/tools/clang-fuzzer/cxx_proto.proto

93 ↗

(On Diff #109761)

Interesting idea. This would allow for protobuf-mutator to choose different option combinations, if I understand correctly.

Is that worth adding to this initial patch, though?

[vitalybuka]

clang/tools/clang-fuzzer/CMakeLists.txt

12 ↗

(On Diff #109761)

It's ok to share code, but I don't see fuzzer with accept string as is.

clang/tools/clang-fuzzer/cxx_proto.proto

16 ↗

(On Diff #109761)

I'd suggest proto3

93 ↗

(On Diff #109761)

yes, instead of CXX_FUZZ_MORE

[morehouse]

clang/tools/clang-fuzzer/ClangFuzzer.cpp

20 ↗

(On Diff #109761)

It's ok to share code, but I don't see fuzzer with accept string as is.

That's exactly what this fuzzer is doing.

[vitalybuka]

clang/tools/clang-fuzzer/ClangFuzzer.cpp

20 ↗

(On Diff #109761)

my mistake, I see now.

clang/tools/clang-fuzzer/cxx_proto.proto

16 ↗

(On Diff #109761)

proto3 has no required, to avoid backward compatibility issues.
Same is useful for us, we don't wont to discard corpus if we drop some field in the future.

[kcc]

+bogner@ FYI

clang/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp

25 ↗

(On Diff #109761)

this is debug code, not worth having here, plz remove

34 ↗

(On Diff #109761)

Remove "-mllvm", "-scalar-evolution-max-arith-depth=4".
It's there as a workaround for a performance bug (https://bugs.llvm.org/show_bug.cgi?id=33494) but it shouldn't be here.

35 ↗

(On Diff #109761)

Remove this section.
In a later change, please allow to change the tripple (and any other flags) similar to https://reviews.llvm.org/D36275

clang/tools/clang-fuzzer/cxx_proto.proto

16 ↗

(On Diff #109761)

I'm afraid it's much more convenient to have 'required' here.
How else could you express a binary op node?

93 ↗

(On Diff #109761)

For now, keep it as is, please (see my other comment about flags)

[morehouse]

• Build protobuf-mutator with same build type as current build.
• Remove unnecessary options from clang-proto-fuzzer.
• Expand macro.

[kcc]

Why do we need LLVM_ENABLE_RTTI=ON here?

[vitalybuka]

clang/tools/clang-fuzzer/cxx_proto.proto

17 ↗

(On Diff #110111)

//option cc_api_version = 2;

Please remove

clang/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt

7 ↗

(On Diff #110111)

Formatting of this statement looks weird

clang/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp

46 ↗

(On Diff #110111)

OP looks to trivial to potentially get into conflicts with some 3rd party macro

50 ↗

(On Diff #110111)

switch (x.op()) {
    BinaryOp::PLUS: os << "+"; break
    BinaryOp::MINUS: os << "-"; break
    BinaryOp::MUL: os << "*"; break

does not look bad to me

102 ↗

(On Diff #110111)

please remove commented code

[morehouse]

In D36324#834660, @kcc wrote:

Why do we need LLVM_ENABLE_RTTI=ON here?

Attempting to build without it yields all kinds of protobuf errors. For example:

[morehouse]

• Formatting and code cleanup.

[kcc]

In D36324#835415, @morehouse wrote:

In D36324#834660, @kcc wrote:

Why do we need LLVM_ENABLE_RTTI=ON here?

Attempting to build without it yields all kinds of protobuf errors. For example:

This is very strange, I'd like to understand more about this LLVM_ENABLE_RTTI.
ideally, we should avoid it.

[morehouse]

• Define GOOGLE_PROTOBUF_NO_RTTI to remove RTTI requirement.

[kcc]

Looks good!
Now, please add a clang/tools/clang-fuzzer/README.txt describing how to build the fuzzers (both the old one and the new one) and how to run them.
For the new one explain how to install the deps

[morehouse]

• Add README.txt.

[kcc]

LGTM with a couple if nits in the README

Thanks!

clang/tools/clang-fuzzer/README.txt

11 ↗

(On Diff #110262)

.. of of Clang and LLVM

36 ↗

(On Diff #110262)

(linux-only instructions)

51 ↗

(On Diff #110262)

You may also build clang-fuzzer with this setup

[morehouse]

• Add run instructions to README.

[morehouse]

• README tweaks.

[vitalybuka]

clang/cmake/modules/ProtobufMutator.cmake

6 ↗

(On Diff #109761)

Just noticed, for cmake projects shorter syntax can be used.
Example: https://github.com/google/libprotobuf-mutator/blob/master/cmake/external/googletest.cmake

cfe/trunk/CMakeLists.txt

option(CLANG_BUILD_TOOLS
   "Build the Clang tools. If OFF, just generate build targets." ON)
 
 option(CLANG_ENABLE_ARCMT "Build ARCMT." ON)
 option(CLANG_ENABLE_STATIC_ANALYZER "Build static analyzer." ON)
 
 option(CLANG_ANALYZER_BUILD_Z3
   "Build the static analyzer with the Z3 constraint manager." OFF)
 
+option(CLANG_ENABLE_PROTO_FUZZER "Build Clang protobuf fuzzer." OFF)
+
 if(NOT CLANG_ENABLE_STATIC_ANALYZER AND (CLANG_ENABLE_ARCMT OR CLANG_ANALYZER_BUILD_Z3))
   message(FATAL_ERROR "Cannot disable static analyzer while enabling ARCMT or Z3")
 endif()
 
 if(CLANG_ANALYZER_BUILD_Z3)
   if(Z3_FOUND)
     set(CLANG_ANALYZER_WITH_Z3 1)
   else()

cfe/trunk/cmake/modules/ProtobufMutator.cmake

+set(PBM_PREFIX protobuf_mutator)
+set(PBM_PATH ${CMAKE_CURRENT_BINARY_DIR}/${PBM_PREFIX}/src/${PBM_PREFIX})
+set(PBM_LIB_PATH ${PBM_PATH}/src/libprotobuf-mutator.a)
+set(PBM_FUZZ_LIB_PATH ${PBM_PATH}/src/libfuzzer/libprotobuf-mutator-libfuzzer.a)
+
+ExternalProject_Add(${PBM_PREFIX}
+  PREFIX ${PBM_PREFIX}
+  GIT_REPOSITORY https://github.com/google/libprotobuf-mutator.git
+  GIT_TAG 34287f8
+  CONFIGURE_COMMAND ${CMAKE_COMMAND} -G${CMAKE_GENERATOR}
+    -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+    -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+    -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
+  BUILD_COMMAND ${CMAKE_MAKE_PROGRAM}
+  BUILD_BYPRODUCTS ${PBM_LIB_PATH} ${PBM_FUZZ_LIB_PATH}
+  BUILD_IN_SOURCE 1
+  INSTALL_COMMAND ""
+  LOG_DOWNLOAD 1
+  LOG_CONFIGURE 1
+  LOG_BUILD 1
+  )
+
+set(ProtobufMutator_INCLUDE_DIRS ${PBM_PATH})
+set(ProtobufMutator_LIBRARIES ${PBM_FUZZ_LIB_PATH} ${PBM_LIB_PATH})

cfe/trunk/tools/clang-fuzzer/CMakeLists.txt

 if( LLVM_USE_SANITIZE_COVERAGE )
   set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
 
+  if(CLANG_ENABLE_PROTO_FUZZER)
+    # Create protobuf .h and .cc files, and put them in a library for use by
+    # clang-proto-fuzzer components.
+    find_package(Protobuf REQUIRED)
+    add_definitions(-DGOOGLE_PROTOBUF_NO_RTTI)
+    include_directories(${PROTOBUF_INCLUDE_DIRS})
+    include_directories(${CMAKE_CURRENT_BINARY_DIR})
+    protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS cxx_proto.proto)
+    # Hack to bypass LLVM's cmake sources check and allow multiple libraries and
+    # executables from this directory.
+    set(LLVM_OPTIONAL_SOURCES
+      ClangFuzzer.cpp
+      ExampleClangProtoFuzzer.cpp
+      ${PROTO_SRCS}
+      )
+    add_clang_library(clangCXXProto
+      ${PROTO_SRCS}
+      ${PROTO_HDRS}
+
+      LINK_LIBS
+      ${PROTOBUF_LIBRARIES}
+      )
+
+    # Build and include libprotobuf-mutator
+    include(ProtobufMutator)
+    include_directories(${ProtobufMutator_INCLUDE_DIRS})
+
+    # Build the protobuf->C++ translation library and driver.
+    add_clang_subdirectory(proto-to-cxx)
+
+    # Build the protobuf fuzzer
+    add_clang_executable(clang-proto-fuzzer ExampleClangProtoFuzzer.cpp)
+    target_link_libraries(clang-proto-fuzzer
+      ${ProtobufMutator_LIBRARIES}
+      clangCXXProto
+      clangHandleCXX
+      clangProtoToCXX
+      LLVMFuzzer
+      )
+  else()
+    # Hack to bypass LLVM's cmake sources check and allow multiple libraries and
+    # executables from this directory.
+    set(LLVM_OPTIONAL_SOURCES ClangFuzzer.cpp ExampleClangProtoFuzzer.cpp)
+  endif()
+
+  add_clang_subdirectory(handle-cxx)
+
   add_clang_executable(clang-fuzzer
     EXCLUDE_FROM_ALL
     ClangFuzzer.cpp
     )
 
   target_link_libraries(clang-fuzzer
-    ${CLANG_FORMAT_LIB_DEPS}
-    clangAST
-    clangBasic
-    clangCodeGen
-    clangDriver
-    clangFrontend
-    clangRewriteFrontend
-    clangStaticAnalyzerFrontend
-    clangTooling
+    clangHandleCXX
     LLVMFuzzer
     )
 endif()

cfe/trunk/tools/clang-fuzzer/ClangFuzzer.cpp

 //===-- ClangFuzzer.cpp - Fuzz Clang --------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// \brief This file implements a function that runs Clang on a single
 ///  input. This function is then linked into the Fuzzer library.
 ///
 //===----------------------------------------------------------------------===//
 
-#include "clang/Tooling/Tooling.h"
-#include "clang/CodeGen/CodeGenAction.h"
-#include "clang/Frontend/CompilerInstance.h"
-#include "clang/Lex/PreprocessorOptions.h"
-#include "llvm/Option/Option.h"
-#include "llvm/Support/TargetSelect.h"
+#include "handle-cxx/handle_cxx.h"
 
-using namespace clang;
+using namespace clang_fuzzer;
 
 extern "C" int LLVMFuzzerTestOneInput(uint8_t *data, size_t size) {
   std::string s((const char *)data, size);
-  llvm::InitializeAllTargets();
-  llvm::InitializeAllTargetMCs();
-  llvm::InitializeAllAsmPrinters();
-  llvm::InitializeAllAsmParsers();
-
-  llvm::opt::ArgStringList CC1Args;
-  CC1Args.push_back("-cc1");
-  CC1Args.push_back("./test.cc");
-  CC1Args.push_back("-O2");
-  llvm::IntrusiveRefCntPtr<FileManager> Files(
-      new FileManager(FileSystemOptions()));
-  IgnoringDiagConsumer Diags;
-  IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
-  DiagnosticsEngine Diagnostics(
-      IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()), &*DiagOpts,
-      &Diags, false);
-  std::unique_ptr<clang::CompilerInvocation> Invocation(
-      tooling::newInvocation(&Diagnostics, CC1Args));
-  std::unique_ptr<llvm::MemoryBuffer> Input =
-      llvm::MemoryBuffer::getMemBuffer(s);
-  Invocation->getPreprocessorOpts().addRemappedFile("./test.cc", Input.release());
-  std::unique_ptr<tooling::ToolAction> action(
-      tooling::newFrontendActionFactory<clang::EmitObjAction>());
-  std::shared_ptr<PCHContainerOperations> PCHContainerOps =
-      std::make_shared<PCHContainerOperations>();
-  action->runInvocation(std::move(Invocation), Files.get(), PCHContainerOps,
-                        &Diags);
+  HandleCXX(s, {"-O2"});
   return 0;
 }

cfe/trunk/tools/clang-fuzzer/ExampleClangProtoFuzzer.cpp

+//===-- ExampleClangProtoFuzzer.cpp - Fuzz Clang --------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief This file implements a function that runs Clang on a single
+///  input and uses libprotobuf-mutator to find new inputs. This function is
+///  then linked into the Fuzzer library.
+///
+//===----------------------------------------------------------------------===//
+
+#include "cxx_proto.pb.h"
+#include "handle-cxx/handle_cxx.h"
+#include "proto-to-cxx/proto_to_cxx.h"
+
+#include "src/libfuzzer/libfuzzer_macro.h"
+
+using namespace clang_fuzzer;
+
+DEFINE_BINARY_PROTO_FUZZER(const Function& input) {
+  auto S = FunctionToString(input);
+  HandleCXX(S, {"-O2"});
+}

cfe/trunk/tools/clang-fuzzer/README.txt

+This directory contains two utilities for fuzzing Clang: clang-fuzzer and
+clang-proto-fuzzer.  Both use libFuzzer to generate inputs to clang via
+coverage-guided mutation.
+
+The two utilities differ, however, in how they structure inputs to Clang.
+clang-fuzzer makes no attempt to generate valid C++ programs and is therefore
+primarily useful for stressing the surface layers of Clang (i.e. lexer, parser).
+clang-proto-fuzzer uses a protobuf class to describe a subset of the C++
+language and then uses libprotobuf-mutator to mutate instantiations of that
+class, producing valid C++ programs in the process.  As a result,
+clang-proto-fuzzer is better at stressing deeper layers of Clang and LLVM.
+
+===================================
+ Building clang-fuzzer
+===================================
+Within your LLVM build directory, run CMake with the following variable
+definitions:
+- CMAKE_C_COMPILER=clang
+- CMAKE_CXX_COMPILER=clang++
+- LLVM_USE_SANITIZE_COVERAGE=YES
+- LLVM_USE_SANITIZER=Address
+
+Then build the clang-fuzzer target.
+
+Example:
+  cd $LLVM_SOURCE_DIR
+  mkdir build && cd build
+  cmake .. -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
+    -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address
+  ninja clang-fuzzer
+
+
+=======================================================
+ Building clang-proto-fuzzer (Linux-only instructions)
+=======================================================
+Install the necessary dependencies:
+- binutils  // needed for libprotobuf-mutator
+- liblzma-dev  // needed for libprotobuf-mutator
+- libz-dev  // needed for libprotobuf-mutator
+- docbook2x  // needed for libprotobuf-mutator
+- Recent version of protobuf [3.3.0 is known to work]
+
+Within your LLVM build directory, run CMake with the following variable
+definitions:
+- CMAKE_C_COMPILER=clang
+- CMAKE_CXX_COMPILER=clang++
+- LLVM_USE_SANITIZE_COVERAGE=YES
+- LLVM_USE_SANITIZER=Address
+- CLANG_ENABLE_PROTO_FUZZER=ON
+
+Then build the clang-proto-fuzzer and clang-proto-to-cxx targets.  Optionally,
+you may also build clang-fuzzer with this setup.
+
+Example:
+  cd $LLVM_SOURCE_DIR
+  mkdir build && cd build
+  cmake .. -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
+    -DLLVM_USE_SANITIZE_COVERAGE=YES -DLLVM_USE_SANITIZER=Address \
+    -DCLANG_ENABLE_PROTO_FUZZER=ON
+  ninja clang-proto-fuzzer clang-proto-to-cxx
+
+
+=====================
+ Running the fuzzers
+=====================
+clang-fuzzer:
+  bin/clang-fuzzer CORPUS_DIR
+
+clang-proto-fuzzer:
+  bin/clang-proto-fuzzer CORPUS_DIR
+
+Translating a clang-proto-fuzzer corpus output to C++:
+  bin/clang-proto-to-cxx CORPUS_OUTPUT_FILE

cfe/trunk/tools/clang-fuzzer/cxx_proto.proto

+//===-- cxx_proto.proto - Protobuf description of C++ ---------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief This file describes a subset of C++ as a protobuf.  It is used to
+///  more easily find interesting inputs for fuzzing Clang.
+///
+//===----------------------------------------------------------------------===//
+
+syntax = "proto2";
+
+message VarRef {
+  required int32 varnum = 1;
+}
+
+message Lvalue {
+  required VarRef varref = 1;
+}
+
+message Const {
+  required int32 val = 1;
+}
+
+message BinaryOp {
+  enum Op {
+    PLUS = 0;
+    MINUS = 1;
+    MUL = 2;
+    DIV = 3;
+    MOD = 4;
+    XOR = 5;
+    AND = 6;
+    OR = 7;
+    EQ = 8;
+    NE = 9;
+    LE = 10;
+    GE = 11;
+    LT = 12;
+    GT = 13;
+  };
+  required Op op = 1;
+  required Rvalue left = 2;
+  required Rvalue right = 3;
+}
+
+message Rvalue {
+  oneof rvalue_oneof {
+    VarRef varref = 1;
+    Const cons = 2;
+    BinaryOp binop = 3;
+  }
+}
+
+message AssignmentStatement {
+  required Lvalue lvalue = 1;
+  required Rvalue rvalue = 2;
+}
+
+
+message IfElse {
+  required Rvalue cond = 1;
+  required StatementSeq if_body = 2;
+  required StatementSeq else_body = 3;
+}
+
+message While {
+  required Rvalue cond = 1;
+  required StatementSeq body = 2;
+}
+
+message Statement {
+  oneof stmt_oneof {
+    AssignmentStatement assignment = 1;
+    IfElse              ifelse     = 2;
+    While               while_loop = 3;
+  }
+}
+
+message StatementSeq {
+  repeated Statement statements = 1;
+}
+
+message Function {
+  required StatementSeq statements = 1;
+}
+
+package clang_fuzzer;

cfe/trunk/tools/clang-fuzzer/handle-cxx/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
+
+add_clang_library(clangHandleCXX
+  handle_cxx.cpp
+
+  LINK_LIBS
+  clangCodeGen
+  clangFrontend
+  clangLex
+  clangTooling
+  )

cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.h

+//==-- handle_cxx.h - Helper function for Clang fuzzers --------------------==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Defines HandleCXX for use by the Clang fuzzers.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H
+#define LLVM_CLANG_TOOLS_CLANG_FUZZER_HANDLE_CXX_HANDLECXX_H
+
+#include <string>
+#include <vector>
+
+namespace clang_fuzzer {
+void HandleCXX(const std::string &S,
+               const std::vector<const char *> &ExtraArgs);
+} // namespace clang_fuzzer
+
+#endif

cfe/trunk/tools/clang-fuzzer/handle-cxx/handle_cxx.cpp

+//==-- handle_cxx.cpp - Helper function for Clang fuzzers ------------------==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements HandleCXX for use by the Clang fuzzers.
+//
+//===----------------------------------------------------------------------===//
+
+#include "handle_cxx.h"
+
+#include "clang/CodeGen/CodeGenAction.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Lex/PreprocessorOptions.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Support/TargetSelect.h"
+
+using namespace clang;
+
+void clang_fuzzer::HandleCXX(const std::string &S,
+                             const std::vector<const char *> &ExtraArgs) {
+  llvm::InitializeAllTargets();
+  llvm::InitializeAllTargetMCs();
+  llvm::InitializeAllAsmPrinters();
+  llvm::InitializeAllAsmParsers();
+
+  llvm::opt::ArgStringList CC1Args;
+  CC1Args.push_back("-cc1");
+  for (auto &A : ExtraArgs)
+    CC1Args.push_back(A);
+  CC1Args.push_back("./test.cc");
+
+  llvm::IntrusiveRefCntPtr<FileManager> Files(
+      new FileManager(FileSystemOptions()));
+  IgnoringDiagConsumer Diags;
+  IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
+  DiagnosticsEngine Diagnostics(
+      IntrusiveRefCntPtr<clang::DiagnosticIDs>(new DiagnosticIDs()), &*DiagOpts,
+      &Diags, false);
+  std::unique_ptr<clang::CompilerInvocation> Invocation(
+      tooling::newInvocation(&Diagnostics, CC1Args));
+  std::unique_ptr<llvm::MemoryBuffer> Input =
+      llvm::MemoryBuffer::getMemBuffer(S);
+  Invocation->getPreprocessorOpts().addRemappedFile("./test.cc",
+                                                    Input.release());
+  std::unique_ptr<tooling::ToolAction> action(
+      tooling::newFrontendActionFactory<clang::EmitObjAction>());
+  std::shared_ptr<PCHContainerOperations> PCHContainerOps =
+      std::make_shared<PCHContainerOperations>();
+  action->runInvocation(std::move(Invocation), Files.get(), PCHContainerOps,
+                        &Diags);
+}
+

cfe/trunk/tools/clang-fuzzer/proto-to-cxx/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD})
+
+# Hack to bypass LLVM's CMake source checks so we can have both a library and
+# an executable built from this directory.
+set(LLVM_OPTIONAL_SOURCES proto_to_cxx.cpp proto_to_cxx_main.cpp)
+
+add_clang_library(clangProtoToCXX proto_to_cxx.cpp LINK_LIBS clangCXXProto)
+
+add_clang_executable(clang-proto-to-cxx proto_to_cxx_main.cpp)
+target_link_libraries(clang-proto-to-cxx clangProtoToCXX)

cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.h

+//==-- proto_to_cxx.h - Protobuf-C++ conversion ----------------------------==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Defines functions for converting between protobufs and C++.
+//
+//===----------------------------------------------------------------------===//
+
+#include <cstdint>
+#include <cstddef>
+#include <string>
+
+namespace clang_fuzzer {
+class Function;
+std::string FunctionToString(const Function &input);
+std::string ProtoToCxx(const uint8_t *data, size_t size);
+}

cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx.cpp

+//==-- proto_to_cxx.cpp - Protobuf-C++ conversion --------------------------==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements functions for converting between protobufs and C++.
+//
+//===----------------------------------------------------------------------===//
+
+#include "proto_to_cxx.h"
+#include "cxx_proto.pb.h"
+
+#include <ostream>
+#include <sstream>
+
+namespace clang_fuzzer {
+
+// Forward decls.
+std::ostream &operator<<(std::ostream &os, const BinaryOp &x);
+std::ostream &operator<<(std::ostream &os, const StatementSeq &x);
+
+// Proto to C++.
+std::ostream &operator<<(std::ostream &os, const Const &x) {
+  return os << "(" << x.val() << ")";
+}
+std::ostream &operator<<(std::ostream &os, const VarRef &x) {
+  return os << "a[" << (static_cast<uint32_t>(x.varnum()) % 100) << "]";
+}
+std::ostream &operator<<(std::ostream &os, const Lvalue &x) {
+  return os << x.varref();
+}
+std::ostream &operator<<(std::ostream &os, const Rvalue &x) {
+    if (x.has_varref()) return os << x.varref();
+    if (x.has_cons())   return os << x.cons();
+    if (x.has_binop())  return os << x.binop();
+    return os << "1";
+}
+std::ostream &operator<<(std::ostream &os, const BinaryOp &x) {
+  os << "(" << x.left();
+  switch (x.op()) {
+    case BinaryOp::PLUS: os << "+"; break;
+    case BinaryOp::MINUS: os << "-"; break;
+    case BinaryOp::MUL: os << "*"; break;
+    case BinaryOp::DIV: os << "/"; break;
+    case BinaryOp::MOD: os << "%"; break;
+    case BinaryOp::XOR: os << "^"; break;
+    case BinaryOp::AND: os << "&"; break;
+    case BinaryOp::OR: os << "|"; break;
+    case BinaryOp::EQ: os << "=="; break;
+    case BinaryOp::NE: os << "!="; break;
+    case BinaryOp::LE: os << "<="; break;
+    case BinaryOp::GE: os << ">="; break;
+    case BinaryOp::LT: os << "<"; break;
+    case BinaryOp::GT: os << ">"; break;
+  }
+  return os << x.right() << ")";
+}
+std::ostream &operator<<(std::ostream &os, const AssignmentStatement &x) {
+  return os << x.lvalue() << "=" << x.rvalue() << ";\n";
+}
+std::ostream &operator<<(std::ostream &os, const IfElse &x) {
+  return os << "if (" << x.cond() << "){\n"
+            << x.if_body() << "} else { \n"
+            << x.else_body() << "}\n";
+}
+std::ostream &operator<<(std::ostream &os, const While &x) {
+  return os << "while (" << x.cond() << "){\n" << x.body() << "}\n";
+}
+std::ostream &operator<<(std::ostream &os, const Statement &x) {
+  if (x.has_assignment()) return os << x.assignment();
+  if (x.has_ifelse())     return os << x.ifelse();
+  if (x.has_while_loop()) return os << x.while_loop();
+  return os << "(void)0;\n";
+}
+std::ostream &operator<<(std::ostream &os, const StatementSeq &x) {
+  for (auto &st : x.statements()) os << st;
+  return os;
+}
+std::ostream &operator<<(std::ostream &os, const Function &x) {
+  return os << "void foo(int *a) {\n" << x.statements() << "}\n";
+}
+
+// ---------------------------------
+
+std::string FunctionToString(const Function &input) {
+  std::ostringstream os;
+  os << input;
+  return os.str();
+
+}
+std::string ProtoToCxx(const uint8_t *data, size_t size) {
+  Function message;
+  if (!message.ParseFromArray(data, size))
+    return "#error invalid proto\n";
+  return FunctionToString(message);
+}
+
+} // namespace clang_fuzzer

cfe/trunk/tools/clang-fuzzer/proto-to-cxx/proto_to_cxx_main.cpp

+//==-- proto_to_cxx_main.cpp - Driver for protobuf-C++ conversion ----------==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements a simple driver to print a C++ program from a protobuf.
+//
+//===----------------------------------------------------------------------===//
+#include <fstream>
+#include <iostream>
+#include <streambuf>
+#include <string>
+
+#include "proto_to_cxx.h"
+
+int main(int argc, char **argv) {
+  for (int i = 1; i < argc; i++) {
+    std::fstream in(argv[i]);
+    std::string str((std::istreambuf_iterator<char>(in)),
+                    std::istreambuf_iterator<char>());
+    std::cout << "// " << argv[i] << std::endl;
+    std::cout << clang_fuzzer::ProtoToCxx(
+        reinterpret_cast<const uint8_t *>(str.data()), str.size());
+  }
+}
+