Updated llvm-proto-fuzzer to execute the compiled code

[emmettneyman]

Summary

Made changes to the llvm-proto-fuzzer

• Added loop vectorizer optimization pass in order to have two IR versions
• Updated old fuzz target to handle two different IR versions
• Wrote code to execute both versions in memory

[emmettneyman]

The files

Object.h
Object.cpp
llvm-objcopy.h

are from llvm/tools/llvm-obj-copy with only slight modifications, mostly deleting irrelevant parts.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

209 ↗

(On Diff #156194)

Did you look at using LLVM's JIT infrastructure to do this?

[morehouse]

You can probably get rid of the llvm-objcopy code and make this a lot simpler with something like:

1. Call getSection() on the Binary object to get the text section.
2. Read the sh_offset and sh_size of that section.
3. Copy sh_size bytes from the start of the binary buffer + sh_offset into your executable memory.
4. Run it.

[emmettneyman]

• Switched to JIT for compilation and execution

[emmettneyman]

• Cleaned up leftover code from mmap memcpy

[emmettneyman]

• Fixed typo that broke build

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

21 ↗

(On Diff #156370)

How are you doing your diff? Some of these changes are already upstream. Please rebase

clang/tools/clang-fuzzer/handle-llvm/fuzzer_initialize.cpp

52 ↗

(On Diff #156370)

Can we use fuzzer-initialize/ now?

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

54 ↗

(On Diff #156370)

Unless there's a good reason for global, let's make this a local.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

211–224 ↗

(On Diff #156370)

Can you move this code (and maybe more of the duplicated code below) into a function and call it twice?

[emmettneyman]

Made fixes to patch, rebased CMake file

[emmettneyman]

Cleaned up code

Tried to get rid of ParseCommandLineOptions() call but could not figure out
how to initialize a PassInfo object without it.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

17 ↗

(On Diff #157138)

Typo introduced here.

25 ↗

(On Diff #157138)

Please don't add whitespace to a previously empty line.

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

10 ↗

(On Diff #157138)

an -> a

112 ↗

(On Diff #157138)

Maybe this should be const std::string &IR or StringRef.

113 ↗

(On Diff #157138)

Does this need to be called every time we get a new input? Can we call this from LLVMFuzzerInitialize()?

123 ↗

(On Diff #157138)

This shouldn't be required. You should be able to directly add the passes you want. See llvm/lib/Passes/PassBuilder.cpp.

132 ↗

(On Diff #157138)

Move this closer to where it's used.

133 ↗

(On Diff #157138)

Options is never used.

136 ↗

(On Diff #157138)

TM is never used.

137 ↗

(On Diff #157138)

Does this do anything since CPUStr and FeaturesStr are empty?

145 ↗

(On Diff #157138)

Can we just make FPasses on stack instead?

190 ↗

(On Diff #157138)

These 3 lines can be combined to builder.setMCJITMemoryManager(new SectionMemoryManager())

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190 ↗

(On Diff #157138)

I use RTDyldMM on line 208. Should I just keep these lines as they are?

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190 ↗

(On Diff #157138)

Ah, missed that. In that case, you can probably simplify this line to
builder.setMCJITMemoryManager(RTDyldMM).

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190 ↗

(On Diff #157138)

It looks like set MCJITMemoryManager() needs to take a unique_ptr. I'm not sure how to clean it up any more than it already is.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190 ↗

(On Diff #157138)

Does it not implicitly cast?

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190 ↗

(On Diff #157138)

No, I think it only works in the other direction.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190 ↗

(On Diff #157138)

Ok, I see. The constructor we need is marked explicit...

208 ↗

(On Diff #157138)

This cast shouldn't be necessary.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

208 ↗

(On Diff #157138)

Turns out this line is redundant anyways. EE->finalizeObject()calls invalidateInstructionCache().

[emmettneyman]

• cleaned up code and moved initialization code
• removed fake command line parsing

[morehouse]

clang/tools/clang-fuzzer/fuzzer-initialize/fuzzer_initialize.cpp

44 ↗

(On Diff #157335)

Unnecessary llvm::

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

89 ↗

(On Diff #157335)

If we do this, do we need to explicitly add the vectorizer pass below?

95 ↗

(On Diff #157335)

OLvl is never used.

105 ↗

(On Diff #157335)

Let's simplify to setFunctionAttributes(getCPUStr(), getFeaturesStr(), *M).

110 ↗

(On Diff #157335)

Can simplify to Passes.add(new TargetLibraryInfoWrapperPass(M->getTargetTriple)).

115 ↗

(On Diff #157335)

Why do we add a TargetTransformInfoWrapperPass to both Passes and FPasses?

115 ↗

(On Diff #157335)

Do we need both Passes and FPasses?

121 ↗

(On Diff #157335)

Let's simplify to Passes.add(createLoopVectorizePass()).

127 ↗

(On Diff #157335)

Why do we keep adding passes in different places?

144 ↗

(On Diff #157335)

Why not just rename Owner to M and remove this line?

158 ↗

(On Diff #157335)

Please simplify to builder.setMCJITMemoryManager(std::make_unique<RTDyldMemoryManager>()).

159 ↗

(On Diff #157335)

If the JIT does optimization already, do we need to call OptLLVM at all? Will the vectorizer kick in without OptLLVM?

168 ↗

(On Diff #157335)

Move this closer to where it's used.

171 ↗

(On Diff #157335)

Do we need to call this again to run destructors after we execute f() ?

181 ↗

(On Diff #157335)

I think f(a, b, c, 1) will also work.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

89 ↗

(On Diff #157335)

No we don't. I've deleted the line below.

110 ↗

(On Diff #157335)

Contrary to the function's name, getTargetTriple() actually returns a string, not a Triple. But I changed it to new TargetLibraryInfoWrapperPass(ModuleTriple) and deleted line 109.

115 ↗

(On Diff #157335)

I think because we're just adding a loop vectorize pass, we don't need FPasses. The loop vectorize pass lives within the regular ModulePassManager, not the FunctionPassManager. I decided to put it in the code so, in the future, it would be easier to add different kinds of passes to the fuzz target. Should I remove it?

159 ↗

(On Diff #157335)

I'll look into this more. I couldn't find an answer quickly.

[emmettneyman]

Fixed some things, made code cleaner

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

159 ↗

(On Diff #157335)

No, the JIT doesn't run any optimization passes. The optimization level given just controls CodeGen.

Source: http://lists.llvm.org/pipermail/llvm-dev/2016-May/099631.html

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

135 ↗

(On Diff #157388)

The indentation looks off. Please fix.

147 ↗

(On Diff #157388)

This uses llvm:make_unique, which was written when LLVM used C++11. But since LLVM now uses C++14, I think we should use std::make_unique instead.

169 ↗

(On Diff #157388)

Remove whitespace on empty lines.

Actually, you could probably remove this line altogether since the call to f() can be grouped with the parameter setup above.

184 ↗

(On Diff #157388)

We're allowing the JIT opt-level to be specified on the command line, but we're hard-coding OptLevel=3 for the optimization passes.

Do we need to allow the opt-level to be specified on the command line?

115 ↗

(On Diff #157335)

Let's remove it since it's currently unnecessary.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

147 ↗

(On Diff #157388)

LLVM doesn't use C++14 yet.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

147 ↗

(On Diff #157388)

We talked offline, but just to put it in writing: My current LLVM build uses C++11 so I'm keeping it llvm::make_unique.

184 ↗

(On Diff #157388)

Good point. Will change to make OptLLVM() use the same optimization level as the JIT Engine.

144 ↗

(On Diff #157335)

Reverted it back since the change caused the fuzzer to crash.

[emmettneyman]

• Code style fixes
• Removed FPasses
• Allowed CL Args to specify opt level for OptLLVM()

[emmettneyman]

Small change to fix line length

[morehouse]

Do we need to parse the arguments for opt-level, or can we just hardcode -O2 and remove the argument parsing code?

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

90 ↗

(On Diff #157545)

Shouldn't OLvl be a CodeGenOpt::Level?

144 ↗

(On Diff #157335)

You will need to move any uses of M above the call to std::move, since that leaves M in an invalid state.

[emmettneyman]

In D49526#1177208, @morehouse wrote:

Do we need to parse the arguments for opt-level, or can we just hardcode -O2 and remove the argument parsing code?

I have the argument parsing code since the original clang-proto-fuzzer code had it and @kcc mentioned we might want to change the command line arguments in the future.

[emmettneyman]

Changed int to CodeGenOpt::Level and fixed unique_ptr issue

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

125 ↗

(On Diff #157553)

We should be able to get rid of this line now, and rename Owner again.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

152 ↗

(On Diff #157545)

Can we use reinterpret_cast here?

[emmettneyman]

Made some minor fixes

cfe/trunk/tools/clang-fuzzer/fuzzer-initialize/fuzzer_initialize.cpp

 /// This file implements two functions: one that returns the command line
 /// arguments for a given call to the fuzz target and one that initializes
 /// the fuzzer with the correct command line arguments.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "fuzzer_initialize.h"
 
+#include "llvm/InitializePasses.h"
+#include "llvm/PassRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include <cstring>
 
 using namespace clang_fuzzer;
+using namespace llvm;
 
 
 namespace clang_fuzzer {
 
 static std::vector<const char *> CLArgs;
 
 const std::vector<const char *>& GetCLArgs() {
   return CLArgs;
 }
 
 }
 
 extern "C" int LLVMFuzzerInitialize(int *argc, char ***argv) {
-  llvm::InitializeAllTargets();
-  llvm::InitializeAllTargetMCs();
-  llvm::InitializeAllAsmPrinters();
-  llvm::InitializeAllAsmParsers();
+  InitializeAllTargets();
+  InitializeAllTargetMCs();
+  InitializeAllAsmPrinters();
+  InitializeAllAsmParsers();
+  
+  PassRegistry &Registry = *PassRegistry::getPassRegistry();
+  initializeCore(Registry);
+  initializeScalarOpts(Registry);
+  initializeVectorization(Registry);
+  initializeIPO(Registry);
+  initializeAnalysis(Registry);
+  initializeTransformUtils(Registry);
+  initializeInstCombine(Registry);
+  initializeAggressiveInstCombine(Registry);
+  initializeInstrumentation(Registry);
+  initializeTarget(Registry);
 
   CLArgs.push_back("-O2");
   for (int I = 1; I < *argc; I++) {
     if (strcmp((*argv)[I], "-ignore_remaining_args=1") == 0) {
       for (I++; I < *argc; I++)
         CLArgs.push_back((*argv)[I]);
       break;
     }
   }
   return 0;
 }

cfe/trunk/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
+  CodeGen
   Core
+  ExecutionEngine
   IRReader
   MC
+  MCJIT
+  Object
+  RuntimeDyld
+  SelectionDAG
   Support
-  Analysis
-  )
+  Target
+  TransformUtils
+  native
 
 # Depend on LLVM IR intrinsic generation.
 set(handle_llvm_deps intrinsics_gen)
 if (CLANG_BUILT_STANDALONE)
   set(handle_llvm_deps)
 endif()
 
 add_clang_library(clangHandleLLVM
   handle_llvm.cpp
 
   DEPENDS
   ${handle_llvm_deps}
   )

cfe/trunk/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

 //==-- handle_llvm.cpp - Helper function for Clang fuzzers -----------------==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
-// Implements HandleLLVM for use by the Clang fuzzers. Mimics the llc tool to
-// compile an LLVM IR file to X86_64 assembly.
+// Implements HandleLLVM for use by the Clang fuzzers. First runs a loop
+// vectorizer optimization pass over the given IR code. Then mimics lli on both
+// versions to JIT the generated code and execute it. Currently, functions are 
+// executed on dummy inputs.
 //
 //===----------------------------------------------------------------------===//
 
 #include "handle_llvm.h"
 
 #include "llvm/ADT/Triple.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/CodeGen/CommandFlags.inc"
 #include "llvm/CodeGen/MachineModuleInfo.h"
+#include "llvm/CodeGen/TargetPassConfig.h"
+#include "llvm/ExecutionEngine/JITEventListener.h"
+#include "llvm/ExecutionEngine/JITSymbol.h"
+#include "llvm/ExecutionEngine/MCJIT.h"
+#include "llvm/ExecutionEngine/ObjectCache.h"
+#include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
+#include "llvm/ExecutionEngine/SectionMemoryManager.h"
+#include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/LegacyPassNameParser.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/IRReader/IRReader.h"
+#include "llvm/Pass.h"
 #include "llvm/PassRegistry.h"
-#include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/TargetSelect.h"
 #include "llvm/Target/TargetMachine.h"
-
-#include <cstdlib>
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+#include "llvm/Transforms/IPO.h"
+#include "llvm/Transforms/Vectorize.h"
 
 using namespace llvm;
 
+// Helper function to parse command line args and find the optimization level
 static void getOptLevel(const std::vector<const char *> &ExtraArgs,
                               CodeGenOpt::Level &OLvl) {
   // Find the optimization level from the command line args
   OLvl = CodeGenOpt::Default;
   for (auto &A : ExtraArgs) {
     if (A[0] == '-' && A[1] == 'O') {
       switch(A[2]) {
         case '0': OLvl = CodeGenOpt::None; break;
         case '1': OLvl = CodeGenOpt::Less; break;
         case '2': OLvl = CodeGenOpt::Default; break;
         case '3': OLvl = CodeGenOpt::Aggressive; break;
         default:
           errs() << "error: opt level must be between 0 and 3.\n";
           std::exit(1);
       }
     }
   }
 }
 
-void clang_fuzzer::HandleLLVM(const std::string &S,
-                              const std::vector<const char *> &ExtraArgs) {
-  // Parse ExtraArgs to set the optimization level
-  CodeGenOpt::Level OLvl;
-  getOptLevel(ExtraArgs, OLvl);
+void ErrorAndExit(std::string message) {
+  errs()<< "ERROR: " << message << "\n";
+  std::exit(1);
+}
 
-  // Set the Module to include the the IR code to be compiled
+// Helper function to add optimization passes to the TargetMachine at the 
+// specified optimization level, OptLevel
+static void AddOptimizationPasses(legacy::PassManagerBase &MPM,
+                                  CodeGenOpt::Level OptLevel,
+                                  unsigned SizeLevel) {
+  // Create and initialize a PassManagerBuilder
+  PassManagerBuilder Builder;
+  Builder.OptLevel = OptLevel;
+  Builder.SizeLevel = SizeLevel;
+  Builder.Inliner = createFunctionInliningPass(OptLevel, SizeLevel, false);
+  Builder.LoopVectorize = true;
+  Builder.populateModulePassManager(MPM);
+}
+
+// Mimics the opt tool to run an optimization pass over the provided IR
+std::string OptLLVM(const std::string &IR, CodeGenOpt::Level OLvl) {
+  // Create a module that will run the optimization passes
   SMDiagnostic Err;
-
   LLVMContext Context;
-  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(S, "IR"), Err, Context);
-  if (!M) {
-    errs() << "error: could not parse IR!\n";
-    std::exit(1);
-  }
+  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(IR, "IR"), Err, Context);
+  if (!M || verifyModule(*M, &errs()))
+    ErrorAndExit("Could not parse IR");
 
-  // Create a new Target
-  std::string Error;
-  const Target *TheTarget = TargetRegistry::lookupTarget(
-      sys::getDefaultTargetTriple(), Error);
-  if (!TheTarget) {
-    errs() << Error;
-    std::exit(1);
-  }
+  setFunctionAttributes(getCPUStr(), getFeaturesStr(), *M);
   
-  TargetOptions Options = InitTargetOptionsFromCodeGenFlags();
+  legacy::PassManager Passes;
+  Triple ModuleTriple(M->getTargetTriple());
   
-  // Create a new Machine
-  std::string CPUStr = getCPUStr();
-  std::string FeaturesStr = getFeaturesStr();
-  std::unique_ptr<TargetMachine> Target(TheTarget->createTargetMachine(
-      sys::getDefaultTargetTriple(), CPUStr, FeaturesStr, Options,
-      getRelocModel(), getCodeModel(), OLvl));
-
-  // Create a new PassManager
-  legacy::PassManager PM;
-  TargetLibraryInfoImpl TLII(Triple(M->getTargetTriple()));
-  PM.add(new TargetLibraryInfoWrapperPass(TLII));
-  M->setDataLayout(Target->createDataLayout());
- 
-  // Make sure the Module has no errors
-  if (verifyModule(*M, &errs())) {
-    errs() << "error: input module is broken!\n";
-    std::exit(1);
-  } 
+  Passes.add(new TargetLibraryInfoWrapperPass(ModuleTriple));
+  Passes.add(createTargetTransformInfoWrapperPass(TargetIRAnalysis()));
+  Passes.add(createVerifierPass());
+
+  AddOptimizationPasses(Passes, OLvl, 0);
+  
+  // Add a pass that writes the optimized IR to an output stream
+  std::string outString;
+  raw_string_ostream OS(outString);
+  Passes.add(createPrintModulePass(OS, "", false));
+
+  Passes.run(*M);
+  return OS.str();
+}
 
-  setFunctionAttributes(CPUStr, FeaturesStr, *M);
-  
-  raw_null_ostream OS;
-  Target->addPassesToEmitFile(PM, OS, nullptr, TargetMachine::CGFT_ObjectFile,
-                              false);
-  PM.run(*M);
+void CreateAndRunJITFun(const std::string &IR, CodeGenOpt::Level OLvl) {
+  SMDiagnostic Err;
+  LLVMContext Context;
+  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(IR, "IR"), Err,
+                                          Context);
+  if (!M)
+    ErrorAndExit("Could not parse IR");
+
+  Function *EntryFunc = M->getFunction("foo");
+  if (!EntryFunc)
+    ErrorAndExit("Function not found in module");
+
+  std::string ErrorMsg;
+  EngineBuilder builder(std::move(M));
+  builder.setMArch(MArch);
+  builder.setMCPU(getCPUStr());
+  builder.setMAttrs(getFeatureList());
+  builder.setErrorStr(&ErrorMsg);
+  builder.setEngineKind(EngineKind::JIT);
+  builder.setUseOrcMCJITReplacement(false);
+  builder.setMCJITMemoryManager(make_unique<SectionMemoryManager>());
+  builder.setOptLevel(OLvl);
+  builder.setTargetOptions(InitTargetOptionsFromCodeGenFlags());
+
+  std::unique_ptr<ExecutionEngine> EE(builder.create());
+  if (!EE)
+    ErrorAndExit("Could not create execution engine");
+
+  EE->finalizeObject();
+  EE->runStaticConstructorsDestructors(false);
+
+  typedef void (*func)(int*, int*, int*, int);
+  func f = reinterpret_cast<func>(EE->getPointerToFunction(EntryFunc)); 
+
+  // Define some dummy arrays to use an input for now
+  int a[] = {1};
+  int b[] = {1};
+  int c[] = {1};
+  f(a, b, c, 1);
+
+  EE->runStaticConstructorsDestructors(true);
+}
+
+// Main fuzz target called by ExampleClangLLVMProtoFuzzer.cpp
+// Mimics the lli tool to JIT the LLVM IR code and execute it
+void clang_fuzzer::HandleLLVM(const std::string &IR,
+                              const std::vector<const char *> &ExtraArgs) {
+  // Parse ExtraArgs to set the optimization level
+  CodeGenOpt::Level OLvl;
+  getOptLevel(ExtraArgs, OLvl);
+
+  // First we optimize the IR by running a loop vectorizer pass
+  std::string OptIR = OptLLVM(IR, OLvl);
+  CreateAndRunJITFun(OptIR, OLvl);
+  CreateAndRunJITFun(IR, CodeGenOpt::None);
   
   return;
 }
-