Updated llvm-proto-fuzzer to execute the compiled code

[emmettneyman]

Summary

Made changes to the llvm-proto-fuzzer

• Added loop vectorizer optimization pass in order to have two IR versions
• Updated old fuzz target to handle two different IR versions
• Wrote code to execute both versions in memory

[emmettneyman]

The files

Object.h
Object.cpp
llvm-objcopy.h

are from llvm/tools/llvm-obj-copy with only slight modifications, mostly deleting irrelevant parts.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

200

Did you look at using LLVM's JIT infrastructure to do this?

[morehouse]

You can probably get rid of the llvm-objcopy code and make this a lot simpler with something like:

1. Call getSection() on the Binary object to get the text section.
2. Read the sh_offset and sh_size of that section.
3. Copy sh_size bytes from the start of the binary buffer + sh_offset into your executable memory.
4. Run it.

[emmettneyman]

• Switched to JIT for compilation and execution

[emmettneyman]

• Cleaned up leftover code from mmap memcpy

[emmettneyman]

• Fixed typo that broke build

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

21

How are you doing your diff? Some of these changes are already upstream. Please rebase

clang/tools/clang-fuzzer/handle-llvm/fuzzer_initialize.cpp

52 ↗

(On Diff #156370)

Can we use fuzzer-initialize/ now?

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

54

Unless there's a good reason for global, let's make this a local.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

101–114

Can you move this code (and maybe more of the duplicated code below) into a function and call it twice?

[emmettneyman]

Made fixes to patch, rebased CMake file

[emmettneyman]

Cleaned up code

Tried to get rid of ParseCommandLineOptions() call but could not figure out
how to initialize a PassInfo object without it.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

17

Typo introduced here.

25

Please don't add whitespace to a previously empty line.

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

10–13

an -> a

82

Move this closer to where it's used.

83

Options is never used.

86

TM is never used.

87

Does this do anything since CPUStr and FeaturesStr are empty?

95

Can we just make FPasses on stack instead?

109

These 3 lines can be combined to builder.setMCJITMemoryManager(new SectionMemoryManager())

110

Maybe this should be const std::string &IR or StringRef.

111

Does this need to be called every time we get a new input? Can we call this from LLVMFuzzerInitialize()?

121

This shouldn't be required. You should be able to directly add the passes you want. See llvm/lib/Passes/PassBuilder.cpp.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

109

I use RTDyldMM on line 208. Should I just keep these lines as they are?

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

109

Ah, missed that. In that case, you can probably simplify this line to
builder.setMCJITMemoryManager(RTDyldMM).

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

109

It looks like set MCJITMemoryManager() needs to take a unique_ptr. I'm not sure how to clean it up any more than it already is.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

109

Does it not implicitly cast?

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

109

No, I think it only works in the other direction.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

109

Ok, I see. The constructor we need is marked explicit...

114

This cast shouldn't be necessary.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

114

Turns out this line is redundant anyways. EE->finalizeObject()calls invalidateInstructionCache().

[emmettneyman]

• cleaned up code and moved initialization code
• removed fake command line parsing

[morehouse]

clang/tools/clang-fuzzer/fuzzer-initialize/fuzzer_initialize.cpp

44

Unnecessary llvm::

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

80

If we do this, do we need to explicitly add the vectorizer pass below?

84

Let's simplify to setFunctionAttributes(getCPUStr(), getFeaturesStr(), *M).

86

OLvl is never used.

89

Can simplify to Passes.add(new TargetLibraryInfoWrapperPass(M->getTargetTriple)).

94

Why do we add a TargetTransformInfoWrapperPass to both Passes and FPasses?

94

Do we need both Passes and FPasses?

97

Why not just rename Owner to M and remove this line?

100

Let's simplify to Passes.add(createLoopVectorizePass()).

106

Why do we keep adding passes in different places?

109

Please simplify to builder.setMCJITMemoryManager(std::make_unique<RTDyldMemoryManager>()).

109

Move this closer to where it's used.

109

Do we need to call this again to run destructors after we execute f() ?

110

If the JIT does optimization already, do we need to call OptLLVM at all? Will the vectorizer kick in without OptLLVM?

110–113

I think f(a, b, c, 1) will also work.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

80

No we don't. I've deleted the line below.

89

Contrary to the function's name, getTargetTriple() actually returns a string, not a Triple. But I changed it to new TargetLibraryInfoWrapperPass(ModuleTriple) and deleted line 109.

94

I think because we're just adding a loop vectorize pass, we don't need FPasses. The loop vectorize pass lives within the regular ModulePassManager, not the FunctionPassManager. I decided to put it in the code so, in the future, it would be easier to add different kinds of passes to the fuzz target. Should I remove it?

110

I'll look into this more. I couldn't find an answer quickly.

[emmettneyman]

Fixed some things, made code cleaner

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

110

No, the JIT doesn't run any optimization passes. The optimization level given just controls CodeGen.

Source: http://lists.llvm.org/pipermail/llvm-dev/2016-May/099631.html

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

94

Let's remove it since it's currently unnecessary.

96

The indentation looks off. Please fix.

107

This uses llvm:make_unique, which was written when LLVM used C++11. But since LLVM now uses C++14, I think we should use std::make_unique instead.

117

We're allowing the JIT opt-level to be specified on the command line, but we're hard-coding OptLevel=3 for the optimization passes.

Do we need to allow the opt-level to be specified on the command line?

122

Remove whitespace on empty lines.

Actually, you could probably remove this line altogether since the call to f() can be grouped with the parameter setup above.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

107

LLVM doesn't use C++14 yet.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

97

Reverted it back since the change caused the fuzzer to crash.

107

We talked offline, but just to put it in writing: My current LLVM build uses C++11 so I'm keeping it llvm::make_unique.

117

Good point. Will change to make OptLLVM() use the same optimization level as the JIT Engine.

[emmettneyman]

• Code style fixes
• Removed FPasses
• Allowed CL Args to specify opt level for OptLLVM()

[emmettneyman]

Small change to fix line length

[morehouse]

Do we need to parse the arguments for opt-level, or can we just hardcode -O2 and remove the argument parsing code?

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

91

Shouldn't OLvl be a CodeGenOpt::Level?

97

You will need to move any uses of M above the call to std::move, since that leaves M in an invalid state.

[emmettneyman]

In D49526#1177208, @morehouse wrote:

Do we need to parse the arguments for opt-level, or can we just hardcode -O2 and remove the argument parsing code?

I have the argument parsing code since the original clang-proto-fuzzer code had it and @kcc mentioned we might want to change the command line arguments in the future.

[emmettneyman]

Changed int to CodeGenOpt::Level and fixed unique_ptr issue

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

125

We should be able to get rid of this line now, and rename Owner again.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

153

Can we use reinterpret_cast here?

[emmettneyman]

Made some minor fixes

clang/tools/clang-fuzzer/fuzzer-initialize/fuzzer_initialize.cpp

 /// This file implements two functions: one that returns the command line
 /// arguments for a given call to the fuzz target and one that initializes
 /// the fuzzer with the correct command line arguments.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "fuzzer_initialize.h"
 
+#include "llvm/InitializePasses.h"
+#include "llvm/PassRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include <cstring>
 
 using namespace clang_fuzzer;
+using namespace llvm;
 
 
 namespace clang_fuzzer {
 
 static std::vector<const char *> CLArgs;
 
 const std::vector<const char *>& GetCLArgs() {
   return CLArgs;
 }
 
 }
 
 extern "C" int LLVMFuzzerInitialize(int *argc, char ***argv) {
-  llvm::InitializeAllTargets();
-  llvm::InitializeAllTargetMCs();
-  llvm::InitializeAllAsmPrinters();
-  llvm::InitializeAllAsmParsers();
+  InitializeAllTargets();
+  InitializeAllTargetMCs();
+  InitializeAllAsmPrinters();
+  InitializeAllAsmParsers();
+  
+  PassRegistry &Registry = *PassRegistry::getPassRegistry();
+  initializeCore(Registry);
+  initializeScalarOpts(Registry);
+  initializeVectorization(Registry);
+  initializeIPO(Registry);
+  initializeAnalysis(Registry);
+  initializeTransformUtils(Registry);
+  initializeInstCombine(Registry);
+  initializeAggressiveInstCombine(Registry);
+  initializeInstrumentation(Registry);
+  initializeTarget(Registry);
 
   CLArgs.push_back("-O2");
   for (int I = 1; I < *argc; I++) {
     if (strcmp((*argv)[I], "-ignore_remaining_args=1") == 0) {
       for (I++; I < *argc; I++)
         CLArgs.push_back((*argv)[I]);
       break;
     }
   }
   return 0;
 }

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
+  CodeGen
   Core
+  ExecutionEngine
   IRReader
   MC
+  MCJIT
+  Object
+  RuntimeDyld
+  SelectionDAG
   Support
-  Analysis
-  )
+  Target
+  TransformUtils
+  native
 
 # Depend on LLVM IR intrinsic generation.
 set(handle_llvm_deps intrinsics_gen)
 if (CLANG_BUILT_STANDALONE)
   set(handle_llvm_deps)
 endif()
 
 add_clang_library(clangHandleLLVM
   handle_llvm.cpp
 
   DEPENDS
   ${handle_llvm_deps}
   )

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

 //==-- handle_llvm.cpp - Helper function for Clang fuzzers -----------------==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
-// Implements HandleLLVM for use by the Clang fuzzers. Mimics the llc tool to
-// compile an LLVM IR file to X86_64 assembly.
+// Implements HandleLLVM for use by the Clang fuzzers. First runs a loop
+// vectorizer optimization pass over the given IR code. Then mimics lli on both
+// versions to JIT the generated code and execute it. Currently, functions are 
+// executed on dummy inputs.
 //
 //===----------------------------------------------------------------------===//
 
 #include "handle_llvm.h"
 
 #include "llvm/ADT/Triple.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/CodeGen/CommandFlags.inc"
 #include "llvm/CodeGen/MachineModuleInfo.h"
+#include "llvm/CodeGen/TargetPassConfig.h"
+#include "llvm/ExecutionEngine/JITEventListener.h"
+#include "llvm/ExecutionEngine/JITSymbol.h"
+#include "llvm/ExecutionEngine/MCJIT.h"
+#include "llvm/ExecutionEngine/ObjectCache.h"
+#include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
+#include "llvm/ExecutionEngine/SectionMemoryManager.h"
+#include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/LegacyPassNameParser.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/IRReader/IRReader.h"
+#include "llvm/Pass.h"
 #include "llvm/PassRegistry.h"
-#include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/TargetSelect.h"
 #include "llvm/Target/TargetMachine.h"
-
-#include <cstdlib>
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+#include "llvm/Transforms/IPO.h"
+#include "llvm/Transforms/Vectorize.h"
 
 using namespace llvm;
 
+// Helper function to parse command line args and find the optimization level
 static void getOptLevel(const std::vector<const char *> &ExtraArgs,
                               CodeGenOpt::Level &OLvl) {
   // Find the optimization level from the command line args
   OLvl = CodeGenOpt::Default;
   for (auto &A : ExtraArgs) {
     if (A[0] == '-' && A[1] == 'O') {
       switch(A[2]) {
         case '0': OLvl = CodeGenOpt::None; break;
         case '1': OLvl = CodeGenOpt::Less; break;
         case '2': OLvl = CodeGenOpt::Default; break;
         case '3': OLvl = CodeGenOpt::Aggressive; break;
         default:
           errs() << "error: opt level must be between 0 and 3.\n";
           std::exit(1);
       }
     }
   }
 }
 
-void clang_fuzzer::HandleLLVM(const std::string &S,
-                              const std::vector<const char *> &ExtraArgs) {
-  // Parse ExtraArgs to set the optimization level
-  CodeGenOpt::Level OLvl;
-  getOptLevel(ExtraArgs, OLvl);
+void ErrorAndExit(std::string message) {
+  errs()<< "ERROR: " << message << "\n";
+  std::exit(1);
+}
 
-  // Set the Module to include the the IR code to be compiled
+// Helper function to add optimization passes to the TargetMachine at the 
+// specified optimization level, OptLevel
+static void AddOptimizationPasses(legacy::PassManagerBase &MPM,
+                                  CodeGenOpt::Level OptLevel,
+                                  unsigned SizeLevel) {
+  // Create and initialize a PassManagerBuilder
+  PassManagerBuilder Builder;
+  Builder.OptLevel = OptLevel;
+  Builder.SizeLevel = SizeLevel;
+  Builder.Inliner = createFunctionInliningPass(OptLevel, SizeLevel, false);
+  Builder.LoopVectorize = true;
+  Builder.populateModulePassManager(MPM);
+}
+
+// Mimics the opt tool to run an optimization pass over the provided IR
+std::string OptLLVM(const std::string &IR, CodeGenOpt::Level OLvl) {
+  // Create a module that will run the optimization passes
   SMDiagnostic Err;
-
   LLVMContext Context;
-  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(S, "IR"), Err, Context);
-  if (!M) {
-    errs() << "error: could not parse IR!\n";
-    std::exit(1);
-  }
+  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(IR, "IR"), Err, Context);
+  if (!M || verifyModule(*M, &errs()))
+    ErrorAndExit("Could not parse IR");
 
-  // Create a new Target
-  std::string Error;
-  const Target *TheTarget = TargetRegistry::lookupTarget(
-      sys::getDefaultTargetTriple(), Error);
-  if (!TheTarget) {
-    errs() << Error;
-    std::exit(1);
-  }
+  setFunctionAttributes(getCPUStr(), getFeaturesStr(), *M);
   
-  TargetOptions Options = InitTargetOptionsFromCodeGenFlags();
+  legacy::PassManager Passes;
+  Triple ModuleTriple(M->getTargetTriple());
   
-  // Create a new Machine
-  std::string CPUStr = getCPUStr();
-  std::string FeaturesStr = getFeaturesStr();
-  std::unique_ptr<TargetMachine> Target(TheTarget->createTargetMachine(
-      sys::getDefaultTargetTriple(), CPUStr, FeaturesStr, Options,
-      getRelocModel(), getCodeModel(), OLvl));
-
-  // Create a new PassManager
-  legacy::PassManager PM;
-  TargetLibraryInfoImpl TLII(Triple(M->getTargetTriple()));
-  PM.add(new TargetLibraryInfoWrapperPass(TLII));
-  M->setDataLayout(Target->createDataLayout());
- 
-  // Make sure the Module has no errors
-  if (verifyModule(*M, &errs())) {
-    errs() << "error: input module is broken!\n";
-    std::exit(1);
-  } 
+  Passes.add(new TargetLibraryInfoWrapperPass(ModuleTriple));
+  Passes.add(createTargetTransformInfoWrapperPass(TargetIRAnalysis()));
+  Passes.add(createVerifierPass());
+
+  AddOptimizationPasses(Passes, OLvl, 0);
+  
+  // Add a pass that writes the optimized IR to an output stream
+  std::string outString;
+  raw_string_ostream OS(outString);
+  Passes.add(createPrintModulePass(OS, "", false));
+
+  Passes.run(*M);
+  return OS.str();
+}
 
-  setFunctionAttributes(CPUStr, FeaturesStr, *M);
-  
-  raw_null_ostream OS;
-  Target->addPassesToEmitFile(PM, OS, nullptr, TargetMachine::CGFT_ObjectFile,
-                              false);
-  PM.run(*M);
+void CreateAndRunJITFun(const std::string &IR, CodeGenOpt::Level OLvl) {
+  SMDiagnostic Err;
+  LLVMContext Context;
+  std::unique_ptr<Module> Owner = parseIR(MemoryBufferRef(IR, "IR"), Err,
+                                          Context);
+  Module *M = Owner.get();
+  if (!M)
+    ErrorAndExit("Could not parse IR");
+
+  Function *EntryFunc = M->getFunction("foo");
+  if (!EntryFunc)
+    ErrorAndExit("Function not found in module");
+
+  std::string ErrorMsg;
+  EngineBuilder builder(std::move(Owner));
+  builder.setMArch(MArch);
+  builder.setMCPU(getCPUStr());
+  builder.setMAttrs(getFeatureList());
+  builder.setErrorStr(&ErrorMsg);
+  builder.setEngineKind(EngineKind::JIT);
+  builder.setUseOrcMCJITReplacement(false);
+  builder.setMCJITMemoryManager(make_unique<SectionMemoryManager>());
+  builder.setOptLevel(OLvl);
+  builder.setTargetOptions(InitTargetOptionsFromCodeGenFlags());
+
+  std::unique_ptr<ExecutionEngine> EE(builder.create());
+  if (!EE)
+    ErrorAndExit("Could not create execution engine");
+
+  EE->finalizeObject();
+  EE->runStaticConstructorsDestructors(false);
+
+  typedef void (*func)(int*, int*, int*, int);
+  func f = (func) EE->getPointerToFunction(EntryFunc); 
+
+  // Define some dummy arrays to use an input for now
+  int a[] = {1};
+  int b[] = {1};
+  int c[] = {1};
+  f(a, b, c, 1);
+
+  EE->runStaticConstructorsDestructors(true);
+}
+
+// Main fuzz target called by ExampleClangLLVMProtoFuzzer.cpp
+// Mimics the lli tool to JIT the LLVM IR code and execute it
+void clang_fuzzer::HandleLLVM(const std::string &IR,
+                              const std::vector<const char *> &ExtraArgs) {
+  // Parse ExtraArgs to set the optimization level
+  CodeGenOpt::Level OLvl;
+  getOptLevel(ExtraArgs, OLvl);
+
+  // First we optimize the IR by running a loop vectorizer pass
+  std::string OptIR = OptLLVM(IR, OLvl);
+  CreateAndRunJITFun(OptIR, OLvl);
+  CreateAndRunJITFun(IR, CodeGenOpt::None);
   
   return;
 }
-