Updated llvm-proto-fuzzer to execute the compiled code

[emmettneyman]

Summary

Made changes to the llvm-proto-fuzzer

• Added loop vectorizer optimization pass in order to have two IR versions
• Updated old fuzz target to handle two different IR versions
• Wrote code to execute both versions in memory

[emmettneyman]

The files

Object.h
Object.cpp
llvm-objcopy.h

are from llvm/tools/llvm-obj-copy with only slight modifications, mostly deleting irrelevant parts.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

202

Did you look at using LLVM's JIT infrastructure to do this?

[morehouse]

You can probably get rid of the llvm-objcopy code and make this a lot simpler with something like:

1. Call getSection() on the Binary object to get the text section.
2. Read the sh_offset and sh_size of that section.
3. Copy sh_size bytes from the start of the binary buffer + sh_offset into your executable memory.
4. Run it.

[emmettneyman]

• Switched to JIT for compilation and execution

[emmettneyman]

• Cleaned up leftover code from mmap memcpy

[emmettneyman]

• Fixed typo that broke build

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

21

How are you doing your diff? Some of these changes are already upstream. Please rebase

clang/tools/clang-fuzzer/handle-llvm/fuzzer_initialize.cpp

52 ↗

(On Diff #156370)

Can we use fuzzer-initialize/ now?

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

53

Unless there's a good reason for global, let's make this a local.

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

182–195

Can you move this code (and maybe more of the duplicated code below) into a function and call it twice?

[emmettneyman]

Made fixes to patch, rebased CMake file

[emmettneyman]

Cleaned up code

Tried to get rid of ParseCommandLineOptions() call but could not figure out
how to initialize a PassInfo object without it.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

17

Typo introduced here.

25

Please don't add whitespace to a previously empty line.

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

10

an -> a

112

Maybe this should be const std::string &IR or StringRef.

113

Does this need to be called every time we get a new input? Can we call this from LLVMFuzzerInitialize()?

123

This shouldn't be required. You should be able to directly add the passes you want. See llvm/lib/Passes/PassBuilder.cpp.

132

Move this closer to where it's used.

133

Options is never used.

136

TM is never used.

137

Does this do anything since CPUStr and FeaturesStr are empty?

145

Can we just make FPasses on stack instead?

190

These 3 lines can be combined to builder.setMCJITMemoryManager(new SectionMemoryManager())

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190

I use RTDyldMM on line 208. Should I just keep these lines as they are?

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190

Ah, missed that. In that case, you can probably simplify this line to
builder.setMCJITMemoryManager(RTDyldMM).

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190

It looks like set MCJITMemoryManager() needs to take a unique_ptr. I'm not sure how to clean it up any more than it already is.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190

Does it not implicitly cast?

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190

No, I think it only works in the other direction.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

190

Ok, I see. The constructor we need is marked explicit...

208

This cast shouldn't be necessary.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

208

Turns out this line is redundant anyways. EE->finalizeObject()calls invalidateInstructionCache().

[emmettneyman]

• cleaned up code and moved initialization code
• removed fake command line parsing

[morehouse]

clang/tools/clang-fuzzer/fuzzer-initialize/fuzzer_initialize.cpp

44 ↗

(On Diff #157335)

Unnecessary llvm::

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

130

If we do this, do we need to explicitly add the vectorizer pass below?

134

Let's simplify to setFunctionAttributes(getCPUStr(), getFeaturesStr(), *M).

136

OLvl is never used.

139

Can simplify to Passes.add(new TargetLibraryInfoWrapperPass(M->getTargetTriple)).

144

Why do we add a TargetTransformInfoWrapperPass to both Passes and FPasses?

144

Do we need both Passes and FPasses?

150

Let's simplify to Passes.add(createLoopVectorizePass()).

156

Why do we keep adding passes in different places?

176

Why not just rename Owner to M and remove this line?

190

Please simplify to builder.setMCJITMemoryManager(std::make_unique<RTDyldMemoryManager>()).

191

If the JIT does optimization already, do we need to call OptLLVM at all? Will the vectorizer kick in without OptLLVM?

200

Move this closer to where it's used.

203

Do we need to call this again to run destructors after we execute f() ?

215–216

I think f(a, b, c, 1) will also work.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

130

No we don't. I've deleted the line below.

139

Contrary to the function's name, getTargetTriple() actually returns a string, not a Triple. But I changed it to new TargetLibraryInfoWrapperPass(ModuleTriple) and deleted line 109.

144

I think because we're just adding a loop vectorize pass, we don't need FPasses. The loop vectorize pass lives within the regular ModulePassManager, not the FunctionPassManager. I decided to put it in the code so, in the future, it would be easier to add different kinds of passes to the fuzz target. Should I remove it?

191

I'll look into this more. I couldn't find an answer quickly.

[emmettneyman]

Fixed some things, made code cleaner

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

191

No, the JIT doesn't run any optimization passes. The optimization level given just controls CodeGen.

Source: http://lists.llvm.org/pipermail/llvm-dev/2016-May/099631.html

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

144

Let's remove it since it's currently unnecessary.

175

The indentation looks off. Please fix.

188

This uses llvm:make_unique, which was written when LLVM used C++11. But since LLVM now uses C++14, I think we should use std::make_unique instead.

216

Remove whitespace on empty lines.

Actually, you could probably remove this line altogether since the call to f() can be grouped with the parameter setup above.

220

We're allowing the JIT opt-level to be specified on the command line, but we're hard-coding OptLevel=3 for the optimization passes.

Do we need to allow the opt-level to be specified on the command line?

[pcc]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

188

LLVM doesn't use C++14 yet.

[emmettneyman]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

176

Reverted it back since the change caused the fuzzer to crash.

188

We talked offline, but just to put it in writing: My current LLVM build uses C++11 so I'm keeping it llvm::make_unique.

220

Good point. Will change to make OptLLVM() use the same optimization level as the JIT Engine.

[emmettneyman]

• Code style fixes
• Removed FPasses
• Allowed CL Args to specify opt level for OptLLVM()

[emmettneyman]

Small change to fix line length

[morehouse]

Do we need to parse the arguments for opt-level, or can we just hardcode -O2 and remove the argument parsing code?

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

172

Shouldn't OLvl be a CodeGenOpt::Level?

176

You will need to move any uses of M above the call to std::move, since that leaves M in an invalid state.

[emmettneyman]

In D49526#1177208, @morehouse wrote:

Do we need to parse the arguments for opt-level, or can we just hardcode -O2 and remove the argument parsing code?

I have the argument parsing code since the original clang-proto-fuzzer code had it and @kcc mentioned we might want to change the command line arguments in the future.

[emmettneyman]

Changed int to CodeGenOpt::Level and fixed unique_ptr issue

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

239

We should be able to get rid of this line now, and rename Owner again.

[morehouse]

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

267

Can we use reinterpret_cast here?

[emmettneyman]

Made some minor fixes

clang/tools/clang-fuzzer/handle-llvm/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
+  CodeGen
   Core
+  ExecutionEngine
   IRReader
   MC
+  MCJIT
+  Object
+  RuntimeDyld
+  SelectionDAG
   Support
-  Analysis
-  )
+  Target
+  TransformUtils
+  native
 
-# Depend on LLVM IR intrinsic generation.
+# Depend on LLVM IR instrinsic generation.
 set(handle_llvm_deps intrinsics_gen)
 if (CLANG_BUILT_STANDALONE)
   set(handle_llvm_deps)
 endif()
 
 add_clang_library(clangHandleLLVM
   handle_llvm.cpp
   
   DEPENDS
   ${handle_llvm_deps}
   )

clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp

 //==-- handle_llvm.cpp - Helper function for Clang fuzzers -----------------==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
-// Implements HandleLLVM for use by the Clang fuzzers. Mimics the llc tool to
-// compile an LLVM IR file to X86_64 assembly.
+// Implements HandleLLVM for use by the Clang fuzzers. First runs an loop
+// vectorizer optimization pass over the given IR code. Then mimics lli on both
+// versions to JIT the generated code and execute it. Currently, functions are 
+// executed on dummy inputs.
 //
 //===----------------------------------------------------------------------===//
 
 #include "handle_llvm.h"
 
 #include "llvm/ADT/Triple.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/CodeGen/CommandFlags.inc"
 #include "llvm/CodeGen/MachineModuleInfo.h"
+#include "llvm/CodeGen/TargetPassConfig.h"
+#include "llvm/ExecutionEngine/JITEventListener.h"
+#include "llvm/ExecutionEngine/JITSymbol.h"
+#include "llvm/ExecutionEngine/MCJIT.h"
+#include "llvm/ExecutionEngine/ObjectCache.h"
+#include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
+#include "llvm/ExecutionEngine/SectionMemoryManager.h"
+#include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/LegacyPassNameParser.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/IRReader/IRReader.h"
+#include "llvm/Pass.h"
 #include "llvm/PassRegistry.h"
-#include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/TargetSelect.h"
 #include "llvm/Target/TargetMachine.h"
-
-#include <cstdlib>
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+#include "llvm/Transforms/IPO.h"
 
 using namespace llvm;
 
+static cl::list<const PassInfo*, bool, PassNameParser>
+PassList(cl::desc("Optimizations available:"));
+
+// Helper function to parse command line args and find the optimization level
 static void getOptLevel(const std::vector<const char *> &ExtraArgs,
                               CodeGenOpt::Level &OLvl) {
   // Find the optimization level from the command line args
   OLvl = CodeGenOpt::Default;
   for (auto &A : ExtraArgs) {
     if (A[0] == '-' && A[1] == 'O') {
       switch(A[2]) {
         case '0': OLvl = CodeGenOpt::None; break;
         case '1': OLvl = CodeGenOpt::Less; break;
         case '2': OLvl = CodeGenOpt::Default; break;
         case '3': OLvl = CodeGenOpt::Aggressive; break;
         default:
           errs() << "error: opt level must be between 0 and 3.\n";
           std::exit(1);
       }
     }
   }
 }
 
-void clang_fuzzer::HandleLLVM(const std::string &S,
-                              const std::vector<const char *> &ExtraArgs) {
-  // Parse ExtraArgs to set the optimization level
-  CodeGenOpt::Level OLvl;
-  getOptLevel(ExtraArgs, OLvl);
-
-  // Set the Module to include the the IR code to be compiled
-  SMDiagnostic Err;
-
-  LLVMContext Context;
-  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(S, "IR"), Err, Context);
-  if (!M) {
-    errs() << "error: could not parse IR!\n";
-    std::exit(1);
-  }
+// Helper function to call pass initialization functions
+void InitEverything() {
+  PassRegistry &Registry = *PassRegistry::getPassRegistry();
+  initializeCore(Registry);
+  initializeScalarOpts(Registry);
+  initializeVectorization(Registry);
+  initializeIPO(Registry);
+  initializeAnalysis(Registry);
+  initializeTransformUtils(Registry);
+  initializeInstCombine(Registry);
+  initializeAggressiveInstCombine(Registry);
+  initializeInstrumentation(Registry);
+  initializeTarget(Registry);
 
-  // Create a new Target
-  std::string Error;
-  const Target *TheTarget = TargetRegistry::lookupTarget(
-      sys::getDefaultTargetTriple(), Error);
-  if (!TheTarget) {
-    errs() << Error;
-    std::exit(1);
-  }
+void ErrorAndExit(std::string message) {
+  errs()<< "ERROR: " << message << "\n";
 
+// Helper function to add optimization passes to the TargetMachine at the 
+// specified optimization level, OptLevel
+static void AddOptimizationPasses(legacy::PassManagerBase &MPM,
+                                  legacy::FunctionPassManager &FPM,
+                                  unsigned OptLevel, unsigned SizeLevel) {
+  // Verify that input is correct by adding a verifier pass
+  FPM.add(createVerifierPass());
+
+  // Create and initializa a PassManagerBuilder
+  PassManagerBuilder Builder;
+  Builder.OptLevel = OptLevel;
+  Builder.SizeLevel = SizeLevel;
+  Builder.Inliner = createFunctionInliningPass(OptLevel, SizeLevel, false);
+  Builder.LoopVectorize = true;
+  Builder.populateFunctionPassManager(FPM);
+  Builder.populateModulePassManager(MPM);
+}
+
+// Mimics the opt tool to run an optimization pass over the provided IR
+std::string OptLLVM(const std::string IR, CodeGenOpt::Level &OLvl) {
+  InitEverything();
+ 
+  // Mimic argc and argv and pass them to ParseCommandLineOptions to initilize
+  // PassList, ie which optimizations we want to run on the IR
+  // TODO: Find a better way of doing this
+  const char *args[2];
+  std::string arg0 = "llvm-proto-fuzzer";
+  std::string arg1 = "-loop-vectorize";
+  args[0] = arg0.c_str();
+  args[1] = arg1.c_str();
+  cl::ParseCommandLineOptions(2, args);
+
+  // Create a module that will run the optimization passes
+  SMDiagnostic Err;
+  LLVMContext Context;
+  std::unique_ptr<Module> M = parseIR(MemoryBufferRef(IR, "IR"), Err, Context);
+  if (!M || verifyModule(*M, &errs()))
+    ErrorAndExit("Could not parse IR");
+
+  Triple ModuleTriple(M->getTargetTriple());
   TargetOptions Options = InitTargetOptionsFromCodeGenFlags();
+  std::string CPUStr;
+  std::string FeaturesStr;
+  std::unique_ptr<TargetMachine> TM(nullptr);
+  setFunctionAttributes(CPUStr, FeaturesStr, *M);
   
-  // Create a new Machine
-  std::string CPUStr = getCPUStr();
-  std::string FeaturesStr = getFeaturesStr();
-  std::unique_ptr<TargetMachine> Target(TheTarget->createTargetMachine(
-      sys::getDefaultTargetTriple(), CPUStr, FeaturesStr, Options,
-      getRelocModel(), getCodeModel(), OLvl));
-
-  // Create a new PassManager
-  legacy::PassManager PM;
-  TargetLibraryInfoImpl TLII(Triple(M->getTargetTriple()));
-  PM.add(new TargetLibraryInfoWrapperPass(TLII));
-  M->setDataLayout(Target->createDataLayout());
- 
-  // Make sure the Module has no errors
-  if (verifyModule(*M, &errs())) {
-    errs() << "error: input module is broken!\n";
-    std::exit(1);
-  } 
+  legacy::PassManager Passes;
+  TargetLibraryInfoImpl TLII(ModuleTriple);
+  Passes.add(new TargetLibraryInfoWrapperPass(TLII));
+  Passes.add(createTargetTransformInfoWrapperPass(TargetIRAnalysis()));
+  
+  std::unique_ptr<legacy::FunctionPassManager> FPasses;
+  FPasses.reset(new legacy::FunctionPassManager(M.get()));
+  FPasses->add(createTargetTransformInfoWrapperPass(TargetIRAnalysis()));
+
+  AddOptimizationPasses(Passes, *FPasses, 3, 0);
+  const PassInfo *PassInf = PassList[0];
+  Pass *P = PassInf->getNormalCtor()();
+  if (!P)
+    ErrorAndExit("Cannot create IR pass");
+  Passes.add(P);
+  
+  FPasses->doInitialization();
+  for (Function &F : *M)
+    FPasses->run(F);
+  FPasses->doFinalization();
+  Passes.add(createVerifierPass());
+  
+  // Add a pass that writes the optimized IR to an output stream
+  std::string outString;
+  raw_string_ostream OS(outString);
+  Passes.add(createPrintModulePass(OS, "", false));
+
+  Passes.run(*M);
+
+  return OS.str();
+}
 
-  setFunctionAttributes(CPUStr, FeaturesStr, *M);
-  
-  raw_null_ostream OS;
-  Target->addPassesToEmitFile(PM, OS, nullptr, TargetMachine::CGFT_ObjectFile,
-                              false);
-  PM.run(*M);
+void CreateAndRunJITFun(std::string IR, CodeGenOpt::Level OLvl) {
+  SMDiagnostic Err;
+  LLVMContext Context;
+  std::unique_ptr<Module> Owner = parseIR(MemoryBufferRef(IR, "IR"), Err,
+                                           Context);
+  Module *M = Owner.get();
+  if (!M)
+    ErrorAndExit("Could not parse IR");
+
+  std::string ErrorMsg;
+  EngineBuilder builder(std::move(Owner));
+  builder.setMArch(MArch);
+  builder.setMCPU(getCPUStr());
+  builder.setMAttrs(getFeatureList());
+  builder.setErrorStr(&ErrorMsg);
+  builder.setEngineKind(EngineKind::JIT);
+  builder.setUseOrcMCJITReplacement(false);
+  RTDyldMemoryManager *RTDyldMM = new SectionMemoryManager();
+  builder.setMCJITMemoryManager(
+      std::unique_ptr<RTDyldMemoryManager>(RTDyldMM));
+  builder.setOptLevel(OLvl);
+  builder.setTargetOptions(InitTargetOptionsFromCodeGenFlags());
+
+  std::unique_ptr<ExecutionEngine> EE(builder.create());
+  if (!EE)
+    ErrorAndExit("Could not create execution engine");
+
+  Function *EntryFunc = M->getFunction("foo");
+  if (!EntryFunc)
+    ErrorAndExit("Function not found in module");
+
+  EE->finalizeObject();
+  EE->runStaticConstructorsDestructors(false);
+
+  typedef void (*func)(int*, int*, int*, int);
+  func f = (func) EE->getPointerToFunction(EntryFunc); 
+
+  static_cast<SectionMemoryManager*>(RTDyldMM)->invalidateInstructionCache();
+
+  // Define some dummy arrays to use an input for now
+  int a[] = {1};
+  int b[] = {1};
+  int c[] = {1};
+  
+  (*f)(a, b, c, 1);
+}
+
+// Main fuzz target called by ExampleClangLLVMProtoFuzzer.cpp
+// Mimics the lli tool to JIT the LLVM IR code and execute it
+void clang_fuzzer::HandleLLVM(const std::string &IR,
+                              const std::vector<const char *> &ExtraArgs) {
+  // Parse ExtraArgs to set the optimization level
+  CodeGenOpt::Level OLvl;
+  getOptLevel(ExtraArgs, OLvl);
+  
+  // First we optimize the IR by running a loop vectorizer pass
+  std::string OptIR = OptLLVM(IR, OLvl);
+  CreateAndRunJITFun(OptIR, OLvl);
+  CreateAndRunJITFun(IR, CodeGenOpt::None);
   
   return;
 }
-