This is an archive of the discontinued LLVM Phabricator instance.

Implemented Protobuf fuzzer for LLVM RISC-V MC Assembler
Needs ReviewPublic

Authored by apazos on Aug 22 2018, 5:08 PM.

Details

Summary
  • Implements an example fuzzer (llvm-mc-assemble-proto-fuzzer-example), with example Protobuf representation (example_asm_proto.proto) of a subset of RISC-V ASM, and corresponding example_proto_to_asm. Also includes example_mcfuzz.py Python driver script, to run the fuzzer and verify its output against a golden assembler GNU.
  • Also contains fuzzer (llvm-mc-assemble-proto-fuzzer-riscv), which fuzzes RISC-V's RV32 and RV64 base ISAs (the I, M, A, F, D extensions; the C extension; and also pseudo instructions).
  • Contains a third, more constrained version of the fuzzer, (llvm-mc-assemble-proto-fuzzer-riscv-fuzz-opnd-values). Fuzzes the same base ISAs (RISC-V I,M,A,F,D,C, and pseudo instructions), but restricts instructions' operands to the valid types; only mutates their values.
  • Contains driver script mcfuzz.py, which should be used to: (a) run the fuzzer a certain number of times; (b) run the fuzzer a specified number of times, and then verify the fuzzer's results against a golden assembler; or, (c) verify the fuzzer's results on a pre-generated test corpus, generated by previous fuzzer runs.

Diff Detail

Event Timeline

jocewei created this revision.Aug 22 2018, 5:08 PM
jocewei updated this revision to Diff 162211.Aug 23 2018, 9:57 AM

Fixed error in Python script

bcain added a subscriber: bcain.Aug 24 2018, 4:14 PM
bcain added a subscriber: kcc.Aug 24 2018, 4:20 PM

Please remove all paths pointing to /prj/...

tools/llvm-mc-assemble-proto-fuzzer/example_mcfuzz.py
36

Need to remove this default.

mgrang added inline comments.Aug 24 2018, 4:41 PM
tools/llvm-mc-assemble-proto-fuzzer/README.txt
57

Would be more readable if you write one flag per line.

81

AS --> assembler

96

AS --> assembler

jocewei updated this revision to Diff 162695.Aug 27 2018, 9:46 AM
This comment was removed by jocewei.
jocewei updated this revision to Diff 162697.Aug 27 2018, 9:53 AM

Fixed according to Mandeep's comments

jocewei updated this revision to Diff 162698.Aug 27 2018, 9:55 AM

Removed default path to GNU objdump (/prj/.../) from example_mcfuzz script

mgrang added inline comments.Aug 27 2018, 10:30 AM
tools/llvm-mc-assemble-proto-fuzzer/README.txt
14

Please list one ninja target per line.

19

Can remove the redirection "2>&1 | tee log". Let the user decide if and how he wants to redirect the output.

46

Remove this redirection.

62

Check indentation here.

tools/llvm-mc-assemble-proto-fuzzer/mcfuzz.py
80

All of these can be if's instead of elif's since you continue inside each one of them.

326

This for can be replaced with count1 = len(list1).

331

Same here.

341

Same here.

350

Same here.

mgrang added inline comments.Aug 27 2018, 12:44 PM
tools/llvm-mc-assemble-proto-fuzzer/mcfuzz.py
99

Can you remove line 97 and replace line 98 with this?
list_of_gnu_as_fails.append(error_msg[1])

306

file1 and file2 can be closed after this line.

326

In fact we don't even need count1, count2, count3 and count4. You could replace them with len(list1), etc.

mgrang added inline comments.Aug 27 2018, 12:55 PM
tools/llvm-mc-assemble-proto-fuzzer/README.txt
76

ie --> i.e.

83

ouptut --> output

111

assebler --> assembler

112

was any differences --> were any differences

tools/llvm-mc-assemble-proto-fuzzer/example_mcfuzz.py
60

If user specified --> If the user specified

tools/llvm-mc-assemble-proto-fuzzer/mcfuzz.py
46

using proto-to-asm tool --> using the proto-to-asm tool

jocewei updated this revision to Diff 162735.Aug 27 2018, 1:19 PM
jocewei updated this revision to Diff 162990.Aug 28 2018, 5:00 PM
jocewei edited the summary of this revision. (Show Details)

Combined protobuf for RV64 and RV32 base ISAs into single protobuf for RISC-V fuzzer tool.

asb added a comment.Aug 30 2018, 7:27 AM

I've been trying to build this locally. The cmake/modules/ProtobufMutator.cmake in Clang defines GIT_REPOSITORY and GIT_TAG, enabling protobuf to be checked out and built automatically. Is there a reason to not try to support that here? It seems to work for me when I edit to set GIT_REPOSITORY in the came way as the version of this file in Clang. I do need to go and manually add a symlink from lib to lib64 in /tools/llvm-mc-assembly-fuzzer/protobuf_mutator/src/protobuf_mutator-build/external, but I had that seem issue with the clang protobuf fuzzer build process.

The PB mutator tests were failing, so we download the PB mutator and disabled the tests. Are the tests passing for you?

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 5c13d2d..6c5ddb5 100644

  • a/src/CMakeLists.txt

+++ b/src/CMakeLists.txt
@@ -28,6 +28,7 @@ protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS

mutator_test_proto2.proto
mutator_test_proto3.proto)

+if (PB_MUTATOR_ENABLE_TEST)
add_executable(mutator_test

mutator_test.cc
utf8_fix_test.cc

@@ -52,3 +53,4 @@ foreach(SHARD RANGE ${TEST_SHARDS_MAX})
endforeach(SHARD)

add_dependencies(check mutator_test)
+endif()

jocewei updated this revision to Diff 164539.Sep 7 2018, 4:42 PM

Alphabetized opcode enums in .proto files, modified Python script, and added comment to handle_asm.cpp.

jocewei retitled this revision from Implemented Proto Fuzzer tool for LLVM MC Assembler to Implemented Protobuf fuzzer for LLVM MC Assembler.Sep 11 2018, 1:59 PM
apazos added inline comments.Sep 11 2018, 2:53 PM
tools/llvm-mc-assemble-proto-fuzzer/README.txt
22

Need to fix target name, replace _ with -

jocewei updated this revision to Diff 164982.EditedSep 11 2018, 3:00 PM

Fixed ninja target names in README

apazos commandeered this revision.Oct 3 2018, 7:09 PM
apazos edited reviewers, added: jocewei; removed: apazos.

Taken over the patch as Jocelyn is back to school.

apazos retitled this revision from Implemented Protobuf fuzzer for LLVM MC Assembler to Implemented Protobuf fuzzer for LLVM RISC-V MC Assembler.Oct 11 2018, 4:05 PM
apazos updated this revision to Diff 169345.Oct 11 2018, 6:03 PM

Removed PBM_FUZZ_PATH and PBM_REPO from build steps.
Relying on building these fuzzers with the latest version of libprotobuf (and libprotobuf-mutator), which should be installed in the system.

apazos updated this revision to Diff 169523.Oct 12 2018, 5:10 PM

Some cleanup.
Removed bracket field from Register Message. The Statement types add the brackets when needed.
Added PCREL_HI/LO modifiers.
Fixed mesage Label definition, added suffix to numeric label.
Added Label to Immediate Message handling in proto-to-asm-fuzz-opnd-values.