This is an archive of the discontinued LLVM Phabricator instance.

[Fuzzer][safestack][scudo] Add Fuchsia as supported OS
AbandonedPublic

Authored by aarongreen on Jan 24 2018, 1:58 PM.

Details

Reviewers
phosek
Summary

This adds Fuchsia to the list of possible operating systems when setting the COMPILER_RT_HAS_* variables for libFuzzer, scudo, and safestack.

Diff Detail

Event Timeline

aarongreen created this revision.Jan 24 2018, 1:58 PM
Herald added subscribers: Restricted Project, llvm-commits, cryptoad and 2 others. · View Herald TranscriptJan 24 2018, 1:58 PM

Adding is fine, but can you run tests for each feature? What are the results?

We (@flowerhack & I) have a couple of items left to straighten out for Fuchsia.
You can compile it but it won't be fully functional as of now.

Well, unfortunately Fuchsia isn't self-hosting yet, and IIUC llvm-lit doesn't quite support running the tests remotely over SSH. I know phosek is working on that. I can run the existing tests and confirm they still pass for other platforms.

/src/llvm-project/build$ ./bin/llvm-lit --verbose ../llvm/runtimes/compiler-rt/test/fuzzer/
llvm-lit: /src/llvm-project/llvm/runtimes/compiler-rt/test/fuzzer/lit.cfg:33: note: lsan feature available
llvm-lit: /src/llvm-project/llvm/runtimes/compiler-rt/test/fuzzer/lit.cfg:45: note: linux feature available

  • Testing: 113 tests, 48 threads --

PASS: LLVMFuzzer :: max-number-of-runs.test (1 of 113)
PASS: LLVMFuzzer :: memcmp64.test (2 of 113)
PASS: LLVMFuzzer :: gc-sections.test (3 of 113)
PASS: LLVMFuzzer :: bad-strcmp.test (4 of 113)
PASS: LLVMFuzzer :: recommended-dictionary.test (5 of 113)
PASS: LLVMFuzzer :: memcmp.test (6 of 113)
PASS: LLVMFuzzer :: cleanse.test (7 of 113)
PASS: LLVMFuzzer :: fuzzer-flags.test (8 of 113)
PASS: LLVMFuzzer :: inline-8bit-counters.test (9 of 113)
PASS: LLVMFuzzer :: fuzzer-custommutator.test (10 of 113)
PASS: LLVMFuzzer :: exit-report.test (11 of 113)
PASS: LLVMFuzzer :: fuzzer-seed.test (12 of 113)
PASS: LLVMFuzzer :: equivalence.test (13 of 113)
PASS: LLVMFuzzer :: merge-posix.test (14 of 113)
PASS: LLVMFuzzer :: fuzzer-dirs.test (15 of 113)
PASS: LLVMFuzzer :: fuzzer-finalstats.test (16 of 113)
PASS: LLVMFuzzer :: fuzzer-ubsan.test (17 of 113)
PASS: LLVMFuzzer :: fuzzer-printcovpcs.test (18 of 113)
PASS: LLVMFuzzer :: fuzzer-fdmask.test (19 of 113)
PASS: LLVMFuzzer :: cxxstring.test (20 of 113)
PASS: LLVMFuzzer :: fuzzer-segv.test (21 of 113)
PASS: LLVMFuzzer :: caller-callee.test (22 of 113)
PASS: LLVMFuzzer :: merge-summary.test (23 of 113)
PASS: LLVMFuzzer :: fuzzer-runs.test (24 of 113)
PASS: LLVMFuzzer :: fuzzer-customcrossoverandmutate.test (25 of 113)
PASS: LLVMFuzzer :: reduce_inputs.test (26 of 113)
PASS: LLVMFuzzer :: overwrite-input.test (27 of 113)
PASS: LLVMFuzzer :: merge-control-file.test (28 of 113)
PASS: LLVMFuzzer :: standalone.test (29 of 113)
PASS: LLVMFuzzer :: swap-cmp.test (30 of 113)
PASS: LLVMFuzzer :: coverage.test (31 of 113)
PASS: LLVMFuzzer :: merge.test (32 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Corpus.Distribution (33 of 113)
PASS: LLVMFuzzer :: strcmp.test (34 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Fuzzer.CrossOver (35 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Fuzzer.ForEachNonZeroByte (36 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerCommand.Create (37 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Fuzzer.Hash (38 of 113)
PASS: LLVMFuzzer :: print-func.test (39 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerCommand.ModifyFlags (40 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerCommand.ModifyArguments (41 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerCommand.SetOutput (42 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerDictionary.ParseOneDictionaryEntry (43 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerDictionary.ParseDictionaryFile (44 of 113)
PASS: LLVMFuzzer :: value-profile-cmp2.test (45 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeASCIIInteger1 (46 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.AddWordFromDictionary1 (47 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.AddWordFromDictionary2 (48 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeASCIIInteger2 (49 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeBinaryInteger1 (50 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeBinaryInteger2 (51 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeBit1 (52 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.CopyPart1 (53 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeByte1 (54 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeBit2 (55 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.CopyPart2 (56 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ChangeByte2 (57 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.EraseBytes1 (58 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.EraseBytes2 (59 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.InsertByte1 (60 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.InsertRepeatedBytes1 (61 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.InsertByte2 (62 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ShuffleBytes1 (63 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.InsertRepeatedBytes2 (64 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Merge.Good (65 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerUtil.Base64 (66 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Merge.Bad (67 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/Merge.Merge (68 of 113)
PASS: LLVMFuzzer :: dump_coverage.test (69 of 113)
PASS: LLVMFuzzer :: exit_on_src_pos.test (70 of 113)
PASS: LLVMFuzzer :: afl-driver-extra-stats.test (71 of 113)
PASS: LLVMFuzzer :: afl-driver.test (72 of 113)
PASS: LLVMFuzzer :: afl-driver-stderr.test (73 of 113)
PASS: LLVMFuzzer-Unittest :: ./Fuzzer-x86_64-Test/FuzzerMutate.ShuffleBytes2 (74 of 113)
PASS: LLVMFuzzer :: minimize_two_crashes.test (75 of 113)
PASS: LLVMFuzzer :: value-profile-cmp3.test (76 of 113)
PASS: LLVMFuzzer :: fuzzer-dict.test (77 of 113)
PASS: LLVMFuzzer :: strstr.test (78 of 113)
PASS: LLVMFuzzer :: deep-recursion.test (79 of 113)
PASS: LLVMFuzzer :: fprofile-instr-generate.test (80 of 113)
PASS: LLVMFuzzer :: simple-cmp.test (81 of 113)
PASS: LLVMFuzzer :: extra-counters.test (82 of 113)
PASS: LLVMFuzzer :: value-profile-strncmp.test (83 of 113)
PASS: LLVMFuzzer :: strncmp.test (84 of 113)
PASS: LLVMFuzzer :: repeated-bytes.test (85 of 113)
PASS: LLVMFuzzer :: fuzzer-singleinputs.test (86 of 113)
PASS: LLVMFuzzer :: shrink.test (87 of 113)
PASS: LLVMFuzzer :: trace-malloc-unbalanced.test (88 of 113)
PASS: LLVMFuzzer :: ulimit.test (89 of 113)
PASS: LLVMFuzzer :: trace-malloc.test (90 of 113)
PASS: LLVMFuzzer :: trace-pc.test (91 of 113)
PASS: LLVMFuzzer :: value-profile-div.test (92 of 113)
PASS: LLVMFuzzer :: trace-malloc-2.test (93 of 113)
PASS: LLVMFuzzer :: fuzzer-customcrossover.test (94 of 113)
PASS: LLVMFuzzer :: fuzzer-oom-with-profile.test (95 of 113)
PASS: LLVMFuzzer :: trace-malloc-threaded.test (96 of 113)
PASS: LLVMFuzzer :: merge-sigusr.test (97 of 113)
PASS: LLVMFuzzer :: fuzzer-threaded.test (98 of 113)
PASS: LLVMFuzzer :: equivalence-signals.test (99 of 113)
PASS: LLVMFuzzer :: value-profile-mem.test (100 of 113)
PASS: LLVMFuzzer :: value-profile-strcmp.test (101 of 113)
PASS: LLVMFuzzer :: value-profile-cmp4.test (102 of 113)
PASS: LLVMFuzzer :: value-profile-set.test (103 of 113)
PASS: LLVMFuzzer :: fuzzer-leak.test (104 of 113)
PASS: LLVMFuzzer :: fuzzer-timeout.test (105 of 113)
PASS: LLVMFuzzer :: disable-leaks.test (106 of 113)
PASS: LLVMFuzzer :: sigusr.test (107 of 113)
PASS: LLVMFuzzer :: minimize_crash.test (108 of 113)
PASS: LLVMFuzzer :: value-profile-load.test (109 of 113)
PASS: LLVMFuzzer :: fuzzer-oom.test (110 of 113)
PASS: LLVMFuzzer :: value-profile-cmp.test (111 of 113)
PASS: LLVMFuzzer :: value-profile-switch.test (112 of 113)
PASS: LLVMFuzzer :: fuzzer.test (113 of 113)
Testing Time: 31.91s

Expected Passes    : 113

/src/llvm-project/build$ ./bin/llvm-lit --verbose ../llvm/runtimes/compiler-rt/test/scudo/

  • Testing: 20 tests, 20 threads --

PASS: Scudo-x86_64 :: preload.cpp (1 of 20)
PASS: Scudo-x86_64 :: overflow.c (2 of 20)
PASS: Scudo-x86_64 :: preinit.c (3 of 20)
PASS: Scudo-x86_64 :: alignment.c (4 of 20)
PASS: Scudo-x86_64 :: valloc.c (5 of 20)
PASS: Scudo-x86_64 :: tsd_destruction.c (6 of 20)
PASS: Scudo-x86_64 :: memalign.c (7 of 20)
PASS: Scudo-x86_64 :: secondary.c (8 of 20)
PASS: Scudo-x86_64 :: quarantine.c (9 of 20)
PASS: Scudo-x86_64 :: threads.c (10 of 20)
PASS: Scudo-x86_64 :: options.cpp (11 of 20)
PASS: Scudo-x86_64 :: double-free.cpp (12 of 20)
PASS: Scudo-x86_64 :: random_shuffle.cpp (13 of 20)
PASS: Scudo-x86_64 :: mismatch.cpp (14 of 20)
PASS: Scudo-x86_64 :: sized-delete.cpp (15 of 20)
PASS: Scudo-x86_64 :: sizes.cpp (16 of 20)
PASS: Scudo-x86_64 :: realloc.cpp (17 of 20)
PASS: Scudo-x86_64 :: malloc.cpp (18 of 20)
PASS: Scudo-x86_64 :: interface.cpp (19 of 20)
PASS: Scudo-x86_64 :: rss.c (20 of 20)
Testing Time: 4.95s

Expected Passes    : 20

/src/llvm-project/build$ ./bin/llvm-lit --verbose ../llvm/runtimes/compiler-rt/test/safestack/

  • Testing: 8 tests, 8 threads --

UNSUPPORTED: SafeStack :: lto.c (1 of 8)
PASS: SafeStack :: buffer-copy.c (2 of 8)
PASS: SafeStack :: init.c (3 of 8)
PASS: SafeStack :: buffer-copy-vla.c (4 of 8)
PASS: SafeStack :: overflow.c (5 of 8)
PASS: SafeStack :: pthread-cleanup.c (6 of 8)
PASS: SafeStack :: pthread.c (7 of 8)
PASS: SafeStack :: canary.c (8 of 8)
Testing Time: 0.52s

Expected Passes    : 7
Unsupported Tests  : 1

We (@flowerhack & I) have a couple of items left to straighten out for Fuchsia.
You can compile it but it won't be fully functional as of now.

I should have double checked with flowerhack first; last I talked with her I was under the impression it was ready enough. I certainly can pull that if you want (and safestack, if mcgrathr wants). TBH, I'm mainly after getting libFuzzer turned on, and figured I'd flip all three while here.

I've got a local port of Scudo to NetBSD, but the tests were badly unportable (introspection through GLIBC's <malloc.h>). This is why I keep it locally.

Your results look well.

I think those tests results are for Linux (at least for Scudo).
We don't pass everything on Fuchsia yet.

aarongreen abandoned this revision.Jan 24 2018, 2:46 PM

I think those tests results are for Linux (at least for Scudo).
We don't pass everything on Fuchsia yet.

Yeah, that was what my remark before the results was trying to say. phosek is working on a way to run them on Fuchsia. It isn't possible to do so via llvm-lit today. Standalone unit tests can be built and run on Fuchsia; that's the approach I took with libFuzzer. I'll work with flowerhack and mcgrathr to enable only what we can test and then resubmit.

I think those tests results are for Linux (at least for Scudo).
We don't pass everything on Fuchsia yet.

Yeah, that was what my remark before the results was trying to say. phosek is working on a way to run them on Fuchsia. It isn't possible to do so via llvm-lit today. Standalone unit tests can be built and run on Fuchsia; that's the approach I took with libFuzzer. I'll work with flowerhack and mcgrathr to enable only what we can test and then resubmit.

What's your malloc implementation? jemalloc? A custom one?